agora inbox for [email protected]
help / color / mirror / Atom feed[PATCH v19] Avoid orphaned objects dependencies
142+ messages / 3 participants
[nested] [flat]
* [PATCH v19] Avoid orphaned objects dependencies
@ 2024-03-29 15:43 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2024-03-29 15:43 UTC (permalink / raw)
It's currently possible to create orphaned objects dependencies, for example:
Scenario 1:
session 1: begin; drop schema schem;
session 2: create a function in the schema schem
session 1: commit;
With the above, the function created in session 2 would be linked to a non
existing schema.
Scenario 2:
session 1: begin; create a function in the schema schem
session 2: drop schema schem;
session 1: commit;
With the above, the function created in session 1 would be linked to a non
existing schema.
To avoid those scenarios, a new lock (that conflicts with a lock taken by DROP)
has been put in place before the dependencies are being recorded. With this in
place, the drop schema in scenario 2 would be locked.
Also, after the new lock attempt, the patch checks that the object still exists:
with this in place session 2 in scenario 1 would be locked and would report an
error once session 1 committs (that would not be the case should session 1 abort
the transaction).
The patch takes into account any type of objects except the ones that are pinned
(they are not droppable because the system requires it).
A special case is done for objects that belong to the RelationRelationId class.
For those, we should be in one of the two following cases that would already
prevent the relation to be dropped:
1. The relation is already locked (could be an existing relation or a relation
that we are creating).
2. The relation is protected indirectly (i.e an index protected by a lock on
its table, a table protected by a lock on a function that depends of the table...)
To avoid any risks for the RelationRelationId class case, we acquire a lock if
there is none. That may add unnecessary lock for 2. but that's worth it.
The patch adds a few tests for some dependency cases (that would currently produce
orphaned objects):
- schema and function (as the above scenarios)
- alter a dependency (function and schema)
- function and arg type
- function and return type
- function and function
- domain and domain
- table and type
- server and foreign data wrapper
---
src/backend/catalog/aclchk.c | 1 +
src/backend/catalog/dependency.c | 201 ++++++++++++++++++
src/backend/catalog/heap.c | 7 +
src/backend/catalog/index.c | 26 +++
src/backend/catalog/objectaddress.c | 57 +++++
src/backend/catalog/pg_aggregate.c | 9 +
src/backend/catalog/pg_attrdef.c | 1 +
src/backend/catalog/pg_cast.c | 5 +
src/backend/catalog/pg_collation.c | 1 +
src/backend/catalog/pg_constraint.c | 26 +++
src/backend/catalog/pg_conversion.c | 2 +
src/backend/catalog/pg_depend.c | 39 +++-
src/backend/catalog/pg_operator.c | 19 ++
src/backend/catalog/pg_proc.c | 17 +-
src/backend/catalog/pg_publication.c | 11 +
src/backend/catalog/pg_range.c | 6 +
src/backend/catalog/pg_type.c | 39 ++++
src/backend/catalog/toasting.c | 1 +
src/backend/commands/alter.c | 4 +
src/backend/commands/amcmds.c | 1 +
src/backend/commands/cluster.c | 7 +
src/backend/commands/event_trigger.c | 1 +
src/backend/commands/extension.c | 5 +
src/backend/commands/foreigncmds.c | 7 +
src/backend/commands/functioncmds.c | 6 +
src/backend/commands/indexcmds.c | 2 +
src/backend/commands/opclasscmds.c | 18 ++
src/backend/commands/operatorcmds.c | 30 +++
src/backend/commands/policy.c | 2 +
src/backend/commands/proclang.c | 3 +
src/backend/commands/sequence.c | 2 +
src/backend/commands/statscmds.c | 10 +
src/backend/commands/tablecmds.c | 32 ++-
src/backend/commands/trigger.c | 29 ++-
src/backend/commands/tsearchcmds.c | 73 ++++++-
src/backend/commands/typecmds.c | 84 ++++++++
src/backend/rewrite/rewriteDefine.c | 1 +
src/backend/utils/errcodes.txt | 1 +
src/include/catalog/dependency.h | 3 +
src/include/catalog/objectaddress.h | 1 +
.../expected/test_dependencies_locks.out | 129 +++++++++++
src/test/isolation/isolation_schedule | 1 +
.../specs/test_dependencies_locks.spec | 89 ++++++++
src/test/regress/expected/alter_table.out | 11 +-
44 files changed, 998 insertions(+), 22 deletions(-)
39.7% src/backend/catalog/
30.5% src/backend/commands/
17.0% src/test/isolation/expected/
10.5% src/test/isolation/specs/
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index 02a754cc30a..420641633c5 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -1341,6 +1341,7 @@ SetDefaultACL(InternalDefaultACL *iacls)
referenced.objectId = iacls->nspid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(NamespaceRelationId, iacls->nspid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
}
}
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 18316a3968b..34884aa8950 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1519,6 +1519,69 @@ AcquireDeletionLock(const ObjectAddress *object, int flags)
}
}
+/*
+ * LockNotPinnedObjectById
+ *
+ * Lock the object that we are about to record a dependency on.
+ * After it's locked, verify that it hasn't been dropped while we
+ * weren't looking. If the object has been dropped, this function
+ * does not return!
+ */
+void
+LockNotPinnedObjectById(const ObjectAddress *object)
+{
+ if (isObjectPinned(object))
+ return;
+
+ if (object->classId == RelationRelationId)
+ {
+ Assert(!IsSharedRelation(object->objectId));
+
+ /*
+ * We must be in one of the two following cases that would already
+ * prevent the relation to be dropped: 1. The relation is already
+ * locked (could be an existing relation or a relation that we are
+ * creating). 2. The relation is protected indirectly (i.e an index
+ * protected by a lock on its table, a table protected by a lock on a
+ * function that depends of the table...). To avoid any risks, acquire
+ * a lock if there is none. That may add unnecessary lock for 2. but
+ * that's worth it.
+ */
+ if (!CheckRelationOidLockedByMe(object->objectId, AccessShareLock, true))
+ LockRelationOid(object->objectId, AccessShareLock);
+ }
+ else
+ {
+ /* assume we should lock the whole object not a sub-object */
+ LockDatabaseObject(object->classId, object->objectId, 0, AccessShareLock);
+ }
+
+ /* check if object still exists */
+ if (!ObjectByIdExist(object))
+ ereport(ERROR,
+ (errcode(ERRCODE_DEPENDENT_OBJECTS_DOES_NOT_EXIST),
+ errmsg("dependent object does not exist"),
+ errdetail("Class OID is %u and object OID is %u",
+ object->classId, object->objectId)));
+
+ return;
+}
+
+/*
+ * LockNotPinnedObject
+ *
+ * Lock the object that we are about to record a dependency on.
+ */
+void
+LockNotPinnedObject(Oid classid, Oid objid)
+{
+ ObjectAddress object;
+
+ ObjectAddressSet(object, classid, objid);
+
+ LockNotPinnedObjectById(&object);
+}
+
/*
* ReleaseDeletionLock - release an object deletion lock
*
@@ -1730,6 +1793,7 @@ find_expr_references_walker(Node *node,
if (rte->rtekind == RTE_RELATION)
{
/* If it's a plain relation, reference this column */
+ LockNotPinnedObject(RelationRelationId, rte->relid);
add_object_address(RelationRelationId, rte->relid, var->varattno,
context->addrs);
}
@@ -1756,6 +1820,7 @@ find_expr_references_walker(Node *node,
Oid objoid;
/* A constant must depend on the constant's datatype */
+ LockNotPinnedObject(TypeRelationId, con->consttype);
add_object_address(TypeRelationId, con->consttype, 0,
context->addrs);
@@ -1767,8 +1832,11 @@ find_expr_references_walker(Node *node,
*/
if (OidIsValid(con->constcollid) &&
con->constcollid != DEFAULT_COLLATION_OID)
+ {
+ LockNotPinnedObject(CollationRelationId, con->constcollid);
add_object_address(CollationRelationId, con->constcollid, 0,
context->addrs);
+ }
/*
* If it's a regclass or similar literal referring to an existing
@@ -1785,59 +1853,83 @@ find_expr_references_walker(Node *node,
objoid = DatumGetObjectId(con->constvalue);
if (SearchSysCacheExists1(PROCOID,
ObjectIdGetDatum(objoid)))
+ {
+ LockNotPinnedObject(ProcedureRelationId, objoid);
add_object_address(ProcedureRelationId, objoid, 0,
context->addrs);
+ }
break;
case REGOPEROID:
case REGOPERATOROID:
objoid = DatumGetObjectId(con->constvalue);
if (SearchSysCacheExists1(OPEROID,
ObjectIdGetDatum(objoid)))
+ {
+ LockNotPinnedObject(OperatorRelationId, objoid);
add_object_address(OperatorRelationId, objoid, 0,
context->addrs);
+ }
break;
case REGCLASSOID:
objoid = DatumGetObjectId(con->constvalue);
if (SearchSysCacheExists1(RELOID,
ObjectIdGetDatum(objoid)))
+ {
+ LockNotPinnedObject(RelationRelationId, objoid);
add_object_address(RelationRelationId, objoid, 0,
context->addrs);
+ }
break;
case REGTYPEOID:
objoid = DatumGetObjectId(con->constvalue);
if (SearchSysCacheExists1(TYPEOID,
ObjectIdGetDatum(objoid)))
+ {
+ LockNotPinnedObject(TypeRelationId, objoid);
add_object_address(TypeRelationId, objoid, 0,
context->addrs);
+ }
break;
case REGCOLLATIONOID:
objoid = DatumGetObjectId(con->constvalue);
if (SearchSysCacheExists1(COLLOID,
ObjectIdGetDatum(objoid)))
+ {
+ LockNotPinnedObject(CollationRelationId, objoid);
add_object_address(CollationRelationId, objoid, 0,
context->addrs);
+ }
break;
case REGCONFIGOID:
objoid = DatumGetObjectId(con->constvalue);
if (SearchSysCacheExists1(TSCONFIGOID,
ObjectIdGetDatum(objoid)))
+ {
+ LockNotPinnedObject(TSConfigRelationId, objoid);
add_object_address(TSConfigRelationId, objoid, 0,
context->addrs);
+ }
break;
case REGDICTIONARYOID:
objoid = DatumGetObjectId(con->constvalue);
if (SearchSysCacheExists1(TSDICTOID,
ObjectIdGetDatum(objoid)))
+ {
+ LockNotPinnedObject(TSDictionaryRelationId, objoid);
add_object_address(TSDictionaryRelationId, objoid, 0,
context->addrs);
+ }
break;
case REGNAMESPACEOID:
objoid = DatumGetObjectId(con->constvalue);
if (SearchSysCacheExists1(NAMESPACEOID,
ObjectIdGetDatum(objoid)))
+ {
+ LockNotPinnedObject(NamespaceRelationId, objoid);
add_object_address(NamespaceRelationId, objoid, 0,
context->addrs);
+ }
break;
/*
@@ -1859,18 +1951,23 @@ find_expr_references_walker(Node *node,
Param *param = (Param *) node;
/* A parameter must depend on the parameter's datatype */
+ LockNotPinnedObject(TypeRelationId, param->paramtype);
add_object_address(TypeRelationId, param->paramtype, 0,
context->addrs);
/* and its collation, just as for Consts */
if (OidIsValid(param->paramcollid) &&
param->paramcollid != DEFAULT_COLLATION_OID)
+ {
+ LockNotPinnedObject(CollationRelationId, param->paramcollid);
add_object_address(CollationRelationId, param->paramcollid, 0,
context->addrs);
+ }
}
else if (IsA(node, FuncExpr))
{
FuncExpr *funcexpr = (FuncExpr *) node;
+ LockNotPinnedObject(ProcedureRelationId, funcexpr->funcid);
add_object_address(ProcedureRelationId, funcexpr->funcid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -1879,6 +1976,7 @@ find_expr_references_walker(Node *node,
{
OpExpr *opexpr = (OpExpr *) node;
+ LockNotPinnedObject(OperatorRelationId, opexpr->opno);
add_object_address(OperatorRelationId, opexpr->opno, 0,
context->addrs);
/* fall through to examine arguments */
@@ -1887,6 +1985,7 @@ find_expr_references_walker(Node *node,
{
DistinctExpr *distinctexpr = (DistinctExpr *) node;
+ LockNotPinnedObject(OperatorRelationId, distinctexpr->opno);
add_object_address(OperatorRelationId, distinctexpr->opno, 0,
context->addrs);
/* fall through to examine arguments */
@@ -1895,6 +1994,7 @@ find_expr_references_walker(Node *node,
{
NullIfExpr *nullifexpr = (NullIfExpr *) node;
+ LockNotPinnedObject(OperatorRelationId, nullifexpr->opno);
add_object_address(OperatorRelationId, nullifexpr->opno, 0,
context->addrs);
/* fall through to examine arguments */
@@ -1903,6 +2003,7 @@ find_expr_references_walker(Node *node,
{
ScalarArrayOpExpr *opexpr = (ScalarArrayOpExpr *) node;
+ LockNotPinnedObject(OperatorRelationId, opexpr->opno);
add_object_address(OperatorRelationId, opexpr->opno, 0,
context->addrs);
/* fall through to examine arguments */
@@ -1911,6 +2012,7 @@ find_expr_references_walker(Node *node,
{
Aggref *aggref = (Aggref *) node;
+ LockNotPinnedObject(ProcedureRelationId, aggref->aggfnoid);
add_object_address(ProcedureRelationId, aggref->aggfnoid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -1919,6 +2021,7 @@ find_expr_references_walker(Node *node,
{
WindowFunc *wfunc = (WindowFunc *) node;
+ LockNotPinnedObject(ProcedureRelationId, wfunc->winfnoid);
add_object_address(ProcedureRelationId, wfunc->winfnoid, 0,
context->addrs);
/* fall through to examine arguments */
@@ -1935,8 +2038,11 @@ find_expr_references_walker(Node *node,
*/
if (sbsref->refrestype != sbsref->refcontainertype &&
sbsref->refrestype != sbsref->refelemtype)
+ {
+ LockNotPinnedObject(TypeRelationId, sbsref->refrestype);
add_object_address(TypeRelationId, sbsref->refrestype, 0,
context->addrs);
+ }
/* fall through to examine arguments */
}
else if (IsA(node, SubPlan))
@@ -1960,16 +2066,25 @@ find_expr_references_walker(Node *node,
* anywhere else in the expression.
*/
if (OidIsValid(reltype))
+ {
+ LockNotPinnedObject(RelationRelationId, reltype);
add_object_address(RelationRelationId, reltype, fselect->fieldnum,
context->addrs);
+ }
else
+ {
+ LockNotPinnedObject(TypeRelationId, fselect->resulttype);
add_object_address(TypeRelationId, fselect->resulttype, 0,
context->addrs);
+ }
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(fselect->resultcollid) &&
fselect->resultcollid != DEFAULT_COLLATION_OID)
+ {
+ LockNotPinnedObject(CollationRelationId, fselect->resultcollid);
add_object_address(CollationRelationId, fselect->resultcollid, 0,
context->addrs);
+ }
}
else if (IsA(node, FieldStore))
{
@@ -1980,53 +2095,76 @@ find_expr_references_walker(Node *node,
if (OidIsValid(reltype))
{
ListCell *l;
+ bool locked = false;
foreach(l, fstore->fieldnums)
+ {
+ if (!locked)
+ {
+ LockNotPinnedObject(RelationRelationId, reltype);
+ locked = true;
+ }
add_object_address(RelationRelationId, reltype, lfirst_int(l),
context->addrs);
+ }
}
else
+ {
+ LockNotPinnedObject(TypeRelationId, fstore->resulttype);
add_object_address(TypeRelationId, fstore->resulttype, 0,
context->addrs);
+ }
}
else if (IsA(node, RelabelType))
{
RelabelType *relab = (RelabelType *) node;
/* since there is no function dependency, need to depend on type */
+ LockNotPinnedObject(TypeRelationId, relab->resulttype);
add_object_address(TypeRelationId, relab->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(relab->resultcollid) &&
relab->resultcollid != DEFAULT_COLLATION_OID)
+ {
+ LockNotPinnedObject(CollationRelationId, relab->resultcollid);
add_object_address(CollationRelationId, relab->resultcollid, 0,
context->addrs);
+ }
}
else if (IsA(node, CoerceViaIO))
{
CoerceViaIO *iocoerce = (CoerceViaIO *) node;
/* since there is no exposed function, need to depend on type */
+ LockNotPinnedObject(TypeRelationId, iocoerce->resulttype);
add_object_address(TypeRelationId, iocoerce->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(iocoerce->resultcollid) &&
iocoerce->resultcollid != DEFAULT_COLLATION_OID)
+ {
+ LockNotPinnedObject(CollationRelationId, iocoerce->resultcollid);
add_object_address(CollationRelationId, iocoerce->resultcollid, 0,
context->addrs);
+ }
}
else if (IsA(node, ArrayCoerceExpr))
{
ArrayCoerceExpr *acoerce = (ArrayCoerceExpr *) node;
/* as above, depend on type */
+ LockNotPinnedObject(TypeRelationId, acoerce->resulttype);
add_object_address(TypeRelationId, acoerce->resulttype, 0,
context->addrs);
/* the collation might not be referenced anywhere else, either */
if (OidIsValid(acoerce->resultcollid) &&
acoerce->resultcollid != DEFAULT_COLLATION_OID)
+ {
+ LockNotPinnedObject(CollationRelationId, acoerce->resultcollid);
add_object_address(CollationRelationId, acoerce->resultcollid, 0,
context->addrs);
+ }
/* fall through to examine arguments */
}
else if (IsA(node, ConvertRowtypeExpr))
@@ -2034,6 +2172,7 @@ find_expr_references_walker(Node *node,
ConvertRowtypeExpr *cvt = (ConvertRowtypeExpr *) node;
/* since there is no function dependency, need to depend on type */
+ LockNotPinnedObject(TypeRelationId, cvt->resulttype);
add_object_address(TypeRelationId, cvt->resulttype, 0,
context->addrs);
}
@@ -2041,6 +2180,7 @@ find_expr_references_walker(Node *node,
{
CollateExpr *coll = (CollateExpr *) node;
+ LockNotPinnedObject(CollationRelationId, coll->collOid);
add_object_address(CollationRelationId, coll->collOid, 0,
context->addrs);
}
@@ -2048,6 +2188,7 @@ find_expr_references_walker(Node *node,
{
RowExpr *rowexpr = (RowExpr *) node;
+ LockNotPinnedObject(TypeRelationId, rowexpr->row_typeid);
add_object_address(TypeRelationId, rowexpr->row_typeid, 0,
context->addrs);
}
@@ -2058,11 +2199,13 @@ find_expr_references_walker(Node *node,
foreach(l, rcexpr->opnos)
{
+ LockNotPinnedObject(OperatorRelationId, lfirst_oid(l));
add_object_address(OperatorRelationId, lfirst_oid(l), 0,
context->addrs);
}
foreach(l, rcexpr->opfamilies)
{
+ LockNotPinnedObject(OperatorFamilyRelationId, lfirst_oid(l));
add_object_address(OperatorFamilyRelationId, lfirst_oid(l), 0,
context->addrs);
}
@@ -2072,6 +2215,7 @@ find_expr_references_walker(Node *node,
{
CoerceToDomain *cd = (CoerceToDomain *) node;
+ LockNotPinnedObject(TypeRelationId, cd->resulttype);
add_object_address(TypeRelationId, cd->resulttype, 0,
context->addrs);
}
@@ -2079,6 +2223,7 @@ find_expr_references_walker(Node *node,
{
NextValueExpr *nve = (NextValueExpr *) node;
+ LockNotPinnedObject(RelationRelationId, nve->seqid);
add_object_address(RelationRelationId, nve->seqid, 0,
context->addrs);
}
@@ -2087,19 +2232,26 @@ find_expr_references_walker(Node *node,
OnConflictExpr *onconflict = (OnConflictExpr *) node;
if (OidIsValid(onconflict->constraint))
+ {
+ LockNotPinnedObject(ConstraintRelationId, onconflict->constraint);
add_object_address(ConstraintRelationId, onconflict->constraint, 0,
context->addrs);
+ }
/* fall through to examine arguments */
}
else if (IsA(node, SortGroupClause))
{
SortGroupClause *sgc = (SortGroupClause *) node;
+ LockNotPinnedObject(OperatorRelationId, sgc->eqop);
add_object_address(OperatorRelationId, sgc->eqop, 0,
context->addrs);
if (OidIsValid(sgc->sortop))
+ {
+ LockNotPinnedObject(OperatorRelationId, sgc->sortop);
add_object_address(OperatorRelationId, sgc->sortop, 0,
context->addrs);
+ }
return false;
}
else if (IsA(node, WindowClause))
@@ -2107,15 +2259,24 @@ find_expr_references_walker(Node *node,
WindowClause *wc = (WindowClause *) node;
if (OidIsValid(wc->startInRangeFunc))
+ {
+ LockNotPinnedObject(ProcedureRelationId, wc->startInRangeFunc);
add_object_address(ProcedureRelationId, wc->startInRangeFunc, 0,
context->addrs);
+ }
if (OidIsValid(wc->endInRangeFunc))
+ {
+ LockNotPinnedObject(ProcedureRelationId, wc->endInRangeFunc);
add_object_address(ProcedureRelationId, wc->endInRangeFunc, 0,
context->addrs);
+ }
if (OidIsValid(wc->inRangeColl) &&
wc->inRangeColl != DEFAULT_COLLATION_OID)
+ {
+ LockNotPinnedObject(CollationRelationId, wc->inRangeColl);
add_object_address(CollationRelationId, wc->inRangeColl, 0,
context->addrs);
+ }
/* fall through to examine substructure */
}
else if (IsA(node, CTECycleClause))
@@ -2123,14 +2284,23 @@ find_expr_references_walker(Node *node,
CTECycleClause *cc = (CTECycleClause *) node;
if (OidIsValid(cc->cycle_mark_type))
+ {
+ LockNotPinnedObject(TypeRelationId, cc->cycle_mark_type);
add_object_address(TypeRelationId, cc->cycle_mark_type, 0,
context->addrs);
+ }
if (OidIsValid(cc->cycle_mark_collation))
+ {
+ LockNotPinnedObject(CollationRelationId, cc->cycle_mark_collation);
add_object_address(CollationRelationId, cc->cycle_mark_collation, 0,
context->addrs);
+ }
if (OidIsValid(cc->cycle_mark_neop))
+ {
+ LockNotPinnedObject(OperatorRelationId, cc->cycle_mark_neop);
add_object_address(OperatorRelationId, cc->cycle_mark_neop, 0,
context->addrs);
+ }
/* fall through to examine substructure */
}
else if (IsA(node, Query))
@@ -2163,6 +2333,7 @@ find_expr_references_walker(Node *node,
switch (rte->rtekind)
{
case RTE_RELATION:
+ LockNotPinnedObject(RelationRelationId, rte->relid);
add_object_address(RelationRelationId, rte->relid, 0,
context->addrs);
break;
@@ -2230,12 +2401,19 @@ find_expr_references_walker(Node *node,
rte = rt_fetch(query->resultRelation, query->rtable);
if (rte->rtekind == RTE_RELATION)
{
+ bool locked = false;
+
foreach(lc, query->targetList)
{
TargetEntry *tle = (TargetEntry *) lfirst(lc);
if (tle->resjunk)
continue; /* ignore junk tlist items */
+ if (!locked)
+ {
+ LockNotPinnedObject(RelationRelationId, rte->relid);
+ locked = true;
+ }
add_object_address(RelationRelationId, rte->relid, tle->resno,
context->addrs);
}
@@ -2247,6 +2425,7 @@ find_expr_references_walker(Node *node,
*/
foreach(lc, query->constraintDeps)
{
+ LockNotPinnedObject(ConstraintRelationId, lfirst_oid(lc));
add_object_address(ConstraintRelationId, lfirst_oid(lc), 0,
context->addrs);
}
@@ -2281,16 +2460,25 @@ find_expr_references_walker(Node *node,
*/
foreach(ct, rtfunc->funccoltypes)
{
+ LockNotPinnedObject(TypeRelationId, lfirst_oid(ct));
add_object_address(TypeRelationId, lfirst_oid(ct), 0,
context->addrs);
}
foreach(ct, rtfunc->funccolcollations)
{
Oid collid = lfirst_oid(ct);
+ bool locked = false;
if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ {
+ if (!locked)
+ {
+ LockNotPinnedObject(CollationRelationId, collid);
+ locked = true;
+ }
add_object_address(CollationRelationId, collid, 0,
context->addrs);
+ }
}
}
else if (IsA(node, TableFunc))
@@ -2303,22 +2491,32 @@ find_expr_references_walker(Node *node,
*/
foreach(ct, tf->coltypes)
{
+ LockNotPinnedObject(TypeRelationId, lfirst_oid(ct));
add_object_address(TypeRelationId, lfirst_oid(ct), 0,
context->addrs);
}
foreach(ct, tf->colcollations)
{
Oid collid = lfirst_oid(ct);
+ bool locked = false;
if (OidIsValid(collid) && collid != DEFAULT_COLLATION_OID)
+ {
+ if (!locked)
+ {
+ LockNotPinnedObject(CollationRelationId, collid);
+ locked = true;
+ }
add_object_address(CollationRelationId, collid, 0,
context->addrs);
+ }
}
}
else if (IsA(node, TableSampleClause))
{
TableSampleClause *tsc = (TableSampleClause *) node;
+ LockNotPinnedObject(ProcedureRelationId, tsc->tsmhandler);
add_object_address(ProcedureRelationId, tsc->tsmhandler, 0,
context->addrs);
/* fall through to examine arguments */
@@ -2369,9 +2567,12 @@ process_function_rte_ref(RangeTblEntry *rte, AttrNumber attnum,
Assert(attnum - atts_done <= tupdesc->natts);
if (OidIsValid(reltype)) /* can this fail? */
+ {
+ LockNotPinnedObject(RelationRelationId, reltype);
add_object_address(RelationRelationId, reltype,
attnum - atts_done,
context->addrs);
+ }
return;
}
/* Nothing to do; function's result type is handled elsewhere */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 57ef466acce..aaf6949a226 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -837,6 +837,7 @@ AddNewAttributeTuples(Oid new_rel_oid,
/* Add dependency info */
ObjectAddressSubSet(myself, RelationRelationId, new_rel_oid, i + 1);
ObjectAddressSet(referenced, TypeRelationId, attr->atttypid);
+ LockNotPinnedObject(TypeRelationId, attr->atttypid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
/* The default collation is pinned, so don't bother recording it */
@@ -845,6 +846,7 @@ AddNewAttributeTuples(Oid new_rel_oid,
{
ObjectAddressSet(referenced, CollationRelationId,
attr->attcollation);
+ LockNotPinnedObject(CollationRelationId, attr->attcollation);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
}
@@ -1452,11 +1454,13 @@ heap_create_with_catalog(const char *relname,
ObjectAddressSet(referenced, NamespaceRelationId, relnamespace);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(NamespaceRelationId, relnamespace);
if (reloftypeid)
{
ObjectAddressSet(referenced, TypeRelationId, reloftypeid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(TypeRelationId, reloftypeid);
}
/*
@@ -1472,6 +1476,7 @@ heap_create_with_catalog(const char *relname,
{
ObjectAddressSet(referenced, AccessMethodRelationId, accessmtd);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(AccessMethodRelationId, accessmtd);
}
record_object_address_dependencies(&myself, addrs, DEPENDENCY_NORMAL);
@@ -3799,6 +3804,7 @@ StorePartitionKey(Relation rel,
{
ObjectAddressSet(referenced, OperatorClassRelationId, partopclass[i]);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(OperatorClassRelationId, partopclass[i]);
/* The default collation is pinned, so don't bother recording it */
if (OidIsValid(partcollation[i]) &&
@@ -3806,6 +3812,7 @@ StorePartitionKey(Relation rel,
{
ObjectAddressSet(referenced, CollationRelationId, partcollation[i]);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(CollationRelationId, partcollation[i]);
}
}
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 7377912b41e..cf442e7f21f 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -1117,6 +1117,7 @@ index_create(Relation heapRelation,
else
{
bool have_simple_col = false;
+ bool locked_object = false;
addrs = new_object_addresses();
@@ -1129,6 +1130,12 @@ index_create(Relation heapRelation,
heapRelationId,
indexInfo->ii_IndexAttrNumbers[i]);
add_exact_object_address(&referenced, addrs);
+
+ if (!locked_object)
+ {
+ LockNotPinnedObject(RelationRelationId, heapRelationId);
+ locked_object = true;
+ }
have_simple_col = true;
}
}
@@ -1144,6 +1151,8 @@ index_create(Relation heapRelation,
ObjectAddressSet(referenced, RelationRelationId,
heapRelationId);
add_exact_object_address(&referenced, addrs);
+
+ LockNotPinnedObject(RelationRelationId, heapRelationId);
}
record_object_address_dependencies(&myself, addrs, DEPENDENCY_AUTO);
@@ -1159,9 +1168,13 @@ index_create(Relation heapRelation,
if (OidIsValid(parentIndexRelid))
{
ObjectAddressSet(referenced, RelationRelationId, parentIndexRelid);
+
+ LockNotPinnedObject(RelationRelationId, parentIndexRelid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_PRI);
ObjectAddressSet(referenced, RelationRelationId, heapRelationId);
+
+ LockNotPinnedObject(RelationRelationId, heapRelationId);
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
@@ -1177,6 +1190,7 @@ index_create(Relation heapRelation,
{
ObjectAddressSet(referenced, CollationRelationId, collationIds[i]);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(CollationRelationId, collationIds[i]);
}
}
@@ -1185,6 +1199,7 @@ index_create(Relation heapRelation,
{
ObjectAddressSet(referenced, OperatorClassRelationId, opclassIds[i]);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(OperatorClassRelationId, opclassIds[i]);
}
record_object_address_dependencies(&myself, addrs, DEPENDENCY_NORMAL);
@@ -1995,6 +2010,14 @@ index_constraint_create(Relation heapRelation,
*/
ObjectAddressSet(myself, ConstraintRelationId, conOid);
ObjectAddressSet(idxaddr, RelationRelationId, indexRelationId);
+
+ /*
+ * CommandCounterIncrement() here to ensure the new constraint entry is
+ * visible when LockNotPinnedObject() will check its existence before
+ * recording the dependencies.
+ */
+ CommandCounterIncrement();
+ LockNotPinnedObject(ConstraintRelationId, conOid);
recordDependencyOn(&idxaddr, &myself, DEPENDENCY_INTERNAL);
/*
@@ -2006,9 +2029,12 @@ index_constraint_create(Relation heapRelation,
ObjectAddress referenced;
ObjectAddressSet(referenced, ConstraintRelationId, parentConstraintId);
+ LockNotPinnedObject(ConstraintRelationId, parentConstraintId);
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_PRI);
ObjectAddressSet(referenced, RelationRelationId,
RelationGetRelid(heapRelation));
+
+ LockNotPinnedObject(RelationRelationId, RelationGetRelid(heapRelation));
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index d8eb8d3deaa..f3e82671df5 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -2595,6 +2595,63 @@ get_object_namespace(const ObjectAddress *address)
return oid;
}
+/*
+ * ObjectByIdExist
+ *
+ * Return whether the given object exists.
+ *
+ * Works for most catalogs, if no special processing is needed.
+ */
+bool
+ObjectByIdExist(const ObjectAddress *address)
+{
+ HeapTuple tuple;
+ int cache;
+ const ObjectPropertyType *property;
+
+ property = get_object_property_data(address->classId);
+
+ cache = property->oid_catcache_id;
+
+ if (cache >= 0)
+ {
+ /* Fetch tuple from syscache. */
+ tuple = SearchSysCache1(cache, ObjectIdGetDatum(address->objectId));
+
+ if (!HeapTupleIsValid(tuple))
+ {
+ return false;
+ }
+
+ ReleaseSysCache(tuple);
+
+ return true;
+ }
+ else
+ {
+ Relation rel;
+ ScanKeyData skey[1];
+ SysScanDesc scan;
+
+ rel = table_open(address->classId, AccessShareLock);
+
+ ScanKeyInit(&skey[0],
+ get_object_attnum_oid(address->classId),
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(address->objectId));
+
+ scan = systable_beginscan(rel, get_object_oid_index(address->classId), true,
+ NULL, 1, skey);
+
+ /* we expect exactly one match */
+ tuple = systable_getnext(scan);
+ systable_endscan(scan);
+ table_close(rel, AccessShareLock);
+
+ return (HeapTupleIsValid(tuple));
+ }
+}
+
/*
* Return ObjectType for the given object type as given by
* getObjectTypeDescription; if no valid ObjectType code exists, but it's a
diff --git a/src/backend/catalog/pg_aggregate.c b/src/backend/catalog/pg_aggregate.c
index bcf4050f5b1..ce5865fac69 100644
--- a/src/backend/catalog/pg_aggregate.c
+++ b/src/backend/catalog/pg_aggregate.c
@@ -748,12 +748,14 @@ AggregateCreate(const char *aggName,
/* Depends on transition function */
ObjectAddressSet(referenced, ProcedureRelationId, transfn);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, transfn);
/* Depends on final function, if any */
if (OidIsValid(finalfn))
{
ObjectAddressSet(referenced, ProcedureRelationId, finalfn);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, finalfn);
}
/* Depends on combine function, if any */
@@ -761,6 +763,7 @@ AggregateCreate(const char *aggName,
{
ObjectAddressSet(referenced, ProcedureRelationId, combinefn);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, combinefn);
}
/* Depends on serialization function, if any */
@@ -768,6 +771,7 @@ AggregateCreate(const char *aggName,
{
ObjectAddressSet(referenced, ProcedureRelationId, serialfn);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, serialfn);
}
/* Depends on deserialization function, if any */
@@ -775,6 +779,7 @@ AggregateCreate(const char *aggName,
{
ObjectAddressSet(referenced, ProcedureRelationId, deserialfn);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, deserialfn);
}
/* Depends on forward transition function, if any */
@@ -782,6 +787,7 @@ AggregateCreate(const char *aggName,
{
ObjectAddressSet(referenced, ProcedureRelationId, mtransfn);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, mtransfn);
}
/* Depends on inverse transition function, if any */
@@ -789,6 +795,7 @@ AggregateCreate(const char *aggName,
{
ObjectAddressSet(referenced, ProcedureRelationId, minvtransfn);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, minvtransfn);
}
/* Depends on final function, if any */
@@ -796,6 +803,7 @@ AggregateCreate(const char *aggName,
{
ObjectAddressSet(referenced, ProcedureRelationId, mfinalfn);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, mfinalfn);
}
/* Depends on sort operator, if any */
@@ -803,6 +811,7 @@ AggregateCreate(const char *aggName,
{
ObjectAddressSet(referenced, OperatorRelationId, sortop);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(OperatorRelationId, sortop);
}
record_object_address_dependencies(&myself, addrs, DEPENDENCY_NORMAL);
diff --git a/src/backend/catalog/pg_attrdef.c b/src/backend/catalog/pg_attrdef.c
index 91dafc81021..91da30c504e 100644
--- a/src/backend/catalog/pg_attrdef.c
+++ b/src/backend/catalog/pg_attrdef.c
@@ -178,6 +178,7 @@ StoreAttrDefault(Relation rel, AttrNumber attnum,
colobject.objectId = RelationGetRelid(rel);
colobject.objectSubId = attnum;
+ LockNotPinnedObject(RelationRelationId, RelationGetRelid(rel));
recordDependencyOn(&defobject, &colobject,
attgenerated ? DEPENDENCY_INTERNAL : DEPENDENCY_AUTO);
diff --git a/src/backend/catalog/pg_cast.c b/src/backend/catalog/pg_cast.c
index 1773c9c5491..90513a39ac6 100644
--- a/src/backend/catalog/pg_cast.c
+++ b/src/backend/catalog/pg_cast.c
@@ -97,16 +97,19 @@ CastCreate(Oid sourcetypeid, Oid targettypeid,
/* dependency on source type */
ObjectAddressSet(referenced, TypeRelationId, sourcetypeid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(TypeRelationId, sourcetypeid);
/* dependency on target type */
ObjectAddressSet(referenced, TypeRelationId, targettypeid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(TypeRelationId, targettypeid);
/* dependency on function */
if (OidIsValid(funcid))
{
ObjectAddressSet(referenced, ProcedureRelationId, funcid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, funcid);
}
/* dependencies on casts required for function */
@@ -114,11 +117,13 @@ CastCreate(Oid sourcetypeid, Oid targettypeid,
{
ObjectAddressSet(referenced, CastRelationId, incastid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(CastRelationId, incastid);
}
if (OidIsValid(outcastid))
{
ObjectAddressSet(referenced, CastRelationId, outcastid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(CastRelationId, outcastid);
}
record_object_address_dependencies(&myself, addrs, behavior);
diff --git a/src/backend/catalog/pg_collation.c b/src/backend/catalog/pg_collation.c
index 469635b3580..fa6fd7bf791 100644
--- a/src/backend/catalog/pg_collation.c
+++ b/src/backend/catalog/pg_collation.c
@@ -218,6 +218,7 @@ CollationCreate(const char *collname, Oid collnamespace,
referenced.classId = NamespaceRelationId;
referenced.objectId = collnamespace;
referenced.objectSubId = 0;
+ LockNotPinnedObject(NamespaceRelationId, collnamespace);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
/* create dependency on owner */
diff --git a/src/backend/catalog/pg_constraint.c b/src/backend/catalog/pg_constraint.c
index ac80652baf2..79b2df100e6 100644
--- a/src/backend/catalog/pg_constraint.c
+++ b/src/backend/catalog/pg_constraint.c
@@ -264,17 +264,26 @@ CreateConstraintEntry(const char *constraintName,
if (constraintNTotalKeys > 0)
{
+ bool locked_object = false;
+
for (i = 0; i < constraintNTotalKeys; i++)
{
ObjectAddressSubSet(relobject, RelationRelationId, relId,
constraintKey[i]);
add_exact_object_address(&relobject, addrs_auto);
+
+ if (!locked_object)
+ {
+ LockNotPinnedObject(RelationRelationId, relId);
+ locked_object = true;
+ }
}
}
else
{
ObjectAddressSet(relobject, RelationRelationId, relId);
add_exact_object_address(&relobject, addrs_auto);
+ LockNotPinnedObject(RelationRelationId, relId);
}
}
@@ -287,6 +296,7 @@ CreateConstraintEntry(const char *constraintName,
ObjectAddressSet(domobject, TypeRelationId, domainId);
add_exact_object_address(&domobject, addrs_auto);
+ LockNotPinnedObject(TypeRelationId, domainId);
}
record_object_address_dependencies(&conobject, addrs_auto,
@@ -306,17 +316,26 @@ CreateConstraintEntry(const char *constraintName,
if (foreignNKeys > 0)
{
+ bool locked_object = false;
+
for (i = 0; i < foreignNKeys; i++)
{
ObjectAddressSubSet(relobject, RelationRelationId,
foreignRelId, foreignKey[i]);
add_exact_object_address(&relobject, addrs_normal);
+
+ if (!locked_object)
+ {
+ LockNotPinnedObject(RelationRelationId, foreignRelId);
+ locked_object = true;
+ }
}
}
else
{
ObjectAddressSet(relobject, RelationRelationId, foreignRelId);
add_exact_object_address(&relobject, addrs_normal);
+ LockNotPinnedObject(RelationRelationId, foreignRelId);
}
}
@@ -332,6 +351,7 @@ CreateConstraintEntry(const char *constraintName,
ObjectAddressSet(relobject, RelationRelationId, indexRelId);
add_exact_object_address(&relobject, addrs_normal);
+ LockNotPinnedObject(RelationRelationId, indexRelId);
}
if (foreignNKeys > 0)
@@ -351,15 +371,18 @@ CreateConstraintEntry(const char *constraintName,
{
oprobject.objectId = pfEqOp[i];
add_exact_object_address(&oprobject, addrs_normal);
+ LockNotPinnedObject(OperatorRelationId, pfEqOp[i]);
if (ppEqOp[i] != pfEqOp[i])
{
oprobject.objectId = ppEqOp[i];
add_exact_object_address(&oprobject, addrs_normal);
+ LockNotPinnedObject(OperatorRelationId, ppEqOp[i]);
}
if (ffEqOp[i] != pfEqOp[i])
{
oprobject.objectId = ffEqOp[i];
add_exact_object_address(&oprobject, addrs_normal);
+ LockNotPinnedObject(OperatorRelationId, ffEqOp[i]);
}
}
}
@@ -1119,9 +1142,12 @@ ConstraintSetParentConstraint(Oid childConstrId,
ObjectAddressSet(depender, ConstraintRelationId, childConstrId);
ObjectAddressSet(referenced, ConstraintRelationId, parentConstrId);
+ LockNotPinnedObject(ConstraintRelationId, parentConstrId);
recordDependencyOn(&depender, &referenced, DEPENDENCY_PARTITION_PRI);
ObjectAddressSet(referenced, RelationRelationId, childTableId);
+
+ LockNotPinnedObject(RelationRelationId, childTableId);
recordDependencyOn(&depender, &referenced, DEPENDENCY_PARTITION_SEC);
}
else
diff --git a/src/backend/catalog/pg_conversion.c b/src/backend/catalog/pg_conversion.c
index 04cc375caea..5ac365ebd04 100644
--- a/src/backend/catalog/pg_conversion.c
+++ b/src/backend/catalog/pg_conversion.c
@@ -116,12 +116,14 @@ ConversionCreate(const char *conname, Oid connamespace,
referenced.classId = ProcedureRelationId;
referenced.objectId = conproc;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ProcedureRelationId, conproc);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
/* create dependency on namespace */
referenced.classId = NamespaceRelationId;
referenced.objectId = connamespace;
referenced.objectSubId = 0;
+ LockNotPinnedObject(NamespaceRelationId, connamespace);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
/* create dependency on owner */
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index c8b11f887e2..fdafffb8934 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -20,19 +20,20 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/pg_auth_members.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
#include "catalog/pg_extension.h"
#include "catalog/partition.h"
#include "commands/extension.h"
#include "miscadmin.h"
+#include "storage/lmgr.h"
+#include "storage/lock.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
#include "utils/rel.h"
-static bool isObjectPinned(const ObjectAddress *object);
-
/*
* Record a dependency between 2 objects via their respective ObjectAddress.
@@ -99,6 +100,37 @@ recordMultipleDependencies(const ObjectAddress *depender,
slot_init_count = 0;
for (i = 0; i < nreferenced; i++, referenced++)
{
+#ifdef USE_ASSERT_CHECKING
+ if (!isObjectPinned(referenced))
+ {
+ if (referenced->classId != RelationRelationId)
+ {
+ LOCKTAG tag;
+
+ SET_LOCKTAG_OBJECT(tag,
+ MyDatabaseId,
+ referenced->classId,
+ referenced->objectId,
+ 0);
+ /* assert the referenced object is locked */
+ Assert(LockHeldByMe(&tag, AccessShareLock, false));
+ }
+ else
+ {
+ Assert(!IsSharedRelation(referenced->objectId));
+
+ /*
+ * Assert the referenced object is locked if it should be
+ * visible (see the comment related to LockNotPinnedObject()
+ * in TypeCreate()).
+ */
+ Assert(!ObjectByIdExist(referenced) ||
+ CheckRelationOidLockedByMe(referenced->objectId,
+ AccessShareLock, true));
+ }
+ }
+#endif
+
/*
* If the referenced object is pinned by the system, there's no real
* need to record dependencies on it. This saves lots of space in
@@ -238,6 +270,7 @@ recordDependencyOnCurrentExtension(const ObjectAddress *object,
extension.objectId = CurrentExtensionObject;
extension.objectSubId = 0;
+ LockNotPinnedObject(ExtensionRelationId, CurrentExtensionObject);
recordDependencyOn(object, &extension, DEPENDENCY_EXTENSION);
}
}
@@ -705,7 +738,7 @@ changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
* The passed subId, if any, is ignored; we assume that only whole objects
* are pinned (and that this implies pinning their components).
*/
-static bool
+bool
isObjectPinned(const ObjectAddress *object)
{
return IsPinnedObject(object->classId, object->objectId);
diff --git a/src/backend/catalog/pg_operator.c b/src/backend/catalog/pg_operator.c
index bfcfa643464..ba6aa332187 100644
--- a/src/backend/catalog/pg_operator.c
+++ b/src/backend/catalog/pg_operator.c
@@ -251,6 +251,16 @@ OperatorShellMake(const char *operatorName,
values[Anum_pg_operator_oprrest - 1] = ObjectIdGetDatum(InvalidOid);
values[Anum_pg_operator_oprjoin - 1] = ObjectIdGetDatum(InvalidOid);
+ /* Lock dependent objects */
+ if (OidIsValid(operatorNamespace))
+ LockNotPinnedObject(NamespaceRelationId, operatorNamespace);
+
+ if (OidIsValid(leftTypeId))
+ LockNotPinnedObject(TypeRelationId, leftTypeId);
+
+ if (OidIsValid(rightTypeId))
+ LockNotPinnedObject(TypeRelationId, rightTypeId);
+
/*
* create a new operator tuple
*/
@@ -513,6 +523,15 @@ OperatorCreate(const char *operatorName,
CatalogTupleInsert(pg_operator_desc, tup);
}
+ /* Lock dependent objects */
+ LockNotPinnedObject(NamespaceRelationId, operatorNamespace);
+ LockNotPinnedObject(TypeRelationId, leftTypeId);
+ LockNotPinnedObject(TypeRelationId, rightTypeId);
+ LockNotPinnedObject(TypeRelationId, operResultType);
+ LockNotPinnedObject(ProcedureRelationId, procedureId);
+ LockNotPinnedObject(ProcedureRelationId, restrictionId);
+ LockNotPinnedObject(ProcedureRelationId, joinId);
+
/* Add dependencies for the entry */
address = makeOperatorDependencies(tup, true, isUpdate);
diff --git a/src/backend/catalog/pg_proc.c b/src/backend/catalog/pg_proc.c
index fe0490259e9..c3999bdce87 100644
--- a/src/backend/catalog/pg_proc.c
+++ b/src/backend/catalog/pg_proc.c
@@ -593,6 +593,13 @@ ProcedureCreate(const char *procedureName,
if (is_update)
deleteDependencyRecordsFor(ProcedureRelationId, retval, true);
+ /*
+ * CommandCounterIncrement() here to ensure the new function entry is
+ * visible when LockNotPinnedObject() will check its existence before
+ * recording the dependencies.
+ */
+ CommandCounterIncrement();
+
addrs = new_object_addresses();
ObjectAddressSet(myself, ProcedureRelationId, retval);
@@ -600,20 +607,24 @@ ProcedureCreate(const char *procedureName,
/* dependency on namespace */
ObjectAddressSet(referenced, NamespaceRelationId, procNamespace);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(NamespaceRelationId, procNamespace);
/* dependency on implementation language */
ObjectAddressSet(referenced, LanguageRelationId, languageObjectId);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(LanguageRelationId, languageObjectId);
/* dependency on return type */
ObjectAddressSet(referenced, TypeRelationId, returnType);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(TypeRelationId, returnType);
/* dependency on transform used by return type, if any */
if ((trfid = get_transform_oid(returnType, languageObjectId, true)))
{
ObjectAddressSet(referenced, TransformRelationId, trfid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(TransformRelationId, trfid);
}
/* dependency on parameter types */
@@ -621,12 +632,14 @@ ProcedureCreate(const char *procedureName,
{
ObjectAddressSet(referenced, TypeRelationId, allParams[i]);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(TypeRelationId, allParams[i]);
/* dependency on transform used by parameter type, if any */
if ((trfid = get_transform_oid(allParams[i], languageObjectId, true)))
{
ObjectAddressSet(referenced, TransformRelationId, trfid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(TransformRelationId, trfid);
}
}
@@ -635,6 +648,7 @@ ProcedureCreate(const char *procedureName,
{
ObjectAddressSet(referenced, ProcedureRelationId, prosupport);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, prosupport);
}
record_object_address_dependencies(&myself, addrs, DEPENDENCY_NORMAL);
@@ -674,9 +688,6 @@ ProcedureCreate(const char *procedureName,
ArrayType *set_items = NULL;
int save_nestlevel = 0;
- /* Advance command counter so new tuple can be seen by validator */
- CommandCounterIncrement();
-
/*
* Set per-function configuration parameters so that the validation is
* done with the environment the function expects. However, if
diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c
index 41ffd494c81..8922b8547e7 100644
--- a/src/backend/catalog/pg_publication.c
+++ b/src/backend/catalog/pg_publication.c
@@ -441,6 +441,7 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
referenced;
List *relids = NIL;
int i;
+ bool locked = false;
rel = table_open(PublicationRelRelationId, RowExclusiveLock);
@@ -503,10 +504,13 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
/* Add dependency on the publication */
ObjectAddressSet(referenced, PublicationRelationId, pubid);
+ LockNotPinnedObject(PublicationRelationId, pubid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
/* Add dependency on the relation */
ObjectAddressSet(referenced, RelationRelationId, relid);
+
+ LockNotPinnedObject(RelationRelationId, relid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
/* Add dependency on the objects mentioned in the qualifications */
@@ -520,6 +524,11 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri,
while ((i = bms_next_member(attnums, i)) >= 0)
{
ObjectAddressSubSet(referenced, RelationRelationId, relid, i);
+ if (!locked)
+ {
+ LockNotPinnedObject(RelationRelationId, relid);
+ locked = true;
+ }
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
@@ -717,10 +726,12 @@ publication_add_schema(Oid pubid, Oid schemaid, bool if_not_exists)
/* Add dependency on the publication */
ObjectAddressSet(referenced, PublicationRelationId, pubid);
+ LockNotPinnedObject(PublicationRelationId, pubid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
/* Add dependency on the schema */
ObjectAddressSet(referenced, NamespaceRelationId, schemaid);
+ LockNotPinnedObject(NamespaceRelationId, schemaid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
/* Close the table */
diff --git a/src/backend/catalog/pg_range.c b/src/backend/catalog/pg_range.c
index 8df73e7ab71..e1538c21468 100644
--- a/src/backend/catalog/pg_range.c
+++ b/src/backend/catalog/pg_range.c
@@ -70,26 +70,31 @@ RangeCreate(Oid rangeTypeOid, Oid rangeSubType, Oid rangeCollation,
ObjectAddressSet(referenced, TypeRelationId, rangeSubType);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(TypeRelationId, rangeSubType);
ObjectAddressSet(referenced, OperatorClassRelationId, rangeSubOpclass);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(OperatorClassRelationId, rangeSubOpclass);
if (OidIsValid(rangeCollation))
{
ObjectAddressSet(referenced, CollationRelationId, rangeCollation);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(CollationRelationId, rangeCollation);
}
if (OidIsValid(rangeCanonical))
{
ObjectAddressSet(referenced, ProcedureRelationId, rangeCanonical);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, rangeCanonical);
}
if (OidIsValid(rangeSubDiff))
{
ObjectAddressSet(referenced, ProcedureRelationId, rangeSubDiff);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, rangeSubDiff);
}
record_object_address_dependencies(&myself, addrs, DEPENDENCY_NORMAL);
@@ -99,6 +104,7 @@ RangeCreate(Oid rangeTypeOid, Oid rangeSubType, Oid rangeCollation,
referencing.classId = TypeRelationId;
referencing.objectId = multirangeTypeOid;
referencing.objectSubId = 0;
+ LockNotPinnedObject(TypeRelationId, rangeTypeOid);
recordDependencyOn(&referencing, &myself, DEPENDENCY_INTERNAL);
table_close(pg_range, RowExclusiveLock);
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index b36f81afb9d..77ff5f06030 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -157,6 +157,12 @@ TypeShellMake(const char *typeName, Oid typeNamespace, Oid ownerId)
* Create dependencies. We can/must skip this in bootstrap mode.
*/
if (!IsBootstrapProcessingMode())
+ {
+ /* Lock dependent objects */
+ LockNotPinnedObject(NamespaceRelationId, typeNamespace);
+ LockNotPinnedObject(ProcedureRelationId, F_SHELL_IN);
+ LockNotPinnedObject(ProcedureRelationId, F_SHELL_OUT);
+
GenerateTypeDependencies(tup,
pg_type_desc,
NULL,
@@ -166,6 +172,7 @@ TypeShellMake(const char *typeName, Oid typeNamespace, Oid ownerId)
false,
true, /* make extension dependency */
false);
+ }
/* Post creation hook for new shell type */
InvokeObjectPostCreateHook(TypeRelationId, typoid, 0);
@@ -494,6 +501,37 @@ TypeCreate(Oid newTypeOid,
* Create dependencies. We can/must skip this in bootstrap mode.
*/
if (!IsBootstrapProcessingMode())
+ {
+ /*
+ * CommandCounterIncrement() here to ensure the new type entry is
+ * visible when LockNotPinnedObject() will check its existence before
+ * recording the dependencies.
+ */
+ CommandCounterIncrement();
+
+ /* Lock dependent objects */
+ LockNotPinnedObject(NamespaceRelationId, typeNamespace);
+ LockNotPinnedObject(ProcedureRelationId, inputProcedure);
+ LockNotPinnedObject(ProcedureRelationId, outputProcedure);
+ LockNotPinnedObject(ProcedureRelationId, receiveProcedure);
+ LockNotPinnedObject(ProcedureRelationId, sendProcedure);
+ LockNotPinnedObject(ProcedureRelationId, typmodinProcedure);
+ LockNotPinnedObject(ProcedureRelationId, typmodoutProcedure);
+ LockNotPinnedObject(ProcedureRelationId, analyzeProcedure);
+ LockNotPinnedObject(ProcedureRelationId, subscriptProcedure);
+ LockNotPinnedObject(TypeRelationId, baseType);
+ LockNotPinnedObject(CollationRelationId, typeCollation);
+ LockNotPinnedObject(TypeRelationId, elementType);
+
+ /*
+ * No need to call LockRelationOid() (through LockNotPinnedObject())
+ * on relationOid as relationOid is set to an InvalidOid or to a new
+ * Oid not added to pg_class yet (In heap_create_with_catalog(),
+ * AddNewRelationType() is called before AddNewRelationTuple()).
+ */
+ if (relationKind == RELKIND_COMPOSITE_TYPE)
+ LockNotPinnedObject(TypeRelationId, typeObjectId);
+
GenerateTypeDependencies(tup,
pg_type_desc,
(defaultTypeBin ?
@@ -505,6 +543,7 @@ TypeCreate(Oid newTypeOid,
isDependentType,
true, /* make extension dependency */
rebuildDeps);
+ }
/* Post creation hook for new type */
InvokeObjectPostCreateHook(TypeRelationId, typeObjectId, 0);
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 874a8fc89ad..b8fb22af6ae 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -383,6 +383,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
toastobject.objectId = toast_relid;
toastobject.objectSubId = 0;
+ LockNotPinnedObject(RelationRelationId, relOid);
recordDependencyOn(&toastobject, &baseobject, DEPENDENCY_INTERNAL);
}
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 78c1d4e1b84..e068b127c28 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -499,7 +499,10 @@ ExecAlterObjectDependsStmt(AlterObjectDependsStmt *stmt, ObjectAddress *refAddre
currexts = getAutoExtensionsOfObject(address.classId,
address.objectId);
if (!list_member_oid(currexts, refAddr.objectId))
+ {
+ LockNotPinnedObject(refAddr.classId, refAddr.objectId);
recordDependencyOn(&address, &refAddr, DEPENDENCY_AUTO_EXTENSION);
+ }
}
return address;
@@ -803,6 +806,7 @@ AlterObjectNamespace_internal(Relation rel, Oid objid, Oid nspOid)
pfree(replaces);
/* update dependency to point to the new schema */
+ LockNotPinnedObject(NamespaceRelationId, nspOid);
if (changeDependencyFor(classId, objid,
NamespaceRelationId, oldNspOid, nspOid) != 1)
elog(ERROR, "could not change schema dependency for object %u",
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 58ed9d216cc..eb9b86d5633 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -104,6 +104,7 @@ CreateAccessMethod(CreateAmStmt *stmt)
referenced.objectId = amhandler;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ProcedureRelationId, amhandler);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
recordDependencyOnCurrentExtension(&myself, false);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 99193f5c886..c1a7ecbfab8 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -1274,6 +1274,7 @@ swap_relation_files(Oid r1, Oid r2, bool target_is_pg_class,
*/
if (relam1 != relam2)
{
+ LockNotPinnedObject(AccessMethodRelationId, relam2);
if (changeDependencyFor(RelationRelationId,
r1,
AccessMethodRelationId,
@@ -1282,6 +1283,8 @@ swap_relation_files(Oid r1, Oid r2, bool target_is_pg_class,
elog(ERROR, "could not change access method dependency for relation \"%s.%s\"",
get_namespace_name(get_rel_namespace(r1)),
get_rel_name(r1));
+
+ LockNotPinnedObject(AccessMethodRelationId, relam1);
if (changeDependencyFor(RelationRelationId,
r2,
AccessMethodRelationId,
@@ -1383,6 +1386,8 @@ swap_relation_files(Oid r1, Oid r2, bool target_is_pg_class,
{
baseobject.objectId = r1;
toastobject.objectId = relform1->reltoastrelid;
+
+ LockNotPinnedObject(RelationRelationId, r1);
recordDependencyOn(&toastobject, &baseobject,
DEPENDENCY_INTERNAL);
}
@@ -1391,6 +1396,8 @@ swap_relation_files(Oid r1, Oid r2, bool target_is_pg_class,
{
baseobject.objectId = r2;
toastobject.objectId = relform2->reltoastrelid;
+
+ LockNotPinnedObject(RelationRelationId, r2);
recordDependencyOn(&toastobject, &baseobject,
DEPENDENCY_INTERNAL);
}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index edc2c988e29..14dab63ef0d 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -327,6 +327,7 @@ insert_event_trigger_tuple(const char *trigname, const char *eventname, Oid evtO
referenced.classId = ProcedureRelationId;
referenced.objectId = funcoid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ProcedureRelationId, funcoid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
/* Depend on extension, if any. */
diff --git a/src/backend/commands/extension.c b/src/backend/commands/extension.c
index ba540e3de5b..01813ac71d5 100644
--- a/src/backend/commands/extension.c
+++ b/src/backend/commands/extension.c
@@ -2035,6 +2035,7 @@ InsertExtensionTuple(const char *extName, Oid extOwner,
ObjectAddressSet(nsp, NamespaceRelationId, schemaOid);
add_exact_object_address(&nsp, refobjs);
+ LockNotPinnedObject(NamespaceRelationId, schemaOid);
foreach(lc, requiredExtensions)
{
@@ -2043,6 +2044,7 @@ InsertExtensionTuple(const char *extName, Oid extOwner,
ObjectAddressSet(otherext, ExtensionRelationId, reqext);
add_exact_object_address(&otherext, refobjs);
+ LockNotPinnedObject(ExtensionRelationId, reqext);
}
/* Record all of them (this includes duplicate elimination) */
@@ -3079,6 +3081,7 @@ AlterExtensionNamespace(const char *extensionName, const char *newschema, Oid *o
table_close(extRel, RowExclusiveLock);
/* update dependency to point to the new schema */
+ LockNotPinnedObject(NamespaceRelationId, nspOid);
if (changeDependencyFor(ExtensionRelationId, extensionOid,
NamespaceRelationId, oldNspOid, nspOid) != 1)
elog(ERROR, "could not change schema dependency for extension %s",
@@ -3369,6 +3372,7 @@ ApplyExtensionUpdates(Oid extensionOid,
otherext.objectId = reqext;
otherext.objectSubId = 0;
+ LockNotPinnedObject(ExtensionRelationId, reqext);
recordDependencyOn(&myself, &otherext, DEPENDENCY_NORMAL);
}
@@ -3525,6 +3529,7 @@ ExecAlterExtensionContentsRecurse(AlterExtensionContentsStmt *stmt,
/*
* OK, add the dependency.
*/
+ LockNotPinnedObject(extension.classId, extension.objectId);
recordDependencyOn(&object, &extension, DEPENDENCY_EXTENSION);
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index c14e038d54f..d6cced22e7a 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -642,6 +642,7 @@ CreateForeignDataWrapper(ParseState *pstate, CreateFdwStmt *stmt)
referenced.classId = ProcedureRelationId;
referenced.objectId = fdwhandler;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ProcedureRelationId, fdwhandler);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
@@ -650,6 +651,7 @@ CreateForeignDataWrapper(ParseState *pstate, CreateFdwStmt *stmt)
referenced.classId = ProcedureRelationId;
referenced.objectId = fdwvalidator;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ProcedureRelationId, fdwvalidator);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
@@ -811,6 +813,7 @@ AlterForeignDataWrapper(ParseState *pstate, AlterFdwStmt *stmt)
referenced.classId = ProcedureRelationId;
referenced.objectId = fdwhandler;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ProcedureRelationId, fdwhandler);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
@@ -819,6 +822,7 @@ AlterForeignDataWrapper(ParseState *pstate, AlterFdwStmt *stmt)
referenced.classId = ProcedureRelationId;
referenced.objectId = fdwvalidator;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ProcedureRelationId, fdwvalidator);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
}
@@ -951,6 +955,7 @@ CreateForeignServer(CreateForeignServerStmt *stmt)
referenced.classId = ForeignDataWrapperRelationId;
referenced.objectId = fdw->fdwid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ForeignDataWrapperRelationId, fdw->fdwid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
recordDependencyOnOwner(ForeignServerRelationId, srvId, ownerId);
@@ -1195,6 +1200,7 @@ CreateUserMapping(CreateUserMappingStmt *stmt)
referenced.classId = ForeignServerRelationId;
referenced.objectId = srv->serverid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ForeignServerRelationId, srv->serverid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
if (OidIsValid(useId))
@@ -1472,6 +1478,7 @@ CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid)
referenced.classId = ForeignServerRelationId;
referenced.objectId = server->serverid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ForeignServerRelationId, server->serverid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
table_close(ftrel, RowExclusiveLock);
diff --git a/src/backend/commands/functioncmds.c b/src/backend/commands/functioncmds.c
index b9fd7683abb..e2155511478 100644
--- a/src/backend/commands/functioncmds.c
+++ b/src/backend/commands/functioncmds.c
@@ -1461,6 +1461,7 @@ AlterFunction(ParseState *pstate, AlterFunctionStmt *stmt)
/* Add or replace dependency on support function */
if (OidIsValid(procForm->prosupport))
{
+ LockNotPinnedObject(ProcedureRelationId, newsupport);
if (changeDependencyFor(ProcedureRelationId, funcOid,
ProcedureRelationId, procForm->prosupport,
newsupport) != 1)
@@ -1474,6 +1475,7 @@ AlterFunction(ParseState *pstate, AlterFunctionStmt *stmt)
referenced.classId = ProcedureRelationId;
referenced.objectId = newsupport;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ProcedureRelationId, newsupport);
recordDependencyOn(&address, &referenced, DEPENDENCY_NORMAL);
}
@@ -1990,21 +1992,25 @@ CreateTransform(CreateTransformStmt *stmt)
/* dependency on language */
ObjectAddressSet(referenced, LanguageRelationId, langid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(LanguageRelationId, langid);
/* dependency on type */
ObjectAddressSet(referenced, TypeRelationId, typeid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(TypeRelationId, typeid);
/* dependencies on functions */
if (OidIsValid(fromsqlfuncid))
{
ObjectAddressSet(referenced, ProcedureRelationId, fromsqlfuncid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, fromsqlfuncid);
}
if (OidIsValid(tosqlfuncid))
{
ObjectAddressSet(referenced, ProcedureRelationId, tosqlfuncid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, tosqlfuncid);
}
record_object_address_dependencies(&myself, addrs, DEPENDENCY_NORMAL);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 5b1753d4681..f601d590d8f 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -4488,8 +4488,10 @@ IndexSetParentIndex(Relation partitionIdx, Oid parentOid)
ObjectAddressSet(parentIdx, RelationRelationId, parentOid);
ObjectAddressSet(partitionTbl, RelationRelationId,
partitionIdx->rd_index->indrelid);
+ LockNotPinnedObject(RelationRelationId, parentOid);
recordDependencyOn(&partIdx, &parentIdx,
DEPENDENCY_PARTITION_PRI);
+ LockNotPinnedObject(RelationRelationId, partitionIdx->rd_index->indrelid);
recordDependencyOn(&partIdx, &partitionTbl,
DEPENDENCY_PARTITION_SEC);
}
diff --git a/src/backend/commands/opclasscmds.c b/src/backend/commands/opclasscmds.c
index 2c325badf94..72b347e9802 100644
--- a/src/backend/commands/opclasscmds.c
+++ b/src/backend/commands/opclasscmds.c
@@ -299,12 +299,14 @@ CreateOpFamily(CreateOpFamilyStmt *stmt, const char *opfname,
referenced.classId = AccessMethodRelationId;
referenced.objectId = amoid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(AccessMethodRelationId, amoid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
/* dependency on namespace */
referenced.classId = NamespaceRelationId;
referenced.objectId = namespaceoid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(NamespaceRelationId, namespaceoid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
/* dependency on owner */
@@ -726,18 +728,21 @@ DefineOpClass(CreateOpClassStmt *stmt)
referenced.classId = NamespaceRelationId;
referenced.objectId = namespaceoid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(NamespaceRelationId, namespaceoid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
/* dependency on opfamily */
referenced.classId = OperatorFamilyRelationId;
referenced.objectId = opfamilyoid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(OperatorFamilyRelationId, opfamilyoid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
/* dependency on indexed datatype */
referenced.classId = TypeRelationId;
referenced.objectId = typeoid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(TypeRelationId, typeoid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
/* dependency on storage datatype */
@@ -746,6 +751,7 @@ DefineOpClass(CreateOpClassStmt *stmt)
referenced.classId = TypeRelationId;
referenced.objectId = storageoid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(TypeRelationId, storageoid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
@@ -1487,6 +1493,13 @@ storeOperators(List *opfamilyname, Oid amoid, Oid opfamilyoid,
heap_freetuple(tup);
+ /*
+ * CommandCounterIncrement() here to ensure the new operator entry is
+ * visible when LockNotPinnedObject() will check its existence before
+ * recording the dependencies.
+ */
+ CommandCounterIncrement();
+
/* Make its dependencies */
myself.classId = AccessMethodOperatorRelationId;
myself.objectId = entryoid;
@@ -1497,6 +1510,7 @@ storeOperators(List *opfamilyname, Oid amoid, Oid opfamilyoid,
referenced.objectSubId = 0;
/* see comments in amapi.h about dependency strength */
+ LockNotPinnedObject(OperatorRelationId, op->object);
recordDependencyOn(&myself, &referenced,
op->ref_is_hard ? DEPENDENCY_NORMAL : DEPENDENCY_AUTO);
@@ -1505,6 +1519,7 @@ storeOperators(List *opfamilyname, Oid amoid, Oid opfamilyoid,
referenced.objectId = op->refobjid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(referenced.classId, op->refobjid);
recordDependencyOn(&myself, &referenced,
op->ref_is_hard ? DEPENDENCY_INTERNAL : DEPENDENCY_AUTO);
@@ -1538,6 +1553,7 @@ storeOperators(List *opfamilyname, Oid amoid, Oid opfamilyoid,
referenced.objectId = op->sortfamily;
referenced.objectSubId = 0;
+ LockNotPinnedObject(OperatorFamilyRelationId, op->sortfamily);
recordDependencyOn(&myself, &referenced,
op->ref_is_hard ? DEPENDENCY_NORMAL : DEPENDENCY_AUTO);
}
@@ -1621,6 +1637,7 @@ storeProcedures(List *opfamilyname, Oid amoid, Oid opfamilyoid,
referenced.objectSubId = 0;
/* see comments in amapi.h about dependency strength */
+ LockNotPinnedObject(ProcedureRelationId, proc->object);
recordDependencyOn(&myself, &referenced,
proc->ref_is_hard ? DEPENDENCY_NORMAL : DEPENDENCY_AUTO);
@@ -1629,6 +1646,7 @@ storeProcedures(List *opfamilyname, Oid amoid, Oid opfamilyoid,
referenced.objectId = proc->refobjid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(referenced.classId, proc->refobjid);
recordDependencyOn(&myself, &referenced,
proc->ref_is_hard ? DEPENDENCY_INTERNAL : DEPENDENCY_AUTO);
diff --git a/src/backend/commands/operatorcmds.c b/src/backend/commands/operatorcmds.c
index 673648f1fc6..59a4715fffe 100644
--- a/src/backend/commands/operatorcmds.c
+++ b/src/backend/commands/operatorcmds.c
@@ -33,6 +33,7 @@
#include "access/htup_details.h"
#include "access/table.h"
+#include "catalog/dependency.h"
#include "catalog/indexing.h"
#include "catalog/objectaccess.h"
#include "catalog/pg_namespace.h"
@@ -656,11 +657,15 @@ AlterOperator(AlterOperatorStmt *stmt)
{
replaces[Anum_pg_operator_oprrest - 1] = true;
values[Anum_pg_operator_oprrest - 1] = ObjectIdGetDatum(restrictionOid);
+ if (OidIsValid(restrictionOid))
+ LockNotPinnedObject(ProcedureRelationId, restrictionOid);
}
if (updateJoin)
{
replaces[Anum_pg_operator_oprjoin - 1] = true;
values[Anum_pg_operator_oprjoin - 1] = ObjectIdGetDatum(joinOid);
+ if (OidIsValid(joinOid))
+ LockNotPinnedObject(ProcedureRelationId, joinOid);
}
if (OidIsValid(commutatorOid))
{
@@ -688,6 +693,31 @@ AlterOperator(AlterOperatorStmt *stmt)
CatalogTupleUpdate(catalog, &tup->t_self, tup);
+
+ /* Lock dependent objects */
+ oprForm = (Form_pg_operator) GETSTRUCT(tup);
+
+ if (OidIsValid(oprForm->oprnamespace))
+ LockNotPinnedObject(NamespaceRelationId, oprForm->oprnamespace);
+
+ if (OidIsValid(oprForm->oprleft))
+ LockNotPinnedObject(TypeRelationId, oprForm->oprleft);
+
+ if (OidIsValid(oprForm->oprright))
+ LockNotPinnedObject(TypeRelationId, oprForm->oprright);
+
+ if (OidIsValid(oprForm->oprresult))
+ LockNotPinnedObject(TypeRelationId, oprForm->oprresult);
+
+ if (OidIsValid(oprForm->oprcode))
+ LockNotPinnedObject(ProcedureRelationId, oprForm->oprcode);
+
+ if (OidIsValid(oprForm->oprrest))
+ LockNotPinnedObject(ProcedureRelationId, oprForm->oprrest);
+
+ if (OidIsValid(oprForm->oprjoin))
+ LockNotPinnedObject(ProcedureRelationId, oprForm->oprjoin);
+
address = makeOperatorDependencies(tup, false, true);
if (OidIsValid(commutatorOid) || OidIsValid(negatorOid))
diff --git a/src/backend/commands/policy.c b/src/backend/commands/policy.c
index 83056960fe4..cf5e1947929 100644
--- a/src/backend/commands/policy.c
+++ b/src/backend/commands/policy.c
@@ -722,6 +722,7 @@ CreatePolicy(CreatePolicyStmt *stmt)
myself.objectId = policy_id;
myself.objectSubId = 0;
+ LockNotPinnedObject(RelationRelationId, table_id);
recordDependencyOn(&myself, &target, DEPENDENCY_AUTO);
recordDependencyOnExpr(&myself, qual, qual_pstate->p_rtable,
@@ -1053,6 +1054,7 @@ AlterPolicy(AlterPolicyStmt *stmt)
myself.objectId = policy_id;
myself.objectSubId = 0;
+ LockNotPinnedObject(RelationRelationId, table_id);
recordDependencyOn(&myself, &target, DEPENDENCY_AUTO);
recordDependencyOnExpr(&myself, qual, qual_parse_rtable, DEPENDENCY_NORMAL);
diff --git a/src/backend/commands/proclang.c b/src/backend/commands/proclang.c
index 5036ac03639..f4a7e46b714 100644
--- a/src/backend/commands/proclang.c
+++ b/src/backend/commands/proclang.c
@@ -190,12 +190,14 @@ CreateProceduralLanguage(CreatePLangStmt *stmt)
/* dependency on the PL handler function */
ObjectAddressSet(referenced, ProcedureRelationId, handlerOid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, handlerOid);
/* dependency on the inline handler function, if any */
if (OidIsValid(inlineOid))
{
ObjectAddressSet(referenced, ProcedureRelationId, inlineOid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, inlineOid);
}
/* dependency on the validator function, if any */
@@ -203,6 +205,7 @@ CreateProceduralLanguage(CreatePLangStmt *stmt)
{
ObjectAddressSet(referenced, ProcedureRelationId, valOid);
add_exact_object_address(&referenced, addrs);
+ LockNotPinnedObject(ProcedureRelationId, valOid);
}
record_object_address_dependencies(&myself, addrs, DEPENDENCY_NORMAL);
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index b13ee2b745d..31dba914b87 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1691,6 +1691,8 @@ process_owned_by(Relation seqrel, List *owned_by, bool for_identity)
depobject.classId = RelationRelationId;
depobject.objectId = RelationGetRelid(seqrel);
depobject.objectSubId = 0;
+
+ LockNotPinnedObject(RelationRelationId, RelationGetRelid(tablerel));
recordDependencyOn(&depobject, &refobject, deptype);
}
diff --git a/src/backend/commands/statscmds.c b/src/backend/commands/statscmds.c
index a817821bf6d..5b7950d5823 100644
--- a/src/backend/commands/statscmds.c
+++ b/src/backend/commands/statscmds.c
@@ -88,6 +88,7 @@ CreateStatistics(CreateStatsStmt *stmt)
bool build_mcv;
bool build_expressions;
bool requested_type = false;
+ bool locked_object = false;
int i;
ListCell *cell;
ListCell *cell2;
@@ -536,6 +537,12 @@ CreateStatistics(CreateStatsStmt *stmt)
for (i = 0; i < nattnums; i++)
{
ObjectAddressSubSet(parentobject, RelationRelationId, relid, attnums[i]);
+
+ if (!locked_object)
+ {
+ LockNotPinnedObject(RelationRelationId, relid);
+ locked_object = true;
+ }
recordDependencyOn(&myself, &parentobject, DEPENDENCY_AUTO);
}
@@ -553,6 +560,8 @@ CreateStatistics(CreateStatsStmt *stmt)
if (!nattnums)
{
ObjectAddressSet(parentobject, RelationRelationId, relid);
+
+ LockNotPinnedObject(RelationRelationId, relid);
recordDependencyOn(&myself, &parentobject, DEPENDENCY_AUTO);
}
@@ -573,6 +582,7 @@ CreateStatistics(CreateStatsStmt *stmt)
* than the underlying table(s).
*/
ObjectAddressSet(parentobject, NamespaceRelationId, namespaceId);
+ LockNotPinnedObject(NamespaceRelationId, namespaceId);
recordDependencyOn(&myself, &parentobject, DEPENDENCY_NORMAL);
recordDependencyOnOwner(StatisticExtRelationId, statoid, stxowner);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 18f64db6e39..77b4d99f711 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -3527,6 +3527,7 @@ StoreCatalogInheritance1(Oid relationId, Oid parentOid,
childobject.objectId = relationId;
childobject.objectSubId = 0;
+ LockNotPinnedObject(RelationRelationId, parentOid);
recordDependencyOn(&childobject, &parentobject,
child_dependency_type(child_is_partition));
@@ -7434,7 +7435,9 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/*
* Add needed dependency entries for the new column.
*/
+ LockNotPinnedObject(TypeRelationId, attribute->atttypid);
add_column_datatype_dependency(myrelid, newattnum, attribute->atttypid);
+ LockNotPinnedObject(CollationRelationId, attribute->attcollation);
add_column_collation_dependency(myrelid, newattnum, attribute->attcollation);
/*
@@ -10491,11 +10494,16 @@ addFkConstraint(addFkConstraintSides fkside,
Assert(fkside != addFkBothSides);
if (fkside == addFkReferencedSide)
+ {
+ LockNotPinnedObject(ConstraintRelationId, parentConstr);
recordDependencyOn(&address, &referenced, DEPENDENCY_INTERNAL);
+ }
else
{
+ LockNotPinnedObject(ConstraintRelationId, parentConstr);
recordDependencyOn(&address, &referenced, DEPENDENCY_PARTITION_PRI);
ObjectAddressSet(referenced, RelationRelationId, RelationGetRelid(rel));
+ LockNotPinnedObject(RelationRelationId, RelationGetRelid(rel));
recordDependencyOn(&address, &referenced, DEPENDENCY_PARTITION_SEC);
}
}
@@ -13969,7 +13977,9 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
table_close(attrelation, RowExclusiveLock);
/* Install dependencies on new datatype and collation */
+ LockNotPinnedObject(TypeRelationId, targettype);
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
+ LockNotPinnedObject(CollationRelationId, targetcollid);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
/*
@@ -15547,6 +15557,7 @@ ATExecSetAccessMethodNoStorage(Relation rel, Oid newAccessMethodId)
*/
ObjectAddressSet(relobj, RelationRelationId, reloid);
ObjectAddressSet(referenced, AccessMethodRelationId, rd_rel->relam);
+ LockNotPinnedObject(AccessMethodRelationId, rd_rel->relam);
recordDependencyOn(&relobj, &referenced, DEPENDENCY_NORMAL);
}
else if (OidIsValid(oldAccessMethodId) &&
@@ -15566,6 +15577,7 @@ ATExecSetAccessMethodNoStorage(Relation rel, Oid newAccessMethodId)
OidIsValid(rd_rel->relam));
/* Both are valid, so update the dependency */
+ LockNotPinnedObject(AccessMethodRelationId, rd_rel->relam);
changeDependencyFor(RelationRelationId, reloid,
AccessMethodRelationId,
oldAccessMethodId, rd_rel->relam);
@@ -17288,6 +17300,7 @@ ATExecAddOf(Relation rel, const TypeName *ofTypename, LOCKMODE lockmode)
typeobj.classId = TypeRelationId;
typeobj.objectId = typeid;
typeobj.objectSubId = 0;
+ LockNotPinnedObject(TypeRelationId, typeid);
recordDependencyOn(&tableobj, &typeobj, DEPENDENCY_NORMAL);
/* Update pg_class.reloftype */
@@ -18060,14 +18073,17 @@ AlterRelationNamespaceInternal(Relation classRel, Oid relOid,
/* Update dependency on schema if caller said so */
- if (hasDependEntry &&
- changeDependencyFor(RelationRelationId,
- relOid,
- NamespaceRelationId,
- oldNspOid,
- newNspOid) != 1)
- elog(ERROR, "could not change schema dependency for relation \"%s\"",
- NameStr(classForm->relname));
+ if (hasDependEntry)
+ {
+ LockNotPinnedObject(NamespaceRelationId, newNspOid);
+ if (changeDependencyFor(RelationRelationId,
+ relOid,
+ NamespaceRelationId,
+ oldNspOid,
+ newNspOid) != 1)
+ elog(ERROR, "could not change schema dependency for relation \"%s\"",
+ NameStr(classForm->relname));
+ }
}
else
UnlockTuple(classRel, &classTup->t_self, InplaceUpdateTupleLock);
diff --git a/src/backend/commands/trigger.c b/src/backend/commands/trigger.c
index 7a5ffe32f60..3c1863c0e5b 100644
--- a/src/backend/commands/trigger.c
+++ b/src/backend/commands/trigger.c
@@ -1020,8 +1020,6 @@ CreateTriggerFiringOn(CreateTrigStmt *stmt, const char *queryString,
((Form_pg_class) GETSTRUCT(tuple))->relhastriggers = true;
CatalogTupleUpdate(pgrel, &tuple->t_self, tuple);
-
- CommandCounterIncrement();
}
else
CacheInvalidateRelcacheByTuple(tuple);
@@ -1029,6 +1027,13 @@ CreateTriggerFiringOn(CreateTrigStmt *stmt, const char *queryString,
heap_freetuple(tuple);
table_close(pgrel, RowExclusiveLock);
+ /*
+ * CommandCounterIncrement() here to ensure the new trigger entry is
+ * visible when LockNotPinnedObject() will check its existence before
+ * recording the dependencies.
+ */
+ CommandCounterIncrement();
+
/*
* If we're replacing a trigger, flush all the old dependencies before
* recording new ones.
@@ -1047,6 +1052,7 @@ CreateTriggerFiringOn(CreateTrigStmt *stmt, const char *queryString,
referenced.classId = ProcedureRelationId;
referenced.objectId = funcoid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ProcedureRelationId, funcoid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
if (isInternal && OidIsValid(constraintOid))
@@ -1060,6 +1066,7 @@ CreateTriggerFiringOn(CreateTrigStmt *stmt, const char *queryString,
referenced.classId = ConstraintRelationId;
referenced.objectId = constraintOid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(ConstraintRelationId, constraintOid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_INTERNAL);
}
else
@@ -1072,6 +1079,8 @@ CreateTriggerFiringOn(CreateTrigStmt *stmt, const char *queryString,
referenced.classId = RelationRelationId;
referenced.objectId = RelationGetRelid(rel);
referenced.objectSubId = 0;
+
+ LockNotPinnedObject(RelationRelationId, RelationGetRelid(rel));
recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
if (OidIsValid(constrrelid))
@@ -1079,6 +1088,8 @@ CreateTriggerFiringOn(CreateTrigStmt *stmt, const char *queryString,
referenced.classId = RelationRelationId;
referenced.objectId = constrrelid;
referenced.objectSubId = 0;
+
+ LockNotPinnedObject(RelationRelationId, constrrelid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_AUTO);
}
/* Not possible to have an index dependency in this case */
@@ -1093,6 +1104,7 @@ CreateTriggerFiringOn(CreateTrigStmt *stmt, const char *queryString,
referenced.classId = ConstraintRelationId;
referenced.objectId = constraintOid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(TriggerRelationId, trigoid);
recordDependencyOn(&referenced, &myself, DEPENDENCY_INTERNAL);
}
@@ -1102,8 +1114,11 @@ CreateTriggerFiringOn(CreateTrigStmt *stmt, const char *queryString,
if (OidIsValid(parentTriggerOid))
{
ObjectAddressSet(referenced, TriggerRelationId, parentTriggerOid);
+ LockNotPinnedObject(TriggerRelationId, parentTriggerOid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_PRI);
ObjectAddressSet(referenced, RelationRelationId, RelationGetRelid(rel));
+
+ LockNotPinnedObject(RelationRelationId, RelationGetRelid(rel));
recordDependencyOn(&myself, &referenced, DEPENDENCY_PARTITION_SEC);
}
}
@@ -1112,12 +1127,19 @@ CreateTriggerFiringOn(CreateTrigStmt *stmt, const char *queryString,
if (columns != NULL)
{
int i;
+ bool locked_object = false;
referenced.classId = RelationRelationId;
referenced.objectId = RelationGetRelid(rel);
for (i = 0; i < ncolumns; i++)
{
referenced.objectSubId = columns[i];
+
+ if (!locked_object)
+ {
+ LockNotPinnedObject(RelationRelationId, RelationGetRelid(rel));
+ locked_object = true;
+ }
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
}
@@ -1257,9 +1279,12 @@ TriggerSetParentTrigger(Relation trigRel,
ObjectAddressSet(depender, TriggerRelationId, childTrigId);
ObjectAddressSet(referenced, TriggerRelationId, parentTrigId);
+ LockNotPinnedObject(TriggerRelationId, parentTrigId);
recordDependencyOn(&depender, &referenced, DEPENDENCY_PARTITION_PRI);
ObjectAddressSet(referenced, RelationRelationId, childTableId);
+
+ LockNotPinnedObject(RelationRelationId, childTableId);
recordDependencyOn(&depender, &referenced, DEPENDENCY_PARTITION_SEC);
}
else
diff --git a/src/backend/commands/tsearchcmds.c b/src/backend/commands/tsearchcmds.c
index ab16d42ad56..e36c3da1023 100644
--- a/src/backend/commands/tsearchcmds.c
+++ b/src/backend/commands/tsearchcmds.c
@@ -214,6 +214,7 @@ DefineTSParser(List *names, List *parameters)
namestrcpy(&pname, prsname);
values[Anum_pg_ts_parser_prsname - 1] = NameGetDatum(&pname);
values[Anum_pg_ts_parser_prsnamespace - 1] = ObjectIdGetDatum(namespaceoid);
+ LockNotPinnedObject(NamespaceRelationId, namespaceoid);
/*
* loop over the definition list and extract the information we need.
@@ -224,28 +225,48 @@ DefineTSParser(List *names, List *parameters)
if (strcmp(defel->defname, "start") == 0)
{
+ Oid procoid;
+
values[Anum_pg_ts_parser_prsstart - 1] =
get_ts_parser_func(defel, Anum_pg_ts_parser_prsstart);
+ procoid = DatumGetObjectId(values[Anum_pg_ts_parser_prsstart - 1]);
+ LockNotPinnedObject(ProcedureRelationId, procoid);
}
else if (strcmp(defel->defname, "gettoken") == 0)
{
+ Oid procoid;
+
values[Anum_pg_ts_parser_prstoken - 1] =
get_ts_parser_func(defel, Anum_pg_ts_parser_prstoken);
+ procoid = DatumGetObjectId(values[Anum_pg_ts_parser_prstoken - 1]);
+ LockNotPinnedObject(ProcedureRelationId, procoid);
}
else if (strcmp(defel->defname, "end") == 0)
{
+ Oid procoid;
+
values[Anum_pg_ts_parser_prsend - 1] =
get_ts_parser_func(defel, Anum_pg_ts_parser_prsend);
+ procoid = DatumGetObjectId(values[Anum_pg_ts_parser_prsend - 1]);
+ LockNotPinnedObject(ProcedureRelationId, procoid);
}
else if (strcmp(defel->defname, "headline") == 0)
{
+ Oid procoid;
+
values[Anum_pg_ts_parser_prsheadline - 1] =
get_ts_parser_func(defel, Anum_pg_ts_parser_prsheadline);
+ procoid = DatumGetObjectId(values[Anum_pg_ts_parser_prsheadline - 1]);
+ LockNotPinnedObject(ProcedureRelationId, procoid);
}
else if (strcmp(defel->defname, "lextypes") == 0)
{
+ Oid procoid;
+
values[Anum_pg_ts_parser_prslextype - 1] =
get_ts_parser_func(defel, Anum_pg_ts_parser_prslextype);
+ procoid = DatumGetObjectId(values[Anum_pg_ts_parser_prslextype - 1]);
+ LockNotPinnedObject(ProcedureRelationId, procoid);
}
else
ereport(ERROR,
@@ -474,6 +495,10 @@ DefineTSDictionary(List *names, List *parameters)
CatalogTupleInsert(dictRel, tup);
+ /* Lock dependent objects */
+ LockNotPinnedObject(NamespaceRelationId, namespaceoid);
+ LockNotPinnedObject(TSTemplateRelationId, templId);
+
address = makeDictionaryDependencies(tup);
/* Post creation hook for new text search dictionary */
@@ -723,6 +748,7 @@ DefineTSTemplate(List *names, List *parameters)
namestrcpy(&dname, tmplname);
values[Anum_pg_ts_template_tmplname - 1] = NameGetDatum(&dname);
values[Anum_pg_ts_template_tmplnamespace - 1] = ObjectIdGetDatum(namespaceoid);
+ LockNotPinnedObject(NamespaceRelationId, namespaceoid);
/*
* loop over the definition list and extract the information we need.
@@ -733,15 +759,23 @@ DefineTSTemplate(List *names, List *parameters)
if (strcmp(defel->defname, "init") == 0)
{
+ Oid procoid;
+
values[Anum_pg_ts_template_tmplinit - 1] =
get_ts_template_func(defel, Anum_pg_ts_template_tmplinit);
nulls[Anum_pg_ts_template_tmplinit - 1] = false;
+ procoid = DatumGetObjectId(values[Anum_pg_ts_template_tmplinit - 1]);
+ LockNotPinnedObject(ProcedureRelationId, procoid);
}
else if (strcmp(defel->defname, "lexize") == 0)
{
+ Oid procoid;
+
values[Anum_pg_ts_template_tmpllexize - 1] =
get_ts_template_func(defel, Anum_pg_ts_template_tmpllexize);
nulls[Anum_pg_ts_template_tmpllexize - 1] = false;
+ procoid = DatumGetObjectId(values[Anum_pg_ts_template_tmpllexize - 1]);
+ LockNotPinnedObject(ProcedureRelationId, procoid);
}
else
ereport(ERROR,
@@ -998,6 +1032,10 @@ DefineTSConfiguration(List *names, List *parameters, ObjectAddress *copied)
values[Anum_pg_ts_config_cfgowner - 1] = ObjectIdGetDatum(GetUserId());
values[Anum_pg_ts_config_cfgparser - 1] = ObjectIdGetDatum(prsOid);
+ /* Lock dependent objects */
+ LockNotPinnedObject(NamespaceRelationId, namespaceoid);
+ LockNotPinnedObject(TSParserRelationId, prsOid);
+
tup = heap_form_tuple(cfgRel->rd_att, values, nulls);
CatalogTupleInsert(cfgRel, tup);
@@ -1063,6 +1101,7 @@ DefineTSConfiguration(List *names, List *parameters, ObjectAddress *copied)
slot[slot_stored_count]->tts_values[Anum_pg_ts_config_map_mapseqno - 1] = cfgmap->mapseqno;
slot[slot_stored_count]->tts_values[Anum_pg_ts_config_map_mapdict - 1] = cfgmap->mapdict;
+ LockNotPinnedObject(TSDictionaryRelationId, cfgmap->mapdict);
ExecStoreVirtualTuple(slot[slot_stored_count]);
slot_stored_count++;
@@ -1156,9 +1195,13 @@ ObjectAddress
AlterTSConfiguration(AlterTSConfigurationStmt *stmt)
{
HeapTuple tup;
+ Form_pg_ts_config cfg;
Oid cfgId;
Relation relMap;
ObjectAddress address;
+ ScanKeyData skey;
+ SysScanDesc scan;
+ HeapTuple maptup;
/* Find the configuration */
tup = GetTSConfigTuple(stmt->cfgname);
@@ -1168,7 +1211,8 @@ AlterTSConfiguration(AlterTSConfigurationStmt *stmt)
errmsg("text search configuration \"%s\" does not exist",
NameListToString(stmt->cfgname))));
- cfgId = ((Form_pg_ts_config) GETSTRUCT(tup))->oid;
+ cfg = (Form_pg_ts_config) GETSTRUCT(tup);
+ cfgId = cfg->oid;
/* must be owner */
if (!object_ownercheck(TSConfigRelationId, cfgId, GetUserId()))
@@ -1183,6 +1227,28 @@ AlterTSConfiguration(AlterTSConfigurationStmt *stmt)
else if (stmt->tokentype)
DropConfigurationMapping(stmt, tup, relMap);
+ /* Lock dependent objects */
+
+ ScanKeyInit(&skey,
+ Anum_pg_ts_config_map_mapcfg,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(cfgId));
+
+ scan = systable_beginscan(relMap, TSConfigMapIndexId, true,
+ NULL, 1, &skey);
+
+ while (HeapTupleIsValid((maptup = systable_getnext(scan))))
+ {
+ Form_pg_ts_config_map cfgmap = (Form_pg_ts_config_map) GETSTRUCT(maptup);
+
+ LockNotPinnedObject(TSDictionaryRelationId, cfgmap->mapdict);
+ }
+
+ systable_endscan(scan);
+
+ LockNotPinnedObject(NamespaceRelationId, cfg->cfgnamespace);
+ LockNotPinnedObject(TSParserRelationId, cfg->cfgparser);
+
/* Update dependencies */
makeConfigurationDependencies(tup, true, relMap);
@@ -1414,6 +1480,8 @@ MakeConfigurationMapping(AlterTSConfigurationStmt *stmt,
repl_val[Anum_pg_ts_config_map_mapdict - 1] = ObjectIdGetDatum(dictNew);
repl_repl[Anum_pg_ts_config_map_mapdict - 1] = true;
+ LockNotPinnedObject(TSDictionaryRelationId, dictNew);
+
newtup = heap_modify_tuple(maptup,
RelationGetDescr(relMap),
repl_val, repl_null, repl_repl);
@@ -1456,6 +1524,9 @@ MakeConfigurationMapping(AlterTSConfigurationStmt *stmt,
slot[slotCount]->tts_values[Anum_pg_ts_config_map_mapseqno - 1] = Int32GetDatum(j + 1);
slot[slotCount]->tts_values[Anum_pg_ts_config_map_mapdict - 1] = ObjectIdGetDatum(dictIds[j]);
+ /* Lock dependent objects */
+ LockNotPinnedObject(TSDictionaryRelationId, dictIds[j]);
+
ExecStoreVirtualTuple(slot[slotCount]);
slotCount++;
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index 3cb3ca1cca1..e36ea6d97e0 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -1820,6 +1820,7 @@ makeRangeConstructors(const char *name, Oid namespace,
* that they go away silently when the type is dropped. Note that
* pg_dump depends on this choice to avoid dumping the constructors.
*/
+ LockNotPinnedObject(TypeRelationId, rangeOid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_INTERNAL);
}
}
@@ -1885,6 +1886,7 @@ makeMultirangeConstructors(const char *name, Oid namespace,
* that they go away silently when the type is dropped. Note that pg_dump
* depends on this choice to avoid dumping the constructors.
*/
+ LockNotPinnedObject(TypeRelationId, multirangeOid);
recordDependencyOn(&myself, &referenced, DEPENDENCY_INTERNAL);
pfree(argtypes);
@@ -2698,6 +2700,45 @@ AlterDomainDefault(List *names, Node *defaultRaw)
CatalogTupleUpdate(rel, &tup->t_self, newtuple);
+ /* Lock dependent objects */
+ typTup = (Form_pg_type) GETSTRUCT(newtuple);
+
+ if (OidIsValid(typTup->typnamespace))
+ LockNotPinnedObject(NamespaceRelationId, typTup->typnamespace);
+
+ if (OidIsValid(typTup->typinput))
+ LockNotPinnedObject(ProcedureRelationId, typTup->typinput);
+
+ if (OidIsValid(typTup->typoutput))
+ LockNotPinnedObject(ProcedureRelationId, typTup->typoutput);
+
+ if (OidIsValid(typTup->typreceive))
+ LockNotPinnedObject(ProcedureRelationId, typTup->typreceive);
+
+ if (OidIsValid(typTup->typsend))
+ LockNotPinnedObject(ProcedureRelationId, typTup->typsend);
+
+ if (OidIsValid(typTup->typmodin))
+ LockNotPinnedObject(ProcedureRelationId, typTup->typmodin);
+
+ if (OidIsValid(typTup->typmodout))
+ LockNotPinnedObject(ProcedureRelationId, typTup->typmodout);
+
+ if (OidIsValid(typTup->typanalyze))
+ LockNotPinnedObject(ProcedureRelationId, typTup->typanalyze);
+
+ if (OidIsValid(typTup->typsubscript))
+ LockNotPinnedObject(ProcedureRelationId, typTup->typsubscript);
+
+ if (OidIsValid(typTup->typbasetype))
+ LockNotPinnedObject(TypeRelationId, typTup->typbasetype);
+
+ if (OidIsValid(typTup->typcollation))
+ LockNotPinnedObject(CollationRelationId, typTup->typcollation);
+
+ if (OidIsValid(typTup->typelem))
+ LockNotPinnedObject(TypeRelationId, typTup->typelem);
+
/* Rebuild dependencies */
GenerateTypeDependencies(newtuple,
rel,
@@ -4263,10 +4304,13 @@ AlterTypeNamespaceInternal(Oid typeOid, Oid nspOid,
if (oldNspOid != nspOid &&
(isCompositeType || typform->typtype != TYPTYPE_COMPOSITE) &&
!isImplicitArray)
+ {
+ LockNotPinnedObject(NamespaceRelationId, nspOid);
if (changeDependencyFor(TypeRelationId, typeOid,
NamespaceRelationId, oldNspOid, nspOid) != 1)
elog(ERROR, "could not change schema dependency for type \"%s\"",
format_type_be(typeOid));
+ }
InvokeObjectPostAlterHook(TypeRelationId, typeOid, 0);
@@ -4558,6 +4602,7 @@ AlterTypeRecurse(Oid typeOid, bool isImplicitArray,
SysScanDesc scan;
ScanKeyData key[1];
HeapTuple domainTup;
+ Form_pg_type typeForm;
/* Since this function recurses, it could be driven to stack overflow */
check_stack_depth();
@@ -4606,6 +4651,45 @@ AlterTypeRecurse(Oid typeOid, bool isImplicitArray,
newtup = heap_modify_tuple(tup, RelationGetDescr(catalog),
values, nulls, replaces);
+ /* Lock dependent objects */
+ typeForm = (Form_pg_type) GETSTRUCT(newtup);
+
+ if (OidIsValid(typeForm->typnamespace))
+ LockNotPinnedObject(NamespaceRelationId, typeForm->typnamespace);
+
+ if (OidIsValid(typeForm->typinput))
+ LockNotPinnedObject(ProcedureRelationId, typeForm->typinput);
+
+ if (OidIsValid(typeForm->typoutput))
+ LockNotPinnedObject(ProcedureRelationId, typeForm->typoutput);
+
+ if (OidIsValid(typeForm->typreceive))
+ LockNotPinnedObject(ProcedureRelationId, typeForm->typreceive);
+
+ if (OidIsValid(typeForm->typsend))
+ LockNotPinnedObject(ProcedureRelationId, typeForm->typsend);
+
+ if (OidIsValid(typeForm->typmodin))
+ LockNotPinnedObject(ProcedureRelationId, typeForm->typmodin);
+
+ if (OidIsValid(typeForm->typmodout))
+ LockNotPinnedObject(ProcedureRelationId, typeForm->typmodout);
+
+ if (OidIsValid(typeForm->typanalyze))
+ LockNotPinnedObject(ProcedureRelationId, typeForm->typanalyze);
+
+ if (OidIsValid(typeForm->typsubscript))
+ LockNotPinnedObject(ProcedureRelationId, typeForm->typsubscript);
+
+ if (OidIsValid(typeForm->typbasetype))
+ LockNotPinnedObject(TypeRelationId, typeForm->typbasetype);
+
+ if (OidIsValid(typeForm->typcollation))
+ LockNotPinnedObject(CollationRelationId, typeForm->typcollation);
+
+ if (OidIsValid(typeForm->typelem))
+ LockNotPinnedObject(TypeRelationId, typeForm->typelem);
+
CatalogTupleUpdate(catalog, &newtup->t_self, newtup);
/* Rebuild dependencies for this type */
diff --git a/src/backend/rewrite/rewriteDefine.c b/src/backend/rewrite/rewriteDefine.c
index 8aa90b0d6fb..14a6468efca 100644
--- a/src/backend/rewrite/rewriteDefine.c
+++ b/src/backend/rewrite/rewriteDefine.c
@@ -155,6 +155,7 @@ InsertRule(const char *rulname,
referenced.objectId = eventrel_oid;
referenced.objectSubId = 0;
+ LockNotPinnedObject(RelationRelationId, eventrel_oid);
recordDependencyOn(&myself, &referenced,
(evtype == CMD_SELECT) ? DEPENDENCY_INTERNAL : DEPENDENCY_AUTO);
diff --git a/src/backend/utils/errcodes.txt b/src/backend/utils/errcodes.txt
index c96aa7c49ef..c664ae4977b 100644
--- a/src/backend/utils/errcodes.txt
+++ b/src/backend/utils/errcodes.txt
@@ -277,6 +277,7 @@ Section: Class 28 - Invalid Authorization Specification
Section: Class 2B - Dependent Privilege Descriptors Still Exist
2B000 E ERRCODE_DEPENDENT_PRIVILEGE_DESCRIPTORS_STILL_EXIST dependent_privilege_descriptors_still_exist
+2BP02 E ERRCODE_DEPENDENT_OBJECTS_DOES_NOT_EXIST dependent_objects_does_not_exist
2BP01 E ERRCODE_DEPENDENT_OBJECTS_STILL_EXIST dependent_objects_still_exist
Section: Class 2D - Invalid Transaction Termination
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 0ea7ccf5243..73ee2c6a787 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -101,6 +101,8 @@ typedef struct ObjectAddresses ObjectAddresses;
/* in dependency.c */
extern void AcquireDeletionLock(const ObjectAddress *object, int flags);
+extern void LockNotPinnedObjectById(const ObjectAddress *object);
+extern void LockNotPinnedObject(Oid classid, Oid objid);
extern void ReleaseDeletionLock(const ObjectAddress *object);
@@ -172,6 +174,7 @@ extern long changeDependenciesOf(Oid classId, Oid oldObjectId,
extern long changeDependenciesOn(Oid refClassId, Oid oldRefObjectId,
Oid newRefObjectId);
+extern bool isObjectPinned(const ObjectAddress *object);
extern Oid getExtensionOfObject(Oid classId, Oid objectId);
extern List *getAutoExtensionsOfObject(Oid classId, Oid objectId);
diff --git a/src/include/catalog/objectaddress.h b/src/include/catalog/objectaddress.h
index 630434b73cf..34d5428433b 100644
--- a/src/include/catalog/objectaddress.h
+++ b/src/include/catalog/objectaddress.h
@@ -53,6 +53,7 @@ extern void check_object_ownership(Oid roleid,
Node *object, Relation relation);
extern Oid get_object_namespace(const ObjectAddress *address);
+extern bool ObjectByIdExist(const ObjectAddress *address);
extern bool is_objectclass_supported(Oid class_id);
extern const char *get_object_class_descr(Oid class_id);
diff --git a/src/test/isolation/expected/test_dependencies_locks.out b/src/test/isolation/expected/test_dependencies_locks.out
new file mode 100644
index 00000000000..820680f5e16
--- /dev/null
+++ b/src/test/isolation/expected/test_dependencies_locks.out
@@ -0,0 +1,129 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1_begin s1_create_function_in_schema s2_drop_schema s1_commit
+step s1_begin: BEGIN;
+step s1_create_function_in_schema: CREATE FUNCTION testschema.foo() RETURNS int AS 'select 1' LANGUAGE sql;
+step s2_drop_schema: DROP SCHEMA testschema; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_schema: <... completed>
+ERROR: cannot drop schema testschema because other objects depend on it
+
+starting permutation: s2_begin s2_drop_schema s1_create_function_in_schema s2_commit
+step s2_begin: BEGIN;
+step s2_drop_schema: DROP SCHEMA testschema;
+step s1_create_function_in_schema: CREATE FUNCTION testschema.foo() RETURNS int AS 'select 1' LANGUAGE sql; <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_function_in_schema: <... completed>
+ERROR: dependent object does not exist
+
+starting permutation: s1_begin s1_alter_function_schema s2_drop_alterschema s1_commit
+step s1_begin: BEGIN;
+step s1_alter_function_schema: ALTER FUNCTION public.falter() SET SCHEMA alterschema;
+step s2_drop_alterschema: DROP SCHEMA alterschema; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_alterschema: <... completed>
+ERROR: cannot drop schema alterschema because other objects depend on it
+
+starting permutation: s2_begin s2_drop_alterschema s1_alter_function_schema s2_commit
+step s2_begin: BEGIN;
+step s2_drop_alterschema: DROP SCHEMA alterschema;
+step s1_alter_function_schema: ALTER FUNCTION public.falter() SET SCHEMA alterschema; <waiting ...>
+step s2_commit: COMMIT;
+step s1_alter_function_schema: <... completed>
+ERROR: dependent object does not exist
+
+starting permutation: s1_begin s1_create_function_with_argtype s2_drop_foo_type s1_commit
+step s1_begin: BEGIN;
+step s1_create_function_with_argtype: CREATE FUNCTION fooargtype(num foo) RETURNS int AS 'select 1' LANGUAGE sql;
+step s2_drop_foo_type: DROP TYPE public.foo; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_foo_type: <... completed>
+ERROR: cannot drop type foo because other objects depend on it
+
+starting permutation: s2_begin s2_drop_foo_type s1_create_function_with_argtype s2_commit
+step s2_begin: BEGIN;
+step s2_drop_foo_type: DROP TYPE public.foo;
+step s1_create_function_with_argtype: CREATE FUNCTION fooargtype(num foo) RETURNS int AS 'select 1' LANGUAGE sql; <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_function_with_argtype: <... completed>
+ERROR: dependent object does not exist
+
+starting permutation: s1_begin s1_create_function_with_rettype s2_drop_foo_rettype s1_commit
+step s1_begin: BEGIN;
+step s1_create_function_with_rettype: CREATE FUNCTION footrettype() RETURNS id LANGUAGE sql RETURN 1;
+step s2_drop_foo_rettype: DROP DOMAIN id; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_foo_rettype: <... completed>
+ERROR: cannot drop type id because other objects depend on it
+
+starting permutation: s2_begin s2_drop_foo_rettype s1_create_function_with_rettype s2_commit
+step s2_begin: BEGIN;
+step s2_drop_foo_rettype: DROP DOMAIN id;
+step s1_create_function_with_rettype: CREATE FUNCTION footrettype() RETURNS id LANGUAGE sql RETURN 1; <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_function_with_rettype: <... completed>
+ERROR: dependent object does not exist
+
+starting permutation: s1_begin s1_create_function_with_function s2_drop_function_f s1_commit
+step s1_begin: BEGIN;
+step s1_create_function_with_function: CREATE FUNCTION foofunc() RETURNS int LANGUAGE SQL RETURN f() + 1;
+step s2_drop_function_f: DROP FUNCTION f(); <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_function_f: <... completed>
+ERROR: cannot drop function f() because other objects depend on it
+
+starting permutation: s2_begin s2_drop_function_f s1_create_function_with_function s2_commit
+step s2_begin: BEGIN;
+step s2_drop_function_f: DROP FUNCTION f();
+step s1_create_function_with_function: CREATE FUNCTION foofunc() RETURNS int LANGUAGE SQL RETURN f() + 1; <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_function_with_function: <... completed>
+ERROR: dependent object does not exist
+
+starting permutation: s1_begin s1_create_domain_with_domain s2_drop_domain_id s1_commit
+step s1_begin: BEGIN;
+step s1_create_domain_with_domain: CREATE DOMAIN idid as id;
+step s2_drop_domain_id: DROP DOMAIN id; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_domain_id: <... completed>
+ERROR: cannot drop type id because other objects depend on it
+
+starting permutation: s2_begin s2_drop_domain_id s1_create_domain_with_domain s2_commit
+step s2_begin: BEGIN;
+step s2_drop_domain_id: DROP DOMAIN id;
+step s1_create_domain_with_domain: CREATE DOMAIN idid as id; <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_domain_with_domain: <... completed>
+ERROR: dependent object does not exist
+
+starting permutation: s1_begin s1_create_table_with_type s2_drop_footab_type s1_commit
+step s1_begin: BEGIN;
+step s1_create_table_with_type: CREATE TABLE tabtype(a footab);
+step s2_drop_footab_type: DROP TYPE public.footab; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_footab_type: <... completed>
+ERROR: cannot drop type footab because other objects depend on it
+
+starting permutation: s2_begin s2_drop_footab_type s1_create_table_with_type s2_commit
+step s2_begin: BEGIN;
+step s2_drop_footab_type: DROP TYPE public.footab;
+step s1_create_table_with_type: CREATE TABLE tabtype(a footab); <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_table_with_type: <... completed>
+ERROR: dependent object does not exist
+
+starting permutation: s1_begin s1_create_server_with_fdw_wrapper s2_drop_fdw_wrapper s1_commit
+step s1_begin: BEGIN;
+step s1_create_server_with_fdw_wrapper: CREATE SERVER srv_fdw_wrapper FOREIGN DATA WRAPPER fdw_wrapper;
+step s2_drop_fdw_wrapper: DROP FOREIGN DATA WRAPPER fdw_wrapper RESTRICT; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_fdw_wrapper: <... completed>
+ERROR: cannot drop foreign-data wrapper fdw_wrapper because other objects depend on it
+
+starting permutation: s2_begin s2_drop_fdw_wrapper s1_create_server_with_fdw_wrapper s2_commit
+step s2_begin: BEGIN;
+step s2_drop_fdw_wrapper: DROP FOREIGN DATA WRAPPER fdw_wrapper RESTRICT;
+step s1_create_server_with_fdw_wrapper: CREATE SERVER srv_fdw_wrapper FOREIGN DATA WRAPPER fdw_wrapper; <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_server_with_fdw_wrapper: <... completed>
+ERROR: dependent object does not exist
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 143109aa4da..0e80dfecfb3 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -115,3 +115,4 @@ test: serializable-parallel-2
test: serializable-parallel-3
test: matview-write-skew
test: lock-nowait
+test: test_dependencies_locks
diff --git a/src/test/isolation/specs/test_dependencies_locks.spec b/src/test/isolation/specs/test_dependencies_locks.spec
new file mode 100644
index 00000000000..5d04dfe9dc6
--- /dev/null
+++ b/src/test/isolation/specs/test_dependencies_locks.spec
@@ -0,0 +1,89 @@
+setup
+{
+ CREATE SCHEMA testschema;
+ CREATE SCHEMA alterschema;
+ CREATE TYPE public.foo as enum ('one', 'two');
+ CREATE TYPE public.footab as enum ('three', 'four');
+ CREATE DOMAIN id AS int;
+ CREATE FUNCTION f() RETURNS int LANGUAGE SQL RETURN 1;
+ CREATE FUNCTION public.falter() RETURNS int LANGUAGE SQL RETURN 1;
+ CREATE FOREIGN DATA WRAPPER fdw_wrapper;
+}
+
+teardown
+{
+ DROP FUNCTION IF EXISTS testschema.foo();
+ DROP FUNCTION IF EXISTS fooargtype(num foo);
+ DROP FUNCTION IF EXISTS footrettype();
+ DROP FUNCTION IF EXISTS foofunc();
+ DROP FUNCTION IF EXISTS public.falter();
+ DROP FUNCTION IF EXISTS alterschema.falter();
+ DROP DOMAIN IF EXISTS idid;
+ DROP SERVER IF EXISTS srv_fdw_wrapper;
+ DROP TABLE IF EXISTS tabtype;
+ DROP SCHEMA IF EXISTS testschema;
+ DROP SCHEMA IF EXISTS alterschema;
+ DROP TYPE IF EXISTS public.foo;
+ DROP TYPE IF EXISTS public.footab;
+ DROP DOMAIN IF EXISTS id;
+ DROP FUNCTION IF EXISTS f();
+ DROP FOREIGN DATA WRAPPER IF EXISTS fdw_wrapper;
+}
+
+session "s1"
+
+step "s1_begin" { BEGIN; }
+step "s1_create_function_in_schema" { CREATE FUNCTION testschema.foo() RETURNS int AS 'select 1' LANGUAGE sql; }
+step "s1_create_function_with_argtype" { CREATE FUNCTION fooargtype(num foo) RETURNS int AS 'select 1' LANGUAGE sql; }
+step "s1_create_function_with_rettype" { CREATE FUNCTION footrettype() RETURNS id LANGUAGE sql RETURN 1; }
+step "s1_create_function_with_function" { CREATE FUNCTION foofunc() RETURNS int LANGUAGE SQL RETURN f() + 1; }
+step "s1_alter_function_schema" { ALTER FUNCTION public.falter() SET SCHEMA alterschema; }
+step "s1_create_domain_with_domain" { CREATE DOMAIN idid as id; }
+step "s1_create_table_with_type" { CREATE TABLE tabtype(a footab); }
+step "s1_create_server_with_fdw_wrapper" { CREATE SERVER srv_fdw_wrapper FOREIGN DATA WRAPPER fdw_wrapper; }
+step "s1_commit" { COMMIT; }
+
+session "s2"
+
+step "s2_begin" { BEGIN; }
+step "s2_drop_schema" { DROP SCHEMA testschema; }
+step "s2_drop_alterschema" { DROP SCHEMA alterschema; }
+step "s2_drop_foo_type" { DROP TYPE public.foo; }
+step "s2_drop_foo_rettype" { DROP DOMAIN id; }
+step "s2_drop_footab_type" { DROP TYPE public.footab; }
+step "s2_drop_function_f" { DROP FUNCTION f(); }
+step "s2_drop_domain_id" { DROP DOMAIN id; }
+step "s2_drop_fdw_wrapper" { DROP FOREIGN DATA WRAPPER fdw_wrapper RESTRICT; }
+step "s2_commit" { COMMIT; }
+
+# function - schema
+permutation "s1_begin" "s1_create_function_in_schema" "s2_drop_schema" "s1_commit"
+permutation "s2_begin" "s2_drop_schema" "s1_create_function_in_schema" "s2_commit"
+
+# alter function - schema
+permutation "s1_begin" "s1_alter_function_schema" "s2_drop_alterschema" "s1_commit"
+permutation "s2_begin" "s2_drop_alterschema" "s1_alter_function_schema" "s2_commit"
+
+# function - argtype
+permutation "s1_begin" "s1_create_function_with_argtype" "s2_drop_foo_type" "s1_commit"
+permutation "s2_begin" "s2_drop_foo_type" "s1_create_function_with_argtype" "s2_commit"
+
+# function - rettype
+permutation "s1_begin" "s1_create_function_with_rettype" "s2_drop_foo_rettype" "s1_commit"
+permutation "s2_begin" "s2_drop_foo_rettype" "s1_create_function_with_rettype" "s2_commit"
+
+# function - function
+permutation "s1_begin" "s1_create_function_with_function" "s2_drop_function_f" "s1_commit"
+permutation "s2_begin" "s2_drop_function_f" "s1_create_function_with_function" "s2_commit"
+
+# domain - domain
+permutation "s1_begin" "s1_create_domain_with_domain" "s2_drop_domain_id" "s1_commit"
+permutation "s2_begin" "s2_drop_domain_id" "s1_create_domain_with_domain" "s2_commit"
+
+# table - type
+permutation "s1_begin" "s1_create_table_with_type" "s2_drop_footab_type" "s1_commit"
+permutation "s2_begin" "s2_drop_footab_type" "s1_create_table_with_type" "s2_commit"
+
+# server - foreign data wrapper
+permutation "s1_begin" "s1_create_server_with_fdw_wrapper" "s2_drop_fdw_wrapper" "s1_commit"
+permutation "s2_begin" "s2_drop_fdw_wrapper" "s1_create_server_with_fdw_wrapper" "s2_commit"
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 362f38856d2..7c162c0d455 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2862,11 +2862,12 @@ begin;
alter table alterlock2
add constraint alterlock2nv foreign key (f1) references alterlock (f1) NOT VALID;
select * from my_locks order by 1;
- relname | max_lockmode
-------------+-----------------------
- alterlock | ShareRowExclusiveLock
- alterlock2 | ShareRowExclusiveLock
-(2 rows)
+ relname | max_lockmode
+----------------+-----------------------
+ alterlock | ShareRowExclusiveLock
+ alterlock2 | ShareRowExclusiveLock
+ alterlock_pkey | AccessShareLock
+(3 rows)
commit;
begin;
--
2.34.1
--jlrXVwdftypL847/--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45 Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)
This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:
- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).
XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.
Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
doc/src/sgml/monitoring.sgml | 70 +++++++++++++
src/backend/storage/aio/aio.c | 13 +++
src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
src/backend/utils/adt/pgstatfuncs.c | 60 +++++++++++
src/include/catalog/pg_proc.dat | 7 ++
src/include/pgstat.h | 25 +++++
src/include/utils/pgstat_internal.h | 3 +-
src/test/modules/test_aio/t/001_aio.pl | 32 ++++++
src/tools/pgindent/typedefs.list | 1 +
9 files changed, 319 insertions(+), 1 deletion(-)
24.4% doc/src/sgml/
3.2% src/backend/storage/aio/
25.3% src/backend/utils/activity/
20.5% src/backend/utils/adt/
4.9% src/include/catalog/
10.0% src/include/
11.1% src/test/modules/test_aio/t/
diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
</para></entry>
</row>
+ <row>
+ <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>pg_stat_get_backend_aio</primary>
+ </indexterm>
+ <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+ <returnvalue>record</returnvalue>
+ </para>
+ <para>
+ Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+ backend with the specified process ID. The returned values are:
+ <itemizedlist>
+ <listitem>
+ <para>
+ <literal>started</literal>: Total IOs initiated.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_sync</literal>: IOs executed synchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>executed_async</literal>: IOs submitted asynchronously.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_self</literal>: IO completions processed by this
+ backend.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>completed_other</literal>: IO completions processed on
+ behalf of other backends.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>handle_waits</literal>: Times waited for a free AIO handle.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>submitted</literal>: Number of submit calls to the IO
+ method. Compare with <literal>executed_async</literal> to determine
+ average batch size.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ <literal>stats_reset</literal>: Timestamp of last stats reset.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ The <literal>completed_other</literal> column is only meaningful
+ when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+ with <literal>worker</literal> mode, IO completions are processed by
+ IO worker processes which do not track these statistics.
+ </para>
+ <para>
+ The function does not return AIO statistics for the checkpointer,
+ the background writer, the startup process and the autovacuum launcher.
+ </para></entry>
+ </row>
+
<row>
<entry role="func_table_entry"><para role="func_signature">
<indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
#include "lib/ilist.h"
#include "miscadmin.h"
+#include "pgstat.h"
#include "port/atomics.h"
#include "storage/aio.h"
#include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
"staged (synchronous: %d, in_batch: %d)",
needs_synchronous, pgaio_my_backend->in_batchmode);
+ pgstat_count_backend_aio_start(needs_synchronous);
+
if (!needs_synchronous)
{
pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
/* condition variable broadcast ensures state is visible before wakeup */
ConditionVariableBroadcast(&ioh->cv);
+ /* Track AIO completion stats */
+ if (ioh->owner_procno == MyProcNumber)
+ pgstat_count_backend_aio_complete_self();
+ else
+ pgstat_count_backend_aio_complete_other();
+
/* contains call to pgaio_io_call_complete_local() */
if (ioh->owner_procno == MyProcNumber)
pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
{
int reclaimed = 0;
+ pgstat_count_backend_aio_handle_wait();
+
pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
pgaio_my_backend->num_staged_ios,
dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
Assert(total_submitted == did_submit);
+ pgstat_count_backend_aio_submitted();
+
pgaio_my_backend->num_staged_ios = 0;
pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
static PgStat_BackendPending PendingBackendStats;
static bool backend_has_iostats = false;
static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
/*
* WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
pgstat_report_fixed = true;
}
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.started++;
+ if (synchronous)
+ PendingBackendStats.aio_counters.executed_sync++;
+ else
+ PendingBackendStats.aio_counters.executed_async++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_self++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.completed_other++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.handle_waits++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+ if (!pgstat_tracks_backend_bktype(MyBackendType))
+ return;
+
+ PendingBackendStats.aio_counters.submitted++;
+
+ backend_has_aiostats = true;
+ pgstat_report_fixed = true;
+}
+
/*
* Returns statistics of a backend by proc number.
*/
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
backend_has_lockstats = false;
}
+/*
+ * Flush out locally pending backend AIO statistics. Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+ PgStatShared_Backend *shbackendent;
+ PgStat_AioCounters *bktype_shstats;
+
+ if (!backend_has_aiostats)
+ return;
+
+ shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+ bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+ (bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+ AIOSTAT_ACC(started);
+ AIOSTAT_ACC(executed_sync);
+ AIOSTAT_ACC(executed_async);
+ AIOSTAT_ACC(completed_self);
+ AIOSTAT_ACC(completed_other);
+ AIOSTAT_ACC(handle_waits);
+ AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+ MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+ backend_has_aiostats = false;
+}
+
/*
* Flush out locally pending backend statistics
*
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
has_pending_data = true;
+ /* Some AIO data pending? */
+ if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+ has_pending_data = true;
+
if (!has_pending_data)
return false;
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
pgstat_flush_backend_entry_lock(entry_ref);
+ if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+ pgstat_flush_backend_entry_aio(entry_ref);
+
pgstat_unlock_entry(entry_ref);
return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
backend_has_iostats = false;
backend_has_lockstats = false;
+ backend_has_aiostats = false;
/*
* Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
}
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS 8
+ TupleDesc tupdesc;
+ Datum values[PG_STAT_BACKEND_AIO_COLS] = {0};
+ bool nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+ int pid;
+ PgStat_Backend *backend_stats;
+ PgStat_AioCounters aio_counters;
+
+ pid = PG_GETARG_INT32(0);
+ backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+ if (!backend_stats)
+ PG_RETURN_NULL();
+
+ aio_counters = backend_stats->aio_counters;
+
+ /* Initialise attributes information in the tuple descriptor */
+ tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+ INT8OID, -1, 0);
+ TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+ TIMESTAMPTZOID, -1, 0);
+ TupleDescFinalize(tupdesc);
+ BlessTupleDesc(tupdesc);
+
+ /* Fill values */
+ values[0] = Int64GetDatum(aio_counters.started);
+ values[1] = Int64GetDatum(aio_counters.executed_sync);
+ values[2] = Int64GetDatum(aio_counters.executed_async);
+ values[3] = Int64GetDatum(aio_counters.completed_self);
+ values[4] = Int64GetDatum(aio_counters.completed_other);
+ values[5] = Int64GetDatum(aio_counters.handle_waits);
+ values[6] = Int64GetDatum(aio_counters.submitted);
+
+ if (backend_stats->stat_reset_timestamp != 0)
+ values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+ else
+ nulls[7] = true;
+
+ PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
/*
* Returns statistics of WAL activity
*/
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
proargmodes => '{i,o,o,o,o,o}',
proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+ proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+ prorettype => 'record', proargtypes => 'int4',
+ proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o}',
+ proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+ prosrc => 'pg_stat_get_backend_aio' },
{ oid => '6248', descr => 'statistics: information about WAL prefetching',
proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
TimestampTz stat_reset_timestamp;
} PgStat_WalStats;
+/* -------
+ * PgStat_AioCounters AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+ PgStat_Counter started;
+ PgStat_Counter executed_sync;
+ PgStat_Counter executed_async;
+ PgStat_Counter completed_self;
+ PgStat_Counter completed_other;
+ PgStat_Counter handle_waits;
+ PgStat_Counter submitted;
+} PgStat_AioCounters;
+
/* -------
* PgStat_Backend Backend statistics
* -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
PgStat_BktypeIO io_stats;
PgStat_WalCounters wal_counters;
PgStat_PendingLock lock_stats;
+ PgStat_AioCounters aio_counters;
} PgStat_Backend;
/* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
* PGSTAT_KIND_LOCK.
*/
PgStat_PendingLock pending_lock;
+ /* Store the AIO statistics counters */
+ PgStat_AioCounters aio_counters;
} PgStat_BackendPending;
/*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
#define PGSTAT_BACKEND_FLUSH_IO (1 << 0) /* Flush I/O statistics */
#define PGSTAT_BACKEND_FLUSH_WAL (1 << 1) /* Flush WAL statistics */
#define PGSTAT_BACKEND_FLUSH_LOCK (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
extern bool pgstat_flush_backend(bool nowait, uint32 flags);
extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
$psql_c->quit();
}
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+ my $io_method = shift;
+ my $node = shift;
+
+ my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+ # Reset backend stats, evict relation, then read it back to force
+ # physical IO through the AIO layer.
+ $psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+ $psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+ $psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+ $psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+ # started must be > 0 after physical IO
+ my $started = $psql->query_safe(
+ qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ cmp_ok($started, '>', 0,
+ "$io_method: AIO stats: started > 0 after physical IO");
+
+ # invariant: started = executed_sync + executed_async
+ my $consistent = $psql->query_safe(
+ qq(SELECT started = executed_sync + executed_async
+ FROM pg_stat_get_backend_aio(pg_backend_pid())));
+ is($consistent, 't',
+ "$io_method: AIO stats: started = executed_sync + executed_async");
+
+ $psql->quit();
+}
+
# Run all tests that for the specified node / io_method
sub test_io_method
{
@@ -1878,6 +1909,7 @@ CHECKPOINT;
test_ignore_checksum($io_method, $node);
test_checksum_createdb($io_method, $node);
test_read_buffers($io_method, $node);
+ test_aio_stats($io_method, $node);
# generic injection tests
SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
PgStatShared_SLRU
PgStatShared_Subscription
PgStatShared_Wal
+PgStat_AioCounters
PgStat_ArchiverStats
PgStat_Backend
PgStat_BackendPending
--
2.34.1
--Pi0/Nu3LVhi0Ij+H--
^ permalink raw reply [nested|flat] 142+ messages in thread
* Add per-backend AIO statistics
@ 2026-07-07 11:02 Bertrand Drouvot <[email protected]>
2026-07-08 06:00 ` Re: Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-07-08 06:52 ` Re: Add per-backend AIO statistics Michael Paquier <[email protected]>
0 siblings, 2 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-07-07 11:02 UTC (permalink / raw)
To: [email protected]
Hi hackers,
Currently to monitor AIO we can use:
1/ pg_aios that lists all AIO handles that are currently in use. That shows
what's happening right now, but not what has happened.
2/ pg_stat_get_backend_io() that shows how much IO was done, but not how it
was done. There's no way to see whether IOs ran synchronously or
asynchronously, whether a backend was stalling on handle exhaustion, or how
completions are distributed across backends.
This patch helps answering those questions by exposing cumulative per-backend
AIO counters:
- started: total AIO operations initiated
- executed_sync: IOs executed synchronously (fallback path)
- executed_async: IOs submitted asynchronously
- completed_self: IO completions processed by the issuing backend
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times waited for a free AIO handle
- submitted: number of submit calls to the IO method
These counters are useful for understanding and tuning AIO behavior:
- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).
- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.
- completed_self vs completed_other reveals cross-backend completion patterns.
That helps see how IO completion work is distributed and could help interpret
per backend IO statistics values.
- executed_async / submitted gives the average batch size per submit call.
As far as the technical implementation:
This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns one row based on the PID provided in input.
pgstat_flush_backend() gains a new flag value, able to control the flush of the
AIO stats.
This patch relies mostly on the infrastructure provided by 9aea73fc61d4, that
has introduced backend statistics.
The overhead (4 functions calls and counters increments) kind of follow the same
patterns as pgstat_count_backend_io_op() and I did not observe measurable
regression (I did not expect to). Also that does not add that much memory
per-backend: PgStat_AioCounters is 56 bytes.
There is no "double" counting as a global view to show those counters does not
exist. I think that's better to start with the per-backend side of it and see
if we want to also add a global view. For example, completed_other identifies
which backends did IOs for other backends. Also this allows correlating with
pg_stat_activity and pg_stat_get_backend_io().
Examples based on Franck's blog post [1]:
1/ query the smalldocs table:
postgres=# select count(*),avg(length(data)) from smalldocs;
count | avg
---------+-----------------------
1024000 | 1024.0000000000000000
(1 row)
postgres=# SELECT * FROM pg_stat_get_backend_aio(pg_backend_pid());
started | executed_sync | executed_async | completed_self | completed_other | handle_waits | submitted | stats_reset
---------+---------------+----------------+----------------+-----------------+--------------+-----------+-------------------------------
3125 | 46 | 3079 | 46 | 0 | 0 | 3078 | 2026-07-07 09:28:27.412136+00
We can see that the sequential scan fully benefits from AIO.
2/ query the largedocs table:
postgres=# select count(*),avg(length(data)) from largedocs;
count | avg
-------+----------------------
1000 | 1048576.000000000000
(1 row)
postgres=# SELECT * FROM pg_stat_get_backend_aio(pg_backend_pid());
started | executed_sync | executed_async | completed_self | completed_other | handle_waits | submitted | stats_reset
---------+---------------+----------------+----------------+-----------------+--------------+-----------+-------------------------------
121154 | 121150 | 4 | 121150 | 0 | 0 | 4 | 2026-07-07 09:35:00.504872+00
We can see that the sequential scan bypasses AIO.
Looking forward to your feedback.
[1]: https://dev.to/franckpachot/iouring-buffered-reads-in-postgresql-19-iouring-mcn
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: Add per-backend AIO statistics
2026-07-07 11:02 Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
@ 2026-07-08 06:00 ` Bertrand Drouvot <[email protected]>
1 sibling, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-07-08 06:00 UTC (permalink / raw)
To: [email protected]
Hi,
On Tue, Jul 07, 2026 at 11:02:03AM +0000, Bertrand Drouvot wrote:
> postgres=# select count(*),avg(length(data)) from smalldocs;
> count | avg
> ---------+-----------------------
> 1024000 | 1024.0000000000000000
> (1 row)
>
> postgres=# SELECT * FROM pg_stat_get_backend_aio(pg_backend_pid());
> started | executed_sync | executed_async | completed_self | completed_other | handle_waits | submitted | stats_reset
> ---------+---------------+----------------+----------------+-----------------+--------------+-----------+-------------------------------
> 3125 | 46 | 3079 | 46 | 0 | 0 | 3078 | 2026-07-07 09:28:27.412136+00
>
> We can see that the sequential scan fully benefits from AIO.
>
> 2/ query the largedocs table:
>
> postgres=# select count(*),avg(length(data)) from largedocs;
> count | avg
> -------+----------------------
> 1000 | 1048576.000000000000
> (1 row)
>
> postgres=# SELECT * FROM pg_stat_get_backend_aio(pg_backend_pid());
> started | executed_sync | executed_async | completed_self | completed_other | handle_waits | submitted | stats_reset
> ---------+---------------+----------------+----------------+-----------------+--------------+-----------+-------------------------------
> 121154 | 121150 | 4 | 121150 | 0 | 0 | 4 | 2026-07-07 09:35:00.504872+00
>
> We can see that the sequential scan bypasses AIO.
I was just doing some AIO experiments and was using the new pg_stat_get_backend_aio()
function.
So, while at it, sharing more examples here:
3/ pg_stat_get_backend_aio() and pg_stat_get_backend_io() correlation
postgres=# SELECT executed_sync, executed_async FROM pg_stat_get_backend_aio(pg_backend_pid());
executed_sync | executed_async
---------------+----------------
46 | 3088
(1 row)
postgres=# SELECT object, context, reads, read_bytes FROM pg_stat_get_backend_io(pg_backend_pid());
object | context | reads | read_bytes
---------------+-----------+-------+------------
relation | bulkread | 3088 | 401580032
relation | bulkwrite | 0 | 0
relation | init | 0 | 0
relation | normal | 46 | 376832
relation | vacuum | 0 | 0
temp relation | normal | 0 | 0
wal | init | |
wal | normal | 0 | 0
(8 rows)
We can see that the "executed_sync" matches the reads "normal" context and that
the "executed_async" matches the reads "bulkread" context.
4/ io_uring and multiple backends
postgres=# SELECT a.pid,
(pg_stat_get_backend_aio(a.pid)).completed_other
FROM pg_stat_activity a
WHERE a.backend_type = 'client backend';
pid | completed_other
---------+-----------------
1911889 | 245
1911892 | 511
1911912 | 147
1911933 | 161
(4 rows)
We can see that the backends completed AIO on behalf of other backends, which
makes fully sense in io_uring mode.
5/ io_max_concurrency = 4
postgres=# SELECT started, handle_waits FROM pg_stat_get_backend_aio(pg_backend_pid());
started | handle_waits
---------+--------------
3139 | 3026
(1 row)
We can see that the backend had to wait for free AIO handles on 96% of its IOs.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: Add per-backend AIO statistics
2026-07-07 11:02 Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
@ 2026-07-08 06:52 ` Michael Paquier <[email protected]>
2026-07-08 08:15 ` Re: Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-07-08 18:08 ` Re: Add per-backend AIO statistics Andres Freund <[email protected]>
1 sibling, 2 replies; 142+ messages in thread
From: Michael Paquier @ 2026-07-08 06:52 UTC (permalink / raw)
To: Bertrand Drouvot <[email protected]>; +Cc: [email protected]; Andres Freund <[email protected]>
On Tue, Jul 07, 2026 at 11:02:03AM +0000, Bertrand Drouvot wrote:
> 1/ pg_aios that lists all AIO handles that are currently in use. That shows
> what's happening right now, but not what has happened.
>
> 2/ pg_stat_get_backend_io() that shows how much IO was done, but not how it
> was done. There's no way to see whether IOs ran synchronously or
> asynchronously, whether a backend was stalling on handle exhaustion, or how
> completions are distributed across backends.
While the information may be useful, one thing that sounds very
important to me is how this impacts workloads by default.
Andres is usually able to catch bottlenecks that everybody else is
unable to see, so perhaps checking with him the location of these
extra function calls would be a good first step. Your proposal goes
down to pgaio_io_stage(), pgaio_io_process_completion() and
pgaio_submit_staged() to track these counter increments.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: Add per-backend AIO statistics
2026-07-07 11:02 Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-07-08 06:52 ` Re: Add per-backend AIO statistics Michael Paquier <[email protected]>
@ 2026-07-08 08:15 ` Bertrand Drouvot <[email protected]>
1 sibling, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-07-08 08:15 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: [email protected]; Andres Freund <[email protected]>
Hi,
On Wed, Jul 08, 2026 at 03:52:20PM +0900, Michael Paquier wrote:
> On Tue, Jul 07, 2026 at 11:02:03AM +0000, Bertrand Drouvot wrote:
> > 1/ pg_aios that lists all AIO handles that are currently in use. That shows
> > what's happening right now, but not what has happened.
> >
> > 2/ pg_stat_get_backend_io() that shows how much IO was done, but not how it
> > was done. There's no way to see whether IOs ran synchronously or
> > asynchronously, whether a backend was stalling on handle exhaustion, or how
> > completions are distributed across backends.
>
> While the information may be useful,
Thanks for looking at it!
> Andres is usually able to catch bottlenecks that everybody else is
> unable to see, so perhaps checking with him the location of these
> extra function calls would be a good first step. Your proposal goes
> down to pgaio_io_stage(), pgaio_io_process_completion() and
> pgaio_submit_staged() to track these counter increments.
yeah, and also to 1/ confirm that I did understand this area of the AIO code
correctly and 2/ see if other counters could make sense.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: Add per-backend AIO statistics
2026-07-07 11:02 Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-07-08 06:52 ` Re: Add per-backend AIO statistics Michael Paquier <[email protected]>
@ 2026-07-08 18:08 ` Andres Freund <[email protected]>
2026-07-09 04:19 ` Re: Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
1 sibling, 1 reply; 142+ messages in thread
From: Andres Freund @ 2026-07-08 18:08 UTC (permalink / raw)
To: Michael Paquier <[email protected]>; +Cc: Bertrand Drouvot <[email protected]>; [email protected]
Hi,
On 2026-07-08 15:52:20 +0900, Michael Paquier wrote:
> On Tue, Jul 07, 2026 at 11:02:03AM +0000, Bertrand Drouvot wrote:
> > 1/ pg_aios that lists all AIO handles that are currently in use. That shows
> > what's happening right now, but not what has happened.
> >
> > 2/ pg_stat_get_backend_io() that shows how much IO was done, but not how it
> > was done. There's no way to see whether IOs ran synchronously or
> > asynchronously, whether a backend was stalling on handle exhaustion, or how
> > completions are distributed across backends.
>
> While the information may be useful, one thing that sounds very
> important to me is how this impacts workloads by default.
> Andres is usually able to catch bottlenecks that everybody else is
> unable to see, so perhaps checking with him the location of these
> extra function calls would be a good first step. Your proposal goes
> down to pgaio_io_stage(), pgaio_io_process_completion() and
> pgaio_submit_staged() to track these counter increments.
I think the overhead might be ok, but I am rather doubtful that all of this
information is actually useful. You're adding quite a few counters for each
IO, do we actually need that?
E.g. what do we gain from counting:
- started (if you want to see the number of IOs that are in progress,
cumulative stats are the wrong tool)
- executed_async (that's just the number of IOs minus executed_sync)
- completed_self (that's just the number of IOs minus executed_other)
Separately, I'm doubtful it makes sense to have only per-backend stats for
this. I think you'd almost always want the stats for exited backend
(e.g. parallel workers) too.
Unfortunately I'm pretty doubtful that pgstat_backend.c is the right
architectural direction. It'll just end up implementing all kinds of stats,
since we'll incrementally want more and more per-backend stats. I think what
we'd want is rather something where for each applicable stats kind we have a
shared counter for all exited backends and then per-backend counters for live
backends, with helpers to aggregate the exited + live stats to a total.
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: Add per-backend AIO statistics
2026-07-07 11:02 Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-07-08 06:52 ` Re: Add per-backend AIO statistics Michael Paquier <[email protected]>
2026-07-08 18:08 ` Re: Add per-backend AIO statistics Andres Freund <[email protected]>
@ 2026-07-09 04:19 ` Bertrand Drouvot <[email protected]>
2026-07-10 04:56 ` Re: Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
0 siblings, 1 reply; 142+ messages in thread
From: Bertrand Drouvot @ 2026-07-09 04:19 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Michael Paquier <[email protected]>; [email protected]
Hi,
On Wed, Jul 08, 2026 at 02:08:00PM -0400, Andres Freund wrote:
> Hi,
>
> On 2026-07-08 15:52:20 +0900, Michael Paquier wrote:
> > On Tue, Jul 07, 2026 at 11:02:03AM +0000, Bertrand Drouvot wrote:
> > > 1/ pg_aios that lists all AIO handles that are currently in use. That shows
> > > what's happening right now, but not what has happened.
> > >
> > > 2/ pg_stat_get_backend_io() that shows how much IO was done, but not how it
> > > was done. There's no way to see whether IOs ran synchronously or
> > > asynchronously, whether a backend was stalling on handle exhaustion, or how
> > > completions are distributed across backends.
> >
> > While the information may be useful, one thing that sounds very
> > important to me is how this impacts workloads by default.
>
>
> > Andres is usually able to catch bottlenecks that everybody else is
> > unable to see, so perhaps checking with him the location of these
> > extra function calls would be a good first step. Your proposal goes
> > down to pgaio_io_stage(), pgaio_io_process_completion() and
> > pgaio_submit_staged() to track these counter increments.
>
> I think the overhead might be ok,
Thanks for the feedback.
> but I am rather doubtful that all of this
> information is actually useful. You're adding quite a few counters for each
> IO, do we actually need that?
>
> E.g. what do we gain from counting:
> - started (if you want to see the number of IOs that are in progress,
> cumulative stats are the wrong tool)
> - executed_async (that's just the number of IOs minus executed_sync)
> - completed_self (that's just the number of IOs minus executed_other)
Yeah, we can remove some fields (as they're derivable).
> Separately, I'm doubtful it makes sense to have only per-backend stats for
> this. I think you'd almost always want the stats for exited backend
> (e.g. parallel workers) too.
Indeed, adding a global view would capture their activity.
> Unfortunately I'm pretty doubtful that pgstat_backend.c is the right
> architectural direction. It'll just end up implementing all kinds of stats,
> since we'll incrementally want more and more per-backend stats. I think what
> we'd want is rather something where for each applicable stats kind we have a
> shared counter for all exited backends and then per-backend counters for live
> backends, with helpers to aggregate the exited + live stats to a total.
That's a very nice proposal that would avoid the double counting. OTOH, that's
also a major re-design that would benefit all existing per-backend stats kinds.
I can see 2 options:
1/
step 1: Implement per-backend AIO stats (like proposed taking into account your
remark about useless, derivable fields) + a global view.
step 2: work on the re-design
2/
step 1: work on the redesign
step 2: Add AIO stats based on the re-design
The pros of 1/ is that step 1 would most probably land in 20, providing more user
visibility (+ it could be used or improved during the AIO write project). Step 2
is a much larger project that might not land in 20.
The cons, would be double counting (as there is no need to try to implement
something like [1] as we are going to re-design anyway).
I'll be tempted to vote for 1/ to provide faster added value. What do you (Andres,
Michael) think?
[1]: https://postgr.es/m/[email protected]
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 142+ messages in thread
* Re: Add per-backend AIO statistics
2026-07-07 11:02 Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-07-08 06:52 ` Re: Add per-backend AIO statistics Michael Paquier <[email protected]>
2026-07-08 18:08 ` Re: Add per-backend AIO statistics Andres Freund <[email protected]>
2026-07-09 04:19 ` Re: Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
@ 2026-07-10 04:56 ` Bertrand Drouvot <[email protected]>
0 siblings, 0 replies; 142+ messages in thread
From: Bertrand Drouvot @ 2026-07-10 04:56 UTC (permalink / raw)
To: Andres Freund <[email protected]>; +Cc: Michael Paquier <[email protected]>; [email protected]
Hi,
On Thu, Jul 09, 2026 at 04:19:26AM +0000, Bertrand Drouvot wrote:
> Hi,
>
> On Wed, Jul 08, 2026 at 02:08:00PM -0400, Andres Freund wrote:
>
> > Unfortunately I'm pretty doubtful that pgstat_backend.c is the right
> > architectural direction. It'll just end up implementing all kinds of stats,
> > since we'll incrementally want more and more per-backend stats. I think what
> > we'd want is rather something where for each applicable stats kind we have a
> > shared counter for all exited backends and then per-backend counters for live
> > backends, with helpers to aggregate the exited + live stats to a total.
>
> That's a very nice proposal that would avoid the double counting. OTOH, that's
> also a major re-design that would benefit all existing per-backend stats kinds.
>
> I can see 2 options:
>
> 1/
>
> step 1: Implement per-backend AIO stats (like proposed taking into account your
> remark about useless, derivable fields) + a global view.
> step 2: work on the re-design
>
> 2/
>
> step 1: work on the redesign
> step 2: Add AIO stats based on the re-design
>
> The pros of 1/ is that step 1 would most probably land in 20, providing more user
> visibility (+ it could be used or improved during the AIO write project). Step 2
> is a much larger project that might not land in 20.
>
> The cons, would be double counting (as there is no need to try to implement
> something like [1] as we are going to re-design anyway).
>
> I'll be tempted to vote for 1/ to provide faster added value. What do you (Andres,
> Michael) think?
Actually, there is no rush to merge the per-backend AIO stats (we still have
plenty of time for 20). So let's try option 2 and implement the new design first
and see where it goes. I'll create a dedicated thread once ready.
Regards,
--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com
^ permalink raw reply [nested|flat] 142+ messages in thread
end of thread, other threads:[~2026-07-10 04:56 UTC | newest]
Thread overview: 142+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-03-29 15:43 [PATCH v19] Avoid orphaned objects dependencies Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-06-11 09:45 [PATCH v1] Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-07-07 11:02 Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-07-08 06:00 ` Re: Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-07-08 06:52 ` Re: Add per-backend AIO statistics Michael Paquier <[email protected]>
2026-07-08 08:15 ` Re: Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-07-08 18:08 ` Re: Add per-backend AIO statistics Andres Freund <[email protected]>
2026-07-09 04:19 ` Re: Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
2026-07-10 04:56 ` Re: Add per-backend AIO statistics Bertrand Drouvot <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox