agora inbox for [email protected]  

help / color / mirror / Atom feed

[PATCH v5] Avoid orphaned objects dependencies
323+ messages / 4 participants
[nested] [flat]

* [PATCH v5] Avoid orphaned objects dependencies
@ 2024-03-29 15:43  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2024-03-29 15:43 UTC (permalink / raw)

It's currently possible to create orphaned objects dependencies, for example:

Scenario 1:

session 1: begin; drop schema schem;
session 2: create a function in the schema schem
session 1: commit;

With the above, the function created in session 2 would be linked to a non
existing schema.

Scenario 2:

session 1: begin; create a function in the schema schem
session 2: drop schema schem;
session 1: commit;

With the above, the function created in session 1 would be linked to a non
existing schema.

To avoid those scenarios, a new lock (that conflicts with a lock taken by DROP)
has been put in place when the dependencies are being recorded. With this in
place, the drop schema in scenario 2 would be locked.

Also, after the new lock attempt, the patch checks that the object still exists:
with this in place session 2 in scenario 1 would be locked and would report an
error once session 1 committs (that would not be the case should session 1 abort
the transaction).

If the object is dropped before the new lock attempt is triggered then the patch
would also report an error (but with less details).

The patch adds a few tests for some dependency cases (that would currently produce
orphaned objects):

- schema and function (as the above scenarios)
- alter a dependency (function and schema)
- function and arg type
- function and return type
- function and function
- domain and domain
- table and type
- server and foreign data wrapper
---
 src/backend/catalog/dependency.c              |  54 ++++++++
 src/backend/catalog/objectaddress.c           |  70 ++++++++++
 src/backend/catalog/pg_depend.c               |  12 ++
 src/include/catalog/dependency.h              |   1 +
 src/include/catalog/objectaddress.h           |   1 +
 src/test/modules/Makefile                     |   1 +
 src/test/modules/meson.build                  |   1 +
 .../test_dependencies_locks/.gitignore        |   3 +
 .../modules/test_dependencies_locks/Makefile  |  14 ++
 .../expected/test_dependencies_locks.out      | 129 ++++++++++++++++++
 .../test_dependencies_locks/meson.build       |  12 ++
 .../specs/test_dependencies_locks.spec        |  89 ++++++++++++
 12 files changed, 387 insertions(+)
  22.9% src/backend/catalog/
  43.7% src/test/modules/test_dependencies_locks/expected/
  27.2% src/test/modules/test_dependencies_locks/specs/
   4.5% src/test/modules/test_dependencies_locks/

diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index d4b5b2ade1..a49357bbe2 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1517,6 +1517,60 @@ AcquireDeletionLock(const ObjectAddress *object, int flags)
 	}
 }
 
+/*
+ * depLockAndCheckObject
+ *
+ * Lock the object that we are about to record a dependency on.
+ * After it's locked, verify that it hasn't been dropped while we
+ * weren't looking.  If the object has been dropped, this function
+ * does not return!
+ */
+void
+depLockAndCheckObject(const ObjectAddress *object)
+{
+	char	   *object_description;
+
+	/*
+	 * Those don't rely on LockDatabaseObject() when being dropped (see
+	 * AcquireDeletionLock()). Also it looks like they can not produce
+	 * orphaned dependent objects when being dropped.
+	 */
+	if (object->classId == RelationRelationId || object->classId == AuthMemRelationId)
+		return;
+
+	object_description = getObjectDescription(object, true);
+
+	/* assume we should lock the whole object not a sub-object */
+	LockDatabaseObject(object->classId, object->objectId, 0, AccessShareLock);
+
+	/* check if object still exists */
+	if (!ObjectByIdExist(object, false))
+	{
+		/*
+		 * It might be possible that we are creating it (for example creating
+		 * a composite type while creating a relation), so bypass the syscache
+		 * lookup and use a dirty snaphot instead to cover this scenario.
+		 */
+		if (!ObjectByIdExist(object, true))
+		{
+			/*
+			 * If the object has been dropped before we get a chance to get
+			 * its description, then emit a generic error message. That looks
+			 * like a good compromise over extra complexity.
+			 */
+			if (object_description)
+				ereport(ERROR, errmsg("%s does not exist", object_description));
+			else
+				ereport(ERROR, errmsg("a dependent object does not exist"));
+		}
+	}
+
+	if (object_description)
+		pfree(object_description);
+
+	return;
+}
+
 /*
  * ReleaseDeletionLock - release an object deletion lock
  *
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 7b536ac6fd..c2b873dd81 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -2590,6 +2590,76 @@ get_object_namespace(const ObjectAddress *address)
 	return oid;
 }
 
+/*
+ * ObjectByIdExist
+ *
+ * Return whether the given object exists.
+ *
+ * Works for most catalogs, if no special processing is needed.
+ */
+bool
+ObjectByIdExist(const ObjectAddress *address, bool use_dirty_snapshot)
+{
+	HeapTuple	tuple;
+	int			cache = -1;
+	const ObjectPropertyType *property;
+
+	if (!use_dirty_snapshot)
+	{
+		property = get_object_property_data(address->classId);
+
+		cache = property->oid_catcache_id;
+	}
+
+	if (cache >= 0)
+	{
+		/* Fetch tuple from syscache. */
+		tuple = SearchSysCache1(cache, ObjectIdGetDatum(address->objectId));
+
+		if (!HeapTupleIsValid(tuple))
+		{
+			return false;
+		}
+
+		ReleaseSysCache(tuple);
+
+		return true;
+	}
+	else
+	{
+		Relation	rel;
+		ScanKeyData skey[1];
+		SysScanDesc scan;
+		SnapshotData DirtySnapshot;
+		Snapshot	snapshot;
+
+		if (use_dirty_snapshot)
+		{
+			InitDirtySnapshot(DirtySnapshot);
+			snapshot = &DirtySnapshot;
+		}
+		else
+			snapshot = NULL;
+
+		rel = table_open(address->classId, AccessShareLock);
+
+		ScanKeyInit(&skey[0],
+					get_object_attnum_oid(address->classId),
+					BTEqualStrategyNumber, F_OIDEQ,
+					ObjectIdGetDatum(address->objectId));
+
+		scan = systable_beginscan(rel, get_object_oid_index(address->classId), true,
+								  snapshot, 1, skey);
+
+		/* we expect exactly one match */
+		tuple = systable_getnext(scan);
+		systable_endscan(scan);
+		table_close(rel, AccessShareLock);
+
+		return (HeapTupleIsValid(tuple));
+	}
+}
+
 /*
  * Return ObjectType for the given object type as given by
  * getObjectTypeDescription; if no valid ObjectType code exists, but it's a
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 5366f7820c..f16af28429 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -108,6 +108,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
 		if (isObjectPinned(referenced))
 			continue;
 
+		/*
+		 * Acquire a lock and check object still exists while recording the
+		 * dependency. XXX - Should we do so only for DEPENDENCY_NORMAL?
+		 */
+		depLockAndCheckObject(referenced);
+
 		if (slot_init_count < max_slots)
 		{
 			slot[slot_stored_count] = MakeSingleTupleTableSlot(RelationGetDescr(dependDesc),
@@ -506,6 +512,12 @@ changeDependencyFor(Oid classId, Oid objectId,
 		return 1;
 	}
 
+	/*
+	 * Acquire a lock and check object still exists while changing the
+	 * dependency.
+	 */
+	depLockAndCheckObject(&objAddr);
+
 	depRel = table_open(DependRelationId, RowExclusiveLock);
 
 	/* There should be existing dependency record(s), so search. */
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 7eee66f810..5619b6f55e 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -101,6 +101,7 @@ typedef struct ObjectAddresses ObjectAddresses;
 /* in dependency.c */
 
 extern void AcquireDeletionLock(const ObjectAddress *object, int flags);
+extern void depLockAndCheckObject(const ObjectAddress *object);
 
 extern void ReleaseDeletionLock(const ObjectAddress *object);
 
diff --git a/src/include/catalog/objectaddress.h b/src/include/catalog/objectaddress.h
index 3a70d80e32..04891abcc1 100644
--- a/src/include/catalog/objectaddress.h
+++ b/src/include/catalog/objectaddress.h
@@ -53,6 +53,7 @@ extern void check_object_ownership(Oid roleid,
 								   Node *object, Relation relation);
 
 extern Oid	get_object_namespace(const ObjectAddress *address);
+extern bool ObjectByIdExist(const ObjectAddress *address, bool use_dirty_snapshot);
 
 extern bool is_objectclass_supported(Oid class_id);
 extern const char *get_object_class_descr(Oid class_id);
diff --git a/src/test/modules/Makefile b/src/test/modules/Makefile
index 256799f520..75f357100f 100644
--- a/src/test/modules/Makefile
+++ b/src/test/modules/Makefile
@@ -17,6 +17,7 @@ SUBDIRS = \
 		  test_copy_callbacks \
 		  test_custom_rmgrs \
 		  test_ddl_deparse \
+		  test_dependencies_locks \
 		  test_dsa \
 		  test_dsm_registry \
 		  test_extensions \
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index d8fe059d23..60305dcccd 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -16,6 +16,7 @@ subdir('test_bloomfilter')
 subdir('test_copy_callbacks')
 subdir('test_custom_rmgrs')
 subdir('test_ddl_deparse')
+subdir('test_dependencies_locks')
 subdir('test_dsa')
 subdir('test_dsm_registry')
 subdir('test_extensions')
diff --git a/src/test/modules/test_dependencies_locks/.gitignore b/src/test/modules/test_dependencies_locks/.gitignore
new file mode 100644
index 0000000000..bf000faac4
--- /dev/null
+++ b/src/test/modules/test_dependencies_locks/.gitignore
@@ -0,0 +1,3 @@
+# Generated subdirectories
+/log/
+/output_iso
diff --git a/src/test/modules/test_dependencies_locks/Makefile b/src/test/modules/test_dependencies_locks/Makefile
new file mode 100644
index 0000000000..7491048380
--- /dev/null
+++ b/src/test/modules/test_dependencies_locks/Makefile
@@ -0,0 +1,14 @@
+# src/test/modules/test_dependencies_locks/Makefile
+
+ISOLATION = test_dependencies_locks
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/test_dependencies_locks
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/test_dependencies_locks/expected/test_dependencies_locks.out b/src/test/modules/test_dependencies_locks/expected/test_dependencies_locks.out
new file mode 100644
index 0000000000..9b645d7aa5
--- /dev/null
+++ b/src/test/modules/test_dependencies_locks/expected/test_dependencies_locks.out
@@ -0,0 +1,129 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s1_begin s1_create_function_in_schema s2_drop_schema s1_commit
+step s1_begin: BEGIN;
+step s1_create_function_in_schema: CREATE FUNCTION testschema.foo() RETURNS int AS 'select 1' LANGUAGE sql;
+step s2_drop_schema: DROP SCHEMA testschema; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_schema: <... completed>
+ERROR:  cannot drop schema testschema because other objects depend on it
+
+starting permutation: s2_begin s2_drop_schema s1_create_function_in_schema s2_commit
+step s2_begin: BEGIN;
+step s2_drop_schema: DROP SCHEMA testschema;
+step s1_create_function_in_schema: CREATE FUNCTION testschema.foo() RETURNS int AS 'select 1' LANGUAGE sql; <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_function_in_schema: <... completed>
+ERROR:  schema testschema does not exist
+
+starting permutation: s1_begin s1_alter_function_schema s2_drop_alterschema s1_commit
+step s1_begin: BEGIN;
+step s1_alter_function_schema: ALTER FUNCTION public.falter() SET SCHEMA alterschema;
+step s2_drop_alterschema: DROP SCHEMA alterschema; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_alterschema: <... completed>
+ERROR:  cannot drop schema alterschema because other objects depend on it
+
+starting permutation: s2_begin s2_drop_alterschema s1_alter_function_schema s2_commit
+step s2_begin: BEGIN;
+step s2_drop_alterschema: DROP SCHEMA alterschema;
+step s1_alter_function_schema: ALTER FUNCTION public.falter() SET SCHEMA alterschema; <waiting ...>
+step s2_commit: COMMIT;
+step s1_alter_function_schema: <... completed>
+ERROR:  schema alterschema does not exist
+
+starting permutation: s1_begin s1_create_function_with_argtype s2_drop_foo_type s1_commit
+step s1_begin: BEGIN;
+step s1_create_function_with_argtype: CREATE FUNCTION fooargtype(num foo) RETURNS int AS 'select 1' LANGUAGE sql;
+step s2_drop_foo_type: DROP TYPE public.foo; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_foo_type: <... completed>
+ERROR:  cannot drop type foo because other objects depend on it
+
+starting permutation: s2_begin s2_drop_foo_type s1_create_function_with_argtype s2_commit
+step s2_begin: BEGIN;
+step s2_drop_foo_type: DROP TYPE public.foo;
+step s1_create_function_with_argtype: CREATE FUNCTION fooargtype(num foo) RETURNS int AS 'select 1' LANGUAGE sql; <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_function_with_argtype: <... completed>
+ERROR:  type foo does not exist
+
+starting permutation: s1_begin s1_create_function_with_rettype s2_drop_foo_rettype s1_commit
+step s1_begin: BEGIN;
+step s1_create_function_with_rettype: CREATE FUNCTION footrettype() RETURNS id LANGUAGE sql RETURN 1;
+step s2_drop_foo_rettype: DROP DOMAIN id; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_foo_rettype: <... completed>
+ERROR:  cannot drop type id because other objects depend on it
+
+starting permutation: s2_begin s2_drop_foo_rettype s1_create_function_with_rettype s2_commit
+step s2_begin: BEGIN;
+step s2_drop_foo_rettype: DROP DOMAIN id;
+step s1_create_function_with_rettype: CREATE FUNCTION footrettype() RETURNS id LANGUAGE sql RETURN 1; <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_function_with_rettype: <... completed>
+ERROR:  type id does not exist
+
+starting permutation: s1_begin s1_create_function_with_function s2_drop_function_f s1_commit
+step s1_begin: BEGIN;
+step s1_create_function_with_function: CREATE FUNCTION foofunc() RETURNS int LANGUAGE SQL RETURN f() + 1;
+step s2_drop_function_f: DROP FUNCTION f(); <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_function_f: <... completed>
+ERROR:  cannot drop function f() because other objects depend on it
+
+starting permutation: s2_begin s2_drop_function_f s1_create_function_with_function s2_commit
+step s2_begin: BEGIN;
+step s2_drop_function_f: DROP FUNCTION f();
+step s1_create_function_with_function: CREATE FUNCTION foofunc() RETURNS int LANGUAGE SQL RETURN f() + 1; <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_function_with_function: <... completed>
+ERROR:  function f() does not exist
+
+starting permutation: s1_begin s1_create_domain_with_domain s2_drop_domain_id s1_commit
+step s1_begin: BEGIN;
+step s1_create_domain_with_domain: CREATE DOMAIN idid as id;
+step s2_drop_domain_id: DROP DOMAIN id; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_domain_id: <... completed>
+ERROR:  cannot drop type id because other objects depend on it
+
+starting permutation: s2_begin s2_drop_domain_id s1_create_domain_with_domain s2_commit
+step s2_begin: BEGIN;
+step s2_drop_domain_id: DROP DOMAIN id;
+step s1_create_domain_with_domain: CREATE DOMAIN idid as id; <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_domain_with_domain: <... completed>
+ERROR:  type id does not exist
+
+starting permutation: s1_begin s1_create_table_with_type s2_drop_footab_type s1_commit
+step s1_begin: BEGIN;
+step s1_create_table_with_type: CREATE TABLE tabtype(a footab);
+step s2_drop_footab_type: DROP TYPE public.footab; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_footab_type: <... completed>
+ERROR:  cannot drop type footab because other objects depend on it
+
+starting permutation: s2_begin s2_drop_footab_type s1_create_table_with_type s2_commit
+step s2_begin: BEGIN;
+step s2_drop_footab_type: DROP TYPE public.footab;
+step s1_create_table_with_type: CREATE TABLE tabtype(a footab); <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_table_with_type: <... completed>
+ERROR:  type footab does not exist
+
+starting permutation: s1_begin s1_create_server_with_fdw_wrapper s2_drop_fdw_wrapper s1_commit
+step s1_begin: BEGIN;
+step s1_create_server_with_fdw_wrapper: CREATE SERVER srv_fdw_wrapper FOREIGN DATA WRAPPER fdw_wrapper;
+step s2_drop_fdw_wrapper: DROP FOREIGN DATA WRAPPER fdw_wrapper RESTRICT; <waiting ...>
+step s1_commit: COMMIT;
+step s2_drop_fdw_wrapper: <... completed>
+ERROR:  cannot drop foreign-data wrapper fdw_wrapper because other objects depend on it
+
+starting permutation: s2_begin s2_drop_fdw_wrapper s1_create_server_with_fdw_wrapper s2_commit
+step s2_begin: BEGIN;
+step s2_drop_fdw_wrapper: DROP FOREIGN DATA WRAPPER fdw_wrapper RESTRICT;
+step s1_create_server_with_fdw_wrapper: CREATE SERVER srv_fdw_wrapper FOREIGN DATA WRAPPER fdw_wrapper; <waiting ...>
+step s2_commit: COMMIT;
+step s1_create_server_with_fdw_wrapper: <... completed>
+ERROR:  foreign-data wrapper fdw_wrapper does not exist
diff --git a/src/test/modules/test_dependencies_locks/meson.build b/src/test/modules/test_dependencies_locks/meson.build
new file mode 100644
index 0000000000..92a978ab93
--- /dev/null
+++ b/src/test/modules/test_dependencies_locks/meson.build
@@ -0,0 +1,12 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+tests += {
+  'name': 'test_dependencies_locks',
+  'sd': meson.current_source_dir(),
+  'bd': meson.current_build_dir(),
+  'isolation': {
+    'specs': [
+      'test_dependencies_locks',
+    ],
+  },
+}
diff --git a/src/test/modules/test_dependencies_locks/specs/test_dependencies_locks.spec b/src/test/modules/test_dependencies_locks/specs/test_dependencies_locks.spec
new file mode 100644
index 0000000000..5d04dfe9dc
--- /dev/null
+++ b/src/test/modules/test_dependencies_locks/specs/test_dependencies_locks.spec
@@ -0,0 +1,89 @@
+setup
+{
+  CREATE SCHEMA testschema;
+  CREATE SCHEMA alterschema;
+  CREATE TYPE public.foo as enum ('one', 'two');
+  CREATE TYPE public.footab as enum ('three', 'four');
+  CREATE DOMAIN id AS int;
+  CREATE FUNCTION f() RETURNS int LANGUAGE SQL RETURN 1;
+  CREATE FUNCTION public.falter() RETURNS int LANGUAGE SQL RETURN 1;
+  CREATE FOREIGN DATA WRAPPER fdw_wrapper;
+}
+
+teardown
+{
+  DROP FUNCTION IF EXISTS testschema.foo();
+  DROP FUNCTION IF EXISTS fooargtype(num foo);
+  DROP FUNCTION IF EXISTS footrettype();
+  DROP FUNCTION IF EXISTS foofunc();
+  DROP FUNCTION IF EXISTS public.falter();
+  DROP FUNCTION IF EXISTS alterschema.falter();
+  DROP DOMAIN IF EXISTS idid;
+  DROP SERVER IF EXISTS srv_fdw_wrapper;
+  DROP TABLE IF EXISTS tabtype;
+  DROP SCHEMA IF EXISTS testschema;
+  DROP SCHEMA IF EXISTS alterschema;
+  DROP TYPE IF EXISTS public.foo;
+  DROP TYPE IF EXISTS public.footab;
+  DROP DOMAIN IF EXISTS id;
+  DROP FUNCTION IF EXISTS f();
+  DROP FOREIGN DATA WRAPPER IF EXISTS fdw_wrapper;
+}
+
+session "s1"
+
+step "s1_begin" { BEGIN; }
+step "s1_create_function_in_schema" { CREATE FUNCTION testschema.foo() RETURNS int AS 'select 1' LANGUAGE sql; }
+step "s1_create_function_with_argtype" { CREATE FUNCTION fooargtype(num foo) RETURNS int AS 'select 1' LANGUAGE sql; }
+step "s1_create_function_with_rettype" { CREATE FUNCTION footrettype() RETURNS id LANGUAGE sql RETURN 1; }
+step "s1_create_function_with_function" { CREATE FUNCTION foofunc() RETURNS int LANGUAGE SQL RETURN f() + 1; }
+step "s1_alter_function_schema" { ALTER FUNCTION public.falter() SET SCHEMA alterschema; }
+step "s1_create_domain_with_domain" { CREATE DOMAIN idid as id; }
+step "s1_create_table_with_type" { CREATE TABLE tabtype(a footab); }
+step "s1_create_server_with_fdw_wrapper" { CREATE SERVER srv_fdw_wrapper FOREIGN DATA WRAPPER fdw_wrapper; }
+step "s1_commit" { COMMIT; }
+
+session "s2"
+
+step "s2_begin" { BEGIN; }
+step "s2_drop_schema" { DROP SCHEMA testschema; }
+step "s2_drop_alterschema" { DROP SCHEMA alterschema; }
+step "s2_drop_foo_type" { DROP TYPE public.foo; }
+step "s2_drop_foo_rettype" { DROP DOMAIN id; }
+step "s2_drop_footab_type" { DROP TYPE public.footab; }
+step "s2_drop_function_f" { DROP FUNCTION f(); }
+step "s2_drop_domain_id" { DROP DOMAIN id; }
+step "s2_drop_fdw_wrapper" { DROP FOREIGN DATA WRAPPER fdw_wrapper RESTRICT; }
+step "s2_commit" { COMMIT; }
+
+# function - schema
+permutation "s1_begin" "s1_create_function_in_schema" "s2_drop_schema" "s1_commit"
+permutation "s2_begin" "s2_drop_schema" "s1_create_function_in_schema" "s2_commit"
+
+# alter function - schema
+permutation "s1_begin" "s1_alter_function_schema" "s2_drop_alterschema" "s1_commit"
+permutation "s2_begin" "s2_drop_alterschema" "s1_alter_function_schema" "s2_commit"
+
+# function - argtype
+permutation "s1_begin" "s1_create_function_with_argtype" "s2_drop_foo_type" "s1_commit"
+permutation "s2_begin" "s2_drop_foo_type" "s1_create_function_with_argtype" "s2_commit"
+
+# function - rettype
+permutation "s1_begin" "s1_create_function_with_rettype" "s2_drop_foo_rettype" "s1_commit"
+permutation "s2_begin" "s2_drop_foo_rettype" "s1_create_function_with_rettype" "s2_commit"
+
+# function - function
+permutation "s1_begin" "s1_create_function_with_function" "s2_drop_function_f" "s1_commit"
+permutation "s2_begin" "s2_drop_function_f" "s1_create_function_with_function" "s2_commit"
+
+# domain - domain
+permutation "s1_begin" "s1_create_domain_with_domain" "s2_drop_domain_id" "s1_commit"
+permutation "s2_begin" "s2_drop_domain_id" "s1_create_domain_with_domain" "s2_commit"
+
+# table - type
+permutation "s1_begin" "s1_create_table_with_type" "s2_drop_footab_type" "s1_commit"
+permutation "s2_begin" "s2_drop_footab_type" "s1_create_table_with_type" "s2_commit"
+
+# server - foreign data wrapper
+permutation "s1_begin" "s1_create_server_with_fdw_wrapper" "s2_drop_fdw_wrapper" "s1_commit"
+permutation "s2_begin" "s2_drop_fdw_wrapper" "s1_create_server_with_fdw_wrapper" "s2_commit"
-- 
2.34.1


--fbv/uQrEQtVlJXri--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread

* [PATCH v1] Add per-backend AIO statistics
@ 2026-06-11 09:45  Bertrand Drouvot <[email protected]>
  0 siblings, 0 replies; 323+ messages in thread

From: Bertrand Drouvot @ 2026-06-11 09:45 UTC (permalink / raw)

This commit adds per-backend AIO statistics, providing per-backend AIO behavior
details.

This data can be retrieved with a new system function called
pg_stat_get_backend_aio(), that returns the following counters based on the PID
provided in input:

- started: total number of AIO operations initiated
- executed_sync: IOs that were executed synchronously (fallback path)
- executed_async: IOs that were submitted asynchronously to the IO method
- completed_self: IO completions processed by the issuing backend itself
- completed_other: IO completions processed on behalf of another backend
- handle_waits: times the backend had to wait for a free AIO handle
- submitted: number of submitted calls to the IO method

These counters are useful for understanding and tuning AIO behavior:

- executed_async / started. A ratio near zero means the backend is falling back
to synchronous execution (TOAST chunk fetches, temp buffers, ...).

- a non-zero handle_waits means the backend exhausted all its AIO handles. That
could mean that io_max_concurrency is too low.

- completed_self vs completed_other reveals cross-backend completion
patterns. That helps see how IO completion work is distributed and could
help interpret per backend IO statistics values.

- executed_async / submitted gives the average batch size per submit call.

This commit is straight-forward, relying on the infrastructure provided
by 9aea73fc61d4 (backend-level pgstats).

XXX: Bump catalog version. No need to touch PGSTAT_FILE_FORMAT_ID as backend
statistics are never written to disk.

Author: Bertrand Drouvot <[email protected]>
Reviewed-by:
Discussion:
---
 doc/src/sgml/monitoring.sgml                |  70 +++++++++++++
 src/backend/storage/aio/aio.c               |  13 +++
 src/backend/utils/activity/pgstat_backend.c | 109 ++++++++++++++++++++
 src/backend/utils/adt/pgstatfuncs.c         |  60 +++++++++++
 src/include/catalog/pg_proc.dat             |   7 ++
 src/include/pgstat.h                        |  25 +++++
 src/include/utils/pgstat_internal.h         |   3 +-
 src/test/modules/test_aio/t/001_aio.pl      |  32 ++++++
 src/tools/pgindent/typedefs.list            |   1 +
 9 files changed, 319 insertions(+), 1 deletion(-)
  24.4% doc/src/sgml/
   3.2% src/backend/storage/aio/
  25.3% src/backend/utils/activity/
  20.5% src/backend/utils/adt/
   4.9% src/include/catalog/
  10.0% src/include/
  11.1% src/test/modules/test_aio/t/

diff --git a/doc/src/sgml/monitoring.sgml b/doc/src/sgml/monitoring.sgml
index 12b9ee20d4a..1c566cf0e0e 100644
--- a/doc/src/sgml/monitoring.sgml
+++ b/doc/src/sgml/monitoring.sgml
@@ -5641,6 +5641,76 @@ description | Waiting for a newly initialized WAL file to reach durable storage
        </para></entry>
       </row>
 
+      <row>
+       <entry id="pg-stat-get-backend-aio" role="func_table_entry"><para role="func_signature">
+        <indexterm>
+         <primary>pg_stat_get_backend_aio</primary>
+        </indexterm>
+        <function>pg_stat_get_backend_aio</function> ( <type>integer</type> )
+        <returnvalue>record</returnvalue>
+       </para>
+       <para>
+        Returns <acronym>AIO</acronym> (Asynchronous I/O) statistics about the
+        backend with the specified process ID. The returned values are:
+       <itemizedlist>
+        <listitem>
+         <para>
+          <literal>started</literal>: Total IOs initiated.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+           <literal>executed_sync</literal>: IOs executed synchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>executed_async</literal>: IOs submitted asynchronously.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_self</literal>: IO completions processed by this
+          backend.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>completed_other</literal>: IO completions processed on
+          behalf of other backends.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>handle_waits</literal>: Times waited for a free AIO handle.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>submitted</literal>: Number of submit calls to the IO
+          method. Compare with <literal>executed_async</literal> to determine
+          average batch size.
+         </para>
+        </listitem>
+        <listitem>
+         <para>
+          <literal>stats_reset</literal>: Timestamp of last stats reset.
+         </para>
+        </listitem>
+       </itemizedlist>
+       </para>
+       <para>
+        The <literal>completed_other</literal> column is only meaningful
+        when <varname>io_method</varname> is set to <literal>io_uring</literal>;
+        with <literal>worker</literal> mode, IO completions are processed by
+        IO worker processes which do not track these statistics.
+       </para>
+       <para>
+        The function does not return AIO statistics for the checkpointer,
+        the background writer, the startup process and the autovacuum launcher.
+       </para></entry>
+      </row>
+
       <row>
        <entry role="func_table_entry"><para role="func_signature">
         <indexterm>
diff --git a/src/backend/storage/aio/aio.c b/src/backend/storage/aio/aio.c
index 8f7e26607b9..507f1727d41 100644
--- a/src/backend/storage/aio/aio.c
+++ b/src/backend/storage/aio/aio.c
@@ -40,6 +40,7 @@
 
 #include "lib/ilist.h"
 #include "miscadmin.h"
+#include "pgstat.h"
 #include "port/atomics.h"
 #include "storage/aio.h"
 #include "storage/aio_internal.h"
@@ -458,6 +459,8 @@ pgaio_io_stage(PgAioHandle *ioh, PgAioOp op)
 				   "staged (synchronous: %d, in_batch: %d)",
 				   needs_synchronous, pgaio_my_backend->in_batchmode);
 
+	pgstat_count_backend_aio_start(needs_synchronous);
+
 	if (!needs_synchronous)
 	{
 		pgaio_my_backend->staged_ios[pgaio_my_backend->num_staged_ios++] = ioh;
@@ -544,6 +547,12 @@ pgaio_io_process_completion(PgAioHandle *ioh, int result)
 	/* condition variable broadcast ensures state is visible before wakeup */
 	ConditionVariableBroadcast(&ioh->cv);
 
+	/* Track AIO completion stats */
+	if (ioh->owner_procno == MyProcNumber)
+		pgstat_count_backend_aio_complete_self();
+	else
+		pgstat_count_backend_aio_complete_other();
+
 	/* contains call to pgaio_io_call_complete_local() */
 	if (ioh->owner_procno == MyProcNumber)
 		pgaio_io_reclaim(ioh);
@@ -762,6 +771,8 @@ pgaio_io_wait_for_free(void)
 {
 	int			reclaimed = 0;
 
+	pgstat_count_backend_aio_handle_wait();
+
 	pgaio_debug(DEBUG2, "waiting for free IO with %d pending, %u in-flight, %u idle IOs",
 				pgaio_my_backend->num_staged_ios,
 				dclist_count(&pgaio_my_backend->in_flight_ios),
@@ -1150,6 +1161,8 @@ pgaio_submit_staged(void)
 
 	Assert(total_submitted == did_submit);
 
+	pgstat_count_backend_aio_submitted();
+
 	pgaio_my_backend->num_staged_ios = 0;
 
 	pgaio_debug(DEBUG4,
diff --git a/src/backend/utils/activity/pgstat_backend.c b/src/backend/utils/activity/pgstat_backend.c
index b736b2ccc6f..4ad755f7f18 100644
--- a/src/backend/utils/activity/pgstat_backend.c
+++ b/src/backend/utils/activity/pgstat_backend.c
@@ -40,6 +40,7 @@
 static PgStat_BackendPending PendingBackendStats;
 static bool backend_has_iostats = false;
 static bool backend_has_lockstats = false;
+static bool backend_has_aiostats = false;
 
 /*
  * WAL usage counters saved from pgWalUsage at the previous call to
@@ -120,6 +121,74 @@ pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type)
 	pgstat_report_fixed = true;
 }
 
+/*
+ * Utility routines to report AIO stats for backends, kept here to avoid
+ * exposing PendingBackendStats to the outside world.
+ */
+void
+pgstat_count_backend_aio_start(bool synchronous)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.started++;
+	if (synchronous)
+		PendingBackendStats.aio_counters.executed_sync++;
+	else
+		PendingBackendStats.aio_counters.executed_async++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_self(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_self++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_complete_other(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.completed_other++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_handle_wait(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.handle_waits++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
+void
+pgstat_count_backend_aio_submitted(void)
+{
+	if (!pgstat_tracks_backend_bktype(MyBackendType))
+		return;
+
+	PendingBackendStats.aio_counters.submitted++;
+
+	backend_has_aiostats = true;
+	pgstat_report_fixed = true;
+}
+
 /*
  * Returns statistics of a backend by proc number.
  */
@@ -326,6 +395,38 @@ pgstat_flush_backend_entry_lock(PgStat_EntryRef *entry_ref)
 	backend_has_lockstats = false;
 }
 
+/*
+ * Flush out locally pending backend AIO statistics.  Locking is managed
+ * by the caller.
+ */
+static void
+pgstat_flush_backend_entry_aio(PgStat_EntryRef *entry_ref)
+{
+	PgStatShared_Backend *shbackendent;
+	PgStat_AioCounters *bktype_shstats;
+
+	if (!backend_has_aiostats)
+		return;
+
+	shbackendent = (PgStatShared_Backend *) entry_ref->shared_stats;
+	bktype_shstats = &shbackendent->stats.aio_counters;
+
+#define AIOSTAT_ACC(fld) \
+	(bktype_shstats->fld += PendingBackendStats.aio_counters.fld)
+	AIOSTAT_ACC(started);
+	AIOSTAT_ACC(executed_sync);
+	AIOSTAT_ACC(executed_async);
+	AIOSTAT_ACC(completed_self);
+	AIOSTAT_ACC(completed_other);
+	AIOSTAT_ACC(handle_waits);
+	AIOSTAT_ACC(submitted);
+#undef AIOSTAT_ACC
+
+	MemSet(&PendingBackendStats.aio_counters, 0, sizeof(PgStat_AioCounters));
+
+	backend_has_aiostats = false;
+}
+
 /*
  * Flush out locally pending backend statistics
  *
@@ -354,6 +455,10 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if ((flags & PGSTAT_BACKEND_FLUSH_LOCK) && backend_has_lockstats)
 		has_pending_data = true;
 
+	/* Some AIO data pending? */
+	if ((flags & PGSTAT_BACKEND_FLUSH_AIO) && backend_has_aiostats)
+		has_pending_data = true;
+
 	if (!has_pending_data)
 		return false;
 
@@ -372,6 +477,9 @@ pgstat_flush_backend(bool nowait, uint32 flags)
 	if (flags & PGSTAT_BACKEND_FLUSH_LOCK)
 		pgstat_flush_backend_entry_lock(entry_ref);
 
+	if (flags & PGSTAT_BACKEND_FLUSH_AIO)
+		pgstat_flush_backend_entry_aio(entry_ref);
+
 	pgstat_unlock_entry(entry_ref);
 
 	return false;
@@ -411,6 +519,7 @@ pgstat_create_backend(ProcNumber procnum)
 	MemSet(&PendingBackendStats, 0, sizeof(PgStat_BackendPending));
 	backend_has_iostats = false;
 	backend_has_lockstats = false;
+	backend_has_aiostats = false;
 
 	/*
 	 * Initialize prevBackendWalUsage with pgWalUsage so that
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 565d0e70768..fb62a56f3fe 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1722,6 +1722,66 @@ pg_stat_get_backend_wal(PG_FUNCTION_ARGS)
 	return (pg_stat_wal_build_tuple(bktype_stats, backend_stats->stat_reset_timestamp));
 }
 
+/*
+ * Returns AIO statistics for a backend with given PID.
+ */
+Datum
+pg_stat_get_backend_aio(PG_FUNCTION_ARGS)
+{
+#define PG_STAT_BACKEND_AIO_COLS	8
+	TupleDesc	tupdesc;
+	Datum		values[PG_STAT_BACKEND_AIO_COLS] = {0};
+	bool		nulls[PG_STAT_BACKEND_AIO_COLS] = {0};
+	int			pid;
+	PgStat_Backend *backend_stats;
+	PgStat_AioCounters aio_counters;
+
+	pid = PG_GETARG_INT32(0);
+	backend_stats = pgstat_fetch_stat_backend_by_pid(pid, NULL);
+
+	if (!backend_stats)
+		PG_RETURN_NULL();
+
+	aio_counters = backend_stats->aio_counters;
+
+	/* Initialise attributes information in the tuple descriptor */
+	tupdesc = CreateTemplateTupleDesc(PG_STAT_BACKEND_AIO_COLS);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 1, "started",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 2, "executed_sync",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 3, "executed_async",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 4, "completed_self",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 5, "completed_other",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 6, "handle_waits",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 7, "submitted",
+					   INT8OID, -1, 0);
+	TupleDescInitEntry(tupdesc, (AttrNumber) 8, "stats_reset",
+					   TIMESTAMPTZOID, -1, 0);
+	TupleDescFinalize(tupdesc);
+	BlessTupleDesc(tupdesc);
+
+	/* Fill values */
+	values[0] = Int64GetDatum(aio_counters.started);
+	values[1] = Int64GetDatum(aio_counters.executed_sync);
+	values[2] = Int64GetDatum(aio_counters.executed_async);
+	values[3] = Int64GetDatum(aio_counters.completed_self);
+	values[4] = Int64GetDatum(aio_counters.completed_other);
+	values[5] = Int64GetDatum(aio_counters.handle_waits);
+	values[6] = Int64GetDatum(aio_counters.submitted);
+
+	if (backend_stats->stat_reset_timestamp != 0)
+		values[7] = TimestampTzGetDatum(backend_stats->stat_reset_timestamp);
+	else
+		nulls[7] = true;
+
+	PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+}
+
 /*
  * Returns statistics of WAL activity
  */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3cb84359adf..a089860ada4 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6108,6 +6108,13 @@
   proargmodes => '{i,o,o,o,o,o}',
   proargnames => '{backend_pid,locktype,waits,wait_time,fastpath_exceeded,stats_reset}',
   prosrc => 'pg_stat_get_backend_lock' },
+{ oid => '9082', descr => 'statistics: backend AIO activity',
+  proname => 'pg_stat_get_backend_aio', provolatile => 'v', proparallel => 'r',
+  prorettype => 'record', proargtypes => 'int4',
+  proallargtypes => '{int4,int8,int8,int8,int8,int8,int8,int8,timestamptz}',
+  proargmodes => '{i,o,o,o,o,o,o,o,o}',
+  proargnames => '{backend_pid,started,executed_sync,executed_async,completed_self,completed_other,handle_waits,submitted,stats_reset}',
+  prosrc => 'pg_stat_get_backend_aio' },
 { oid => '6248', descr => 'statistics: information about WAL prefetching',
   proname => 'pg_stat_get_recovery_prefetch', prorows => '1', proretset => 't',
   provolatile => 'v', prorettype => 'record', proargtypes => '',
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 58a44857f13..64afb2bc082 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -514,6 +514,21 @@ typedef struct PgStat_WalStats
 	TimestampTz stat_reset_timestamp;
 } PgStat_WalStats;
 
+/* -------
+ * PgStat_AioCounters	AIO activity counters
+ * -------
+ */
+typedef struct PgStat_AioCounters
+{
+	PgStat_Counter started;
+	PgStat_Counter executed_sync;
+	PgStat_Counter executed_async;
+	PgStat_Counter completed_self;
+	PgStat_Counter completed_other;
+	PgStat_Counter handle_waits;
+	PgStat_Counter submitted;
+} PgStat_AioCounters;
+
 /* -------
  * PgStat_Backend		Backend statistics
  * -------
@@ -524,6 +539,7 @@ typedef struct PgStat_Backend
 	PgStat_BktypeIO io_stats;
 	PgStat_WalCounters wal_counters;
 	PgStat_PendingLock lock_stats;
+	PgStat_AioCounters aio_counters;
 } PgStat_Backend;
 
 /* ---------
@@ -542,6 +558,8 @@ typedef struct PgStat_BackendPending
 	 * PGSTAT_KIND_LOCK.
 	 */
 	PgStat_PendingLock pending_lock;
+	/* Store the AIO statistics counters */
+	PgStat_AioCounters aio_counters;
 } PgStat_BackendPending;
 
 /*
@@ -598,6 +616,13 @@ extern void pgstat_count_backend_io_op(IOObject io_object,
 extern void pgstat_count_backend_lock_waits(uint8 locktag_type, PgStat_Counter usecs);
 extern void pgstat_count_backend_lock_fastpath_exceeded(uint8 locktag_type);
 
+/* used by aio.c for AIO stats tracked in backends */
+extern void pgstat_count_backend_aio_start(bool synchronous);
+extern void pgstat_count_backend_aio_complete_self(void);
+extern void pgstat_count_backend_aio_complete_other(void);
+extern void pgstat_count_backend_aio_handle_wait(void);
+extern void pgstat_count_backend_aio_submitted(void);
+
 extern PgStat_Backend *pgstat_fetch_stat_backend(ProcNumber procNumber);
 extern PgStat_Backend *pgstat_fetch_stat_backend_by_pid(int pid,
 														BackendType *bktype);
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index b3dc3ff7d8b..b2092fb42b9 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -706,7 +706,8 @@ extern void pgstat_archiver_snapshot_cb(void);
 #define PGSTAT_BACKEND_FLUSH_IO		(1 << 0)	/* Flush I/O statistics */
 #define PGSTAT_BACKEND_FLUSH_WAL   (1 << 1) /* Flush WAL statistics */
 #define PGSTAT_BACKEND_FLUSH_LOCK  (1 << 2) /* Flush lock statistics */
-#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK)
+#define PGSTAT_BACKEND_FLUSH_AIO   (1 << 3) /* Flush AIO statistics */
+#define PGSTAT_BACKEND_FLUSH_ALL   (PGSTAT_BACKEND_FLUSH_IO | PGSTAT_BACKEND_FLUSH_WAL | PGSTAT_BACKEND_FLUSH_LOCK | PGSTAT_BACKEND_FLUSH_AIO)
 
 extern bool pgstat_flush_backend(bool nowait, uint32 flags);
 extern bool pgstat_backend_flush_cb(bool nowait);
diff --git a/src/test/modules/test_aio/t/001_aio.pl b/src/test/modules/test_aio/t/001_aio.pl
index 63cadd64c15..8ecf3d3ad91 100644
--- a/src/test/modules/test_aio/t/001_aio.pl
+++ b/src/test/modules/test_aio/t/001_aio.pl
@@ -1842,6 +1842,37 @@ read_buffers('$table', 0, 4)|,
 	$psql_c->quit();
 }
 
+# Test per-backend AIO statistics counters
+sub test_aio_stats
+{
+	my $io_method = shift;
+	my $node = shift;
+
+	my $psql = $node->background_psql('postgres', on_error_stop => 0);
+
+	# Reset backend stats, evict relation, then read it back to force
+	# physical IO through the AIO layer.
+	$psql->query_safe(qq(SELECT pg_stat_reset_backend_stats(pg_backend_pid())));
+	$psql->query_safe(qq(SELECT evict_rel('tbl_ok')));
+	$psql->query_safe(qq(SELECT count(*) FROM tbl_ok));
+	$psql->query_safe(qq(SELECT pg_stat_force_next_flush()));
+
+	# started must be > 0 after physical IO
+	my $started = $psql->query_safe(
+		qq(SELECT started FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	cmp_ok($started, '>', 0,
+		"$io_method: AIO stats: started > 0 after physical IO");
+
+	# invariant: started = executed_sync + executed_async
+	my $consistent = $psql->query_safe(
+		qq(SELECT started = executed_sync + executed_async
+		   FROM pg_stat_get_backend_aio(pg_backend_pid())));
+	is($consistent, 't',
+		"$io_method: AIO stats: started = executed_sync + executed_async");
+
+	$psql->quit();
+}
+
 # Run all tests that for the specified node / io_method
 sub test_io_method
 {
@@ -1878,6 +1909,7 @@ CHECKPOINT;
 	test_ignore_checksum($io_method, $node);
 	test_checksum_createdb($io_method, $node);
 	test_read_buffers($io_method, $node);
+	test_aio_stats($io_method, $node);
 
 	# generic injection tests
   SKIP:
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 117e7379f10..6ce3077fa5b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2329,6 +2329,7 @@ PgStatShared_ReplSlot
 PgStatShared_SLRU
 PgStatShared_Subscription
 PgStatShared_Wal
+PgStat_AioCounters
 PgStat_ArchiverStats
 PgStat_Backend
 PgStat_BackendPending
-- 
2.34.1


--Pi0/Nu3LVhi0Ij+H--





^ permalink  raw  reply  [nested|flat] 323+ messages in thread