Date: Thu, 11 Sep 2025 13:36:14 +0200
From: Christoph Berg <myon@debian.org>
To: Tomas Vondra <tomas@vondra.me>
Cc: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>,
	Andres Freund <andres@anarazel.de>,
	Tomas Vondra <tomas.vondra@postgresql.org>,
	pgsql-hackers@lists.postgresql.org
Subject: Re: pgsql: Introduce pg_shmem_allocations_numa view
Message-ID: <aMK0Lt7o74pMcPfK@msg.df7cb.de>
References: <b54df21b-6595-4031-baf8-7c87851794c8@vondra.me>
 <aFuRoUieUVh+pMfZ@ip-10-97-1-34.eu-west-3.compute.internal>
 <aFu6tmQ7EVaaNV9K@msg.df7cb.de>
 <aFzadttaX+Qnb5s3@ip-10-97-1-34.eu-west-3.compute.internal>
 <aed7fd1f-4691-45d6-a002-6eba297443ca@vondra.me>
 <aF7V5hLDOc54QKfP@ip-10-97-1-34.eu-west-3.compute.internal>
 <132f85de-75c8-4e21-b875-b806596c9214@vondra.me>
 <aGNex9zyxmXqCHXy@ip-10-97-1-34.eu-west-3.compute.internal>
 <64e8ec11-6bb8-488b-abfc-8e67324a03fa@vondra.me>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="wIOga72bHU5UUpAV"
Content-Disposition: inline
In-Reply-To: <64e8ec11-6bb8-488b-abfc-8e67324a03fa@vondra.me>
Archived-At: 
 <https://www.postgresql.org/message-id/aMK0Lt7o74pMcPfK%40msg.df7cb.de>
Precedence: bulk


--wIOga72bHU5UUpAV
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

Re: Tomas Vondra
> Thanks! Pushed, with both adjustments (link to kernel thread, adding the
> commit hash).

The PG18 Debian package is still carrying the contrib complement of
this patch (see attachment).

Should that be addressed before 18.0?

Christoph

--wIOga72bHU5UUpAV
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename=move-pages32

Work around a Linux bug in move_pages

In 32-bit mode on 64-bit kernels, move_pages() does not correctly advance to
the next chunk. Work around by not asking for more than 16 pages at once so
move_pages() internal loop is not executed more than once.

https://www.postgresql.org/message-id/flat/a3a4fe3d-1a80-4e03-aa8e-150ee15f6c35%40vondra.me#6abe7eaa802b5b07bb70cc3229e63a9f
https://marc.info/?l=linux-mm&m=175077821909222&w=2

--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -390,8 +390,15 @@ pg_buffercache_numa_pages(PG_FUNCTION_AR
 		memset(os_page_status, 0xff, sizeof(int) * os_page_count);
 
 		/* Query NUMA status for all the pointers */
-		if (pg_numa_query_pages(0, os_page_count, os_page_ptrs, os_page_status) == -1)
-			elog(ERROR, "failed NUMA pages inquiry: %m");
+#define NUMA_QUERY_CHUNK_SIZE 16  /* has to be <= DO_PAGES_STAT_CHUNK_NR (do_pages_stat())*/
+		for (uint64 chunk_start = 0; chunk_start < os_page_count; chunk_start += NUMA_QUERY_CHUNK_SIZE) {
+			uint64 chunk_size = Min(NUMA_QUERY_CHUNK_SIZE, os_page_count - chunk_start);
+
+			if (pg_numa_query_pages(0, chunk_size, &os_page_ptrs[chunk_start],
+						&os_page_status[chunk_start]) == -1)
+				elog(ERROR, "failed NUMA pages inquiry status: %m");
+		}
+#undef NUMA_QUERY_CHUNK_SIZE
 
 		/* Initialize the multi-call context, load entries about buffers */
 
--- a/src/backend/storage/ipc/shmem.c
+++ b/src/backend/storage/ipc/shmem.c
@@ -689,8 +689,15 @@ pg_get_shmem_allocations_numa(PG_FUNCTIO
 			CHECK_FOR_INTERRUPTS();
 		}
 
-		if (pg_numa_query_pages(0, shm_ent_page_count, page_ptrs, pages_status) == -1)
-			elog(ERROR, "failed NUMA pages inquiry status: %m");
+#define NUMA_QUERY_CHUNK_SIZE 16  /* has to be <= DO_PAGES_STAT_CHUNK_NR (do_pages_stat())*/
+		for (uint64 chunk_start = 0; chunk_start < shm_ent_page_count; chunk_start += NUMA_QUERY_CHUNK_SIZE) {
+			uint64 chunk_size = Min(NUMA_QUERY_CHUNK_SIZE, shm_ent_page_count - chunk_start);
+
+			if (pg_numa_query_pages(0, chunk_size, &page_ptrs[chunk_start],
+						&pages_status[chunk_start]) == -1)
+				elog(ERROR, "failed NUMA pages inquiry status: %m");
+		}
+#undef NUMA_QUERY_CHUNK_SIZE
 
 		/* Count number of NUMA nodes used for this shared memory entry */
 		memset(nodes, 0, sizeof(Size) * (max_nodes + 1));

--wIOga72bHU5UUpAV--