Date: Wed, 30 Mar 2022 16:54:50 +0900
From: Michael Paquier <michael@paquier.xyz>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Julien Rouhaud <rjuju123@gmail.com>, okano.naoki@jp.fujitsu.com,
	PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Subject: Re: BUG #17448: In Windows 10, version 1703 and later, huge_pages
 doesn't work.
Message-ID: <YkQMyhsP7xgLDDT/@paquier.xyz>
References: <17448-0a96583a67edb1f7@postgresql.org>
 <Yj6oxYsBE5LXA8ZP@paquier.xyz>
 <20220326082408.moh55zuecxjmewnm@jrouhaud>
 <CA+hUKGLwOSf-QwVtCcKx9xvugN2BSzEBBeD5DNyTXgcSRp5sTw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="LD3pfApFPl6CapiV"
Content-Disposition: inline
In-Reply-To: 
 <CA+hUKGLwOSf-QwVtCcKx9xvugN2BSzEBBeD5DNyTXgcSRp5sTw@mail.gmail.com>
Archived-At: 
 <https://www.postgresql.org/message-id/YkQMyhsP7xgLDDT/%40paquier.xyz>
Precedence: bulk


--LD3pfApFPl6CapiV
Content-Type: multipart/mixed; boundary="8jnowtt6rp7hcNn9"
Content-Disposition: inline


--8jnowtt6rp7hcNn9
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Sun, Mar 27, 2022 at 12:07:57AM +1300, Thomas Munro wrote:
> There are traces of method to the madness:  It's basically YYMM, but
> then after 2004 they switched to H1 and H2 (first/second half of the
> year) instead of MM, perhaps to avoid confusion with YYYY format year.
> Note also that Windows 10 has a 21H2 and Windows 11 has a 21H2.
>
> Some question I have: is FILE_MAP_LARGE PAGES a macro?  We claim to
> support all those ancient zombie OSes like Windows 7, or maybe it's
> even XP for 11, and this has to be back-patched to 11, so we might
> need to make it conditional.  But conditional on what?  For example,
> does something like the attached work (untested)?  What happens if a <
> 1703 kernel sees this flag, does it reject it or ignore it?

I don't have an answer about how much Windows gets angry if we pass
down to MapViewOfFileEx() the flag FILE_MAP_LARGE_PAGES when running
the code on a version of Windows that does not support it.

Anyway, I think that we could just play it safe.  See for example the
attached, where I use PG_FILE_MAP_LARGE_PAGES at compile time to find
if the value is set.  Then, at run-time, I am just relying on=20
IsWindowsVersionOrGreater() to do the job, something useful when
huge_pages=3Don as I guess we should fail hard if we did not know about
FILE_MAP_LARGE_PAGES at compile-time, but try to use huge pages at run
time with version >=3D 10.0.1703.

Perhaps there is a better thing to do?
--
Michael

--8jnowtt6rp7hcNn9
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment; filename="win32-hugepages-michael.patch"
Content-Transfer-Encoding: quoted-printable

diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 6cf69411db..49b9fa0322 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -12,6 +12,8 @@
  */
 #include "postgres.h"
=20
+#include <versionhelpers.h>
+
 #include "miscadmin.h"
 #include "storage/dsm.h"
 #include "storage/ipc.h"
@@ -39,6 +41,17 @@
 #define PROTECTIVE_REGION_SIZE (10 * WIN32_STACK_RLIMIT)
 void	   *ShmemProtectiveRegion =3D NULL;
=20
+/*
+ * FILE_MAP_LARGE_PAGES is needed for the support of huge pages, and
+ * it is only available since Windows 10.0.1703.  PG_FILE_MAP_LARGE_PAGES
+ * saves its value at compile-time.
+ */
+#ifdef FILE_MAP_LARGE_PAGES
+#define PG_FILE_MAP_LARGE_PAGES FILE_MAP_LARGE_PAGES
+#else
+#define PG_FILE_MAP_LARGE_PAGES 0
+#endif
+
 HANDLE		UsedShmemSegID =3D INVALID_HANDLE_VALUE;
 void	   *UsedShmemSegAddr =3D NULL;
 static Size UsedShmemSegSize =3D 0;
@@ -216,6 +229,7 @@ PGSharedMemoryCreate(Size size,
 	SIZE_T		largePageSize =3D 0;
 	Size		orig_size =3D size;
 	DWORD		flProtect =3D PAGE_READWRITE;
+	DWORD		desiredAccess;
=20
 	ShmemProtectiveRegion =3D VirtualAlloc(NULL, PROTECTIVE_REGION_SIZE,
 										 MEM_RESERVE, PAGE_NOACCESS);
@@ -258,6 +272,22 @@ PGSharedMemoryCreate(Size size,
 		}
 	}
=20
+	/*
+	 * If this binary has been compiled without FILE_MAP_LARGE_PAGES
+	 * available and huge pages are wanted, there is nothing do do
+	 * if running on a version older than 10.0.1703 as this flag is a
+	 * requirement to allow the use of huge pages.
+	 */
+	if (huge_pages =3D=3D HUGE_PAGES_ON &&
+		PG_FILE_MAP_LARGE_PAGES =3D=3D 0 &&
+		IsWindowsVersionOrGreater(10, 0, 1703))
+	{
+		ereport(FATAL,
+				(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+				 errmsg("the processor does not support large pages"),
+				 errdetail("The code has been compiled without huge page support for t=
his version of Windows.")));
+	}
+
 retry:
 #ifdef _WIN64
 	size_high =3D size >> 32;
@@ -353,12 +383,21 @@ retry:
 	if (!CloseHandle(hmap))
 		elog(LOG, "could not close handle to shared memory: error code %lu", Get=
LastError());
=20
+	desiredAccess =3D FILE_MAP_WRITE | FILE_MAP_READ;
+
+	/*
+	 * Set large pages if wanted.  FILE_MAP_LARGE_PAGES is needed when
+	 * running at least Windows 10.0.1703.
+	 */
+	if ((flProtect & SEC_LARGE_PAGES) !=3D 0 &&
+		IsWindowsVersionOrGreater(10, 0, 1703))
+		desiredAccess |=3D PG_FILE_MAP_LARGE_PAGES;
=20
 	/*
 	 * Get a pointer to the new shared memory segment. Map the whole segment
 	 * at once, and let the system decide on the initial address.
 	 */
-	memAddress =3D MapViewOfFileEx(hmap2, FILE_MAP_WRITE | FILE_MAP_READ, 0, =
0, 0, NULL);
+	memAddress =3D MapViewOfFileEx(hmap2, desiredAccess, 0, 0, 0, NULL);
 	if (!memAddress)
 		ereport(FATAL,
 				(errmsg("could not create shared memory segment: error code %lu", GetL=
astError()),

--8jnowtt6rp7hcNn9--

--LD3pfApFPl6CapiV
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmJEDMoACgkQnvQgOdby
QH1mwBAApu/wKMUtdXG6pQlzzpQUlbJH4U17pU8Pd8YsuRr5HAzkaJW0DFOrFdTA
9AFlg9mjD9QWLWMYe5dJ80myBZqNjyIXN7CwpSFzqwPYEMSqzpG/FF96Ed7yUtp3
QfEicqZZCjnFPRbAJTpunSQj+Ry8zQhRF/ZUGZuIWTZAw1MeorHcDsgGAsKAnf3E
SbyK37ZJVNLgE+jAauGtX8tNsDUl0HNZno8EG5uEg7v2P/EUIh4nf4Gl/e/hiEGl
5JfBVhZXaO7o3moFU2HzCfduB3ZtAiZfxN3JRsbW4Z1O85/kkgjIOoqjYdj3yPsw
NhStW0aQffGwcI6kp5DiKs173wzfqUzKlkr7AwPp0roTGm9Nl9erGgARbuR/XxYy
NnbW9IovhCDKvHOBxbCU6GmHj82VsENfqv8MmcX+YLd6pP8kYRyUGfBKccg7HA+0
ri15jiaEXTVW/ZkAKJxlHr1MYLdDDm59kZmyyaDhWYHQclYgtifl3Ih5W4wVAQnG
4Sf1Gmi3Nlu1Bd3VgSQIPavvSZ9w/06GetCF4RYliN1GdCPHihbpDmrIqd9Aavcd
F/w6nfAAmvnoDmUxy6ixku7HNHsmJaKWkIZp9FlGZtdTCOUdb/pgMCU3Hky6vuMx
EEbk7E1j/txwvV96OHpL1zYNJOuI4RaK3ZC1uiwx/PL7lYbjnCM=
=IF2N
-----END PGP SIGNATURE-----

--LD3pfApFPl6CapiV--