From: Michael Paquier <michael@paquier.xyz>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Julien Rouhaud <rjuju123@gmail.com>
Cc: okano.naoki@jp.fujitsu.com, PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Subject: Re: BUG #17448: In Windows 10, version 1703 and later, huge_pages doesn't work.
Date: Wed, 30 Mar 2022 16:54:50 +0900
Message-ID: <YkQMyhsP7xgLDDT/@paquier.xyz> (raw)
In-Reply-To: <CA+hUKGLwOSf-QwVtCcKx9xvugN2BSzEBBeD5DNyTXgcSRp5sTw@mail.gmail.com>
References: <17448-0a96583a67edb1f7@postgresql.org>
<Yj6oxYsBE5LXA8ZP@paquier.xyz>
<20220326082408.moh55zuecxjmewnm@jrouhaud>
<CA+hUKGLwOSf-QwVtCcKx9xvugN2BSzEBBeD5DNyTXgcSRp5sTw@mail.gmail.com>
On Sun, Mar 27, 2022 at 12:07:57AM +1300, Thomas Munro wrote:
> There are traces of method to the madness: It's basically YYMM, but
> then after 2004 they switched to H1 and H2 (first/second half of the
> year) instead of MM, perhaps to avoid confusion with YYYY format year.
> Note also that Windows 10 has a 21H2 and Windows 11 has a 21H2.
>
> Some question I have: is FILE_MAP_LARGE PAGES a macro? We claim to
> support all those ancient zombie OSes like Windows 7, or maybe it's
> even XP for 11, and this has to be back-patched to 11, so we might
> need to make it conditional. But conditional on what? For example,
> does something like the attached work (untested)? What happens if a <
> 1703 kernel sees this flag, does it reject it or ignore it?
I don't have an answer about how much Windows gets angry if we pass
down to MapViewOfFileEx() the flag FILE_MAP_LARGE_PAGES when running
the code on a version of Windows that does not support it.
Anyway, I think that we could just play it safe. See for example the
attached, where I use PG_FILE_MAP_LARGE_PAGES at compile time to find
if the value is set. Then, at run-time, I am just relying on
IsWindowsVersionOrGreater() to do the job, something useful when
huge_pages=on as I guess we should fail hard if we did not know about
FILE_MAP_LARGE_PAGES at compile-time, but try to use huge pages at run
time with version >= 10.0.1703.
Perhaps there is a better thing to do?
--
Michael
Attachments:win32-hugepages-michael.patch (text/x-diff, 2.7K) downloaddiff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.cindex 6cf69411db..49b9fa0322 100644--- a/src/backend/port/win32_shmem.c+++ b/src/backend/port/win32_shmem.c@@ -12,6 +12,8 @@
*/
#include "postgres.h"
+#include <versionhelpers.h>+
#include "miscadmin.h"
#include "storage/dsm.h"
#include "storage/ipc.h"
@@ -39,6 +41,17 @@
#define PROTECTIVE_REGION_SIZE (10 * WIN32_STACK_RLIMIT)
void *ShmemProtectiveRegion = NULL;
+/*+ * FILE_MAP_LARGE_PAGES is needed for the support of huge pages, and+ * it is only available since Windows 10.0.1703. PG_FILE_MAP_LARGE_PAGES+ * saves its value at compile-time.+ */+#ifdef FILE_MAP_LARGE_PAGES+#define PG_FILE_MAP_LARGE_PAGES FILE_MAP_LARGE_PAGES+#else+#define PG_FILE_MAP_LARGE_PAGES 0+#endif+
HANDLE UsedShmemSegID = INVALID_HANDLE_VALUE;
void *UsedShmemSegAddr = NULL;
static Size UsedShmemSegSize = 0;
@@ -216,6 +229,7 @@ PGSharedMemoryCreate(Size size,
SIZE_T largePageSize = 0;
Size orig_size = size;
DWORD flProtect = PAGE_READWRITE;
+ DWORD desiredAccess;
ShmemProtectiveRegion = VirtualAlloc(NULL, PROTECTIVE_REGION_SIZE,
MEM_RESERVE, PAGE_NOACCESS);
@@ -258,6 +272,22 @@ PGSharedMemoryCreate(Size size,
}
}
+ /*+ * If this binary has been compiled without FILE_MAP_LARGE_PAGES+ * available and huge pages are wanted, there is nothing do do+ * if running on a version older than 10.0.1703 as this flag is a+ * requirement to allow the use of huge pages.+ */+ if (huge_pages == HUGE_PAGES_ON &&+ PG_FILE_MAP_LARGE_PAGES == 0 &&+ IsWindowsVersionOrGreater(10, 0, 1703))+ {+ ereport(FATAL,+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),+ errmsg("the processor does not support large pages"),+ errdetail("The code has been compiled without huge page support for this version of Windows.")));+ }+
retry:
#ifdef _WIN64
size_high = size >> 32;
@@ -353,12 +383,21 @@ retry:
if (!CloseHandle(hmap))
elog(LOG, "could not close handle to shared memory: error code %lu", GetLastError());
+ desiredAccess = FILE_MAP_WRITE | FILE_MAP_READ;++ /*+ * Set large pages if wanted. FILE_MAP_LARGE_PAGES is needed when+ * running at least Windows 10.0.1703.+ */+ if ((flProtect & SEC_LARGE_PAGES) != 0 &&+ IsWindowsVersionOrGreater(10, 0, 1703))+ desiredAccess |= PG_FILE_MAP_LARGE_PAGES;
/*
* Get a pointer to the new shared memory segment. Map the whole segment
* at once, and let the system decide on the initial address.
*/
- memAddress = MapViewOfFileEx(hmap2, FILE_MAP_WRITE | FILE_MAP_READ, 0, 0, 0, NULL);+ memAddress = MapViewOfFileEx(hmap2, desiredAccess, 0, 0, 0, NULL);
if (!memAddress)
ereport(FATAL,
(errmsg("could not create shared memory segment: error code %lu", GetLastError()),
signature.asc (application/pgp-signature, 832B) download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-bugs@postgresql.org
Cc: michael@paquier.xyz, thomas.munro@gmail.com, rjuju123@gmail.com
Subject: Re: BUG #17448: In Windows 10, version 1703 and later, huge_pages doesn't work.
In-Reply-To: <YkQMyhsP7xgLDDT/@paquier.xyz>
References: <17448-0a96583a67edb1f7@postgresql.org> <Yj6oxYsBE5LXA8ZP@paquier.xyz> <20220326082408.moh55zuecxjmewnm@jrouhaud> <CA+hUKGLwOSf-QwVtCcKx9xvugN2BSzEBBeD5DNyTXgcSRp5sTw@mail.gmail.com> <YkQMyhsP7xgLDDT/@paquier.xyz>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox