pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Michael Paquier <michael@paquier.xyz>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Julien Rouhaud <rjuju123@gmail.com>
Cc: okano.naoki@jp.fujitsu.com, PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Subject: Re: BUG #17448: In Windows 10, version 1703 and later, huge_pages doesn't work.
Date: Wed, 30 Mar 2022 16:54:50 +0900
Message-ID: <YkQMyhsP7xgLDDT/@paquier.xyz> (raw)
In-Reply-To: <CA+hUKGLwOSf-QwVtCcKx9xvugN2BSzEBBeD5DNyTXgcSRp5sTw@mail.gmail.com>
References: <17448-0a96583a67edb1f7@postgresql.org>
	<Yj6oxYsBE5LXA8ZP@paquier.xyz>
	<20220326082408.moh55zuecxjmewnm@jrouhaud>
	<CA+hUKGLwOSf-QwVtCcKx9xvugN2BSzEBBeD5DNyTXgcSRp5sTw@mail.gmail.com>

On Sun, Mar 27, 2022 at 12:07:57AM +1300, Thomas Munro wrote:
> There are traces of method to the madness:  It's basically YYMM, but
> then after 2004 they switched to H1 and H2 (first/second half of the
> year) instead of MM, perhaps to avoid confusion with YYYY format year.
> Note also that Windows 10 has a 21H2 and Windows 11 has a 21H2.
>
> Some question I have: is FILE_MAP_LARGE PAGES a macro?  We claim to
> support all those ancient zombie OSes like Windows 7, or maybe it's
> even XP for 11, and this has to be back-patched to 11, so we might
> need to make it conditional.  But conditional on what?  For example,
> does something like the attached work (untested)?  What happens if a <
> 1703 kernel sees this flag, does it reject it or ignore it?

I don't have an answer about how much Windows gets angry if we pass
down to MapViewOfFileEx() the flag FILE_MAP_LARGE_PAGES when running
the code on a version of Windows that does not support it.

Anyway, I think that we could just play it safe.  See for example the
attached, where I use PG_FILE_MAP_LARGE_PAGES at compile time to find
if the value is set.  Then, at run-time, I am just relying on 
IsWindowsVersionOrGreater() to do the job, something useful when
huge_pages=on as I guess we should fail hard if we did not know about
FILE_MAP_LARGE_PAGES at compile-time, but try to use huge pages at run
time with version >= 10.0.1703.

Perhaps there is a better thing to do?
--
Michael

Attachments:

win32-hugepages-michael.patch (text/x-diff, 2.7K) download diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c index 6cf69411db..49b9fa0322 100644 --- a/src/backend/port/win32_shmem.c +++ b/src/backend/port/win32_shmem.c @@ -12,6 +12,8 @@ */ #include "postgres.h" +#include <versionhelpers.h> + #include "miscadmin.h" #include "storage/dsm.h" #include "storage/ipc.h" @@ -39,6 +41,17 @@ #define PROTECTIVE_REGION_SIZE (10 * WIN32_STACK_RLIMIT) void *ShmemProtectiveRegion = NULL; +/* + * FILE_MAP_LARGE_PAGES is needed for the support of huge pages, and + * it is only available since Windows 10.0.1703. PG_FILE_MAP_LARGE_PAGES + * saves its value at compile-time. + */ +#ifdef FILE_MAP_LARGE_PAGES +#define PG_FILE_MAP_LARGE_PAGES FILE_MAP_LARGE_PAGES +#else +#define PG_FILE_MAP_LARGE_PAGES 0 +#endif + HANDLE UsedShmemSegID = INVALID_HANDLE_VALUE; void *UsedShmemSegAddr = NULL; static Size UsedShmemSegSize = 0; @@ -216,6 +229,7 @@ PGSharedMemoryCreate(Size size, SIZE_T largePageSize = 0; Size orig_size = size; DWORD flProtect = PAGE_READWRITE; + DWORD desiredAccess; ShmemProtectiveRegion = VirtualAlloc(NULL, PROTECTIVE_REGION_SIZE, MEM_RESERVE, PAGE_NOACCESS); @@ -258,6 +272,22 @@ PGSharedMemoryCreate(Size size, } } + /* + * If this binary has been compiled without FILE_MAP_LARGE_PAGES + * available and huge pages are wanted, there is nothing do do + * if running on a version older than 10.0.1703 as this flag is a + * requirement to allow the use of huge pages. + */ + if (huge_pages == HUGE_PAGES_ON && + PG_FILE_MAP_LARGE_PAGES == 0 && + IsWindowsVersionOrGreater(10, 0, 1703)) + { + ereport(FATAL, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("the processor does not support large pages"), + errdetail("The code has been compiled without huge page support for this version of Windows."))); + } + retry: #ifdef _WIN64 size_high = size >> 32; @@ -353,12 +383,21 @@ retry: if (!CloseHandle(hmap)) elog(LOG, "could not close handle to shared memory: error code %lu", GetLastError()); + desiredAccess = FILE_MAP_WRITE | FILE_MAP_READ; + + /* + * Set large pages if wanted. FILE_MAP_LARGE_PAGES is needed when + * running at least Windows 10.0.1703. + */ + if ((flProtect & SEC_LARGE_PAGES) != 0 && + IsWindowsVersionOrGreater(10, 0, 1703)) + desiredAccess |= PG_FILE_MAP_LARGE_PAGES; /* * Get a pointer to the new shared memory segment. Map the whole segment * at once, and let the system decide on the initial address. */ - memAddress = MapViewOfFileEx(hmap2, FILE_MAP_WRITE | FILE_MAP_READ, 0, 0, 0, NULL); + memAddress = MapViewOfFileEx(hmap2, desiredAccess, 0, 0, 0, NULL); if (!memAddress) ereport(FATAL, (errmsg("could not create shared memory segment: error code %lu", GetLastError()),
signature.asc (application/pgp-signature, 832B) download

view thread (25+ messages)  latest in thread

Message-ID: <YkQMyhsP7xgLDDT/@paquier.xyz>
Permalink:  ../YkQMyhsP7xgLDDT%2F@paquier.xyz/
Also on:    postgresql.org/message-id/YkQMyhsP7xgLDDT/@paquier.xyz

 ·  · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-bugs@postgresql.org
  Cc: michael@paquier.xyz, thomas.munro@gmail.com, rjuju123@gmail.com
  Subject: Re: BUG #17448: In Windows 10, version 1703 and later, huge_pages doesn't work.
  In-Reply-To: <YkQMyhsP7xgLDDT/@paquier.xyz>
  References: <17448-0a96583a67edb1f7@postgresql.org> <Yj6oxYsBE5LXA8ZP@paquier.xyz> <20220326082408.moh55zuecxjmewnm@jrouhaud> <CA+hUKGLwOSf-QwVtCcKx9xvugN2BSzEBBeD5DNyTXgcSRp5sTw@mail.gmail.com> <YkQMyhsP7xgLDDT/@paquier.xyz>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox