Date: Mon, 23 Jan 2023 19:21:00 -0600
From: Justin Pryzby <pryzby@telsasoft.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>, thomas.munro@gmail.com,
	tgl@sss.pgh.pa.us, andres@anarazel.de, john.naylor@enterprisedb.com,
	noriyoshi.shinoda@hpe.com, jchampion@timescale.com,
	sawada.mshk@gmail.com, masao.fujii@oss.nttdata.com,
	pgsql-hackers@postgresql.org, rjuju123@gmail.com
Subject: Re: Improve logging when using Huge Pages
Message-ID: <20230124012100.GB13860@telsasoft.com>
References: <20221107151458.6puzamerxti2y5in@awork3.anarazel.de>
 <3239882.1667836576@sss.pgh.pa.us>
 <CA+hUKGKRfzOH9gji-W7j=iKDQ50AjtGSsUfybNxeNVk+uy2gcg@mail.gmail.com>
 <20221109.114757.661491395677294771.horikyota.ntt@gmail.com>
 <Y2s0wCFgDv4LXzIW@paquier.xyz>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="UlVJffcvxoiEqYs2"
Content-Disposition: inline
In-Reply-To: <Y2s0wCFgDv4LXzIW@paquier.xyz>
User-Agent: Mutt/1.9.4 (2018-02-28)
Archived-At: 
 <https://www.postgresql.org/message-id/20230124012100.GB13860%40telsasoft.com>
Precedence: bulk


--UlVJffcvxoiEqYs2
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Wed, Nov 09, 2022 at 02:04:00PM +0900, Michael Paquier wrote:
> On Wed, Nov 09, 2022 at 11:47:57AM +0900, Kyotaro Horiguchi wrote:
> > Honestly I don't come up with other users of the new
> > log-level. Another possible issue is it might be a bit hard for people
> > to connect that level to huge_pages=try, whereas I think we shouldn't
> > put a description about the concrete impact range of that log-level.
> > 
> > I came up with an alternative idea that add a new huge_pages value
> > try_report or try_verbose, which tell postgresql to *always* report
> > the result of huge_pages = try.
> 
> Here is an extra idea for the bucket of ideas: switch the user-visible
> value of huge_pages to 'on' when we are at "try" but success in using
> huge pages, and switch the visible value to "off".  The idea of Justin
> in [1] to use an internal runtime-computed GUC sounds sensible, as well
> (say a boolean effective_huge_pages?).
> 
> [1]: https://www.postgresql.org/message-id/20221106130426.GG16921@telsasoft.com
> --
> Michael

Michael seemed to support this idea and nobody verbalized hatred of it,
so I implemented it.  In v15, we have shared_memory_size_in_huge_pages,
so this adds effective_huge_pages.

Feel free to suggest a better name.

-- 
Justin

--UlVJffcvxoiEqYs2
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
 filename="0001-add-GUC-effective_huge_pages.patch"

From 2bb0c48dfefff78325b1b9d31a3e54e982d44e4e Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 23 Jan 2023 18:33:51 -0600
Subject: [PATCH] add GUC: effective_huge_pages

This is useful to show the current state of huge pages when
huge_pages=try.
---
 doc/src/sgml/config.sgml            | 17 ++++++++++++++++-
 src/backend/port/sysv_shmem.c       | 12 +++++++++---
 src/backend/port/win32_shmem.c      |  4 ++++
 src/backend/utils/misc/guc_tables.c | 12 ++++++++++++
 4 files changed, 41 insertions(+), 4 deletions(-)

diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index f985afc009d..383139d5266 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -1708,7 +1708,8 @@ include_dir 'conf.d'
         server will try to request huge pages, but fall back to the default if
         that fails. With <literal>on</literal>, failure to request huge pages
         will prevent the server from starting up. With <literal>off</literal>,
-        huge pages will not be requested.
+        huge pages will not be requested.  The actual state of huge pages is
+        indicated by the server variable <xref linkend="guc-effective-huge-pages"/>.
        </para>
 
        <para>
@@ -10687,6 +10688,20 @@ dynamic_library_path = 'C:\tools\postgresql;H:\my_project\lib;$libdir'
       </listitem>
      </varlistentry>
 
+     <varlistentry id="guc-effective-huge-pages" xreflabel="effective_huge_pages">
+      <term><varname>effective_huge_pages</varname> (<type>boolean</type>)
+      <indexterm>
+       <primary><varname>effective_huge_pages</varname> configuration parameter</primary>
+      </indexterm>
+      </term>
+      <listitem>
+       <para>
+        Reports whether huge pages are in use by the current process.
+        See <xref linkend="guc-huge-pages"/> for more information.
+       </para>
+      </listitem>
+     </varlistentry>
+
      <varlistentry id="guc-integer-datetimes" xreflabel="integer_datetimes">
       <term><varname>integer_datetimes</varname> (<type>boolean</type>)
       <indexterm>
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.c
index eaba244bc9c..1e6adc97533 100644
--- a/src/backend/port/sysv_shmem.c
+++ b/src/backend/port/sysv_shmem.c
@@ -621,9 +621,15 @@ CreateAnonymousSegment(Size *size)
 		ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
 				   PG_MMAP_FLAGS | mmap_flags, -1, 0);
 		mmap_errno = errno;
-		if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)
-			elog(DEBUG1, "mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
-				 allocsize);
+		if (huge_pages == HUGE_PAGES_TRY)
+		{
+			if (ptr == MAP_FAILED)
+				elog(DEBUG1, "mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",
+					 allocsize);
+			else
+				SetConfigOption("effective_huge_pages", "on",
+						PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+		}
 	}
 #endif
 
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.c
index 62e08074770..8c5ffd7af4c 100644
--- a/src/backend/port/win32_shmem.c
+++ b/src/backend/port/win32_shmem.c
@@ -314,6 +314,10 @@ retry:
 						 errdetail("Failed system call was CreateFileMapping(size=%zu, name=%s).",
 								   size, szShareMem)));
 		}
+		else if (huge_pages == HUGE_PAGES_TRY)
+			SetConfigOption("effective_huge_pages", "on",
+					PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);
+
 
 		/*
 		 * If the segment already existed, CreateFileMapping() will return a
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 4454d57322a..88272765479 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -571,6 +571,7 @@ static int	shared_memory_size_mb;
 static int	shared_memory_size_in_huge_pages;
 static int	wal_block_size;
 static bool data_checksums;
+static bool	effective_huge_pages;
 static bool integer_datetimes;
 
 #ifdef USE_ASSERT_CHECKING
@@ -1972,6 +1973,17 @@ struct config_bool ConfigureNamesBool[] =
 		NULL, NULL, NULL
 	},
 
+	{
+		{"effective_huge_pages", PGC_INTERNAL, PRESET_OPTIONS,
+			gettext_noop("Indicates whether huge pages are in use."),
+			NULL,
+			GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_RUNTIME_COMPUTED
+		},
+		&effective_huge_pages,
+		false,
+		NULL, NULL, NULL
+	},
+
 	/* End-of-list marker */
 	{
 		{NULL, 0, 0, NULL, NULL}, NULL, false, NULL, NULL, NULL
-- 
2.25.1


--UlVJffcvxoiEqYs2--