From: Justin Pryzby <pryzby@telsasoft.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: thomas.munro@gmail.com
Cc: tgl@sss.pgh.pa.us
Cc: andres@anarazel.de
Cc: john.naylor@enterprisedb.com
Cc: noriyoshi.shinoda@hpe.com
Cc: jchampion@timescale.com
Cc: sawada.mshk@gmail.com
Cc: masao.fujii@oss.nttdata.com
Cc: pgsql-hackers@postgresql.org
Cc: rjuju123@gmail.com
Subject: Re: Improve logging when using Huge Pages
Date: Mon, 23 Jan 2023 19:21:00 -0600
Message-ID: <20230124012100.GB13860@telsasoft.com> (raw)
In-Reply-To: <Y2s0wCFgDv4LXzIW@paquier.xyz>
References: <20221107151458.6puzamerxti2y5in@awork3.anarazel.de>
<3239882.1667836576@sss.pgh.pa.us>
<CA+hUKGKRfzOH9gji-W7j=iKDQ50AjtGSsUfybNxeNVk+uy2gcg@mail.gmail.com>
<20221109.114757.661491395677294771.horikyota.ntt@gmail.com>
<Y2s0wCFgDv4LXzIW@paquier.xyz>
On Wed, Nov 09, 2022 at 02:04:00PM +0900, Michael Paquier wrote:
> On Wed, Nov 09, 2022 at 11:47:57AM +0900, Kyotaro Horiguchi wrote:
> > Honestly I don't come up with other users of the new
> > log-level. Another possible issue is it might be a bit hard for people
> > to connect that level to huge_pages=try, whereas I think we shouldn't
> > put a description about the concrete impact range of that log-level.
> >
> > I came up with an alternative idea that add a new huge_pages value
> > try_report or try_verbose, which tell postgresql to *always* report
> > the result of huge_pages = try.
>
> Here is an extra idea for the bucket of ideas: switch the user-visible
> value of huge_pages to 'on' when we are at "try" but success in using
> huge pages, and switch the visible value to "off". The idea of Justin
> in [1] to use an internal runtime-computed GUC sounds sensible, as well
> (say a boolean effective_huge_pages?).
>
> [1]: https://www.postgresql.org/message-id/20221106130426.GG16921@telsasoft.com
> --
> Michael
Michael seemed to support this idea and nobody verbalized hatred of it,
so I implemented it. In v15, we have shared_memory_size_in_huge_pages,
so this adds effective_huge_pages.
Feel free to suggest a better name.
--
Justin
Attachments:
[text/x-diff] 0001-add-GUC-effective_huge_pages.patch (4.3K, ../20230124012100.GB13860@telsasoft.com/2-0001-add-GUC-effective_huge_pages.patch)
download | inline diff:
From 2bb0c48dfefff78325b1b9d31a3e54e982d44e4e Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Mon, 23 Jan 2023 18:33:51 -0600
Subject: [PATCH] add GUC: effective_huge_pages
This is useful to show the current state of huge pages when
huge_pages=try.
---
doc/src/sgml/config.sgml | 17 ++++++++++++++++-
src/backend/port/sysv_shmem.c | 12 +++++++++---
src/backend/port/win32_shmem.c | 4 ++++
src/backend/utils/misc/guc_tables.c | 12 ++++++++++++
4 files changed, 41 insertions(+), 4 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgmlindex f985afc009d..383139d5266 100644--- a/doc/src/sgml/config.sgml+++ b/doc/src/sgml/config.sgml@@ -1708,7 +1708,8 @@ include_dir 'conf.d'
server will try to request huge pages, but fall back to the default if
that fails. With <literal>on</literal>, failure to request huge pages
will prevent the server from starting up. With <literal>off</literal>,
- huge pages will not be requested.+ huge pages will not be requested. The actual state of huge pages is+ indicated by the server variable <xref linkend="guc-effective-huge-pages"/>.
</para>
<para>
@@ -10687,6 +10688,20 @@ dynamic_library_path = 'C:\tools\postgresql;H:\my_project\lib;$libdir'
</listitem>
</varlistentry>
+ <varlistentry id="guc-effective-huge-pages" xreflabel="effective_huge_pages">+ <term><varname>effective_huge_pages</varname> (<type>boolean</type>)+ <indexterm>+ <primary><varname>effective_huge_pages</varname> configuration parameter</primary>+ </indexterm>+ </term>+ <listitem>+ <para>+ Reports whether huge pages are in use by the current process.+ See <xref linkend="guc-huge-pages"/> for more information.+ </para>+ </listitem>+ </varlistentry>+
<varlistentry id="guc-integer-datetimes" xreflabel="integer_datetimes">
<term><varname>integer_datetimes</varname> (<type>boolean</type>)
<indexterm>
diff --git a/src/backend/port/sysv_shmem.c b/src/backend/port/sysv_shmem.cindex eaba244bc9c..1e6adc97533 100644--- a/src/backend/port/sysv_shmem.c+++ b/src/backend/port/sysv_shmem.c@@ -621,9 +621,15 @@ CreateAnonymousSegment(Size *size)
ptr = mmap(NULL, allocsize, PROT_READ | PROT_WRITE,
PG_MMAP_FLAGS | mmap_flags, -1, 0);
mmap_errno = errno;
- if (huge_pages == HUGE_PAGES_TRY && ptr == MAP_FAILED)- elog(DEBUG1, "mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",- allocsize);+ if (huge_pages == HUGE_PAGES_TRY)+ {+ if (ptr == MAP_FAILED)+ elog(DEBUG1, "mmap(%zu) with MAP_HUGETLB failed, huge pages disabled: %m",+ allocsize);+ else+ SetConfigOption("effective_huge_pages", "on",+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);+ }
}
#endif
diff --git a/src/backend/port/win32_shmem.c b/src/backend/port/win32_shmem.cindex 62e08074770..8c5ffd7af4c 100644--- a/src/backend/port/win32_shmem.c+++ b/src/backend/port/win32_shmem.c@@ -314,6 +314,10 @@ retry:
errdetail("Failed system call was CreateFileMapping(size=%zu, name=%s).",
size, szShareMem)));
}
+ else if (huge_pages == HUGE_PAGES_TRY)+ SetConfigOption("effective_huge_pages", "on",+ PGC_INTERNAL, PGC_S_DYNAMIC_DEFAULT);+
/*
* If the segment already existed, CreateFileMapping() will return a
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.cindex 4454d57322a..88272765479 100644--- a/src/backend/utils/misc/guc_tables.c+++ b/src/backend/utils/misc/guc_tables.c@@ -571,6 +571,7 @@ static int shared_memory_size_mb;
static int shared_memory_size_in_huge_pages;
static int wal_block_size;
static bool data_checksums;
+static bool effective_huge_pages;
static bool integer_datetimes;
#ifdef USE_ASSERT_CHECKING
@@ -1972,6 +1973,17 @@ struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
+ {+ {"effective_huge_pages", PGC_INTERNAL, PRESET_OPTIONS,+ gettext_noop("Indicates whether huge pages are in use."),+ NULL,+ GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE | GUC_RUNTIME_COMPUTED+ },+ &effective_huge_pages,+ false,+ NULL, NULL, NULL+ },+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, false, NULL, NULL, NULL
--
2.25.1
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: pryzby@telsasoft.com, michael@paquier.xyz, horikyota.ntt@gmail.com, thomas.munro@gmail.com, tgl@sss.pgh.pa.us, andres@anarazel.de, john.naylor@enterprisedb.com, noriyoshi.shinoda@hpe.com, jchampion@timescale.com, sawada.mshk@gmail.com, masao.fujii@oss.nttdata.com, rjuju123@gmail.com
Subject: Re: Improve logging when using Huge Pages
In-Reply-To: <20230124012100.GB13860@telsasoft.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox