From: "Shinoda, Noriyoshi (PN Japan FSIP)" <noriyoshi.shinoda@hpe.com>
To: "masao.fujii@oss.nttdata.com" <masao.fujii@oss.nttdata.com>
CC: "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>,
        "rjuju123@gmail.com" <rjuju123@gmail.com>,
        "tgl@sss.pgh.pa.us"
	<tgl@sss.pgh.pa.us>,
        Kyotaro Horiguchi <horikyota.ntt@gmail.com>,
        "pryzby@telsasoft.com" <pryzby@telsasoft.com>
Subject: RE: Improve logging when using Huge Pages
Thread-Topic: Improve logging when using Huge Pages
Thread-Index: 
 AdeeKUJFq/9tpjh3SqCu3Xs3TusIyQAYT9OAAIBdmEAADznEVwABup+AAAR68QAArqbGIAAu2UB/AAtiutA=
Date: Wed, 8 Sep 2021 07:52:35 +0000
Message-ID: 
 <DF4PR8401MB1147ADD49D7236F6FBDCABF5EED49@DF4PR8401MB1147.NAMPRD84.PROD.OUTLOOK.COM>
References: 
 <TU4PR8401MB115257079749D8958A3450F7EED39@TU4PR8401MB1152.NAMPRD84.PROD.OUTLOOK.COM>
	<b707355a-84ba-9f8b-6026-8fac36121279@oss.nttdata.com>
	<20210907131653.GK26465@telsasoft.com>
 <20210908.111736.355857044349579789.horikyota.ntt@gmail.com>
In-Reply-To: <20210908.111736.355857044349579789.horikyota.ntt@gmail.com>
Accept-Language: ja-JP, en-US
Content-Language: ja-JP
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: 
 =?us-ascii?Q?sxPbvFCpzC8v3o9+qhYv8IBIyqAZO14hXKsFL4GJsV00in5E5lZgPY8EvHuD?=
 =?us-ascii?Q?X/MnQZraFqY3BpftNe2LqhVXrYAi6yQLLs3tz2ykz9ZWOzre+paBvOp/HeFA?=
 =?us-ascii?Q?mNY+rpleXsVwAjShR4TKIyE+COBymPHLA1X1IQa5PV/zAZvlV+KfmE/R+ZXF?=
 =?us-ascii?Q?srrtmsiFeEoVEhiZir2XzSTqkUTBgX3pQq0BpllO/FraO8qePz6gk4B94n4U?=
 =?us-ascii?Q?xFDGp98mcJJXOQRV+W9p316I4k7gQYT5VN2A2S6str2Jcmr5LHY+1ucowp3C?=
 =?us-ascii?Q?ND/6Yg3GtcPhzzNAQko0NDgfD7UNQ34lVuO7fFyvX6Y+faA/7oz8/QAxEpZY?=
 =?us-ascii?Q?PqAw5dbaAqKe2rmBbdLuV7SarKCHU7xIE2oh9x5yQ7Z/2j7heugRh0FlNkV5?=
 =?us-ascii?Q?qeD6CBQyLeiUL+mpor8UmkiLbVP3C6SpFyQlY8yc/9tooOq1CfdADYlsVj1K?=
 =?us-ascii?Q?ChA8gTAitq4syg+XPBiMVJ27NkT9Lo/FUe/iTShShM7Ez/QKc2UpAl1Gp11i?=
 =?us-ascii?Q?4Bp9pLgy5gKcknwHPWh2kBkS+ONpr2ooLLbhEt4ngMO3G2KaKpfBVoB61HSJ?=
 =?us-ascii?Q?qYKL83Tgs022O92WUMenO0n8nw7tMExl5jjfewdbHlsbKkPmPn8zwxk0RWH2?=
 =?us-ascii?Q?ClG6pVz4+DKV+OlYoCXgaTjUnaJnNt6AZWoSag0LUV4jDbC1Vlabp1+4fSE2?=
 =?us-ascii?Q?OckO+iCPZclGNnMcb41wEgneikKjj19XakAqigAKJ0CX+VX0MGYHkKWYiQ1+?=
 =?us-ascii?Q?oV0JSORD/o5zM0y3UlgogYHIlPAJX1LYvMytIBdgOZphsAK+gapqJ/SZxN1e?=
 =?us-ascii?Q?DsQou1x4+uD/poxteHgp4NFDiLfhPVEmEL56ZhxutLJP3KtcI8Gn3jm9MIqu?=
 =?us-ascii?Q?f1VNIjl53/hyo2JFeHsqfHWZYbGhfZaEbmwKgyyc+r8DFmZQoP/GJplf3+CX?=
 =?us-ascii?Q?xMKlwbEL4ttK69KPgDfTCdn28XW8A7vVdxS0o6/W5D0QyTlDlDqfY0CJrhXu?=
 =?us-ascii?Q?hWTtdesU58L0LsDmb5+7mqAo5dcKDxAXbaWa+T4QzheuYe1ooEo2TsJoX9Ff?=
 =?us-ascii?Q?M3Lh41/OKCkp/j3bCrIxldCgb0Bj8/YpNOkIppJEI4T2FLzrfSM+l5g+f4XZ?=
 =?us-ascii?Q?bI4Tmx/NalO2gG24FxeIQ0UVByV+VsNlpvG45uznDEA57mpAa0G/nBj5qweh?=
 =?us-ascii?Q?d698eZuHAxcfv+jqqo2JhSp+IIzk/+vpte77ULOY8QawYyZvZg+qWIstSJpv?=
 =?us-ascii?Q?+keP/byriItoiJ0iKu93ZaIDUh7gcmiJWTcNjh98MwmFaZZWhUyxAVg+5Sf4?=
 =?us-ascii?Q?yHh/o7ctw7wI5xY08l8aWpgo?=
Content-Type: multipart/mixed;
	boundary="_002_DF4PR8401MB1147ADD49D7236F6FBDCABF5EED49DF4PR8401MB1147_"
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: 
 DF4PR8401MB1147.NAMPRD84.PROD.OUTLOOK.COM
X-MS-Exchange-CrossTenant-Network-Message-Id: 
 930f0bed-f5af-459f-2c5a-08d9729d9f15
X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Sep 2021 07:52:35.2263
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 105b2061-b669-4b31-92ac-24d304d195dc
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 
 6YkS3M29TVMaMKjobNJr9wt8HON/BF8OEhM9bIjN1f0j6M5VybUCfjA8XT9CF6HpWo10bVm+JyjlITbGUUMnRtTyKbLr+cCCw9OfVRHiARk=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DF4PR8401MB0444
X-OriginatorOrg: hpe.com
X-Proofpoint-ORIG-GUID: Wa4j0c7SxPelIvjd2QfJITVy2L7EjN4Y
X-Proofpoint-GUID: Wa4j0c7SxPelIvjd2QfJITVy2L7EjN4Y
X-Proofpoint-UnRewURL: 0 URL was un-rewritten
MIME-Version: 1.0
X-HPE-SCL: -1
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.790
 definitions=2021-09-08_02:2021-09-07,2021-09-08 signatures=0
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0
 spamscore=0
 priorityscore=1501 impostorscore=0 mlxscore=0 clxscore=1015 suspectscore=0
 mlxlogscore=999 lowpriorityscore=0 adultscore=0 bulkscore=0 malwarescore=0
 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109080048
List-Id: <pgsql-hackers.lists.postgresql.org>
List-Help: <https://lists.postgresql.org/manage/>
List-Subscribe: <https://lists.postgresql.org/manage/>
List-Post: <mailto:pgsql-hackers@lists.postgresql.org>
List-Owner: <mailto:pgsql-hackers-owner@lists.postgresql.org>
List-Archive: <https://www.postgresql.org/list/pgsql-hackers>
Archived-At: 
 <https://www.postgresql.org/message-id/DF4PR8401MB1147ADD49D7236F6FBDCABF5EED49%40DF4PR8401MB1147.NAMPRD84.PROD.OUTLOOK.COM>
Precedence: bulk

--_002_DF4PR8401MB1147ADD49D7236F6FBDCABF5EED49DF4PR8401MB1147_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

Hello,

Thank you everyone for comments.
I have attached a patch that simply changed the message like the advice fro=
m Horiguchi-san.

> Even with the patch, there are still some cases where huge pages is disab=
led silently. We should report something even in these cases?
> For example, in the platform where huge pages is not supported, it's sile=
ntly disabled when huge_pages=3Dtry.

The area where this patch is written is inside the "#ifdef MAP_HUGETLB #end=
if" block.
For this reason, I think it is excluded from binaries created in an environ=
ment that does not have the MAP_HUGETLB macro.

> One big concern about the patch is that log message is always reported wh=
en shared memory fails to be allocated with huge pages enabled when huge_pa=
ges=3Dtry. Since=20
> huge_pages=3Dtry is the default setting, many users would see this new lo=
g message whenever they start the server. Those who don't need huge pages b=
ut just use the default=20
> setting might think that such log messages would be noisy.

This patch is meant to let the admin know that HugePages isn't being used, =
so I'm sure you're right. I have no idea what to do so far.

Regards,
Noriyoshi Shinoda

-----Original Message-----
From: Kyotaro Horiguchi [mailto:horikyota.ntt@gmail.com]=20
Sent: Wednesday, September 8, 2021 11:18 AM
To: pryzby@telsasoft.com
Cc: masao.fujii@oss.nttdata.com; Shinoda, Noriyoshi (PN Japan FSIP) <noriyo=
shi.shinoda@hpe.com>; pgsql-hackers@postgresql.org; rjuju123@gmail.com; tgl=
@sss.pgh.pa.us
Subject: Re: Improve logging when using Huge Pages

At Tue, 7 Sep 2021 08:16:53 -0500, Justin Pryzby <pryzby@telsasoft.com> wro=
te in=20
> On Tue, Sep 07, 2021 at 07:12:36PM +0900, Fujii Masao wrote:
> > One big concern about the patch is that log message is always=20
> > reported when shared memory fails to be allocated with huge pages=20
> > enabled when huge_pages=3Dtry. Since huge_pages=3Dtry is the default=20
> > setting, many users would see this new log message whenever they=20
> > start the server. Those who don't need huge pages but just use the=20
> > default setting might think that such log messages would be noisy.
>=20
> I don't see this as any issue.  We're only talking about a single=20
> message on each restart, which would be added in a major release.  If=20
> it's a problem, the message could be a NOTICE or INFO, and it won't be sh=
own by default.
>=20
> I think it should say "with/out huge pages" without=20
> "enabled/disabled", without "again", and without "The server", like:
>=20
> +                                       (errmsg("could not map anonymous =
shared memory (%zu bytes)"
> +                                               " with huge pages.", allo=
csize),
> +                                        errdetail("Anonymous shared memo=
ry will be mapped "
> +                                               "without huge=20
> + pages.")));

I don't dislike the message, but I'm not sure I like the message is too ver=
bose, especially about it has DETAILS. It seems to me something like the fo=
llowing is sufficient, or I'd like see it even more concise.

"fall back anonymous shared memory to non-huge pages: required %zu bytes fo=
r huge pages"

If we emit an error message for other than mmap failure, it would be like t=
he following.

"fall back anonymous shared memory to non-huge pages: huge pages not availa=
ble"

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center

--_002_DF4PR8401MB1147ADD49D7236F6FBDCABF5EED49DF4PR8401MB1147_
Content-Type: application/octet-stream; name="huge_pages_log_v4.diff"
Content-Description: huge_pages_log_v4.diff
Content-Disposition: attachment; filename="huge_pages_log_v4.diff"; size=1237;
	creation-date="Wed, 08 Sep 2021 07:44:33 GMT";
	modification-date="Wed, 08 Sep 2021 07:44:33 GMT"
Content-Transfer-Encoding: base64

ZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL3BvcnQvc3lzdl9zaG1lbS5jIGIvc3JjL2JhY2tlbmQv
cG9ydC9zeXN2X3NobWVtLmMKaW5kZXggOWRlOTZlZC4uY2Q5ODAwNyAxMDA2NDQKLS0tIGEvc3Jj
L2JhY2tlbmQvcG9ydC9zeXN2X3NobWVtLmMKKysrIGIvc3JjL2JhY2tlbmQvcG9ydC9zeXN2X3No
bWVtLmMKQEAgLTU5Miw4ICs1OTIsOSBAQCBDcmVhdGVBbm9ueW1vdXNTZWdtZW50KFNpemUgKnNp
emUpCiAJCQkJICAgUEdfTU1BUF9GTEFHUyB8IG1tYXBfZmxhZ3MsIC0xLCAwKTsKIAkJbW1hcF9l
cnJubyA9IGVycm5vOwogCQlpZiAoaHVnZV9wYWdlcyA9PSBIVUdFX1BBR0VTX1RSWSAmJiBwdHIg
PT0gTUFQX0ZBSUxFRCkKLQkJCWVsb2coREVCVUcxLCAibW1hcCglenUpIHdpdGggTUFQX0hVR0VU
TEIgZmFpbGVkLCBodWdlIHBhZ2VzIGRpc2FibGVkOiAlbSIsCi0JCQkJIGFsbG9jc2l6ZSk7CisJ
CQllcmVwb3J0KExPRywKKwkJCQkJKGVycm1zZygiZmFsbCBiYWNrIGFub255bW91cyBzaGFyZWQg
bWVtb3J5IHRvIG5vbi1odWdlICIKKwkJCQkJCSJwYWdlczogcmVxdWlyZWQgJXp1IGJ5dGVzIGZv
ciBodWdlIHBhZ2VzIiwgYWxsb2NzaXplKSkpOwogCX0KICNlbmRpZgogCmRpZmYgLS1naXQgYS9z
cmMvYmFja2VuZC9wb3J0L3dpbjMyX3NobWVtLmMgYi9zcmMvYmFja2VuZC9wb3J0L3dpbjMyX3No
bWVtLmMKaW5kZXggZDdhNzE5OS4uOWE5ZTk4YSAxMDA2NDQKLS0tIGEvc3JjL2JhY2tlbmQvcG9y
dC93aW4zMl9zaG1lbS5jCisrKyBiL3NyYy9iYWNrZW5kL3BvcnQvd2luMzJfc2htZW0uYwpAQCAt
MjkzLDkgKzI5Myw5IEBAIHJldHJ5OgogCQkJCWh1Z2VfcGFnZXMgPT0gSFVHRV9QQUdFU19UUlkg
JiYKIAkJCQkoZmxQcm90ZWN0ICYgU0VDX0xBUkdFX1BBR0VTKSAhPSAwKQogCQkJewotCQkJCWVs
b2coREVCVUcxLCAiQ3JlYXRlRmlsZU1hcHBpbmcoJXp1KSB3aXRoIFNFQ19MQVJHRV9QQUdFUyBm
YWlsZWQsICIKLQkJCQkJICJodWdlIHBhZ2VzIGRpc2FibGVkIiwKLQkJCQkJIHNpemUpOworCQkJ
CWVyZXBvcnQoTE9HLAorCQkJCQkJKGVycm1zZygiZmFsbCBiYWNrIGFub255bW91cyBzaGFyZWQg
bWVtb3J5IHRvIG5vbi1odWdlICIKKwkJCQkJCQkicGFnZXM6IHJlcXVpcmVkICV6dSBieXRlcyBm
b3IgaHVnZSBwYWdlcyIsIHNpemUpKSk7CiAKIAkJCQkvKgogCQkJCSAqIFVzZSB0aGUgb3JpZ2lu
YWwgc2l6ZSwgbm90IHRoZSByb3VuZGVkLXVwIHZhbHVlLCB3aGVuCg==

--_002_DF4PR8401MB1147ADD49D7236F6FBDCABF5EED49DF4PR8401MB1147_--