MIME-Version: 1.0
References: <CAJCjmtsEtKMyBxZDg3Do72fr+VVF0eVUdt1aBUctD9BVEt_LNg@mail.gmail.com>
 <CAM=sWa5eM2_M_jqdYnU45nOuB_R=V8i8koS_hX0cAFjjSChv+w@mail.gmail.com>
In-Reply-To: <CAM=sWa5eM2_M_jqdYnU45nOuB_R=V8i8koS_hX0cAFjjSChv+w@mail.gmail.com>
From: Bogdan Siara <bogdan.siara@gmail.com>
Date: Wed, 7 May 2025 18:36:20 +0200
Message-ID: <CAJCjmttdbh6TPYqr+4X5tr+m2m=-QgHq+xe92AOxnXaxMRS6RQ@mail.gmail.com>
Subject: Re: huge_pages=on cause could not map anonymous shared memory: Cannot
 allocate memory
To: Alicja Kucharczyk <zaledwie10minut@gmail.com>
Cc: pgsql-general@lists.postgresql.org
Content-Type: multipart/alternative; boundary="000000000000cbe3ec06348e560e"
Archived-At: <https://www.postgresql.org/message-id/CAJCjmttdbh6TPYqr%2B4X5tr%2Bm2m%3D-QgHq%2Bxe92AOxnXaxMRS6RQ%40mail.gmail.com>
Precedence: bulk

--000000000000cbe3ec06348e560e
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Alicja,
Thanks for your advice, now postgresql works fine with 'huge_pages=3Don'.
Regards
Bogdan

=C5=9Br., 7 maj 2025 o 14:17 Alicja Kucharczyk <zaledwie10minut@gmail.com>
napisa=C5=82(a):

> Hi Bogdan,
> The root cause here is that the number of huge pages you've configured
> (vm.nr_hugepages =3D 980) is not sufficient.
> Each huge page on your system is 2=E2=80=AFMB in size, so 980 pages give =
you
> roughly 1.96=E2=80=AFGB of memory (980 =C3=97 2=E2=80=AFMB). However, Pos=
tgreSQL is clearly
> requesting about 2.2=E2=80=AFGB of shared memory (specifically, 220410675=
2 bytes as
> shown in the error message you provided), which exceeds what's available
> through huge pages.
>
> That=E2=80=99s why PostgreSQL fails to start when huge_pages =3D on - it =
requires
> the entire shared memory segment to come from huge pages and refuses to
> fall back to regular ones.
>
> Earlier, you had the huge_pages setting commented out, which means
> PostgreSQL used the default value: huge_pages =3D try. In that mode, it f=
irst
> attempts to use huge pages, but if that fails (like in your case due to
> insufficient allocation), it falls back to standard memory pages =E2=80=
=94 which is
> why the instance started without issues then.
>
> To fix the issue, you should increase vm.nr_hugepages to at least 1100 to
> fully cover the shared memory request (you can go a bit higher to be safe
> and then reduce it as described in the article I'm pasting the link to).
>
> Also, a side note: max_connections =3D 1000 is quite high for an instance
> with 8=E2=80=AFGB of RAM and only 2 vCPUs. Even if huge pages are properl=
y
> allocated, such a high number of connections can lead to performance
> issues. You might want to consider lowering it or using a connection pool=
er
> like PgBouncer.
>
> If you=E2=80=99d like to understand how huge pages work in PostgreSQL, in=
cluding
> how to calculate memory needs and configure the OS properly, I wrote a
> detailed article some time ago (still valid). It=E2=80=99s in Polish, whi=
ch I
> assume is fine for you:
> https://linuxpolska.com/pl/baza-wiedzy/blog/postgres-pamieci-ram-tipstric=
ks/
>
> best regards,
> Alicja Kucharczyk
>
>>

--000000000000cbe3ec06348e560e
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Alicja,<div>Thanks for your advice, now postgresql work=
s fine with &#39;huge_pages=3Don&#39;.</div><div>Regards</div><div>Bogdan</=
div></div><br><div class=3D"gmail_quote gmail_quote_container"><div dir=3D"=
ltr" class=3D"gmail_attr">=C5=9Br., 7 maj 2025 o 14:17=C2=A0Alicja Kucharcz=
yk &lt;<a href=3D"mailto:zaledwie10minut@gmail.com">zaledwie10minut@gmail.c=
om</a>&gt; napisa=C5=82(a):<br></div><blockquote class=3D"gmail_quote" styl=
e=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);paddin=
g-left:1ex"><div dir=3D"ltr"><div>Hi Bogdan,</div><div>The root cause here =
is that the number of huge pages you&#39;ve configured (vm.nr_hugepages =3D=
 980) is not sufficient.<br>Each huge page on your system is 2=E2=80=AFMB i=
n size, so 980 pages give you roughly 1.96=E2=80=AFGB of memory (980 =C3=97=
 2=E2=80=AFMB). However, PostgreSQL is clearly requesting about 2.2=E2=80=
=AFGB of shared memory (specifically, 2204106752 bytes as shown in the erro=
r message you provided), which exceeds what&#39;s available through huge pa=
ges.<br><br>That=E2=80=99s why PostgreSQL fails to start when huge_pages =
=3D on - it requires the entire shared memory segment to come from huge pag=
es and refuses to fall back to regular ones.<br><br>Earlier, you had the hu=
ge_pages setting commented out, which means PostgreSQL used the default val=
ue: huge_pages =3D try. In that mode, it first attempts to use huge pages, =
but if that fails (like in your case due to insufficient allocation), it fa=
lls back to standard memory pages =E2=80=94 which is why the instance start=
ed without issues then.<br><br>To fix the issue, you should increase vm.nr_=
hugepages to at least 1100 to fully cover the shared memory request (you ca=
n go a bit higher to be safe and then reduce it as described in the article=
 I&#39;m pasting the link to).<br><br>Also, a side note: max_connections =
=3D 1000 is quite high for an instance with 8=E2=80=AFGB of RAM and only 2 =
vCPUs. Even if huge pages are properly allocated, such a high number of con=
nections can lead to performance issues. You might want to consider lowerin=
g it or using a connection pooler like PgBouncer.<br><br>If you=E2=80=99d l=
ike to understand how huge pages work in PostgreSQL, including how to calcu=
late memory needs and configure the OS properly, I wrote a detailed article=
 some time ago (still valid). It=E2=80=99s in Polish, which I assume is fin=
e for you: <a href=3D"https://linuxpolska.com/pl/baza-wiedzy/blog/postgres-=
pamieci-ram-tipstricks/" target=3D"_blank">https://linuxpolska.com/pl/baza-=
wiedzy/blog/postgres-pamieci-ram-tipstricks/</a></div><div><br></div><div>b=
est regards,</div><div>Alicja Kucharczyk</div><div class=3D"gmail_quote"><b=
lockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-le=
ft:1px solid rgb(204,204,204);padding-left:1ex">
</blockquote></div></div>
</blockquote></div>

--000000000000cbe3ec06348e560e--