MIME-Version: 1.0
From: Koen De Groote <kdg.dev@gmail.com>
Date: Tue, 19 Nov 2024 23:22:18 +0100
Message-ID: <CAGbX52GNvsV-sNAS36LwmFOc2Qsn74ZA31T_GNJNeBYATJGBAw@mail.gmail.com>
Subject: Running docker in postgres, SHM size of the docker container in
 postgres 16
To: PostgreSQL General <pgsql-general@lists.postgresql.org>
Content-Type: multipart/alternative; boundary="0000000000000ddd4406274b7895"
Archived-At: <https://www.postgresql.org/message-id/CAGbX52GNvsV-sNAS36LwmFOc2Qsn74ZA31T_GNJNeBYATJGBAw%40mail.gmail.com>
Precedence: bulk

--0000000000000ddd4406274b7895
Content-Type: text/plain; charset="UTF-8"

Hello all,

Assume a machine with 16 CPU cores and 64GB of RAM. 300 max connections.

Postgres is running here, inside a docker container. Docker containers
receive a small amount of shared memory from /dev/shm

Experience teaches that the default 64MB of /dev/shm for the postgres
container, is not enough for databases of even 50GB in size. Previously
I've increased the /dev/shm setting to 256MB for the postgres container.
Issues that made this necessary all came in the form of too many queries
running at once, asking too much memory space, I assume.

I recently upgraded a setup from postgres 11 to postgres 16 via logical
replication, and found that running "vacuum(analyze, verbose)" fails unless
I give the container 1GB of /dev/shm. Where previously 256MB sufficed on
the old postgres 11 version.

Why would that be? It's the exact same data. The install is about 50GB in
size. Is there something wrong with postgres 16, or did some settings
significantly change, that I need to know about? I went over all the
changelogs, nothing stood out as relevant to this, but that might be a lack
of experience on my end.

Regards,
Koen De Groote

--0000000000000ddd4406274b7895
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div>Hello all,</div><div><br></div><div>Assume a machine =
with 16 CPU cores and 64GB of RAM. 300 max connections.<br></div><div><br><=
/div>Postgres is running here, inside a docker container. Docker containers=
 receive a small amount of shared memory from /dev/shm<br><div><div><br></d=
iv><div>Experience teaches that the default 64MB of /dev/shm for the postgr=
es container, is not enough for databases of even 50GB in size. Previously =
I&#39;ve increased the /dev/shm setting to 256MB for the postgres container=
. Issues that made this necessary all came in the form of too many queries =
running at once, asking too much memory space, I assume.<br></div><div><br>=
</div><div>I recently upgraded a setup from postgres 11 to postgres 16 via =
logical replication, and found that running &quot;vacuum(analyze, verbose)&=
quot; fails unless I give the container 1GB of /dev/shm. Where previously 2=
56MB sufficed on the old postgres 11 version.<br></div><div><br></div><div>=
Why would that be? It&#39;s the exact same data. The install is about 50GB =
in size. Is there something wrong with postgres 16, or did some settings si=
gnificantly change, that I need to know about? I went over all the changelo=
gs, nothing stood out as relevant to this, but that might be a lack of expe=
rience on my end.<br></div><div><br></div><div>Regards,</div><div>Koen De G=
roote<br></div></div></div>

--0000000000000ddd4406274b7895--