Date: Mon, 4 Nov 2024 21:45:54 +0100
From: "Peter J. Holzer" <hjp-pgsql@hjp.at>
To: pgsql-general@lists.postgresql.org
Subject: Re: Used memory calculation in containers - docker stats and file
 cache
Message-ID: <20241104204554.gwjvyhqvayi5ec6k@hjp.at>
Mail-Followup-To: pgsql-general@lists.postgresql.org
References: <CAJ+5Ff6qH6hLW3MVSip0eCY7pFdHJQvmFrOXfKRMQkqYVfsMHw@mail.gmail.com>
 <20241101195617.xkexgtwuyeo2hvmx@hjp.at>
 <CAJ+5Ff5vEbwBLFw9LyDG2mmL6Ljkj_r-6iGujYFAUQnPkYP_KQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="h5boekgla2cujddv"
Content-Disposition: inline
In-Reply-To: <CAJ+5Ff5vEbwBLFw9LyDG2mmL6Ljkj_r-6iGujYFAUQnPkYP_KQ@mail.gmail.com>
Archived-At: <https://www.postgresql.org/message-id/20241104204554.gwjvyhqvayi5ec6k%40hjp.at>
Precedence: bulk


--h5boekgla2cujddv
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On 2024-11-04 14:35:23 +0100, Costa Alexoglou wrote:
> > I don't know if Docker does anything strange here.
>=20
> I am not sure if this is docker specific or cgroup comes into play.=A0
> The measurement is implemented in docker CLI, but I would make the
> assumption that the eviction is done within the cgroup scope.

I was trying to come up with possible *causes* for the eviction.


> > A large file (or many smaller files) which is cached is deleted
>=20
> The increase pattern is "incremental" until the huge eviction, and
> this is my question. Couldn't also the eviction happen incrementally
> rather than 15GB of file cache evicted on an instant?

It (usually) takes longer to write a file than to delete it.

If a temporary file is slowly written and then deleted after it is no
longer needed, you would see such a "sawtooth" as your screenshots show:
While the file is written, the kernel will cache the data on the
assumption that it will be read again sometime in the near future. But
when it is deleted, the kernel knows that it can't be read again - so it
will throw away all this (now useless) data.


> Seems like this issue, or the parent one=A0that everyone is linking to th=
is.

That seems to be just about the way it is reported, not the behaviour.

        hp

--=20
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp@hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"

--h5boekgla2cujddv
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEETtJbRjyPwVTYGJ5k8g5IURL+KF0FAmcpMncACgkQ8g5IURL+
KF0fqg//Sd3N9ZCic4C7s+tll9aLJx990BDzEhvRpJV1BylQFJMGjLqDyDtKENZE
HZ+mImT/GTJehhknADB1/67sTbJaQNOl3BTyAS6YvkCL9IcH8BKrSpnyAYfjDnCB
t8hljj4NbykwVHTVAqqErTwr81JIq6ZhMgXrDpcTZK316iiS4L2zMfcmJr7R953h
daSC481Wh5RixTYhcGnzw0FKLAihI+AZYIT5G28Mq6ANWdhJmToIhSjSOph0UCa/
JIAHsje+f5s9m2Eom1jeq4EpcieZ88W6ZvYwwtw6H5DLpBriI4NvQFhcgUHwXn00
2Kbm0UMCL1CBr4FG7h59TNxQaZiHLAQ/SPcys/V+wWq1RuRmRyCwQS9bS7abYjAP
3H9dhy8NlcBfCSpdfLFFxglwuzM1BCi4qs++P50M1gaeCrBCi4uT9axoPGVksmBd
Ul2R6bGJIT4K0XH3g3V6P6jZLxyXncMbEkkrsfPzW6XKyPWuYMxv1yQmlZKt2lTZ
ZVa62DEvyqAcOvPipZodiMyL5QNDHrHAk2VkfeCMDGS45k7bmEGJnQuELHTiCUys
0keFuFy2albCz5LRYw++WcSKPasJfVbXdXq+YgcrFWiJMSxQhRWac985F0FIZant
VccOwk2y57OmIe2r46KwvaRdV0MYcsmof+X+dnghNvnZIYLyFqg=
=Kxmg
-----END PGP SIGNATURE-----

--h5boekgla2cujddv--