Feedback-ID: i0fe9450f:Fastmail
Date: Tue, 31 Dec 2024 10:06:45 +0900
From: Michael Paquier <michael@paquier.xyz>
To: Tomas Vondra <tomas@vondra.me>
Cc: wenhui qiu <qiuwenhuifx@gmail.com>, Robert Treat <rob@xzilla.net>,
	PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: PoC: history of recent vacuum/checkpoint runs (using new hooks)
Message-ID: <Z3NDpUslYaRW4Yo4@paquier.xyz>
References: <b68ab452-c41f-4d04-893f-eaab84f1855b@vondra.me>
 <CAGjGUAJSg3tJGxjeyU6T98ZYCXcAiGpq0eitAUQVaOa+nULDuA@mail.gmail.com>
 <8df7cee1-31aa-4db3-bbb7-83157ca139da@vondra.me>
 <CABV9wwPcnoS+=27o=pHinCy7fegTTfB=yECyK5XyGecCFfrFHg@mail.gmail.com>
 <CAGjGUAKXjFw+zq+9GZ+y5p360pkO7UuTojkKuB6k3MtdDCEf6w@mail.gmail.com>
 <a40fc9ea-ec9a-4f34-a5d5-42629c76fbb2@vondra.me>
 <Z24mZZAPKkw02oy2@paquier.xyz>
 <00d42567-22bd-4ed2-b190-188e9c151e28@vondra.me>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="oa5H/Kg8kVK1GI/B"
Content-Disposition: inline
In-Reply-To: <00d42567-22bd-4ed2-b190-188e9c151e28@vondra.me>
Archived-At: <https://www.postgresql.org/message-id/Z3NDpUslYaRW4Yo4%40paquier.xyz>
Precedence: bulk


--oa5H/Kg8kVK1GI/B
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Sat, Dec 28, 2024 at 02:25:16AM +0100, Tomas Vondra wrote:
> And the more I think about it the more I'm convinced we don't need to
> keep the data about past runs in memory, a file should be enough (except
> maybe for a small buffer). That would mean we don't need to worry about
> dynamic shared memory, etc. I initially rejected this because it seemed
> like a regression to how pgstat worked initially (sharing data through
> files), but I don't think that's actually true - this data is different
> (almost append-only), etc.

Right, I was looking a bit at 0003 that introduces the extension.  I
am wondering if we are not repeating the errors of pgss by using a
different file, and if we should not just use pgstats and its single=20
file instead to store this data through an extension.  You are right
that as an append-only pattern using the dshash of pgstat does not fit
well into this picture.   How about the second type of stats kinds:
the fixed-numbered stats kind?  These allocate a fixed amount of
shared memory, meaning that you could allocate N entries of history
and just manage a queue of them, then do a memcpy() of the whole set
if adding new history at the head of the queue, or just append new
ones at the tail of the queue in shmem, memcpy() once the queue is
full.  The extension gets simpler:
- No need to manage a new file, flush of the stats is controlled by
pgstats itself.
- The extension could register a fixed-numbered custom stats kind.
- Push the stats with the new hooks.
- Perhaps less useful, but it is possible to control the timing where
the data is pushed.
- Add SQL wrappers on top to fetch the data from pgstat.
--
Michael

--oa5H/Kg8kVK1GI/B
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmdzQ6UACgkQnvQgOdby
QH1EvhAAnJjKYc2GbwODwDXvagOsuee0041bN1mNyoauL2iNmH9W8PgSeCVU5msf
14jUpVyBkpvhcK8t/DDPi1UI/i8QsMVgDjQAR/3bbNipGIn3UImgc/PcFBdYZauP
W+HGtYm5XE4bDdRS4rKQ3M6aKsBRX5+LwBsZh6qaUVIPT+QtoxhK0U04HOEDs1ls
Q5xjZfQ5dji4UDQCAaYL18zZZBUcBA5LU46+2UWRSygUz8r9aqaKsohc5Q0gVK3u
+I5kqybq0Jf/nuF7zsadtJzsxXRXvgvrCBarHyB9GUD6TClPYMcA+vlsXHh/OgRS
KqPVtxG8ZhujYFcTcwYuGYd+vI+hdOv7djMEcS3ilYVdJ/3wPt4zkhF8oKNsWAJh
wphLqmDLpqItYSP7fHFITkWBB31/+n3svaMwlW8wnVIjWO6AEQHnG72RJbax4iDC
w61q1igAB8bNMXqfqoOjLjXdrZaK3S7WXtnQS9aoYRekj+vIFTRIGAxZw7Z28UYv
FZWJrtgRy+LXFFVAsdSZcOHw6VczITZh648QOvN9VF/0l7lYwl3mwMnHRlhG5EgR
m4WjXq98gUECnPCVeF3+5cOrfntJ1EtliIiL6bSahQ3gKJW1lVYhrdOTlXECP2DY
eBzvTykGoUeUz0+AiUyh0rD90VGaZM5onDpoRfXzhzWBda8Efbk=
=srQ8
-----END PGP SIGNATURE-----

--oa5H/Kg8kVK1GI/B--