Feedback-ID: i0fe9450f:Fastmail
Date: Wed, 6 Dec 2023 14:38:45 +0900
From: Michael Paquier <michael@paquier.xyz>
To: Matthias van de Meent <boekewurm+postgres@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>,
	Aleksander Alekseev <aleksander@timescale.com>,
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
	Nikita Malakhov <hukutoc@gmail.com>
Subject: Re: RFI: Extending the TOAST Pointer
Message-ID: <ZXAI5TH588Uq2yEm@paquier.xyz>
References: <CAN-LCVMq2X=fhx7KLxfeDyb3P+BXuCkHC0g=9GF+JD4izfVa0Q@mail.gmail.com>
 <CAJ7c6TNAYyeMYKVkiwOZChy7UpE_CkjpYOk73gcWTXMkLkEyzw@mail.gmail.com>
 <CAEze2WibtT+EdGscamqDm3+7qauWi_S6nVUhbK_qg1rkqN_48w@mail.gmail.com>
 <CA+TgmoaVcjUkmtWdc_9QjBzvSShjDBYk-5XFNaOvYLgGROjJMA@mail.gmail.com>
 <CAEze2Wg6TjMmOAOjaN9-J-rYt4nUT9qdE=DhqwRhj0j0Eyirpw@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="K9AqIcJSFJHzYtE+"
Content-Disposition: inline
In-Reply-To: <CAEze2Wg6TjMmOAOjaN9-J-rYt4nUT9qdE=DhqwRhj0j0Eyirpw@mail.gmail.com>
Archived-At: <https://www.postgresql.org/message-id/ZXAI5TH588Uq2yEm%40paquier.xyz>
Precedence: bulk


--K9AqIcJSFJHzYtE+
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Wed, May 24, 2023 at 11:50:21AM +0200, Matthias van de Meent wrote:
> I think you might not have picked up on what I was arguing for, but I
> agree with what you just said.
>=20
> My comment on not needing to invent a new ToastCompressionId was on
> the topic of adding capabilities^ to toast pointers that do things
> differently than the current TOAST and need more info than just sizes,
> 2x 32-bit ID and a compression algorithm.

I am not sure to understand why a new vartag is really required just
for the sake of compression when it comes to VARTAG_EXTERNAL, because
that's where the compression information is stored for ages.  The code
footprint gets more invasive, as well if you add more compression
methods as a new vartag implies more code areas to handle.

> ^ capabilities such as compression dictionaries (which would need to
> store a dictionary ID in the pointer), TOAST IDs that are larger than
> 32 bits, and other such advances.

Saying that, I don't really see why we cannot just do both, because it
is clear that many other projects want to fill in more data into
varlena headers for their own needs.  Hence, I would do:
1) Use the last bit of va_extinfo in varatt_external to link it more
info related to compression, and keep the compression information
close to varatt_external.
2) Add a new kind of "custom" vartag, for what any other requirements
want it to be.
--
Michael

--K9AqIcJSFJHzYtE+
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmVwCOUACgkQnvQgOdby
QH2eEw/+OSk+KcDrT9jotf18Y5IiySoctXWPYlU2MpY8zae56lPYOdGosNYkQ+BG
z/hKR+2yVEu5yC2ofR6k1poZCZODjMyvr38JEEXQI7+CRMpvKOMOg6ud0lvLDJF+
Ta2W12FcIlYBI1wEbUlDxE921pe61zxG5dvWn+eICHgeHfmzFmQguOFBmUFaSiH5
FHUZqf+PIKU4IZl9gAOUiIWV4kFgoCvTr1+NrGlswUwzvtl6DCc4FNbyUXqVZxSH
11ms9J7ydm6zpIaw4jtrEdT+QYJqsT8h+CAPxIFrE+vPAmE9aOpbjsBIKTddqwI3
/qJjssUzRRD/D/7My0KIXDv/F/zaq3Vy3N1URy4NjjQ7OotxRVkbTGook6mgaEAk
BMLEDf5Vjfrp+xUkccaMNdFidLfTkSnDaM50JyK1GZxwUWsL0K1fLypSA0d7Idat
goWE9ObVWbS/9eD4XBw9ZiV1ym0JUYnXGq2XImdNffS8CG1IS4PPCSNGxr8iFvje
FlGHqOJPg2TCtQBQ+jvN/q4UeqPm+o/FltN8RqRwK599mW5Q4VrkgExL+sb7o/GE
0SKrFtljL4geiyD3dKt+woiLz87wNFw/GbmUm0Lxs97MtpPX1ttJyiAIKddStODF
4AzugDXOqsd9nHZVDO9t8hAWu9Xl7ztQaDqlxqLEBPtZzZqSkrg=
=2Dk3
-----END PGP SIGNATURE-----

--K9AqIcJSFJHzYtE+--