MIME-Version: 1.0
References: <CAN-LCVMq2X=fhx7KLxfeDyb3P+BXuCkHC0g=9GF+JD4izfVa0Q@mail.gmail.com>
 <CAJ7c6TNAYyeMYKVkiwOZChy7UpE_CkjpYOk73gcWTXMkLkEyzw@mail.gmail.com>
 <CAEze2WibtT+EdGscamqDm3+7qauWi_S6nVUhbK_qg1rkqN_48w@mail.gmail.com> <CA+TgmoaVcjUkmtWdc_9QjBzvSShjDBYk-5XFNaOvYLgGROjJMA@mail.gmail.com>
In-Reply-To: <CA+TgmoaVcjUkmtWdc_9QjBzvSShjDBYk-5XFNaOvYLgGROjJMA@mail.gmail.com>
From: Matthias van de Meent <boekewurm+postgres@gmail.com>
Date: Wed, 24 May 2023 11:50:21 +0200
Message-ID: <CAEze2Wg6TjMmOAOjaN9-J-rYt4nUT9qdE=DhqwRhj0j0Eyirpw@mail.gmail.com>
Subject: Re: RFI: Extending the TOAST Pointer
To: Robert Haas <robertmhaas@gmail.com>
Cc: Aleksander Alekseev <aleksander@timescale.com>, 
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Nikita Malakhov <hukutoc@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://www.postgresql.org/message-id/CAEze2Wg6TjMmOAOjaN9-J-rYt4nUT9qdE%3DDhqwRhj0j0Eyirpw%40mail.gmail.com>
Precedence: bulk

On Tue, 23 May 2023 at 18:34, Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Thu, May 18, 2023 at 8:06=E2=80=AFAM Matthias van de Meent
> <boekewurm+postgres@gmail.com> wrote:
> > This enum still has many options to go before it exceeds the maximum
> > of the uint8 va_tag field. Therefore, I don't think we have no disk
> > representations left, nor do I think we'll need to add another option
> > to the ToastCompressionId enum.
> > As an example, we can add another VARTAG option for dictionary-enabled
> > external toast; like what the pluggable toast patch worked on. I think
> > we can salvage some ideas from that patch, even if the main idea got
> > stuck.
>
> Adding another VARTAG option is somewhat different from adding another
> ToastCompressionId. I think that right now we have embedded in various
> places the idea that VARTAG_EXTERNAL is the only thing that shows up
> on disk, and we'd need to track down all such places and adjust them
> if we add other VARTAG types in the future. Depending on how it is to
> be used, adding a new ToastCompressionId might be less work. However,
> I don't think we can use the possibility of adding a new VARTAG value
> as a reason why it's OK to use up the last possible ToastCompressionId
> value for something non-extensible.

I think you might not have picked up on what I was arguing for, but I
agree with what you just said.

My comment on not needing to invent a new ToastCompressionId was on
the topic of adding capabilities^ to toast pointers that do things
differently than the current TOAST and need more info than just sizes,
2x 32-bit ID and a compression algorithm.

^ capabilities such as compression dictionaries (which would need to
store a dictionary ID in the pointer), TOAST IDs that are larger than
32 bits, and other such advances.

Kind regards,

Matthias van de Meent
Neon, Inc.