MIME-Version: 1.0
References: <CAN-LCVMV3m+pzSDqOLyzgVRXA2zJ94n_T7kyAc2Z5JJpH018sA@mail.gmail.com>
 <20230206104917.sipa7nzue5lw2e6z@alvherre.pgsql> <CAN-LCVOv5is9dUn1U3KYy4hAhVK7tx6a7goOFmgbrGM7ZSAxRA@mail.gmail.com>
 <CAJ7c6TMZ7xySLzhygvzezPysyM5aZ6OjO8zHSUTO+GMBeL1a3g@mail.gmail.com> <CAEze2WhRJwr7MhkcckxpOAR5=6gUQQQmVdfPtvNu-u8J8RkzmA@mail.gmail.com>
In-Reply-To: <CAEze2WhRJwr7MhkcckxpOAR5=6gUQQQmVdfPtvNu-u8J8RkzmA@mail.gmail.com>
From: Aleksander Alekseev <aleksander@timescale.com>
Date: Tue, 7 Feb 2023 13:05:45 +0300
Message-ID: <CAJ7c6TOrdn8XxieKWFGCnGmEMDXeac_PArdP1Y6Zmio0jJHpkg@mail.gmail.com>
Subject: Re: Pluggable toaster
To: pgsql-hackers <pgsql-hackers@postgresql.org>
Cc: Alvaro Herrera <alvherre@alvh.no-ip.org>, 
	Matthias van de Meent <boekewurm+postgres@gmail.com>, Nikita Malakhov <hukutoc@gmail.com>, 
	vignesh C <vignesh21@gmail.com>, Jacob Champion <jchampion@timescale.com>, 
	Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>, Greg Stark <stark@mit.edu>, 
	Teodor Sigaev <teodor@sigaev.ru>
Content-Type: text/plain; charset="UTF-8"
Archived-At: <https://www.postgresql.org/message-id/CAJ7c6TOrdn8XxieKWFGCnGmEMDXeac_PArdP1Y6Zmio0jJHpkg%40mail.gmail.com>
Precedence: bulk

Hi,

> I believe that is a misrepresentation of the situation. ZSON had
> (has?) several systemic issues and could not be accepted to /contrib/
> in the way it was implemented, and it was commented that it would make
> sense that the feature of compression assisted by dictionaries would
> be implemented in core. Yet still, that feature is only closely
> related to pluggable TOAST, but it is not the same as making TOAST
> pluggable.
>
> > * The community wants the feature to have a simple implementation. You
> > said yourself that the idea of type-aware TOASTers is very invasive,
> > and I completely agree.
>
> I'm not sure that this is correct either. Compression (and TOAST) is
> inherently complex, and I don't think we should reject improvements
> because they are complex.
> The problem that I see being raised here, is that there was little
> discussion and no observed community consensus about the design of
> this complex feature *before* this patch with high complexity was
> provided.

Strictly speaking there is no such thing as "community opinion". There
are different people, everyone has their own opinion. To make things
more interesting the opinions change with time.

I did my best to make a brief summary of 100+ messages from different
people in something like 4 threads. These are things that were
requested and/or no one disagrees with (at least no one said "no, put
all this out of the core! and make it complicated too!"). Focusing on
something (almost) no one disagrees with seems to be more productive
than arguing about something everyone disagrees with.

As I see it, the goal is not to be right, but rather to find a
consensus most of us will be not unhappy with.

> The next action that was requested is to take a step back and decide
> how we would want to implement type-aware TOASTing (and the associated
> patch compression dictionaries) *before* we look into the type-aware
> toasting.

Yes, I thought we already agreed to forget about type-aware TOASTing
and compression dictionaries, and are looking for a consensus now.

To clarify, I don't think that pluggable TOASTing is an absolutely bad
idea. We are just not discussing this particular idea anymore, at
least for now.

> > * People also want this to be simple from the user perspective, as
> > simple as just CREATE COMPRESSED TABLE ... [USING lz4|zstd];
>
> Could you provide a reference to this? I have yet to see a COMPRESSED
> TABLE feature or syntax, let alone users asking for TOAST to be as
> easily usable as that feature or syntax.

I was referring to the recent discussion of the new RFC. Please see
[1] and below.

[1]: https://www.postgresql.org/message-id/flat/20230203095540.zutul5vmsbmantbm%40alvherre.pgsql#7cce6acef0cb7eb2490715ec9d835e74

-- 
Best regards,
Aleksander Alekseev