Subject: Re: Pluggable toaster

From: Nikita Malakhov <[email-masked]>
Date: Mon, 6 Feb 2023 00:10:50 +0300

Hi hackers!

In response to opinion in thread on Compresson dictionaries for JSONb [1]
>The approaches are completely different,

>but they seem to be trying to fix the same problem: the fact that the
>default TOAST stuff isn't good enough for JSONB.

The problem, actually, is that the default TOAST is often not good for

modern loads and amounts of data.Pluggable TOAST is based not only

on pure enthusiasm, but on demands and tickets from production

databases.The main demand is effective and transparent storage subsystem

for large values for some problematic types of data, which we already have,

with proven efficiency.

So we're really quite surprised that it has got so little feedback. We've
got

some opinions on approach but there is no any general one on the approach

itself except doubts about the TOAST mechanism needs revision at all.

Currently we're busy revising the whole Pluggable TOAST API to make it

available as an extension and based on hooks to minimize changes in

the core. It will be available soon.

> [1]  https://postgr.es/m/[email-masked]
>
--
Nikita Malakhov
Postgres Professional
https://postgrespro.ru/

---

From: Alvaro Herrera <[email-masked]>
Date: Mon, 6 Feb 2023 11:49:17 +0100

> the core. It will be available soon.

Hmm, I'm not sure why would PGDG want to accept such a thing.  I read
"minimize changes" as "open source Postgres can keep their crap
implementation and companies will offer good ones for a fee".  I'd
rather have something that can give users direct benefit -- not hide it
behind proprietary licenses forcing each company to implement their own
performant toaster.

-- 
Álvaro Herrera               48°01'N 7°57'E  —  https://www.EnterpriseDB.com/

---

From: Nikita Malakhov <[email-masked]>
Date: Mon, 6 Feb 2023 16:38:01 +0300

Hi!

Existing TOAST has several very painful drawbacks - lack of UPDATE
operation, bloating of TOAST tables, and limits that are implicitly implied
on base tables by their TOAST tables, so it is seems not fair to say that
Pluggable TOAST does not solve any problems but just introduces new
ones.

The main reason behind this decision is that keeping the first
implementation
on the side of the vanilla (I mean rebasing it) over time is very difficult
due
to the very invasive nature of this solution.

So we decided to reduce changes in the core to the minimum necessary
to make it available through the hooks, because the hooks part is very
lightweight and simple to keep rebasing onto the vanilla core. We plan
to keep this extension free with the PostgreSQL license, so any PostgreSQL
user could benefit from the TOAST on steroids, and sometimes in the
future it will be a much simpler task to integrate the Pluggable TOAST into
the vanilla, along with our advanced TOAST implementations which
we plan to keep under Open Source licenses too.

-- 
Regards,
Nikita Malakhov
Postgres Professional
https://postgrespro.ru/

---

From: Andres Freund <[email-masked]>
Date: Mon, 6 Feb 2023 11:24:01 -0800

Hi,

On 2023-02-06 16:38:01 +0300, Nikita Malakhov wrote:
> lightweight and simple to keep rebasing onto the vanilla core.

At least I don't think we should accept such hooks. I don't think I am alone
in that.

Greetings,

Andres Freund

---

From: Matthias van de Meent <[email-masked]>
Date: Mon, 6 Feb 2023 21:11:45 +0100

> At least I don't think we should accept such hooks. I don't think I am alone

+1

Assuming type-aware TOASTing is the goal, I don't think hooks are the
right design to implement that.

-Matthias van de Meent