References: <87ttoyihgm.fsf@163.com>
 <CAEze2Wj2TFZdddFe5dw6Wgudan3__gBHeTuHnZg9-Kv6tJ_z4g@mail.gmail.com>
User-agent: mu4e 1.10.7; emacs 29.1
From: zhihuifan1213@163.com
To: Matthias van de Meent <boekewurm+postgres@gmail.com>
Cc: pgsql-hackers@lists.postgresql.org
Subject: Re: Avoid detoast overhead when possible
Date: Mon, 04 Dec 2023 20:55:05 +0800
In-reply-to: <CAEze2Wj2TFZdddFe5dw6Wgudan3__gBHeTuHnZg9-Kv6tJ_z4g@mail.gmail.com>
Message-ID: <87a5qqhzka.fsf@163.com>
MIME-Version: 1.0
Content-Type: text/plain
Archived-At: <https://www.postgresql.org/message-id/87a5qqhzka.fsf%40163.com>
Precedence: bulk


Hi,

Matthias van de Meent <boekewurm+postgres@gmail.com> writes:

> On Mon, 4 Dec 2023 at 07:56, <zhihuifan1213@163.com> wrote:

> ..It would also add overhead when
> we write results to disk, such as spilling merge sorts, hash join
> spills, or CTE materializations.
>
> Could you find a way to reduce this memory and IO usage when the value
> is not going to be used immediately? Using the toast pointer at such
> points surely will be cheaper than storing the full value again and
> again.

I'm not sure I understand you correctly, I think the issue you raised
here is covered by the below design (not implemented in the patch).

"
However this patch just throws away almost all the benefits of toast, so
how can we draw a line between should vs should not do this code path?
IMO, we should only run the 'eagerly detoast' when we know that we will
have a FuncCall against the toast_col on **the current plan node**. I
think this information can be get from Qual and TargetList. If so, we
can set the slot->detoast_attrs accordingly.
"

Let's see an example of this:

SELECT f(t1.toastable_col) FROM t1 join t2 using(c);

Suppose it is using hash join and t1 should be hashed.  With the above
design, we will NOT detoast toastable_col at the scan of t1 or hash t1
since there is no one "funcall" access it in either SeqScan of t1 or
hash (t1). But when we do the projection on the joinrel, the detoast
would happen.

I'm still working on how to know if a toast_col will be detoast for a
given PlanState. If there is no design error, I think I can work out a
version tomorrow.

-- 
Best Regards
Andy Fan