From: Oleg Bartunov <obartunov@postgrespro.ru>
To: Michael Paquier <michael@paquier.xyz>
Cc: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Cc: Yugo Nagata <nagata@sraoss.co.jp>
Cc: Postgres hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: Support for 8-byte TOAST values, round two
Date: Mon, 14 Sep 2026 07:24:34 +0545
Message-ID: <72d00c47-33e4-4d35-9a3c-b335ef344afd@email.android.com> (raw)
In-Reply-To: <aqcvG0xsljDMQ1on@paquier.xyz>
<div dir='auto'><div><br><div><br><div class="elided-text">On Sep 14, 2026 05:02, Michael Paquier <michael@paquier.xyz> wrote:<br type="attribution"><blockquote style="margin:0 0 0 0.8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">On Sun, Sep 13, 2026 at 10:07:27PM +0545, Oleg Bartunov wrote:
<br>
> I did a quick check of out-of-tree users and found at least two
<br>
> examples. PostGIS has code like this in ToastCacheGetGeometry():
<br>
<br>
postgis holds a direct reference to varatt_external in its
<br>
libpgcommon/lwgeom_cache.c, which is wanted to feed data to
<br>
ToastCacheArgument. As far as I can see, their change is not
<br>
complicated, upgrade ToastCacheArgument.valueid to be an Oid8, then,
<br>
instead of the direct use of varatt_external() switch to a
<br>
toast_external_data combined with a call of toast_external_info_get()
<br>
to make the switch transparent, so as the TOAST value, Oid or Oid8 can
<br>
be retrieved.
<br>
<br>
So it's not a complicated change, and as far as I can see it's mostly
<br>
a matter of two PG_VERSION_NUM blocks plus a few tests. Using
<br>
varatt_external in the code would lead to a compilation failure,
<br>
because it's now renamed varatt_external_oid, one will not see a
<br>
runtime failure. It's going to be hard to miss that something is
<br>
wrong in this case.
<br>
<br>
> MobilityDB carries the same pattern. So an OID8 datum can now pass
<br>
> VARATT_IS_EXTERNAL_ONDISK() and existing code can then interpret it
<br>
> using the old OID pointer layout.
<br>
<br>
EXTERNAL_ONDISK() means either of them.
<br>
<br>
> I noticed this because we have been experimenting with alternative
<br>
> physical representations of large varlena values and tracing which
<br>
> parts of the value lifecycle actually depend on a particular TOAST
<br>
> representation. OID8 turned out to be a useful real-world test of that
<br>
> boundary.
<br>
<br>
MobilityDB is carrying the same patterns because it uses a copy-paste
<br>
of the postgis code. Again, this uses directly varatt_external, which
<br>
would fail at compilation under PG20. It sounds to me that they just
<br>
update their code on a regular basis based on postgis, so it means
<br>
that sending only a patch to adjust postgis would be enough. I'm
<br>
happy to take some time to do that, it</p></blockquote></div></div></div><div dir="auto">Thanks, that makes sense. I had missed that the varatt_external rename turns this particular case into a compile-time failure.</div><div dir="auto"><div><div class="elided-text"><blockquote style="margin:0 0 0 0.8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr"><br>
> Perhaps we need to distinguish explicitly between "any on-disk
<br>
> external pointer" and "the old OID on-disk pointer", rather than
<br>
> changing the meaning of the existing predicate?
<br>
<br>
It seems to me that this difference already exists in the patch, due
<br>
to the rebranding of varatt_external.
<br>
<br>
Note that if somebody has the idea to use VARTAG_SIZE() to copy
<br>
external varlena pointers, then the change is transparent.</p></blockquote></div></div></div><div dir="auto"><br></div><div dir="auto">Just to make sure I understand the intended abstraction correctly: with this patch, should I think of VARATT_IS_EXTERNAL_ONDISK() as identifying a family of on-disk external representations, while the vartag identifies the concrete physical representation within that family?</div><div dir="auto">In other words:</div><div dir="auto">VARATT_IS_EXTERNAL_ONDISK() -> representation family, and </div><div dir="auto">VARTAG_* -> concrete representation</div><div dir="auto">If so, that is actually the distinction I was looking for.</div><div dir="auto"><br></div><div dir="auto">Oleg </div><div dir="auto"><div><div class="elided-text"><blockquote style="margin:0 0 0 0.8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">
<br>
--
<br>
Michael
<br>
</p>
</blockquote></div><br></div></div></div>
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: obartunov@postgrespro.ru, michael@paquier.xyz, bharath.rupireddyforpostgres@gmail.com, nagata@sraoss.co.jp, pgsql-hackers@lists.postgresql.org
Subject: Re: Support for 8-byte TOAST values, round two
In-Reply-To: <72d00c47-33e4-4d35-9a3c-b335ef344afd@email.android.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox