Feedback-ID: i0c21471d:Fastmail
MIME-Version: 1.0
Date: Tue, 08 Apr 2025 23:21:31 -0300
From: "Euler Taveira" <euler@eulerto.com>
To: "Nathan Bossart" <nathandbossart@gmail.com>,
 "Amit Kapila" <amit.kapila16@gmail.com>
Cc: "Michael Paquier" <michael@paquier.xyz>,
 "Alexander Lakhin" <exclusion@gmail.com>, "Tom Lane" <tgl@sss.pgh.pa.us>,
 "Jonathan S. Katz" <jkatz@postgresql.org>,
 "PostgreSQL Hackers" <pgsql-hackers@lists.postgresql.org>
Message-Id: <4c2a6100-738e-48cc-8a11-2d85f95ae0ed@app.fastmail.com>
In-Reply-To: <Z_WGI4cKoTjhUXrs@nathan>
References: <Zw_hxaDoC3TzQYjj@nathan> <ZxGZB2d-RttHwmlt@paquier.xyz>
 <ZyKdCEaUB2lB1rG8@nathan> <ZyLFC24UpcIJJmwT@paquier.xyz>
 <Z0TQG6869v8t-hV5@nathan> <Z0a6IwjW36af71J7@paquier.xyz>
 <Z0dB1ld2iPcS6nC9@nathan>
 <CAA4eK1LbhNxhczGJwuGURURc2pHVhdbmcnstmNaGOvcvdnP40Q@mail.gmail.com>
 <Z-_sixINC08J_SQB@nathan>
 <CAA4eK1LeMM8royHj5UyMBDbspD7+_70AJkV4Uji122hmkn7jOw@mail.gmail.com>
 <Z_WGI4cKoTjhUXrs@nathan>
Subject: Re: Large expressions in indexes can't be stored (non-TOASTable)
Content-Type: multipart/alternative;
 boundary=555e4b82ec384b8a8230f9853ab628ad
Archived-At: 
 <https://www.postgresql.org/message-id/4c2a6100-738e-48cc-8a11-2d85f95ae0ed%40app.fastmail.com>
Precedence: bulk

--555e4b82ec384b8a8230f9853ab628ad
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On Tue, Apr 8, 2025, at 5:25 PM, Nathan Bossart wrote:
> On Tue, Apr 08, 2025 at 04:44:02PM +0530, Amit Kapila wrote:
> > On Fri, Apr 4, 2025 at 7:58=E2=80=AFPM Nathan Bossart <nathandbossar=
t@gmail.com> wrote:
> >> On Fri, Apr 04, 2025 at 05:16:43PM +0530, Amit Kapila wrote:
> >> > Can we dodge adding this push call if we restrict the length of t=
he
> >> > origin name to some reasonable limit like 256 or 512 and avoid the
> >> > need of toast altogether?
> >>
> >> We did consider just removing pg_replication_origin's TOAST table e=
arlier
> >> [0], but we decided against it at that time.  Maybe it's worth
> >> reconsidering...
> >=20
> > I don't see any good reason in that email for not removing the TOAST
> > table for pg_replication_origin. I would prefer to remove it rather
> > than add protection related to its access.
>=20
> The only reason I can think of is that folks might have existing
> replication origins with extremely long names that would cause upgrade=
s to
> fail.  While I think it would be probably be okay to remove the TOAST =
table
> and restrict origin names to 512 bytes (like we did for password hashe=
s in
> commit 8275325), folks routinely complain about NAMEDATALEN, so I thin=
k we
> should be cautious here.

The logical replication creates origin names as pg_SUBOID_RELID or pg_SU=
BOID.
It means the maximum origin name is 24. This limited origin name also ap=
plies
to pglogical that limits the name to 54 IIRC. I think that covers the ma=
jority
of the logical replication setups. There might be a small number of cust=
om
logical replication systems that possibly use long names for replication
origin. I've never seen a replication origin name longer than NAMEDATALE=
N.

If you consider that the maximum number of replication origin is limited=
 to 2
bytes (65k distinct names), it is reasonable to restrict the replication
origin names to 512 due to the high number of combinations. We generally
expects that a catalog string uses "name" as type if it is an identifier=
; it
could be the case for roname if author decided to be strict.

This additional TOAST table has no or rare use. +1 for removing it. It i=
s one
less file, one less table and one less index; in short, one less source =
of data
corruption. ;)


--
Euler Taveira
EDB   https://www.enterprisedb.com/

--555e4b82ec384b8a8230f9853ab628ad
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE html><html><head><title></title><style type=3D"text/css">p.Mso=
Normal,p.MsoNoSpacing{margin:0}</style></head><body><div>On Tue, Apr 8, =
2025, at 5:25 PM, Nathan Bossart wrote:<br></div><blockquote type=3D"cit=
e" id=3D"qt" style=3D""><div>On Tue, Apr 08, 2025 at 04:44:02PM +0530, A=
mit Kapila wrote:<br></div><div>&gt; On Fri, Apr 4, 2025 at 7:58=E2=80=AF=
PM Nathan Bossart &lt;<a href=3D"mailto:nathandbossart@gmail.com">nathan=
dbossart@gmail.com</a>&gt; wrote:<br></div><div>&gt;&gt; On Fri, Apr 04,=
 2025 at 05:16:43PM +0530, Amit Kapila wrote:<br></div><div>&gt;&gt; &gt=
; Can we dodge adding this push call if we restrict the length of the<br=
></div><div>&gt;&gt; &gt; origin name to some reasonable limit like 256 =
or 512 and avoid the<br></div><div>&gt;&gt; &gt; need of toast altogethe=
r?<br></div><div>&gt;&gt;<br></div><div>&gt;&gt; We did consider just re=
moving pg_replication_origin's TOAST table earlier<br></div><div>&gt;&gt=
; [0], but we decided against it at that time.&nbsp; Maybe it's worth<br=
></div><div>&gt;&gt; reconsidering...<br></div><div>&gt;&nbsp;<br></div>=
<div>&gt; I don't see any good reason in that email for not removing the=
 TOAST<br></div><div>&gt; table for pg_replication_origin. I would prefe=
r to remove it rather<br></div><div>&gt; than add protection related to =
its access.<br></div><div><br></div><div>The only reason I can think of =
is that folks might have existing<br></div><div>replication origins with=
 extremely long names that would cause upgrades to<br></div><div>fail.&n=
bsp; While I think it would be probably be okay to remove the TOAST tabl=
e<br></div><div>and restrict origin names to 512 bytes (like we did for =
password hashes in<br></div><div>commit 8275325), folks routinely compla=
in about NAMEDATALEN, so I think we<br></div><div>should be cautious her=
e.<br></div></blockquote><div><br></div><div>The logical replication cre=
ates origin names as pg_SUBOID_RELID or pg_SUBOID.<br></div><div>It mean=
s the maximum origin name is 24. This limited origin name also applies<b=
r></div><div>to pglogical that limits the name to 54 IIRC. I think that =
covers the majority<br></div><div>of the logical replication setups. The=
re might be a small number of custom<br></div><div>logical replication s=
ystems that possibly use long names for replication<br></div><div>origin=
. I've never seen a replication origin name longer than NAMEDATALEN.<br>=
</div><div><br></div><div>If you consider that the maximum number of rep=
lication origin is limited to 2<br></div><div>bytes (65k distinct names)=
, it is reasonable to restrict the replication<br></div><div>origin name=
s to 512 due to the high number of combinations. We generally<br></div><=
div>expects that a catalog string uses "name" as type if it is an identi=
fier; it<br></div><div>could be the case for roname if author decided to=
 be strict.<br></div><div><br></div><div>This additional TOAST table has=
 no or rare use. +1 for removing it. It is one<br></div><div>less file, =
one less table and one less index; in short, one less source of data<br>=
</div><div>corruption. ;)<br></div><div><br></div><div><br></div><div id=
=3D"sig112409080"><div class=3D"signature">--<br></div><div class=3D"sig=
nature">Euler Taveira<br></div><div class=3D"signature">EDB&nbsp;&nbsp; =
<a href=3D"https://www.enterprisedb.com/">https://www.enterprisedb.com/<=
/a><br></div></div><div><br></div></body></html>
--555e4b82ec384b8a8230f9853ab628ad--