MIME-Version: 1.0
References: 
 <CADK3HHKe1PA1U6aB5-7tWBQ0yZGgNvY7H=ECDD9955Pas_zx_Q@mail.gmail.com>
 <CAGECzQRQ5optaG4DPbshKS+zpUtn_oceh8-qdshFbS+-FSb8Dg@mail.gmail.com>
 <CAOYmi+nVQRpSs3vd_v9L8ytO9wnL2ndnzGwU31aDGorVFxrAYA@mail.gmail.com>
 <CAGECzQSZ43JMjA8QEJoF9DCdTO0GQeR2qyhouQciSG2ik40Yhg@mail.gmail.com>
 <CAOYmi+m20jS8zZ2qFpSnvhaqGDX+vtgCsqcu9VhokyLqF8kFag@mail.gmail.com>
In-Reply-To: 
 <CAOYmi+m20jS8zZ2qFpSnvhaqGDX+vtgCsqcu9VhokyLqF8kFag@mail.gmail.com>
From: Dave Cramer <davecramer@gmail.com>
Date: Sun, 14 Dec 2025 07:31:05 -0500
Message-ID: 
 <CADK3HH+DPY_x_H=e0c_AVWoUP9E+YXdyJDVvmzYEYxZXT87Agw@mail.gmail.com>
Subject: Re: Proposal to allow setting cursor options on Portals
To: Jacob Champion <jacob.champion@enterprisedb.com>
Cc: Jelte Fennema-Nio <postgres@jeltef.nl>,
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
 Heikki Linnakangas <hlinnaka@iki.fi>
Content-Type: multipart/alternative; boundary="000000000000ffdace0645e8ab76"
Archived-At: 
 <https://www.postgresql.org/message-id/CADK3HH%2BDPY_x_H%3De0c_AVWoUP9E%2BYXdyJDVvmzYEYxZXT87Agw%40mail.gmail.com>
Precedence: bulk

--000000000000ffdace0645e8ab76
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Thu, 11 Dec 2025 at 14:21, Jacob Champion <
jacob.champion@enterprisedb.com> wrote:

> [I considered splitting this off into a new thread, but I think Dave
> has to wait for it to be resolved before much can happen with the
> patch. Sorry Dave.]
>

No worries, I expected discussion.

>
> On Wed, Dec 10, 2025 at 3:01=E2=80=AFPM Jelte Fennema-Nio <postgres@jelte=
f.nl>
> wrote:
> > If we keep the features that are bundled with a protocol version bump
> > of the kind where a client, either has to do nothing to implement it,
> > or at worst has to ignore the contents of a new message/field. Then
> > implementing support becomes so trivial for clients that I don't think
> > it'd be a hurdle for client authors to implement support for 3.3, 3.4,
> > 3.5 and if they only wanted a feature from the 3.6 protocol.^1 I'll
> > call these things "no-op implementations" from now on.
>
> It's too late for that, isn't it? 3.2's only feature doesn't work that
> way (and couldn't have been designed that way, as far as I can tell).
> So I don't have any confidence that all future features will fall in
> line with this new rule.
>
> NegotiateProtocolVersion is the only in-band tool we have to ratchet
> the protocol forward. Why go through all this pain of getting NPV
> packets working, only to immediately limit its power to the most
> trivial cases?
>
> > I think we disagree on this. I think the downside of using protocol
> > extensions for everything is that we then end up with N*N different
> > combinations of features in the wild that servers and clients need to
> > deal with. We have to start to define what happens when features
> > interact, but either of them is not enabled.
>
> In the worst case? Yes. (That worst case doesn't really bother me.
> Many other protocols regularly navigate extension combinations.)
>
> But! The two extension proposals in flight at the moment -- GoAway and
> cursor options -- are completely orthogonal, no? Both to each other,
> and to the functionality in 3.2. There are no combinatorics yet. So it
> seems strange to optimize for combinatorics out of the gate, by
> burning through a client-mandatory minor version every year.
>
> > With incrementing
> > versions you don't have that problem,
>
> You still have N*M. Implementers have to test each feature of their
> 3.10 client against server versions 3.0-9, rather than testing against
> a single server that turns individual extension support on and off. I
> prefer the latter (but maybe that's just because it's what I'm used
> to). Middleboxes increase the matrix further, as you point out below.
>

As a client author we test against multiple options all the time.  I don't
think this should be an argument against changing the protocol, otherwise
we will never change it.

>
> Paradoxically, if all N features happen to be orthogonal, the testing
> burden for the extension strategy collapses to... N.
> Minor-version-per-year is worse for that case.
>

I had never contemplated features being dependent on one another, only
additive and orthogonal.

>
> > which results in simpler logic
> > in the spec, servers and clients.
>
> I don't want to dissuade a proof of concept for this, because simpler
> logic everywhere sounds amazing. But it sounds like magical thinking
> to me. A bit like telling Christoph that the dpkg dependency graph is
> too complicated, so it should be a straight line instead -- if that
> worked, presumably everyone would have done it that way, right?
> Convince me that you're not just ignoring necessary complexity in an
> attempt to stamp out unnecessary complexity.
>
> An example of an established network protocol that follows this same
> strategy would be helpful. How do their clients deal with the
> minor-version treadmill?
>
> > Finally, because we don't have any protocol extensions yet. All
> > clients still need to build infrastructure for them, including libpq.
>
> For clients still on 3.0 (the vast majority of them), they'd have to
> add infrastructure for sliding minor version ranges, too.
>
> > So I'd argue that if we make such "no-op implementation" features use
> > protocol extensions, then it'd be more work for everyone.
>
> Why advertise a protocol extension if you plan to ignore it? Don't
> advertise it. Do nothing. That's even less work than retrofitting
> packet parsers to correctly ignore a byte range when minorversion > X.
>
> > > Plus I think it's unwise to introduce a 3.3 before we're
> > > confident that 3.2 can be widely deployed, and I'm trying to put
> > > effort into the latter for 19, so that I'm not just sitting here
> > > gatekeeping.
> >
> > I'm not sure what you mean with this. People use libpq18 and PG18, and
> > I've heard no complaints about protocol problems. So I think it was a
> > success. Do you mean widely deployed by default?
>
> Yes. Or even just "deployed". GitHub shows zero hits outside of the
> Postgres fork graph.
>

As mentioned pgjdbc supports 3.2. It was trivial to implement.

>
> Google's results show that an organization called "cardo" tried
> max_protocol_version=3Dlatest. They had to revert it. :( Time for
> grease.
>
> > Why exactly does that
> > matter for 3.3? Anything that stands default deployment in the way for
> > 3.2, will continue to stand default deployment in the way for 3.3.
>
> Exactly. Don't you want to make sure that clients in the ecosystem are
> able to use this _before_ we rev the version again, and again? We
> don't ever get these numbers back.
>

Well there are 97 of them. 1 per year is a long time.

>
> Like, I'm arguing as hard as I can against the very existence of the
> treadmill. But if I'm outvoted on that, *please* don't start the
> treadmill before other people can climb on -- otherwise, they won't be
> able to give us any feedback at all!
>
> > Personally, if we flip the default in e.g. 5 years from now. I'd much
> > rather have it be flipped to a very nice 3.6 protocol, than still only
> > having the single new feature that was added in 3.2.
>
> Those are not the only two choices. I'd rather we get a bunch of nice
> features without any flipping at all, if that's possible. It looks
> possible to me.
>
> > > IETF has a bunch of related case studies [1,2,3] that might be useful
> > > reading, even if we decide that their experience differs heavily from
> > > ours.
> >
> > I gave them a skim and they seem like a good read (which I'll do
> > later). But I'm not sure part of them you thought was actionable for
> > the discussion about version bumps vs protocol extensions. (I did see
> > useful stuff for the grease thread, but that seems better to discuss
> > there)
>
> For this conversation, I'm focused on RFC 8170. Specifically the
> concepts of incremental transitions and incentive alignment
> (cost/benefit to individual community members).
>
> I view minor-version-per-year as violating both of those principles.
> It instead focuses on the ease of the people who are most plugged into
> this mailing list, and who have the most power to change things on a
> whim.
>
> > ^1: You and I only talked about clients above, but obviously there's
> > also proxies and other servers that implement the protocol to
> > consider. If a feature that is "no-op implementation" on the client is
> > a complicated implementation on the proxy/server then maybe a protocol
> > extension is indeed the better choice. I think for GoAway it's trivial
> > to "no-op implement" too on the proxy/server. For this cursor option
> > proposal it's less clear cut imo. Proxies can probably simply forward
> > the message to the server, although maybe PgBouncer would want to
> > throw an error when a client uses a hold cursor (but it also doesn't
> > do that for SQL level hold cursors, so that seems like an optional
> > enhancement).
>

FWIW, HOLDABLE cursors are not the only option this enables. It enables all
of the other cursor options.

>
> I think proposals should attempt to answer those questions as a
> prerequisite to commit, personally. Or at least, we should be moving
> in that direction, if that's too harsh on the first authors who are
> trying to get things moving inside the protocol.
>
> More generally, it bothers me that we still don't have a clear mental
> model of middlebox extensibility. We're just retreading the
> discussions from [1] instead of starting from where we stopped, and
> that's exhausting for me.
>
> (As a reminder: 3.2 broke my testing rig, which relied on implicit
> assumptions around minor-version extensibility for middleboxes. I
> didn't speak up until very late, because it was just a testing rig,
> and I could change it. I should have spoken up immediately, because
> IIRC, pgpool then broke as well.)
>
> > Other servers might not even support hold cursors, but
> > then they could simply throw a clear error (like pgbouncer would do).
> > If throwing an error is an acceptable server implementation, then I
> > think a "no-op implementation" is again trivial.
>
> A server is always free to decide at the _application_ layer that it
> will error out for a particular packet that it can parse at the
> _network_ layer. But it seems a lot more user-friendly to just decline
> the protocol bit, if it's directly tied to an application-level
> feature that isn't implemented. I think we should encourage that when
> possible; otherwise we've traded protocol fragmentation for
> application fragmentation.
>

Are we concerned with servers that are not compatible with Postgres ?
As far as protocol fragmentation goes, I see this more as evolution to a
more complete usable implementation. I do see that we will have to be
careful with interdependent protocol options.

Dave

--000000000000ffdace0645e8ab76
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><div><br></div></div><br><div class=3D"gm=
ail_quote gmail_quote_container"><div dir=3D"ltr" class=3D"gmail_attr">On T=
hu, 11 Dec 2025 at 14:21, Jacob Champion &lt;<a href=3D"mailto:jacob.champi=
on@enterprisedb.com">jacob.champion@enterprisedb.com</a>&gt; wrote:<br></di=
v><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;borde=
r-left:1px solid rgb(204,204,204);padding-left:1ex">[I considered splitting=
 this off into a new thread, but I think Dave<br>
has to wait for it to be resolved before much can happen with the<br>
patch. Sorry Dave.]<br></blockquote><div><br></div><div>No worries, I expec=
ted discussion.=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margi=
n:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex=
">
<br>
On Wed, Dec 10, 2025 at 3:01=E2=80=AFPM Jelte Fennema-Nio &lt;<a href=3D"ma=
ilto:postgres@jeltef.nl" target=3D"_blank">postgres@jeltef.nl</a>&gt; wrote=
:<br>
&gt; If we keep the features that are bundled with a protocol version bump<=
br>
&gt; of the kind where a client, either has to do nothing to implement it,<=
br>
&gt; or at worst has to ignore the contents of a new message/field. Then<br=
>
&gt; implementing support becomes so trivial for clients that I don&#39;t t=
hink<br>
&gt; it&#39;d be a hurdle for client authors to implement support for 3.3, =
3.4,<br>
&gt; 3.5 and if they only wanted a feature from the 3.6 protocol.^1 I&#39;l=
l<br>
&gt; call these things &quot;no-op implementations&quot; from now on.<br>
<br>
It&#39;s too late for that, isn&#39;t it? 3.2&#39;s only feature doesn&#39;=
t work that<br>
way (and couldn&#39;t have been designed that way, as far as I can tell).<b=
r>
So I don&#39;t have any confidence that all future features will fall in<br=
>
line with this new rule.<br>
<br>
NegotiateProtocolVersion is the only in-band tool we have to ratchet<br>
the protocol forward. Why go through all this pain of getting NPV<br>
packets working, only to immediately limit its power to the most<br>
trivial cases?<br>
<br>
&gt; I think we disagree on this. I think the downside of using protocol<br=
>
&gt; extensions for everything is that we then end up with N*N different<br=
>
&gt; combinations of features in the wild that servers and clients need to<=
br>
&gt; deal with. We have to start to define what happens when features<br>
&gt; interact, but either of them is not enabled.<br>
<br>
In the worst case? Yes. (That worst case doesn&#39;t really bother me.<br>
Many other protocols regularly navigate extension combinations.)<br>
<br>
But! The two extension proposals in flight at the moment -- GoAway and<br>
cursor options -- are completely orthogonal, no? Both to each other,<br>
and to the functionality in 3.2. There are no combinatorics yet. So it<br>
seems strange to optimize for combinatorics out of the gate, by<br>
burning through a client-mandatory minor version every year.<br>
<br>
&gt; With incrementing<br>
&gt; versions you don&#39;t have that problem,<br>
<br>
You still have N*M. Implementers have to test each feature of their<br>
3.10 client against server versions 3.0-9, rather than testing against<br>
a single server that turns individual extension support on and off. I<br>
prefer the latter (but maybe that&#39;s just because it&#39;s what I&#39;m =
used<br>
to). Middleboxes increase the matrix further, as you point out below.<br></=
blockquote><div><br></div><div>As a client author we test against multiple=
=C2=A0options all the time.=C2=A0 I don&#39;t think this should be an argum=
ent against changing the protocol, otherwise we will never change it.</div>=
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Paradoxically, if all N features happen to be orthogonal, the testing<br>
burden for the extension strategy collapses to... N.<br>
Minor-version-per-year is worse for that case.<br></blockquote><div><br></d=
iv><div>I had never contemplated features being dependent on one another, o=
nly additive and orthogonal.=C2=A0=C2=A0</div><blockquote class=3D"gmail_qu=
ote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,20=
4);padding-left:1ex">
<br>
&gt; which results in simpler logic<br>
&gt; in the spec, servers and clients.<br>
<br>
I don&#39;t want to dissuade a proof of concept for this, because simpler<b=
r>
logic everywhere sounds amazing. But it sounds like magical thinking<br>
to me. A bit like telling Christoph that the dpkg dependency graph is<br>
too complicated, so it should be a straight line instead -- if that<br>
worked, presumably everyone would have done it that way, right?<br>
Convince me that you&#39;re not just ignoring necessary complexity in an<br=
>
attempt to stamp out unnecessary complexity.<br>
<br>
An example of an established network protocol that follows this same<br>
strategy would be helpful. How do their clients deal with the<br>
minor-version treadmill?<br>
<br>
&gt; Finally, because we don&#39;t have any protocol extensions yet. All<br=
>
&gt; clients still need to build infrastructure for them, including libpq.<=
br>
<br>
For clients still on 3.0 (the vast majority of them), they&#39;d have to<br=
>
add infrastructure for sliding minor version ranges, too.<br>
<br>
&gt; So I&#39;d argue that if we make such &quot;no-op implementation&quot;=
 features use<br>
&gt; protocol extensions, then it&#39;d be more work for everyone.<br>
<br>
Why advertise a protocol extension if you plan to ignore it? Don&#39;t<br>
advertise it. Do nothing. That&#39;s even less work than retrofitting<br>
packet parsers to correctly ignore a byte range when minorversion &gt; X.<b=
r>
<br>
&gt; &gt; Plus I think it&#39;s unwise to introduce a 3.3 before we&#39;re<=
br>
&gt; &gt; confident that 3.2 can be widely deployed, and I&#39;m trying to =
put<br>
&gt; &gt; effort into the latter for 19, so that I&#39;m not just sitting h=
ere<br>
&gt; &gt; gatekeeping.<br>
&gt;<br>
&gt; I&#39;m not sure what you mean with this. People use libpq18 and PG18,=
 and<br>
&gt; I&#39;ve heard no complaints about protocol problems. So I think it wa=
s a<br>
&gt; success. Do you mean widely deployed by default?<br>
<br>
Yes. Or even just &quot;deployed&quot;. GitHub shows zero hits outside of t=
he<br>
Postgres fork graph.<br></blockquote><div><br></div><div>As mentioned pgjdb=
c supports 3.2. It was trivial to implement.=C2=A0</div><blockquote class=
=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rg=
b(204,204,204);padding-left:1ex">
<br>
Google&#39;s results show that an organization called &quot;cardo&quot; tri=
ed<br>
max_protocol_version=3Dlatest. They had to revert it. :( Time for<br>
grease.<br>
<br>
&gt; Why exactly does that<br>
&gt; matter for 3.3? Anything that stands default deployment in the way for=
<br>
&gt; 3.2, will continue to stand default deployment in the way for 3.3.<br>
<br>
Exactly. Don&#39;t you want to make sure that clients in the ecosystem are<=
br>
able to use this _before_ we rev the version again, and again? We<br>
don&#39;t ever get these numbers back.<br></blockquote><div><br></div><div>=
Well there are 97 of them. 1 per year is a long time.=C2=A0</div><blockquot=
e class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px s=
olid rgb(204,204,204);padding-left:1ex">
<br>
Like, I&#39;m arguing as hard as I can against the very existence of the<br=
>
treadmill. But if I&#39;m outvoted on that, *please* don&#39;t start the<br=
>
treadmill before other people can climb on -- otherwise, they won&#39;t be<=
br>
able to give us any feedback at all!<br>
<br>
&gt; Personally, if we flip the default in e.g. 5 years from now. I&#39;d m=
uch<br>
&gt; rather have it be flipped to a very nice 3.6 protocol, than still only=
<br>
&gt; having the single new feature that was added in 3.2.<br>
<br>
Those are not the only two choices. I&#39;d rather we get a bunch of nice<b=
r>
features without any flipping at all, if that&#39;s possible. It looks<br>
possible to me.<br>
<br>
&gt; &gt; IETF has a bunch of related case studies [1,2,3] that might be us=
eful<br>
&gt; &gt; reading, even if we decide that their experience differs heavily =
from<br>
&gt; &gt; ours.<br>
&gt;<br>
&gt; I gave them a skim and they seem like a good read (which I&#39;ll do<b=
r>
&gt; later). But I&#39;m not sure part of them you thought was actionable f=
or<br>
&gt; the discussion about version bumps vs protocol extensions. (I did see<=
br>
&gt; useful stuff for the grease thread, but that seems better to discuss<b=
r>
&gt; there)<br>
<br>
For this conversation, I&#39;m focused on RFC 8170. Specifically the<br>
concepts of incremental transitions and incentive alignment<br>
(cost/benefit to individual community members).<br>
<br>
I view minor-version-per-year as violating both of those principles.<br>
It instead focuses on the ease of the people who are most plugged into<br>
this mailing list, and who have the most power to change things on a<br>
whim.<br>
<br>
&gt; ^1: You and I only talked about clients above, but obviously there&#39=
;s<br>
&gt; also proxies and other servers that implement the protocol to<br>
&gt; consider. If a feature that is &quot;no-op implementation&quot; on the=
 client is<br>
&gt; a complicated implementation on the proxy/server then maybe a protocol=
<br>
&gt; extension is indeed the better choice. I think for GoAway it&#39;s tri=
vial<br>
&gt; to &quot;no-op implement&quot; too on the proxy/server. For this curso=
r option<br>
&gt; proposal it&#39;s less clear cut imo. Proxies can probably simply forw=
ard<br>
&gt; the message to the server, although maybe PgBouncer would want to<br>
&gt; throw an error when a client uses a hold cursor (but it also doesn&#39=
;t<br>
&gt; do that for SQL level hold cursors, so that seems like an optional<br>
&gt; enhancement).<br></blockquote><div><br></div><div>FWIW, HOLDABLE curso=
rs are not the only option this enables. It enables all of the other cursor=
 options.=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px =
0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
I think proposals should attempt to answer those questions as a<br>
prerequisite to commit, personally. Or at least, we should be moving<br>
in that direction, if that&#39;s too harsh on the first authors who are<br>
trying to get things moving inside the protocol.<br>
<br>
More generally, it bothers me that we still don&#39;t have a clear mental<b=
r>
model of middlebox extensibility. We&#39;re just retreading the<br>
discussions from [1] instead of starting from where we stopped, and<br>
that&#39;s exhausting for me.<br>
<br>
(As a reminder: 3.2 broke my testing rig, which relied on implicit<br>
assumptions around minor-version extensibility for middleboxes. I<br>
didn&#39;t speak up until very late, because it was just a testing rig,<br>
and I could change it. I should have spoken up immediately, because<br>
IIRC, pgpool then broke as well.)<br>
<br>
&gt; Other servers might not even support hold cursors, but<br>
&gt; then they could simply throw a clear error (like pgbouncer would do).<=
br>
&gt; If throwing an error is an acceptable server implementation, then I<br=
>
&gt; think a &quot;no-op implementation&quot; is again trivial.<br>
<br>
A server is always free to decide at the _application_ layer that it<br>
will error out for a particular packet that it can parse at the<br>
_network_ layer. But it seems a lot more user-friendly to just decline<br>
the protocol bit, if it&#39;s directly tied to an application-level<br>
feature that isn&#39;t implemented. I think we should encourage that when<b=
r>
possible; otherwise we&#39;ve traded protocol fragmentation for<br>
application fragmentation.<br></blockquote><div><br></div><div>Are we conce=
rned with servers that are not compatible with Postgres ?=C2=A0</div><div>A=
s far as protocol fragmentation goes, I see this more as evolution to a mor=
e complete usable implementation. I do see that we will have to be careful =
with interdependent=C2=A0protocol options.=C2=A0</div><div><br></div><div>D=
ave</div></div></div>

--000000000000ffdace0645e8ab76--