MIME-Version: 1.0
References: <83C44AB4-24B0-437F-B139-B5CBC5821BB1@yesql.se>
 <CAOYmi+=zgJinCoqdQ70fvKmvyrLekd9-supkuj57Poa1FmX8bw@mail.gmail.com>
 <2A1511A0-C04B-47E4-B1C3-54C2A1C765B8@yesql.se> <CAOYmi+=1_AqPL0Y7qdqg1ER0-bBjCUKUZfrLcgob5bHjzSsvbA@mail.gmail.com>
 <CAOYmi+nP8AM9xm+xUW5kDz7XDF7MKBjuDnQ4LjMEm07tpwDgrg@mail.gmail.com>
 <AACF415F-894A-434F-9FE2-272ED8FA4981@yesql.se> <13F329B6-86BC-40A5-96F4-102784A0357A@yesql.se>
 <CAOYmi+nZUivtO3Ed4X3tBh3sLmMQTe+ir7utyc1Gv5L6ZDU5cA@mail.gmail.com>
 <p4bd7mn6dxr2zdak74abocyltpfdxif4pxqzixqpxpetjwt34h@qc6jgfmoddvq>
 <CAOYmi+nePFefTKmeRtajfVhunbXFq64etBr7VwbhxzDBQL-BBw@mail.gmail.com>
 <uqzksikuuprgcu4mrpqdu65s7loh52pghdaq5rkzdruhgy4wse@4ojpvnsfumpc>
 <CAOYmi+=Pyf9EuR7dRn46ymV-P9fhUft3qH8mqdS-m9ZksmooEg@mail.gmail.com>
 <CAOYmi+mhqahb65y1zXtv60T9=mDYTaepV9b-wq-GHey00zuGOg@mail.gmail.com>
 <CA+hUKGJ+WyJ26QGvO_nkgvbxgw+03U4EQ4Hxw+QBft6Np+XW7w@mail.gmail.com>
 <CAOYmi+mf+hw7x+E3XkRypEGy=yR3n6zowjjXeXaSK20WxX3UGw@mail.gmail.com>
 <CAOYmi+kr-Eh7utEDZBFq8Mc9byLLRF3ZNc0TaHt78SKwB4T5KA@mail.gmail.com>
 <CA+hUKGL3yYmKmNHw1wnpuhCbqaiDw1DDUhghEDrh6y97Ygzw9Q@mail.gmail.com>
 <CAOYmi+kOyST1DoF1WOMQvf75L56LiLHZR2+3qTJ+QCg2KC8raQ@mail.gmail.com>
 <CA+hUKGL59uY=UpCHEmTL6dpzuWJBVf1Kc4bQADYDEWG9ZDCffA@mail.gmail.com>
 <CAOYmi+k6HUMK4XQAfnxsmgs1oPOKnchyj2O2a+R7H9jOTU4LPQ@mail.gmail.com>
 <CA+hUKG+mUfH81RuYm5fJrOxdm85hvxLEm=KxXm+n-12FJJH9jA@mail.gmail.com> <CAOYmi+nk-KHGGBHz6BGik4+xHPmyPrwOj3ezSM3F4d=-kH_n2Q@mail.gmail.com>
In-Reply-To: <CAOYmi+nk-KHGGBHz6BGik4+xHPmyPrwOj3ezSM3F4d=-kH_n2Q@mail.gmail.com>
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Thu, 6 Mar 2025 12:57:24 -0800
Message-ID: <CAOYmi+n4EDOOUL27_OqYT2-F2rS6S+3mK-ppWb2Ec92UEoUbYA@mail.gmail.com>
Subject: Re: [PoC] Federated Authn/z with OAUTHBEARER
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, Daniel Gustafsson <daniel@yesql.se>, 
	Peter Eisentraut <peter@eisentraut.org>, Antonin Houska <ah@cybertec.at>, 
	PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://www.postgresql.org/message-id/CAOYmi%2Bn4EDOOUL27_OqYT2-F2rS6S%2B3mK-ppWb2Ec92UEoUbYA%40mail.gmail.com>
Precedence: bulk

On Tue, Mar 4, 2025 at 2:44=E2=80=AFPM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Maybe. My first attempt gets all the BSDs green except macOS -- which
> now fails in a completely different test, haha... -_-

Small update: there is not one bug, but three that interact. =E0=B2=A0_=E0=
=B2=A0

1) The test server advertises an issuer of `https://localhost:<port>`,
but it doesn't listen on all localhost interfaces. When Curl tries to
contact the issuer on IPv6, its Happy Eyeballs handling usually falls
back to IPv4 after discovering that IPv6 is nonfunctional, but
occasionally it contacts something that was temporarily listening
there instead.

Since I don't really want to write a bunch of IPv6 fallback code for
the test server -- this should be testing OAuth, not finding all the
ways that buildfarm OSes can expose dual stack sockets -- I changed
the issuer to be IPv4-only. When I did this, the interval timing tests
immediately failed on macOS.

2) macOS's EVFILT_TIMER implementation seems to be different from the
other BSDs. On Mac, when you re-add a timer to a kqueue, any existing
timer-fired events for it are not cleared out and the kqueue might
remain readable. This breaks a postcondition of our set_timer()
function, which is that new timeouts are supposed to completely
replace previous timeouts.

With a dual stack issuer, the Happy Eyeballs timeouts would be
routinely cleared out by libcurl, setting up a clean slate for the
next call to set_timer(). But with an IPv4-only issuer, libcurl didn't
need to clear out the timeouts (they'd already fired), which meant
that our call to set the ping interval was ineffective.

3) There is a related performance bug on other platforms. If a Curl
timeout happens partway through a request (so libcurl won't clear it),
the timer-expired event will stay set and CPU will be burned to spin
pointlessly on drive_request(). This is much easier to notice after
taking Happy Eyeballs out of the picture. It doesn't cause logical
failures -- Curl basically discards the unnecessary calls -- but it's
definitely unintended.

--

Problem 1 is a simple patch. I am working on a fix for Problem 2, but
I got stuck trying to get a "perfect" solution working yesterday...
Since this is a partial(?) blocker for getting NetBSD going, I'm going
to pivot to an ugly-but-simple approach today.

I plan to defer working on Problem 3, which should just be a
performance bug, until the tests are green again. And I would like to
eventually add some stronger unit tests for the timer behavior, to
catch other potential OS-specific problems in the future.

Thanks,
--Jacob