public inbox for [email protected]help / color / mirror / Atom feed
pgsql: Add support for OAUTHBEARER SASL mechanism 9+ messages / 4 participants [nested] [flat]
* pgsql: Add support for OAUTHBEARER SASL mechanism @ 2025-02-20 15:47 Daniel Gustafsson <[email protected]> 0 siblings, 1 reply; 9+ messages in thread From: Daniel Gustafsson @ 2025-02-20 15:47 UTC (permalink / raw) To: [email protected] Add support for OAUTHBEARER SASL mechanism This commit implements OAUTHBEARER, RFC 7628, and OAuth 2.0 Device Authorization Grants, RFC 8628. In order to use this there is a new pg_hba auth method called oauth. When speaking to a OAuth- enabled server, it looks a bit like this: $ psql 'host=example.org oauth_issuer=... oauth_client_id=...' Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG Device authorization is currently the only supported flow so the OAuth issuer must support that in order for users to authenticate. Third-party clients may however extend this and provide their own flows. The built-in device authorization flow is currently not supported on Windows. In order for validation to happen server side a new framework for plugging in OAuth validation modules is added. As validation is implementation specific, with no default specified in the standard, PostgreSQL does not ship with one built-in. Each pg_hba entry can specify a specific validator or be left blank for the validator installed as default. This adds a requirement on libcurl for the client side support, which is optional to build, but the server side has no additional build requirements. In order to run the tests, Python is required as this adds a https server written in Python. Tests are gated behind PG_TEST_EXTRA as they open ports. This patch has been a multi-year project with many contributors involved with reviews and in-depth discussions: Michael Paquier, Heikki Linnakangas, Zhihong Yu, Mahendrakar Srinivasarao, Andrey Chudnovsky and Stephen Frost to name a few. While Jacob Champion is the main author there have been some levels of hacking by others. Daniel Gustafsson contributed the validation module and various bits and pieces; Thomas Munro wrote the client side support for kqueue. Author: Jacob Champion <[email protected]> Co-authored-by: Daniel Gustafsson <[email protected]> Co-authored-by: Thomas Munro <[email protected]> Reviewed-by: Daniel Gustafsson <[email protected]> Reviewed-by: Peter Eisentraut <[email protected]> Reviewed-by: Antonin Houska <[email protected]> Reviewed-by: Kashif Zeeshan <[email protected]> Discussion: https://postgr.es/m/[email protected] Branch ------ master Details ------- https://git.postgresql.org/pg/commitdiff/b3f0be788afc17d2206e1ae1c731d8aeda1f2f59 Modified Files -------------- .cirrus.tasks.yml | 15 +- config/programs.m4 | 65 + configure | 332 +++ configure.ac | 41 + doc/src/sgml/client-auth.sgml | 252 ++ doc/src/sgml/config.sgml | 26 + doc/src/sgml/filelist.sgml | 1 + doc/src/sgml/installation.sgml | 27 + doc/src/sgml/libpq.sgml | 445 +++ doc/src/sgml/oauth-validators.sgml | 414 +++ doc/src/sgml/postgres.sgml | 1 + doc/src/sgml/protocol.sgml | 133 +- doc/src/sgml/regress.sgml | 10 + meson.build | 100 + meson_options.txt | 3 + src/Makefile.global.in | 1 + src/backend/libpq/Makefile | 1 + src/backend/libpq/auth-oauth.c | 894 ++++++ src/backend/libpq/auth.c | 10 +- src/backend/libpq/hba.c | 64 +- src/backend/libpq/meson.build | 1 + src/backend/libpq/pg_hba.conf.sample | 4 +- src/backend/utils/adt/hbafuncs.c | 19 + src/backend/utils/misc/guc_tables.c | 12 + src/backend/utils/misc/postgresql.conf.sample | 3 + src/include/common/oauth-common.h | 19 + src/include/libpq/auth.h | 1 + src/include/libpq/hba.h | 7 +- src/include/libpq/oauth.h | 101 + src/include/pg_config.h.in | 9 + src/interfaces/libpq/Makefile | 11 +- src/interfaces/libpq/exports.txt | 3 + src/interfaces/libpq/fe-auth-oauth-curl.c | 2883 ++++++++++++++++++++ src/interfaces/libpq/fe-auth-oauth.c | 1163 ++++++++ src/interfaces/libpq/fe-auth-oauth.h | 46 + src/interfaces/libpq/fe-auth.c | 36 +- src/interfaces/libpq/fe-auth.h | 3 + src/interfaces/libpq/fe-connect.c | 48 +- src/interfaces/libpq/libpq-fe.h | 85 + src/interfaces/libpq/libpq-int.h | 13 +- src/interfaces/libpq/meson.build | 5 + src/makefiles/meson.build | 1 + src/test/authentication/t/001_password.pl | 8 +- src/test/modules/Makefile | 1 + src/test/modules/meson.build | 1 + src/test/modules/oauth_validator/.gitignore | 4 + src/test/modules/oauth_validator/Makefile | 40 + src/test/modules/oauth_validator/README | 13 + src/test/modules/oauth_validator/fail_validator.c | 47 + src/test/modules/oauth_validator/magic_validator.c | 48 + src/test/modules/oauth_validator/meson.build | 85 + .../modules/oauth_validator/oauth_hook_client.c | 293 ++ src/test/modules/oauth_validator/t/001_server.pl | 594 ++++ src/test/modules/oauth_validator/t/002_client.pl | 154 ++ src/test/modules/oauth_validator/t/OAuth/Server.pm | 140 + src/test/modules/oauth_validator/t/oauth_server.py | 391 +++ src/test/modules/oauth_validator/validator.c | 143 + src/test/perl/PostgreSQL/Test/Cluster.pm | 22 +- src/tools/pgindent/pgindent | 14 + src/tools/pgindent/typedefs.list | 11 + 60 files changed, 9278 insertions(+), 39 deletions(-) ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Add support for OAUTHBEARER SASL mechanism @ 2025-03-31 21:54 Christoph Berg <[email protected]> parent: Daniel Gustafsson <[email protected]> 0 siblings, 2 replies; 9+ messages in thread From: Christoph Berg @ 2025-03-31 21:54 UTC (permalink / raw) To: Daniel Gustafsson <[email protected]>; Jacob Champion <[email protected]>; Thomas Munro <[email protected]>; +Cc: [email protected] > Add support for OAUTHBEARER SASL mechanism Debian still has this experimental port with a GNU userland and a FreeBSD kernel called kfreebsd. I don't expect anyone to particularly care about it, but it found an actual bug: /build/reproducible-path/postgresql-18-18~~devel.20250331/build/../src/interfaces/libpq/fe-auth-oauth-curl.c: In function ‘register_socket’: /build/reproducible-path/postgresql-18-18~~devel.20250331/build/../src/interfaces/libpq/fe-auth-oauth-curl.c:1317:20: error: ‘actx’ undeclared (first use in this function); did you mean ‘ctx’? 1317 | actx_error(actx, "libpq does not support multiplexer sockets on this platform"); | ^~~~ This should not be a compile-time error; actx is not defined outside the #ifdef blocks there: /* * Adds and removes sockets from the multiplexer set, as directed by the * libcurl multi handle. */ static int register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx, void *socketp) { #ifdef HAVE_SYS_EPOLL_H struct async_ctx *actx = ctx; ... #endif #ifdef HAVE_SYS_EVENT_H struct async_ctx *actx = ctx; ... #endif actx_error(actx, "libpq does not support multiplexer sockets on this platform"); return -1; } https://buildd.debian.org/status/fetch.php?pkg=postgresql-18&arch=hurd-amd64&ver=18%7E%7Edev... Christoph ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Add support for OAUTHBEARER SASL mechanism @ 2025-03-31 22:17 Christoph Berg <[email protected]> parent: Christoph Berg <[email protected]> 1 sibling, 0 replies; 9+ messages in thread From: Christoph Berg @ 2025-03-31 22:17 UTC (permalink / raw) To: Daniel Gustafsson <[email protected]>; Jacob Champion <[email protected]>; Thomas Munro <[email protected]>; +Cc: [email protected] Re: To Daniel Gustafsson > > Add support for OAUTHBEARER SASL mechanism > > Debian still has this experimental port with a GNU userland and a > FreeBSD kernel called kfreebsd. Sorry this part was nonsense, kfreebsd was actually terminated and obviously I didn't even read the port's name. The failing port (still experimental and care-is-optional) is hurd-amd64. The bug is the same, though. Christoph ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Add support for OAUTHBEARER SASL mechanism @ 2025-03-31 23:09 Jacob Champion <[email protected]> parent: Christoph Berg <[email protected]> 1 sibling, 2 replies; 9+ messages in thread From: Jacob Champion @ 2025-03-31 23:09 UTC (permalink / raw) To: Christoph Berg <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Thomas Munro <[email protected]>; [email protected] On Mon, Mar 31, 2025 at 2:54 PM Christoph Berg <[email protected]> wrote: > > > Add support for OAUTHBEARER SASL mechanism > > Debian still has this experimental port with a GNU userland and a > FreeBSD kernel called kfreebsd. I don't expect anyone to particularly > care about it, but it found an actual bug: > > /build/reproducible-path/postgresql-18-18~~devel.20250331/build/../src/interfaces/libpq/fe-auth-oauth-curl.c: In function ‘register_socket’: > /build/reproducible-path/postgresql-18-18~~devel.20250331/build/../src/interfaces/libpq/fe-auth-oauth-curl.c:1317:20: error: ‘actx’ undeclared (first use in this function); did you mean ‘ctx’? > 1317 | actx_error(actx, "libpq does not support multiplexer sockets on this platform"); > | ^~~~ > > This should not be a compile-time error; actx is not defined outside > the #ifdef blocks there: Ah, sorry about that. Thank you for reporting it! (That means that Windows builds --with-libcurl are similarly broken, I think. Not that Windows packagers will want to use --with-libcurl -- it doesn't do anything -- but it should build.) I don't have hurd-amd64 to test, but I'm working on a patch that will build and pass tests if I manually munge pg_config.h. We were skipping the useless tests via a $windows_os check; I think I should use check_pg_config() instead. We could change how this works a bit for the proposed libpq-oauth.so plugin, and only build it if we have a workable implementation. I do like having these other platforms compile the Curl code, though, since we'd prefer to keep the build clean for a future Windows implementation... --Jacob ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Add support for OAUTHBEARER SASL mechanism @ 2025-04-01 00:06 Jacob Champion <[email protected]> parent: Jacob Champion <[email protected]> 1 sibling, 1 reply; 9+ messages in thread From: Jacob Champion @ 2025-04-01 00:06 UTC (permalink / raw) To: Christoph Berg <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Thomas Munro <[email protected]>; [email protected] On Mon, Mar 31, 2025 at 4:09 PM Jacob Champion <[email protected]> wrote: > I don't have hurd-amd64 to test, but I'm working on a patch that will > build and pass tests if I manually munge pg_config.h. We were skipping > the useless tests via a $windows_os check; I think I should use > check_pg_config() instead. Proposed fix attached. Thanks, --Jacob Attachments: [application/octet-stream] 0001-oauth-Fix-build-on-platforms-without-epoll-kqueue.patch (2.3K, 2-0001-oauth-Fix-build-on-platforms-without-epoll-kqueue.patch) download | inline diff: From a6bce74459370e8b81654328d93a8ea4601274ed Mon Sep 17 00:00:00 2001 From: Jacob Champion <[email protected]> Date: Mon, 31 Mar 2025 16:07:33 -0700 Subject: [PATCH] oauth: Fix build on platforms without epoll/kqueue register_socket() missed a variable declaration if neither HAVE_SYS_EPOLL_H nor HAVE_SYS_EVENT_H was defined. While we're fixing that, adjust the tests to check pg_config.h for one of the multiplexer implementations, rather than assuming that Windows is the only platform without support. (Christoph reported this on hurd-amd64, an experimental Debian.) Reported-by: Christoph Berg <[email protected]> Discussion: https://postgr.es/m/Z-sPFl27Y0ZC-VBl%40msg.df7cb.de --- src/interfaces/libpq/fe-auth-oauth-curl.c | 4 ++-- src/test/modules/oauth_validator/t/001_server.pl | 6 ++++-- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/src/interfaces/libpq/fe-auth-oauth-curl.c b/src/interfaces/libpq/fe-auth-oauth-curl.c index 9e0e8a9f2be..cd9c0323bb6 100644 --- a/src/interfaces/libpq/fe-auth-oauth-curl.c +++ b/src/interfaces/libpq/fe-auth-oauth-curl.c @@ -1172,8 +1172,9 @@ static int register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx, void *socketp) { -#ifdef HAVE_SYS_EPOLL_H struct async_ctx *actx = ctx; + +#ifdef HAVE_SYS_EPOLL_H struct epoll_event ev = {0}; int res; int op = EPOLL_CTL_ADD; @@ -1231,7 +1232,6 @@ register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx, return 0; #endif #ifdef HAVE_SYS_EVENT_H - struct async_ctx *actx = ctx; struct kevent ev[2] = {0}; struct kevent ev_out[2]; struct timespec timeout = {0}; diff --git a/src/test/modules/oauth_validator/t/001_server.pl b/src/test/modules/oauth_validator/t/001_server.pl index 30295364ebd..d88994abc24 100644 --- a/src/test/modules/oauth_validator/t/001_server.pl +++ b/src/test/modules/oauth_validator/t/001_server.pl @@ -26,9 +26,11 @@ if (!$ENV{PG_TEST_EXTRA} || $ENV{PG_TEST_EXTRA} !~ /\boauth\b/) 'Potentially unsafe test oauth not enabled in PG_TEST_EXTRA'; } -if ($windows_os) +unless (check_pg_config("#define HAVE_SYS_EVENT_H 1") + or check_pg_config("#define HAVE_SYS_EPOLL_H 1")) { - plan skip_all => 'OAuth server-side tests are not supported on Windows'; + plan skip_all => + 'OAuth server-side tests are not supported on this platform'; } if ($ENV{with_libcurl} ne 'yes') -- 2.34.1 ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Add support for OAUTHBEARER SASL mechanism @ 2025-04-01 13:03 Christoph Berg <[email protected]> parent: Jacob Champion <[email protected]> 1 sibling, 1 reply; 9+ messages in thread From: Christoph Berg @ 2025-04-01 13:03 UTC (permalink / raw) To: Jacob Champion <[email protected]>; +Cc: Daniel Gustafsson <[email protected]>; Thomas Munro <[email protected]>; [email protected] Re: Jacob Champion > (That means that Windows builds --with-libcurl are similarly broken, I > think. Not that Windows packagers will want to use --with-libcurl -- > it doesn't do anything -- but it should build.) Does --with-libcurl still do anything useful if this feature test fails? From what you are saying, the answer is "no", and I can see more "not on this platform" error messages in other callbacks. This should be documented in doc/src/sgml/installation.sgml. > We could change how this works a bit for the proposed libpq-oauth.so > plugin, and only build it if we have a workable implementation. I do > like having these other platforms compile the Curl code, though, since > we'd prefer to keep the build clean for a future Windows > implementation... I would prefer to get an error from configure if the feature doesn't do anything on my platform. The current way is confusing. If future users of libcurl change that, the configure test can still be changed. With the libpq-oauth split, this makes even more sense because building a library that always throws an error isn't very useful. (Don't build that file at all if the feature doesn't work.) Since oauth/curl have some security implications, would it make more sense to call the switch --enable-oauth (-Doauth) so users could control better what features their libpq is going to have? Perhaps some other feature (pg_service as URL?) is going to need libcurl as well, but it should be configurable separately. Christoph ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Add support for OAUTHBEARER SASL mechanism @ 2025-04-01 13:11 Daniel Gustafsson <[email protected]> parent: Christoph Berg <[email protected]> 0 siblings, 1 reply; 9+ messages in thread From: Daniel Gustafsson @ 2025-04-01 13:11 UTC (permalink / raw) To: Christoph Berg <[email protected]>; +Cc: Jacob Champion <[email protected]>; Thomas Munro <[email protected]>; [email protected] > On 1 Apr 2025, at 15:03, Christoph Berg <[email protected]> wrote: > With the libpq-oauth split, this makes even more sense because > building a library that always throws an error isn't very useful. > (Don't build that file at all if the feature doesn't work.) After the split, configure/meson should fail if the libcurl dependency isn't satisfied or if the platform isn't supported. > Since oauth/curl have some security implications, would it make more > sense to call the switch --enable-oauth (-Doauth) so users could > control better what features their libpq is going to have? Perhaps > some other feature (pg_service as URL?) is going to need libcurl as > well, but it should be configurable separately. Perhaps --with-oauth-client for the opt-in libpq-oauth? -- Daniel Gustafsson ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Add support for OAUTHBEARER SASL mechanism @ 2025-04-01 13:47 Daniel Gustafsson <[email protected]> parent: Jacob Champion <[email protected]> 0 siblings, 0 replies; 9+ messages in thread From: Daniel Gustafsson @ 2025-04-01 13:47 UTC (permalink / raw) To: Jacob Champion <[email protected]>; +Cc: Christoph Berg <[email protected]>; Thomas Munro <[email protected]>; [email protected] > On 1 Apr 2025, at 02:06, Jacob Champion <[email protected]> wrote: > > On Mon, Mar 31, 2025 at 4:09 PM Jacob Champion > <[email protected]> wrote: >> I don't have hurd-amd64 to test, but I'm working on a patch that will >> build and pass tests if I manually munge pg_config.h. We were skipping >> the useless tests via a $windows_os check; I think I should use >> check_pg_config() instead. > > Proposed fix attached. Thanks, I agree that this is the right fix. While this is all subject to change, I will go ahead with this patch in the meantime to make the tree properly buildable. -- Daniel Gustafsson ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Add support for OAUTHBEARER SASL mechanism @ 2025-04-01 15:48 Jacob Champion <[email protected]> parent: Daniel Gustafsson <[email protected]> 0 siblings, 0 replies; 9+ messages in thread From: Jacob Champion @ 2025-04-01 15:48 UTC (permalink / raw) To: Daniel Gustafsson <[email protected]>; Peter Eisentraut <[email protected]>; +Cc: Christoph Berg <[email protected]>; Thomas Munro <[email protected]>; [email protected] On Tue, Apr 1, 2025 at 6:12 AM Daniel Gustafsson <[email protected]> wrote: > > > On 1 Apr 2025, at 15:03, Christoph Berg <[email protected]> wrote: > > > With the libpq-oauth split, this makes even more sense because > > building a library that always throws an error isn't very useful. > > (Don't build that file at all if the feature doesn't work.) > > After the split, configure/meson should fail if the libcurl dependency isn't > satisfied or if the platform isn't supported. Yeah, after sleeping on it I agree. If I want a "canary" buildfarm animal to opt into compilation on unsupported platforms, I can instead look into a manual #define or something; it doesn't have to be a supported configure-time thing. > > Since oauth/curl have some security implications, would it make more > > sense to call the switch --enable-oauth (-Doauth) so users could > > control better what features their libpq is going to have? Perhaps > > some other feature (pg_service as URL?) is going to need libcurl as > > well, but it should be configurable separately. > > Perhaps --with-oauth-client for the opt-in libpq-oauth? It started as -Doauth way back when, but was changed as part of the discussion at [1]. Peter, do you have any objections to switching back to an OAuth-related name? --Jacob [1] https://postgr.es/m/6bde5f56-9e7a-4148-b81c-eb6532cb3651%40eisentraut.org ^ permalink raw reply [nested|flat] 9+ messages in thread
end of thread, other threads:[~2025-04-01 15:48 UTC | newest] Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2025-02-20 15:47 pgsql: Add support for OAUTHBEARER SASL mechanism Daniel Gustafsson <[email protected]> 2025-03-31 21:54 ` Christoph Berg <[email protected]> 2025-03-31 22:17 ` Christoph Berg <[email protected]> 2025-03-31 23:09 ` Jacob Champion <[email protected]> 2025-04-01 00:06 ` Jacob Champion <[email protected]> 2025-04-01 13:47 ` Daniel Gustafsson <[email protected]> 2025-04-01 13:03 ` Christoph Berg <[email protected]> 2025-04-01 13:11 ` Daniel Gustafsson <[email protected]> 2025-04-01 15:48 ` Jacob Champion <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox