pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feed[PoC] Federated Authn/z with OAUTHBEARER
424+ messages / 34 participants
[nested] [flat]
Commits from this thread:
2025-02-20 b3f0be788a Add support for OAUTHBEARER SASL mechanism
2024-09-11 0785d1b8b2 common/jsonapi: support libpq as a client
2024-03-21 adcdb2c8dd Explicitly require password for SCRAM exchange
2024-03-21 24178e235e Refactor SASL exchange to return tri-state status
* [PoC] Federated Authn/z with OAUTHBEARER
@ 2021-06-08 16:37 Jacob Champion <pchampion@vmware.com>
0 siblings, 2 replies; 424+ messages in thread
From: Jacob Champion @ 2021-06-08 16:37 UTC (permalink / raw)
To: pgsql-hackers
Hi all,
We've been working on ways to expand the list of third-party auth
methods that Postgres provides. Some example use cases might be "I want
to let anyone with a Google account read this table" or "let anyone who
belongs to this GitHub organization connect as a superuser".
Attached is a proof of concept that implements pieces of OAuth 2.0
federated authorization, via the OAUTHBEARER SASL mechanism from RFC
7628 [1]. Currently, only Linux is supported due to some ugly hacks in
the backend.
The architecture can support the following use cases, as long as your
OAuth issuer of choice implements the necessary specs, and you know how
to write a validator for your issuer's bearer tokens:
- Authentication only, where an external validator uses the bearer
token to determine the end user's identity, and Postgres decides
whether that user ID is authorized to connect via the standard pg_ident
user mapping.
- Authorization only, where the validator uses the bearer token to
determine the allowed roles for the end user, and then checks to make
sure that the connection's role is one of those. This bypasses pg_ident
and allows pseudonymous connections, where Postgres doesn't care who
you are as long as the token proves you're allowed to assume the role
you want.
- A combination, where the validator provides both an authn_id (for
later audits of database access) and an authorization decision based on
the bearer token and role provided.
It looks kinda like this during use:
$ psql 'host=example.org oauth_client_id=f02c6361-0635-...'
Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG
= Quickstart =
For anyone who likes building and seeing green tests ASAP.
Prerequisite software:
- iddawc v0.9.9 [2], library and dev headers, for client support
- Python 3, for the test suite only
(Some newer distributions have dev packages for iddawc, but mine did
not.)
Configure using --with-oauth (and, if you've installed iddawc into a
non-standard location, be sure to use --with-includes and --with-
libraries. Make sure either rpath or LD_LIBRARY_PATH will get you what
you need). Install as usual.
To run the test suite, make sure the contrib/authn_id extension is
installed, then init and start your dev cluster. No other configuration
is required; the test will do it for you. Switch to the src/test/python
directory, point your PG* envvars to a superuser connection on the
cluster (so that a "bare" psql will connect automatically), and run
`make installcheck`.
= Production Setup =
(but don't use this in production, please)
Actually setting up a "real" system requires knowing the specifics of
your third-party issuer of choice. Your issuer MUST implement OpenID
Discovery and the OAuth Device Authorization flow! Seriously, check
this before spending a lot of time writing a validator against an
issuer that can't actually talk to libpq.
The broad strokes are as follows:
1. Register a new public client with your issuer to get an OAuth client
ID for libpq. You'll use this as the oauth_client_id in the connection
string. (If your issuer doesn't support public clients and gives you a
client secret, you can use the oauth_client_secret connection parameter
to provide that too.)
The client you register must be able to use a device authorization
flow; some issuers require additional setup for that.
2. Set up your HBA with the 'oauth' auth method, and set the 'issuer'
and 'scope' options. 'issuer' is the base URL identifying your third-
party issuer (for example, https://accounts.google.com), and 'scope' is
the set of OAuth scopes that the client and server will need to
authenticate and/or authorize the user (e.g. "openid email").
So a sample HBA line might look like
host all all samehost oauth issuer="https://accounts.google.com"; scope="openid email"
3. In postgresql.conf, set up an oauth_validator_command that's capable
of verifying bearer tokens and implements the validator protocol. This
is the hardest part. See below.
= Design =
On the client side, I've implemented the Device Authorization flow (RFC
8628, [3]). What this means in practice is that libpq reaches out to a
third-party issuer (e.g. Google, Azure, etc.), identifies itself with a
client ID, and requests permission to act on behalf of the end user.
The issuer responds with a login URL and a one-time code, which libpq
presents to the user using the notice hook. The end user then navigates
to that URL, presents their code, authenticates to the issuer, and
grants permission for libpq to retrieve a bearer token. libpq grabs a
token and sends it to the server for verification.
(The bearer token, in this setup, is essentially a plaintext password,
and you must secure it like you would a plaintext password. The token
has an expiration date and can be explicitly revoked, which makes it
slightly better than a password, but this is still a step backwards
from something like SCRAM with channel binding. There are ways to bind
a bearer token to a client certificate [4], which would mitigate the
risk of token theft -- but your issuer has to support that, and I
haven't found much support in the wild.)
The server side is where things get more difficult for the DBA. The
OAUTHBEARER spec has this to say about the server side implementation:
The server validates the response according to the specification for
the OAuth Access Token Types used.
And here's what the Bearer Token specification [5] says:
This document does not specify the encoding or the contents of the
token; hence, detailed recommendations about the means of
guaranteeing token integrity protection are outside the scope of
this document.
It's the Wild West. Every issuer does their own thing in their own
special way. Some don't really give you a way to introspect information
about a bearer token at all, because they assume that the issuer of the
token and the consumer of the token are essentially the same service.
Some major players provide their own custom libraries, implemented in
your-language-of-choice, to deal with their particular brand of magic.
So I punted and added the oauth_validator_command GUC. A token
validator command reads the bearer token from a file descriptor that's
passed to it, then does whatever magic is necessary to validate that
token and find out who owns it. Optionally, it can look at the role
that's being connected and make sure that the token authorizes the user
to actually use that role. Then it says yea or nay to Postgres, and
optionally tells the server who the user is so that their ID can be
logged and mapped through pg_ident.
(See the commit message in 0005 for a full description of the protocol.
The test suite also has two toy implementations that illustrate the
protocol, but they provide zero security.)
This is easily the worst part of the patch, not only because my
implementation is a bad hack on OpenPipeStream(), but because it
balances the security of the entire system on the shoulders of a DBA
who does not have time to read umpteen OAuth specifications cover to
cover. More thought and coding effort is needed here, but I didn't want
to gold-plate a bad design. I'm not sure what alternatives there are
within the rules laid out by OAUTHBEARER. And the system is _extremely_
flexible, in the way that only code that's maintained by somebody else
can be.
= Patchset Roadmap =
The seven patches can be grouped into three:
1. Prep
0001 decouples the SASL code from the SCRAM implementation.
0002 makes it possible to use common/jsonapi from the frontend.
0003 lets the json_errdetail() result be freed, to avoid leaks.
2. OAUTHBEARER Implementation
0004 implements the client with libiddawc.
0005 implements server HBA support and oauth_validator_command.
3. Testing
0006 adds a simple test extension to retrieve the authn_id.
0007 adds the Python test suite I've been developing against.
The first three patches are, hopefully, generally useful outside of
this implementation, and I'll plan to register them in the next
commitfest. The middle two patches are the "interesting" pieces, and
I've split them into client and server for ease of understanding,
though neither is particularly useful without the other.
The last two patches grew out of a test suite that I originally built
to be able to exercise NSS corner cases at the protocol/byte level. It
was incredibly helpful during implementation of this new SASL
mechanism, since I could write the client and server independently of
each other and get high coverage of broken/malicious implementations.
It's based on pytest and Construct, and the Python 3 requirement might
turn some away, but I wanted to include it in case anyone else wanted
to hack on the code. src/test/python/README explains more.
= Thoughts/Reflections =
...in no particular order.
I picked OAuth 2.0 as my first experiment in federated auth mostly
because I was already familiar with pieces of it. I think SAML (via the
SAML20 mechanism, RFC 6595) would be a good companion to this proof of
concept, if there is general interest in federated deployments.
I don't really like the OAUTHBEARER spec, but I'm not sure there's a
better alternative. Everything is left as an exercise for the reader.
It's not particularly extensible. Standard OAuth is built for
authorization, not authentication, and from reading the RFC's history,
it feels like it was a hack to just get something working. New
standards like OpenID Connect have begun to fill in the gaps, but the
SASL mechanisms have not kept up. (The OPENID20 mechanism is, to my
understanding, unrelated/obsolete.) And support for helpful OIDC
features seems to be spotty in the real world.
The iddawc dependency for client-side OAuth was extremely helpful to
develop this proof of concept quickly, but I don't think it would be an
appropriate component to build a real feature on. It's extremely
heavyweight -- it incorporates a huge stack of dependencies, including
a logging framework and a web server, to implement features we would
probably never use -- and it's fairly difficult to debug in practice.
If a device authorization flow were the only thing that libpq needed to
support natively, I think we should just depend on a widely used HTTP
client, like libcurl or neon, and implement the minimum spec directly
against the existing test suite.
There are a huge number of other authorization flows besides Device
Authorization; most would involve libpq automatically opening a web
browser for you. I felt like that wasn't an appropriate thing for a
library to do by default, especially when one of the most important
clients is a command-line application. Perhaps there could be a hook
for applications to be able to override the builtin flow and substitute
their own.
Since bearer tokens are essentially plaintext passwords, the relevant
specs require the use of transport-level protection, and I think it'd
be wise for the client to require TLS to be in place before performing
the initial handshake or sending a token.
Not every OAuth issuer is also an OpenID Discovery provider, so it's
frustrating that OAUTHBEARER (which is purportedly an OAuth 2.0
feature) requires OIDD for real-world implementations. Perhaps we could
hack around this with a data: URI or something.
The client currently performs the OAuth login dance every single time a
connection is made, but a proper OAuth client would cache its tokens to
reuse later, and keep an eye on their expiration times. This would make
daily use a little more like that of Kerberos, but we would have to
design a way to create and secure a token cache on disk.
If you've read this far, thank you for your interest, and I hope you
enjoy playing with it!
--Jacob
[1] https://datatracker.ietf.org/doc/html/rfc7628
[2] https://github.com/babelouest/iddawc
[3] https://datatracker.ietf.org/doc/html/rfc8628
[4] https://datatracker.ietf.org/doc/html/rfc8705
[5] https://datatracker.ietf.org/doc/html/rfc6750#section-5.2
Attachments: download all (7 files, .tar.gz)
0001-auth-generalize-SASL-mechanisms.patch (text/x-patch, 16.3K) download
From a6a65b66cc3dc5da7219378dbadb090ff10fd42b Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 13 Apr 2021 10:25:48 -0700
Subject: [PATCH 1/7] auth: generalize SASL mechanisms
Split the SASL logic out from the SCRAM implementation, so that it can
be reused by other mechanisms. New implementations will implement both
a pg_sasl_mech and a pg_be_sasl_mech.
---
src/backend/libpq/auth-scram.c | 34 ++++++++++++++---------
src/backend/libpq/auth.c | 34 ++++++++++++++++-------
src/include/libpq/sasl.h | 34 +++++++++++++++++++++++
src/include/libpq/scram.h | 13 +++------
src/interfaces/libpq/fe-auth-scram.c | 40 +++++++++++++++++++---------
src/interfaces/libpq/fe-auth.c | 16 ++++++++---
src/interfaces/libpq/fe-auth.h | 11 ++------
src/interfaces/libpq/fe-connect.c | 6 +----
src/interfaces/libpq/libpq-int.h | 14 ++++++++++
9 files changed, 139 insertions(+), 63 deletions(-)
create mode 100644 src/include/libpq/sasl.h
diff --git a/src/backend/libpq/auth-scram.c b/src/backend/libpq/auth-scram.c
index f9e1026a12..db3ca75a60 100644
--- a/src/backend/libpq/auth-scram.c
+++ b/src/backend/libpq/auth-scram.c
@@ -101,11 +101,25 @@
#include "common/sha2.h"
#include "libpq/auth.h"
#include "libpq/crypt.h"
+#include "libpq/sasl.h"
#include "libpq/scram.h"
#include "miscadmin.h"
#include "utils/builtins.h"
#include "utils/timestamp.h"
+static void scram_get_mechanisms(Port *port, StringInfo buf);
+static void *scram_init(Port *port, const char *selected_mech,
+ const char *shadow_pass);
+static int scram_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, char **logdetail);
+
+/* Mechanism declaration */
+const pg_be_sasl_mech pg_be_scram_mech = {
+ scram_get_mechanisms,
+ scram_init,
+ scram_exchange,
+};
+
/*
* Status data for a SCRAM authentication exchange. This should be kept
* internal to this file.
@@ -170,16 +184,14 @@ static char *sanitize_str(const char *s);
static char *scram_mock_salt(const char *username);
/*
- * pg_be_scram_get_mechanisms
- *
* Get a list of SASL mechanisms that this module supports.
*
* For the convenience of building the FE/BE packet that lists the
* mechanisms, the names are appended to the given StringInfo buffer,
* separated by '\0' bytes.
*/
-void
-pg_be_scram_get_mechanisms(Port *port, StringInfo buf)
+static void
+scram_get_mechanisms(Port *port, StringInfo buf)
{
/*
* Advertise the mechanisms in decreasing order of importance. So the
@@ -199,8 +211,6 @@ pg_be_scram_get_mechanisms(Port *port, StringInfo buf)
}
/*
- * pg_be_scram_init
- *
* Initialize a new SCRAM authentication exchange status tracker. This
* needs to be called before doing any exchange. It will be filled later
* after the beginning of the exchange with authentication information.
@@ -215,10 +225,8 @@ pg_be_scram_get_mechanisms(Port *port, StringInfo buf)
* an authentication exchange, but it will fail, as if an incorrect password
* was given.
*/
-void *
-pg_be_scram_init(Port *port,
- const char *selected_mech,
- const char *shadow_pass)
+static void *
+scram_init(Port *port, const char *selected_mech, const char *shadow_pass)
{
scram_state *state;
bool got_secret;
@@ -325,9 +333,9 @@ pg_be_scram_init(Port *port,
* string at *logdetail that will be sent to the postmaster log (but not
* the client).
*/
-int
-pg_be_scram_exchange(void *opaq, const char *input, int inputlen,
- char **output, int *outputlen, char **logdetail)
+static int
+scram_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, char **logdetail)
{
scram_state *state = (scram_state *) opaq;
int result;
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 68372fcea8..e20740a7c5 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -26,11 +26,11 @@
#include "commands/user.h"
#include "common/ip.h"
#include "common/md5.h"
-#include "common/scram-common.h"
#include "libpq/auth.h"
#include "libpq/crypt.h"
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
+#include "libpq/sasl.h"
#include "libpq/scram.h"
#include "miscadmin.h"
#include "port/pg_bswap.h"
@@ -51,6 +51,13 @@ static void auth_failed(Port *port, int status, char *logdetail);
static char *recv_password_packet(Port *port);
static void set_authn_id(Port *port, const char *id);
+/*----------------------------------------------------------------
+ * SASL common authentication
+ *----------------------------------------------------------------
+ */
+static int SASL_exchange(const pg_be_sasl_mech *mech, Port *port,
+ char *shadow_pass, char **logdetail);
+
/*----------------------------------------------------------------
* Password-based authentication methods (password, md5, and scram-sha-256)
@@ -912,12 +919,13 @@ CheckMD5Auth(Port *port, char *shadow_pass, char **logdetail)
}
static int
-CheckSCRAMAuth(Port *port, char *shadow_pass, char **logdetail)
+SASL_exchange(const pg_be_sasl_mech *mech, Port *port, char *shadow_pass,
+ char **logdetail)
{
StringInfoData sasl_mechs;
int mtype;
StringInfoData buf;
- void *scram_opaq = NULL;
+ void *opaq = NULL;
char *output = NULL;
int outputlen = 0;
const char *input;
@@ -931,7 +939,7 @@ CheckSCRAMAuth(Port *port, char *shadow_pass, char **logdetail)
*/
initStringInfo(&sasl_mechs);
- pg_be_scram_get_mechanisms(port, &sasl_mechs);
+ mech->get_mechanisms(port, &sasl_mechs);
/* Put another '\0' to mark that list is finished. */
appendStringInfoChar(&sasl_mechs, '\0');
@@ -998,7 +1006,7 @@ CheckSCRAMAuth(Port *port, char *shadow_pass, char **logdetail)
* This is because we don't want to reveal to an attacker what
* usernames are valid, nor which users have a valid password.
*/
- scram_opaq = pg_be_scram_init(port, selected_mech, shadow_pass);
+ opaq = mech->init(port, selected_mech, shadow_pass);
inputlen = pq_getmsgint(&buf, 4);
if (inputlen == -1)
@@ -1022,12 +1030,11 @@ CheckSCRAMAuth(Port *port, char *shadow_pass, char **logdetail)
Assert(input == NULL || input[inputlen] == '\0');
/*
- * we pass 'logdetail' as NULL when doing a mock authentication,
- * because we should already have a better error message in that case
+ * Hand the incoming message to the mechanism implementation.
*/
- result = pg_be_scram_exchange(scram_opaq, input, inputlen,
- &output, &outputlen,
- logdetail);
+ result = mech->exchange(opaq, input, inputlen,
+ &output, &outputlen,
+ logdetail);
/* input buffer no longer used */
pfree(buf.data);
@@ -1039,6 +1046,7 @@ CheckSCRAMAuth(Port *port, char *shadow_pass, char **logdetail)
*/
elog(DEBUG4, "sending SASL challenge of length %u", outputlen);
+ /* TODO: SASL_EXCHANGE_FAILURE with output is forbidden in SASL */
if (result == SASL_EXCHANGE_SUCCESS)
sendAuthRequest(port, AUTH_REQ_SASL_FIN, output, outputlen);
else
@@ -1057,6 +1065,12 @@ CheckSCRAMAuth(Port *port, char *shadow_pass, char **logdetail)
return STATUS_OK;
}
+static int
+CheckSCRAMAuth(Port *port, char *shadow_pass, char **logdetail)
+{
+ return SASL_exchange(&pg_be_scram_mech, port, shadow_pass, logdetail);
+}
+
/*----------------------------------------------------------------
* GSSAPI authentication system
diff --git a/src/include/libpq/sasl.h b/src/include/libpq/sasl.h
new file mode 100644
index 0000000000..8c9c9983d4
--- /dev/null
+++ b/src/include/libpq/sasl.h
@@ -0,0 +1,34 @@
+/*-------------------------------------------------------------------------
+ *
+ * sasl.h
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/libpq/sasl.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_SASL_H
+#define PG_SASL_H
+
+#include "libpq/libpq-be.h"
+
+/* Status codes for message exchange */
+#define SASL_EXCHANGE_CONTINUE 0
+#define SASL_EXCHANGE_SUCCESS 1
+#define SASL_EXCHANGE_FAILURE 2
+
+/* Backend mechanism API */
+typedef void (*pg_be_sasl_mechanism_func)(Port *, StringInfo);
+typedef void *(*pg_be_sasl_init_func)(Port *, const char *, const char *);
+typedef int (*pg_be_sasl_exchange_func)(void *, const char *, int, char **, int *, char **);
+
+typedef struct
+{
+ pg_be_sasl_mechanism_func get_mechanisms;
+ pg_be_sasl_init_func init;
+ pg_be_sasl_exchange_func exchange;
+} pg_be_sasl_mech;
+
+#endif /* PG_SASL_H */
diff --git a/src/include/libpq/scram.h b/src/include/libpq/scram.h
index 2c879150da..9e4540bde3 100644
--- a/src/include/libpq/scram.h
+++ b/src/include/libpq/scram.h
@@ -15,17 +15,10 @@
#include "lib/stringinfo.h"
#include "libpq/libpq-be.h"
+#include "libpq/sasl.h"
-/* Status codes for message exchange */
-#define SASL_EXCHANGE_CONTINUE 0
-#define SASL_EXCHANGE_SUCCESS 1
-#define SASL_EXCHANGE_FAILURE 2
-
-/* Routines dedicated to authentication */
-extern void pg_be_scram_get_mechanisms(Port *port, StringInfo buf);
-extern void *pg_be_scram_init(Port *port, const char *selected_mech, const char *shadow_pass);
-extern int pg_be_scram_exchange(void *opaq, const char *input, int inputlen,
- char **output, int *outputlen, char **logdetail);
+/* Implementation */
+extern const pg_be_sasl_mech pg_be_scram_mech;
/* Routines to handle and check SCRAM-SHA-256 secret */
extern char *pg_be_scram_build_secret(const char *password);
diff --git a/src/interfaces/libpq/fe-auth-scram.c b/src/interfaces/libpq/fe-auth-scram.c
index 5881386e37..04d5703d89 100644
--- a/src/interfaces/libpq/fe-auth-scram.c
+++ b/src/interfaces/libpq/fe-auth-scram.c
@@ -21,6 +21,22 @@
#include "fe-auth.h"
+/* The exported SCRAM callback mechanism. */
+static void *scram_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism);
+static void scram_exchange(void *opaq, char *input, int inputlen,
+ char **output, int *outputlen,
+ bool *done, bool *success);
+static bool scram_channel_bound(void *opaq);
+static void scram_free(void *opaq);
+
+const pg_sasl_mech pg_scram_mech = {
+ scram_init,
+ scram_exchange,
+ scram_channel_bound,
+ scram_free,
+};
+
/*
* Status of exchange messages used for SCRAM authentication via the
* SASL protocol.
@@ -72,10 +88,10 @@ static bool calculate_client_proof(fe_scram_state *state,
/*
* Initialize SCRAM exchange status.
*/
-void *
-pg_fe_scram_init(PGconn *conn,
- const char *password,
- const char *sasl_mechanism)
+static void *
+scram_init(PGconn *conn,
+ const char *password,
+ const char *sasl_mechanism)
{
fe_scram_state *state;
char *prep_password;
@@ -128,8 +144,8 @@ pg_fe_scram_init(PGconn *conn,
* Note that the caller must also ensure that the exchange was actually
* successful.
*/
-bool
-pg_fe_scram_channel_bound(void *opaq)
+static bool
+scram_channel_bound(void *opaq)
{
fe_scram_state *state = (fe_scram_state *) opaq;
@@ -152,8 +168,8 @@ pg_fe_scram_channel_bound(void *opaq)
/*
* Free SCRAM exchange status
*/
-void
-pg_fe_scram_free(void *opaq)
+static void
+scram_free(void *opaq)
{
fe_scram_state *state = (fe_scram_state *) opaq;
@@ -188,10 +204,10 @@ pg_fe_scram_free(void *opaq)
/*
* Exchange a SCRAM message with backend.
*/
-void
-pg_fe_scram_exchange(void *opaq, char *input, int inputlen,
- char **output, int *outputlen,
- bool *done, bool *success)
+static void
+scram_exchange(void *opaq, char *input, int inputlen,
+ char **output, int *outputlen,
+ bool *done, bool *success)
{
fe_scram_state *state = (fe_scram_state *) opaq;
PGconn *conn = state->conn;
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index e8062647e6..d5cbac108e 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -482,7 +482,10 @@ pg_SASL_init(PGconn *conn, int payloadlen)
* channel_binding is not disabled.
*/
if (conn->channel_binding[0] != 'd') /* disable */
+ {
selected_mechanism = SCRAM_SHA_256_PLUS_NAME;
+ conn->sasl = &pg_scram_mech;
+ }
#else
/*
* The client does not support channel binding. If it is
@@ -516,7 +519,10 @@ pg_SASL_init(PGconn *conn, int payloadlen)
}
else if (strcmp(mechanism_buf.data, SCRAM_SHA_256_NAME) == 0 &&
!selected_mechanism)
+ {
selected_mechanism = SCRAM_SHA_256_NAME;
+ conn->sasl = &pg_scram_mech;
+ }
}
if (!selected_mechanism)
@@ -555,20 +561,22 @@ pg_SASL_init(PGconn *conn, int payloadlen)
goto error;
}
+ Assert(conn->sasl);
+
/*
* Initialize the SASL state information with all the information gathered
* during the initial exchange.
*
* Note: Only tls-unique is supported for the moment.
*/
- conn->sasl_state = pg_fe_scram_init(conn,
+ conn->sasl_state = conn->sasl->init(conn,
password,
selected_mechanism);
if (!conn->sasl_state)
goto oom_error;
/* Get the mechanism-specific Initial Client Response, if any */
- pg_fe_scram_exchange(conn->sasl_state,
+ conn->sasl->exchange(conn->sasl_state,
NULL, -1,
&initialresponse, &initialresponselen,
&done, &success);
@@ -649,7 +657,7 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
/* For safety and convenience, ensure the buffer is NULL-terminated. */
challenge[payloadlen] = '\0';
- pg_fe_scram_exchange(conn->sasl_state,
+ conn->sasl->exchange(conn->sasl_state,
challenge, payloadlen,
&output, &outputlen,
&done, &success);
@@ -830,7 +838,7 @@ check_expected_areq(AuthRequest areq, PGconn *conn)
case AUTH_REQ_SASL_FIN:
break;
case AUTH_REQ_OK:
- if (!pg_fe_scram_channel_bound(conn->sasl_state))
+ if (!conn->sasl || !conn->sasl->channel_bound(conn->sasl_state))
{
appendPQExpBufferStr(&conn->errorMessage,
libpq_gettext("channel binding required, but server authenticated client without channel binding\n"));
diff --git a/src/interfaces/libpq/fe-auth.h b/src/interfaces/libpq/fe-auth.h
index 7877dcbd09..1e4fcbff62 100644
--- a/src/interfaces/libpq/fe-auth.h
+++ b/src/interfaces/libpq/fe-auth.h
@@ -22,15 +22,8 @@
extern int pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn);
extern char *pg_fe_getauthname(PQExpBuffer errorMessage);
-/* Prototypes for functions in fe-auth-scram.c */
-extern void *pg_fe_scram_init(PGconn *conn,
- const char *password,
- const char *sasl_mechanism);
-extern bool pg_fe_scram_channel_bound(void *opaq);
-extern void pg_fe_scram_free(void *opaq);
-extern void pg_fe_scram_exchange(void *opaq, char *input, int inputlen,
- char **output, int *outputlen,
- bool *done, bool *success);
+/* Mechanisms in fe-auth-scram.c */
+extern const pg_sasl_mech pg_scram_mech;
extern char *pg_fe_scram_build_secret(const char *password);
#endif /* FE_AUTH_H */
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 80703698b8..10d007582c 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -517,11 +517,7 @@ pqDropConnection(PGconn *conn, bool flushInput)
#endif
if (conn->sasl_state)
{
- /*
- * XXX: if support for more authentication mechanisms is added, this
- * needs to call the right 'free' function.
- */
- pg_fe_scram_free(conn->sasl_state);
+ conn->sasl->free(conn->sasl_state);
conn->sasl_state = NULL;
}
}
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index e81dc37906..25eaa231c5 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -339,6 +339,19 @@ typedef struct pg_conn_host
* found in password file. */
} pg_conn_host;
+typedef void *(*pg_sasl_init_func)(PGconn *, const char *, const char *);
+typedef void (*pg_sasl_exchange_func)(void *, char *, int, char **, int *, bool *, bool *);
+typedef bool (*pg_sasl_channel_bound_func)(void *);
+typedef void (*pg_sasl_free_func)(void *);
+
+typedef struct
+{
+ pg_sasl_init_func init;
+ pg_sasl_exchange_func exchange;
+ pg_sasl_channel_bound_func channel_bound;
+ pg_sasl_free_func free;
+} pg_sasl_mech;
+
/*
* PGconn stores all the state data associated with a single connection
* to a backend.
@@ -500,6 +513,7 @@ struct pg_conn
PGresult *next_result; /* next result (used in single-row mode) */
/* Assorted state for SASL, SSL, GSS, etc */
+ const pg_sasl_mech *sasl;
void *sasl_state;
/* SSL structures */
--
2.25.1
0002-src-common-remove-logging-from-jsonapi-for-shlib.patch (text/x-patch, 1.7K) download
From 0541598e4f0bad1b9ff41a4640ec69491b393d54 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Mon, 3 May 2021 11:15:15 -0700
Subject: [PATCH 2/7] src/common: remove logging from jsonapi for shlib
The can't-happen code in jsonapi was pulling in logging code, which for
libpq is not included.
---
src/common/Makefile | 4 ++++
src/common/jsonapi.c | 11 ++++++++---
2 files changed, 12 insertions(+), 3 deletions(-)
diff --git a/src/common/Makefile b/src/common/Makefile
index 38a8599337..6f1039bc78 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -28,6 +28,10 @@ subdir = src/common
top_builddir = ../..
include $(top_builddir)/src/Makefile.global
+# For use in shared libraries, jsonapi needs to not link in any logging
+# functions.
+override CFLAGS_SL += -DJSONAPI_NO_LOG
+
# don't include subdirectory-path-dependent -I and -L switches
STD_CPPFLAGS := $(filter-out -I$(top_srcdir)/src/include -I$(top_builddir)/src/include,$(CPPFLAGS))
STD_LDFLAGS := $(filter-out -L$(top_builddir)/src/common -L$(top_builddir)/src/port,$(LDFLAGS))
diff --git a/src/common/jsonapi.c b/src/common/jsonapi.c
index 1bf38d7b42..6b6001b118 100644
--- a/src/common/jsonapi.c
+++ b/src/common/jsonapi.c
@@ -27,11 +27,16 @@
#endif
#ifdef FRONTEND
-#define check_stack_depth()
-#define json_log_and_abort(...) \
+# define check_stack_depth()
+# ifdef JSONAPI_NO_LOG
+# define json_log_and_abort(...) \
+ do { fprintf(stderr, __VA_ARGS__); exit(1); } while(0)
+# else
+# define json_log_and_abort(...) \
do { pg_log_fatal(__VA_ARGS__); exit(1); } while(0)
+# endif
#else
-#define json_log_and_abort(...) elog(ERROR, __VA_ARGS__)
+# define json_log_and_abort(...) elog(ERROR, __VA_ARGS__)
#endif
/*
--
2.25.1
0003-common-jsonapi-always-palloc-the-error-strings.patch (text/x-patch, 2.2K) download
From 5ad4b3c7835fe9e0f284702ec7b827c27770854e Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Mon, 3 May 2021 15:38:26 -0700
Subject: [PATCH 3/7] common/jsonapi: always palloc the error strings
...so that client code can pfree() to avoid memory leaks in long-running
operations.
---
src/common/jsonapi.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/src/common/jsonapi.c b/src/common/jsonapi.c
index 6b6001b118..f7304f584f 100644
--- a/src/common/jsonapi.c
+++ b/src/common/jsonapi.c
@@ -1089,7 +1089,7 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
return psprintf(_("Expected JSON value, but found \"%s\"."),
extract_token(lex));
case JSON_EXPECTED_MORE:
- return _("The input string ended unexpectedly.");
+ return pstrdup(_("The input string ended unexpectedly."));
case JSON_EXPECTED_OBJECT_FIRST:
return psprintf(_("Expected string or \"}\", but found \"%s\"."),
extract_token(lex));
@@ -1103,16 +1103,16 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
return psprintf(_("Token \"%s\" is invalid."),
extract_token(lex));
case JSON_UNICODE_CODE_POINT_ZERO:
- return _("\\u0000 cannot be converted to text.");
+ return pstrdup(_("\\u0000 cannot be converted to text."));
case JSON_UNICODE_ESCAPE_FORMAT:
- return _("\"\\u\" must be followed by four hexadecimal digits.");
+ return pstrdup(_("\"\\u\" must be followed by four hexadecimal digits."));
case JSON_UNICODE_HIGH_ESCAPE:
/* note: this case is only reachable in frontend not backend */
- return _("Unicode escape values cannot be used for code point values above 007F when the encoding is not UTF8.");
+ return pstrdup(_("Unicode escape values cannot be used for code point values above 007F when the encoding is not UTF8."));
case JSON_UNICODE_HIGH_SURROGATE:
- return _("Unicode high surrogate must not follow a high surrogate.");
+ return pstrdup(_("Unicode high surrogate must not follow a high surrogate."));
case JSON_UNICODE_LOW_SURROGATE:
- return _("Unicode low surrogate must follow a high surrogate.");
+ return pstrdup(_("Unicode low surrogate must follow a high surrogate."));
}
/*
--
2.25.1
0004-libpq-add-OAUTHBEARER-SASL-mechanism.patch (text/x-patch, 34.5K) download
From e3d95709e147ae3670bd8acd0c265493a6116b9a Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 13 Apr 2021 10:27:27 -0700
Subject: [PATCH 4/7] libpq: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) and OAuth 2.0 Device Authorization
Grants (RFC 8628) on the client side. When speaking to a OAuth-enabled
server, it looks a bit like this:
$ psql 'host=example.org oauth_client_id=f02c6361-0635-...'
Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG
The OAuth issuer must support device authorization. No other OAuth flows
are currently implemented.
The client implementation requires libiddawc and its development
headers. Configure --with-oauth (and --with-includes/--with-libraries to
point at the iddawc installation, if it's in a custom location).
Several TODOs:
- don't retry forever if the server won't accept our token
- perform several sanity checks on the OAuth issuer's responses
- handle cases where the client has been set up with an issuer and
scope, but the Postgres server wants to use something different
- improve error debuggability during the OAuth handshake
- ...and more.
---
configure | 100 ++++
configure.ac | 19 +
src/Makefile.global.in | 1 +
src/include/common/oauth-common.h | 19 +
src/include/pg_config.h.in | 6 +
src/interfaces/libpq/Makefile | 7 +-
src/interfaces/libpq/fe-auth-oauth.c | 724 +++++++++++++++++++++++++++
src/interfaces/libpq/fe-auth-scram.c | 6 +-
src/interfaces/libpq/fe-auth.c | 42 +-
src/interfaces/libpq/fe-auth.h | 3 +
src/interfaces/libpq/fe-connect.c | 38 ++
src/interfaces/libpq/libpq-int.h | 10 +-
12 files changed, 956 insertions(+), 19 deletions(-)
create mode 100644 src/include/common/oauth-common.h
create mode 100644 src/interfaces/libpq/fe-auth-oauth.c
diff --git a/configure b/configure
index e9b98f442f..c3b7a89bf0 100755
--- a/configure
+++ b/configure
@@ -713,6 +713,7 @@ with_uuid
with_readline
with_systemd
with_selinux
+with_oauth
with_ldap
with_krb_srvnam
krb_srvtab
@@ -856,6 +857,7 @@ with_krb_srvnam
with_pam
with_bsd_auth
with_ldap
+with_oauth
with_bonjour
with_selinux
with_systemd
@@ -1562,6 +1564,7 @@ Optional Packages:
--with-pam build with PAM support
--with-bsd-auth build with BSD Authentication support
--with-ldap build with LDAP support
+ --with-oauth build with OAuth 2.0 support
--with-bonjour build with Bonjour support
--with-selinux build with SELinux support
--with-systemd build with systemd support
@@ -8046,6 +8049,42 @@ $as_echo "$with_ldap" >&6; }
+#
+# OAuth 2.0
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with OAuth support" >&5
+$as_echo_n "checking whether to build with OAuth support... " >&6; }
+
+
+
+# Check whether --with-oauth was given.
+if test "${with_oauth+set}" = set; then :
+ withval=$with_oauth;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_OAUTH 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-oauth option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_oauth=no
+
+fi
+
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_oauth" >&5
+$as_echo "$with_oauth" >&6; }
+
+
+
#
# Bonjour
#
@@ -13048,6 +13087,56 @@ fi
+if test "$with_oauth" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for i_init_session in -liddawc" >&5
+$as_echo_n "checking for i_init_session in -liddawc... " >&6; }
+if ${ac_cv_lib_iddawc_i_init_session+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-liddawc $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char i_init_session ();
+int
+main ()
+{
+return i_init_session ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_iddawc_i_init_session=yes
+else
+ ac_cv_lib_iddawc_i_init_session=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_iddawc_i_init_session" >&5
+$as_echo "$ac_cv_lib_iddawc_i_init_session" >&6; }
+if test "x$ac_cv_lib_iddawc_i_init_session" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBIDDAWC 1
+_ACEOF
+
+ LIBS="-liddawc $LIBS"
+
+else
+ as_fn_error $? "library 'iddawc' is required for OAuth support" "$LINENO" 5
+fi
+
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for security_compute_create_name in -lselinux" >&5
@@ -13942,6 +14031,17 @@ fi
done
+fi
+
+if test "$with_oauth" != no; then
+ ac_fn_c_check_header_mongrel "$LINENO" "iddawc.h" "ac_cv_header_iddawc_h" "$ac_includes_default"
+if test "x$ac_cv_header_iddawc_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <iddawc.h> is required for OAuth" "$LINENO" 5
+fi
+
+
fi
if test "$PORTNAME" = "win32" ; then
diff --git a/configure.ac b/configure.ac
index 3b42d8bdc9..f15f6f64d5 100644
--- a/configure.ac
+++ b/configure.ac
@@ -842,6 +842,17 @@ AC_MSG_RESULT([$with_ldap])
AC_SUBST(with_ldap)
+#
+# OAuth 2.0
+#
+AC_MSG_CHECKING([whether to build with OAuth support])
+PGAC_ARG_BOOL(with, oauth, no,
+ [build with OAuth 2.0 support],
+ [AC_DEFINE([USE_OAUTH], 1, [Define to 1 to build with OAuth 2.0 support. (--with-oauth)])])
+AC_MSG_RESULT([$with_oauth])
+AC_SUBST(with_oauth)
+
+
#
# Bonjour
#
@@ -1313,6 +1324,10 @@ fi
AC_SUBST(LDAP_LIBS_FE)
AC_SUBST(LDAP_LIBS_BE)
+if test "$with_oauth" = yes ; then
+ AC_CHECK_LIB(iddawc, i_init_session, [], [AC_MSG_ERROR([library 'iddawc' is required for OAuth support])])
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
AC_CHECK_LIB(selinux, security_compute_create_name, [],
@@ -1523,6 +1538,10 @@ elif test "$with_uuid" = ossp ; then
[AC_MSG_ERROR([header file <ossp/uuid.h> or <uuid.h> is required for OSSP UUID])])])
fi
+if test "$with_oauth" != no; then
+ AC_CHECK_HEADER(iddawc.h, [], [AC_MSG_ERROR([header file <iddawc.h> is required for OAuth])])
+fi
+
if test "$PORTNAME" = "win32" ; then
AC_CHECK_HEADERS(crtdefs.h)
fi
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 8f05840821..3a61dd46d3 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -193,6 +193,7 @@ with_ldap = @with_ldap@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
+with_oauth = @with_oauth@
with_system_tzdata = @with_system_tzdata@
with_uuid = @with_uuid@
with_zlib = @with_zlib@
diff --git a/src/include/common/oauth-common.h b/src/include/common/oauth-common.h
new file mode 100644
index 0000000000..3fa95ac7e8
--- /dev/null
+++ b/src/include/common/oauth-common.h
@@ -0,0 +1,19 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth-common.h
+ * Declarations for helper functions used for OAuth/OIDC authentication
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/common/oauth-common.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef OAUTH_COMMON_H
+#define OAUTH_COMMON_H
+
+/* Name of SASL mechanism per IANA */
+#define OAUTHBEARER_NAME "OAUTHBEARER"
+
+#endif /* OAUTH_COMMON_H */
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 783b8fc1ba..db5bc56ac5 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -319,6 +319,9 @@
/* Define to 1 if you have the `crypto' library (-lcrypto). */
#undef HAVE_LIBCRYPTO
+/* Define to 1 if you have the `iddawc' library (-liddawc). */
+#undef HAVE_LIBIDDAWC
+
/* Define to 1 if you have the `ldap' library (-lldap). */
#undef HAVE_LIBLDAP
@@ -920,6 +923,9 @@
/* Define to select named POSIX semaphores. */
#undef USE_NAMED_POSIX_SEMAPHORES
+/* Define to 1 to build with OAuth 2.0 support. (--with-oauth) */
+#undef USE_OAUTH
+
/* Define to 1 to build with OpenSSL support. (--with-ssl=openssl) */
#undef USE_OPENSSL
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 0c4e55b6ad..8e89d50900 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -62,6 +62,11 @@ OBJS += \
fe-secure-gssapi.o
endif
+ifeq ($(with_oauth),yes)
+OBJS += \
+ fe-auth-oauth.o
+endif
+
ifeq ($(PORTNAME), cygwin)
override shlib = cyg$(NAME)$(DLSUFFIX)
endif
@@ -83,7 +88,7 @@ endif
# that are built correctly for use in a shlib.
SHLIB_LINK_INTERNAL = -lpgcommon_shlib -lpgport_shlib
ifneq ($(PORTNAME), win32)
-SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
+SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -liddawc -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
else
SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi32 -lssl -lsocket -lnsl -lresolv -lintl -lm $(PTHREAD_LIBS), $(LIBS)) $(LDAP_LIBS_FE)
endif
diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
new file mode 100644
index 0000000000..a27f974369
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth.c
@@ -0,0 +1,724 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth.c
+ * The front-end (client) implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include <iddawc.h>
+
+#include "postgres_fe.h"
+
+#include "common/base64.h"
+#include "common/hmac.h"
+#include "common/jsonapi.h"
+#include "common/oauth-common.h"
+#include "fe-auth.h"
+#include "mb/pg_wchar.h"
+
+/* The exported OAuth callback mechanism. */
+static void *oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism);
+static void oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen,
+ bool *done, bool *success);
+static bool oauth_channel_bound(void *opaq);
+static void oauth_free(void *opaq);
+
+const pg_sasl_mech pg_oauth_mech = {
+ oauth_init,
+ oauth_exchange,
+ oauth_channel_bound,
+ oauth_free,
+};
+
+typedef enum
+{
+ FE_OAUTH_INIT,
+ FE_OAUTH_BEARER_SENT,
+ FE_OAUTH_SERVER_ERROR,
+} fe_oauth_state_enum;
+
+typedef struct
+{
+ fe_oauth_state_enum state;
+
+ PGconn *conn;
+} fe_oauth_state;
+
+static void *
+oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism)
+{
+ fe_oauth_state *state;
+
+ /*
+ * We only support one SASL mechanism here; anything else is programmer
+ * error.
+ */
+ Assert(sasl_mechanism != NULL);
+ Assert(!strcmp(sasl_mechanism, OAUTHBEARER_NAME));
+
+ state = malloc(sizeof(*state));
+ if (!state)
+ return NULL;
+
+ state->state = FE_OAUTH_INIT;
+ state->conn = conn;
+
+ return state;
+}
+
+static const char *
+iddawc_error_string(int errcode)
+{
+ switch (errcode)
+ {
+ case I_OK:
+ return "I_OK";
+
+ case I_ERROR:
+ return "I_ERROR";
+
+ case I_ERROR_PARAM:
+ return "I_ERROR_PARAM";
+
+ case I_ERROR_MEMORY:
+ return "I_ERROR_MEMORY";
+
+ case I_ERROR_UNAUTHORIZED:
+ return "I_ERROR_UNAUTHORIZED";
+
+ case I_ERROR_SERVER:
+ return "I_ERROR_SERVER";
+ }
+
+ return "<unknown>";
+}
+
+static void
+iddawc_error(PGconn *conn, int errcode, const char *msg)
+{
+ appendPQExpBufferStr(&conn->errorMessage, libpq_gettext(msg));
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext(" (iddawc error %s)\n"),
+ iddawc_error_string(errcode));
+}
+
+static void
+iddawc_request_error(PGconn *conn, struct _i_session *i, int err, const char *msg)
+{
+ const char *error_code;
+ const char *desc;
+
+ appendPQExpBuffer(&conn->errorMessage, "%s: ", libpq_gettext(msg));
+
+ error_code = i_get_str_parameter(i, I_OPT_ERROR);
+ if (!error_code)
+ {
+ /*
+ * The server didn't give us any useful information, so just print the
+ * error code.
+ */
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("(iddawc error %s)\n"),
+ iddawc_error_string(err));
+ return;
+ }
+
+ /* If the server gave a string description, print that too. */
+ desc = i_get_str_parameter(i, I_OPT_ERROR_DESCRIPTION);
+ if (desc)
+ appendPQExpBuffer(&conn->errorMessage, "%s ", desc);
+
+ appendPQExpBuffer(&conn->errorMessage, "(%s)\n", error_code);
+}
+
+static char *
+get_auth_token(PGconn *conn)
+{
+ PQExpBuffer token_buf = NULL;
+ struct _i_session session;
+ int err;
+ int auth_method;
+ bool user_prompted = false;
+ const char *verification_uri;
+ const char *user_code;
+ const char *access_token;
+ const char *token_type;
+ char *token = NULL;
+
+ if (!conn->oauth_discovery_uri)
+ return strdup(""); /* ask the server for one */
+
+ i_init_session(&session);
+
+ if (!conn->oauth_client_id)
+ {
+ /* We can't talk to a server without a client identifier. */
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("no oauth_client_id is set for the connection"));
+ goto cleanup;
+ }
+
+ token_buf = createPQExpBuffer();
+
+ if (!token_buf)
+ goto cleanup;
+
+ err = i_set_str_parameter(&session, I_OPT_OPENID_CONFIG_ENDPOINT, conn->oauth_discovery_uri);
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set OpenID config endpoint");
+ goto cleanup;
+ }
+
+ err = i_get_openid_config(&session);
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to fetch OpenID discovery document");
+ goto cleanup;
+ }
+
+ if (!i_get_str_parameter(&session, I_OPT_TOKEN_ENDPOINT))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer has no token endpoint"));
+ goto cleanup;
+ }
+
+ if (!i_get_str_parameter(&session, I_OPT_DEVICE_AUTHORIZATION_ENDPOINT))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer does not support device authorization"));
+ goto cleanup;
+ }
+
+ err = i_set_response_type(&session, I_RESPONSE_TYPE_DEVICE_CODE);
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set device code response type");
+ goto cleanup;
+ }
+
+ auth_method = I_TOKEN_AUTH_METHOD_NONE;
+ if (conn->oauth_client_secret && *conn->oauth_client_secret)
+ auth_method = I_TOKEN_AUTH_METHOD_SECRET_BASIC;
+
+ err = i_set_parameter_list(&session,
+ I_OPT_CLIENT_ID, conn->oauth_client_id,
+ I_OPT_CLIENT_SECRET, conn->oauth_client_secret,
+ I_OPT_TOKEN_METHOD, auth_method,
+ I_OPT_SCOPE, conn->oauth_scope,
+ I_OPT_NONE
+ );
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set client identifier");
+ goto cleanup;
+ }
+
+ err = i_run_device_auth_request(&session);
+ if (err)
+ {
+ iddawc_request_error(conn, &session, err,
+ "failed to obtain device authorization");
+ goto cleanup;
+ }
+
+ verification_uri = i_get_str_parameter(&session, I_OPT_DEVICE_AUTH_VERIFICATION_URI);
+ if (!verification_uri)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer did not provide a verification URI"));
+ goto cleanup;
+ }
+
+ user_code = i_get_str_parameter(&session, I_OPT_DEVICE_AUTH_USER_CODE);
+ if (!user_code)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer did not provide a user code"));
+ goto cleanup;
+ }
+
+ /*
+ * Poll the token endpoint until either the user logs in and authorizes the
+ * use of a token, or a hard failure occurs. We perform one ping _before_
+ * prompting the user, so that we don't make them do the work of logging in
+ * only to find that the token endpoint is completely unreachable.
+ */
+ err = i_run_token_request(&session);
+ while (err)
+ {
+ const char *error_code;
+ uint interval;
+
+ error_code = i_get_str_parameter(&session, I_OPT_ERROR);
+
+ /*
+ * authorization_pending and slow_down are the only acceptable errors;
+ * anything else and we bail.
+ */
+ if (!error_code || (strcmp(error_code, "authorization_pending")
+ && strcmp(error_code, "slow_down")))
+ {
+ iddawc_request_error(conn, &session, err,
+ "OAuth token retrieval failed");
+ goto cleanup;
+ }
+
+ if (!user_prompted)
+ {
+ /*
+ * Now that we know the token endpoint isn't broken, give the user
+ * the login instructions.
+ */
+ pqInternalNotice(&conn->noticeHooks,
+ "Visit %s and enter the code: %s",
+ verification_uri, user_code);
+
+ user_prompted = true;
+ }
+
+ /*
+ * We are required to wait between polls; the server tells us how long.
+ * TODO: if interval's not set, we need to default to five seconds
+ * TODO: sanity check the interval
+ */
+ interval = i_get_int_parameter(&session, I_OPT_DEVICE_AUTH_INTERVAL);
+
+ /*
+ * A slow_down error requires us to permanently increase our retry
+ * interval by five seconds. RFC 8628, Sec. 3.5.
+ */
+ if (!strcmp(error_code, "slow_down"))
+ {
+ interval += 5;
+ i_set_int_parameter(&session, I_OPT_DEVICE_AUTH_INTERVAL, interval);
+ }
+
+ sleep(interval);
+
+ /*
+ * XXX Reset the error code before every call, because iddawc won't do
+ * that for us. This matters if the server first sends a "pending" error
+ * code, then later hard-fails without sending an error code to
+ * overwrite the first one.
+ *
+ * That we have to do this at all seems like a bug in iddawc.
+ */
+ i_set_str_parameter(&session, I_OPT_ERROR, NULL);
+
+ err = i_run_token_request(&session);
+ }
+
+ access_token = i_get_str_parameter(&session, I_OPT_ACCESS_TOKEN);
+ token_type = i_get_str_parameter(&session, I_OPT_TOKEN_TYPE);
+
+ if (!access_token || !token_type || strcasecmp(token_type, "Bearer"))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer did not provide a bearer token"));
+ goto cleanup;
+ }
+
+ appendPQExpBufferStr(token_buf, "Bearer ");
+ appendPQExpBufferStr(token_buf, access_token);
+
+ if (PQExpBufferBroken(token_buf))
+ goto cleanup;
+
+ token = strdup(token_buf->data);
+
+cleanup:
+ if (token_buf)
+ destroyPQExpBuffer(token_buf);
+ i_clean_session(&session);
+
+ return token;
+}
+
+#define kvsep "\x01"
+
+static char *
+client_initial_response(PGconn *conn)
+{
+ static const char * const resp_format = "n,," kvsep "auth=%s" kvsep kvsep;
+
+ PQExpBuffer token_buf;
+ PQExpBuffer discovery_buf = NULL;
+ char *token = NULL;
+ char *response = NULL;
+
+ token_buf = createPQExpBuffer();
+ if (!token_buf)
+ goto cleanup;
+
+ /*
+ * If we don't yet have a discovery URI, but the user gave us an explicit
+ * issuer, use the .well-known discovery URI for that issuer.
+ */
+ if (!conn->oauth_discovery_uri && conn->oauth_issuer)
+ {
+ discovery_buf = createPQExpBuffer();
+ if (!discovery_buf)
+ goto cleanup;
+
+ appendPQExpBufferStr(discovery_buf, conn->oauth_issuer);
+ appendPQExpBufferStr(discovery_buf, "/.well-known/openid-configuration");
+
+ if (PQExpBufferBroken(discovery_buf))
+ goto cleanup;
+
+ conn->oauth_discovery_uri = strdup(discovery_buf->data);
+ }
+
+ token = get_auth_token(conn);
+ if (!token)
+ goto cleanup;
+
+ appendPQExpBuffer(token_buf, resp_format, token);
+ if (PQExpBufferBroken(token_buf))
+ goto cleanup;
+
+ response = strdup(token_buf->data);
+
+cleanup:
+ if (token)
+ free(token);
+ if (discovery_buf)
+ destroyPQExpBuffer(discovery_buf);
+ if (token_buf)
+ destroyPQExpBuffer(token_buf);
+
+ return response;
+}
+
+#define ERROR_STATUS_FIELD "status"
+#define ERROR_SCOPE_FIELD "scope"
+#define ERROR_OPENID_CONFIGURATION_FIELD "openid-configuration"
+
+struct json_ctx
+{
+ char *errmsg; /* any non-NULL value stops all processing */
+ int nested; /* nesting level (zero is the top) */
+
+ const char *target_field_name; /* points to a static allocation */
+ char **target_field; /* see below */
+
+ /* target_field, if set, points to one of the following: */
+ char *status;
+ char *scope;
+ char *discovery_uri;
+};
+
+static void
+oauth_json_object_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (ctx->errmsg)
+ return; /* short-circuit */
+
+ if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ ctx->errmsg = psprintf(libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ ++ctx->nested;
+}
+
+static void
+oauth_json_object_end(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (ctx->errmsg)
+ return; /* short-circuit */
+
+ --ctx->nested;
+}
+
+static void
+oauth_json_object_field_start(void *state, char *name, bool isnull)
+{
+ struct json_ctx *ctx = state;
+
+ if (ctx->errmsg)
+ {
+ /* short-circuit */
+ pfree(name);
+ return;
+ }
+
+ if (ctx->nested == 1)
+ {
+ if (!strcmp(name, ERROR_STATUS_FIELD))
+ {
+ ctx->target_field_name = ERROR_STATUS_FIELD;
+ ctx->target_field = &ctx->status;
+ }
+ else if (!strcmp(name, ERROR_SCOPE_FIELD))
+ {
+ ctx->target_field_name = ERROR_SCOPE_FIELD;
+ ctx->target_field = &ctx->scope;
+ }
+ else if (!strcmp(name, ERROR_OPENID_CONFIGURATION_FIELD))
+ {
+ ctx->target_field_name = ERROR_OPENID_CONFIGURATION_FIELD;
+ ctx->target_field = &ctx->discovery_uri;
+ }
+ }
+
+ pfree(name);
+}
+
+static void
+oauth_json_array_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (ctx->errmsg)
+ return; /* short-circuit */
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = pstrdup(libpq_gettext("top-level element must be an object"));
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ ctx->errmsg = psprintf(libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+}
+
+static void
+oauth_json_scalar(void *state, char *token, JsonTokenType type)
+{
+ struct json_ctx *ctx = state;
+
+ if (ctx->errmsg)
+ {
+ /* short-circuit */
+ pfree(token);
+ return;
+ }
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = pstrdup(libpq_gettext("top-level element must be an object"));
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ if (type == JSON_TOKEN_STRING)
+ {
+ *ctx->target_field = token;
+
+ ctx->target_field = NULL;
+ ctx->target_field_name = NULL;
+
+ return; /* don't pfree the token we're using */
+ }
+
+ ctx->errmsg = psprintf(libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ pfree(token);
+}
+
+static bool
+handle_oauth_sasl_error(PGconn *conn, char *msg, int msglen)
+{
+ JsonLexContext *lex;
+ JsonSemAction sem = {0};
+ JsonParseErrorType err;
+ struct json_ctx ctx = {0};
+ char *errmsg = NULL;
+
+ /* Sanity check. */
+ if (strlen(msg) != msglen)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server's error message contained an embedded NULL"));
+ return false;
+ }
+
+ lex = makeJsonLexContextCstringLen(msg, msglen, PG_UTF8, true);
+
+ sem.semstate = &ctx;
+
+ sem.object_start = oauth_json_object_start;
+ sem.object_end = oauth_json_object_end;
+ sem.object_field_start = oauth_json_object_field_start;
+ sem.array_start = oauth_json_array_start;
+ sem.scalar = oauth_json_scalar;
+
+ err = pg_parse_json(lex, &sem);
+
+ if (err != JSON_SUCCESS)
+ {
+ errmsg = json_errdetail(err, lex);
+ }
+ else if (ctx.errmsg)
+ {
+ errmsg = ctx.errmsg;
+ }
+
+ if (errmsg)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("failed to parse server's error response: %s"),
+ errmsg);
+ pfree(errmsg);
+ return false;
+ }
+
+ /* TODO: what if these override what the user already specified? */
+ if (ctx.discovery_uri)
+ {
+ if (conn->oauth_discovery_uri)
+ free(conn->oauth_discovery_uri);
+
+ conn->oauth_discovery_uri = ctx.discovery_uri;
+ }
+
+ if (ctx.scope)
+ {
+ if (conn->oauth_scope)
+ free(conn->oauth_scope);
+
+ conn->oauth_scope = ctx.scope;
+ }
+ /* TODO: missing error scope should clear any existing connection scope */
+
+ if (!ctx.status)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("server sent error response without a status"));
+ return false;
+ }
+
+ if (!strcmp(ctx.status, "invalid_token"))
+ {
+ /*
+ * invalid_token is the only error code we'll automatically retry for,
+ * but only if we have enough information to do so.
+ */
+ if (conn->oauth_discovery_uri)
+ conn->oauth_want_retry = true;
+ }
+ /* TODO: include status in hard failure message */
+
+ return true;
+}
+
+static void
+oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen,
+ bool *done, bool *success)
+{
+ fe_oauth_state *state = opaq;
+ PGconn *conn = state->conn;
+
+ *done = false;
+ *success = false;
+ *output = NULL;
+ *outputlen = 0;
+
+ switch (state->state)
+ {
+ case FE_OAUTH_INIT:
+ Assert(inputlen == -1);
+
+ *output = client_initial_response(conn);
+ if (!*output)
+ goto error;
+
+ *outputlen = strlen(*output);
+ state->state = FE_OAUTH_BEARER_SENT;
+
+ break;
+
+ case FE_OAUTH_BEARER_SENT:
+ if (final)
+ {
+ /* TODO: ensure there is no message content here. */
+ *done = true;
+ *success = true;
+
+ break;
+ }
+
+ /*
+ * Error message sent by the server.
+ */
+ if (!handle_oauth_sasl_error(conn, input, inputlen))
+ goto error;
+
+ /*
+ * Respond with the required dummy message (RFC 7628, sec. 3.2.3).
+ */
+ *output = strdup(kvsep);
+ *outputlen = strlen(*output); /* == 1 */
+
+ state->state = FE_OAUTH_SERVER_ERROR;
+ break;
+
+ case FE_OAUTH_SERVER_ERROR:
+ /*
+ * After an error, the server should send an error response to fail
+ * the SASL handshake, which is handled in higher layers.
+ *
+ * If we get here, the server either sent *another* challenge which
+ * isn't defined in the RFC, or completed the handshake successfully
+ * after telling us it was going to fail. Neither is acceptable.
+ */
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server sent additional OAuth data after error\n"));
+ goto error;
+
+ default:
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("invalid OAuth exchange state\n"));
+ goto error;
+ }
+
+ return;
+
+error:
+ *done = true;
+ *success = false;
+}
+
+static bool
+oauth_channel_bound(void *opaq)
+{
+ /* This mechanism does not support channel binding. */
+ return false;
+}
+
+static void
+oauth_free(void *opaq)
+{
+ fe_oauth_state *state = opaq;
+
+ free(state);
+}
diff --git a/src/interfaces/libpq/fe-auth-scram.c b/src/interfaces/libpq/fe-auth-scram.c
index 04d5703d89..f2ba3bca37 100644
--- a/src/interfaces/libpq/fe-auth-scram.c
+++ b/src/interfaces/libpq/fe-auth-scram.c
@@ -24,7 +24,8 @@
/* The exported SCRAM callback mechanism. */
static void *scram_init(PGconn *conn, const char *password,
const char *sasl_mechanism);
-static void scram_exchange(void *opaq, char *input, int inputlen,
+static void scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen,
bool *done, bool *success);
static bool scram_channel_bound(void *opaq);
@@ -205,7 +206,8 @@ scram_free(void *opaq)
* Exchange a SCRAM message with backend.
*/
static void
-scram_exchange(void *opaq, char *input, int inputlen,
+scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen,
bool *done, bool *success)
{
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index d5cbac108e..690b23b9d9 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -39,6 +39,7 @@
#endif
#include "common/md5.h"
+#include "common/oauth-common.h"
#include "common/scram-common.h"
#include "fe-auth.h"
#include "libpq-fe.h"
@@ -422,7 +423,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
bool success;
const char *selected_mechanism;
PQExpBufferData mechanism_buf;
- char *password;
+ char *password = NULL;
initPQExpBuffer(&mechanism_buf);
@@ -444,8 +445,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
/*
* Parse the list of SASL authentication mechanisms in the
* AuthenticationSASL message, and select the best mechanism that we
- * support. SCRAM-SHA-256-PLUS and SCRAM-SHA-256 are the only ones
- * supported at the moment, listed by order of decreasing importance.
+ * support. Mechanisms are listed by order of decreasing importance.
*/
selected_mechanism = NULL;
for (;;)
@@ -485,6 +485,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
selected_mechanism = SCRAM_SHA_256_PLUS_NAME;
conn->sasl = &pg_scram_mech;
+ conn->password_needed = true;
}
#else
/*
@@ -522,7 +523,17 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
selected_mechanism = SCRAM_SHA_256_NAME;
conn->sasl = &pg_scram_mech;
+ conn->password_needed = true;
}
+#ifdef USE_OAUTH
+ else if (strcmp(mechanism_buf.data, OAUTHBEARER_NAME) == 0 &&
+ !selected_mechanism)
+ {
+ selected_mechanism = OAUTHBEARER_NAME;
+ conn->sasl = &pg_oauth_mech;
+ conn->password_needed = false;
+ }
+#endif
}
if (!selected_mechanism)
@@ -547,18 +558,19 @@ pg_SASL_init(PGconn *conn, int payloadlen)
/*
* First, select the password to use for the exchange, complaining if
- * there isn't one. Currently, all supported SASL mechanisms require a
- * password, so we can just go ahead here without further distinction.
+ * there isn't one and the SASL mechanism needs it.
*/
- conn->password_needed = true;
- password = conn->connhost[conn->whichhost].password;
- if (password == NULL)
- password = conn->pgpass;
- if (password == NULL || password[0] == '\0')
+ if (conn->password_needed)
{
- appendPQExpBufferStr(&conn->errorMessage,
- PQnoPasswordSupplied);
- goto error;
+ password = conn->connhost[conn->whichhost].password;
+ if (password == NULL)
+ password = conn->pgpass;
+ if (password == NULL || password[0] == '\0')
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ PQnoPasswordSupplied);
+ goto error;
+ }
}
Assert(conn->sasl);
@@ -576,7 +588,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
goto oom_error;
/* Get the mechanism-specific Initial Client Response, if any */
- conn->sasl->exchange(conn->sasl_state,
+ conn->sasl->exchange(conn->sasl_state, false,
NULL, -1,
&initialresponse, &initialresponselen,
&done, &success);
@@ -657,7 +669,7 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
/* For safety and convenience, ensure the buffer is NULL-terminated. */
challenge[payloadlen] = '\0';
- conn->sasl->exchange(conn->sasl_state,
+ conn->sasl->exchange(conn->sasl_state, final,
challenge, payloadlen,
&output, &outputlen,
&done, &success);
diff --git a/src/interfaces/libpq/fe-auth.h b/src/interfaces/libpq/fe-auth.h
index 1e4fcbff62..edc748fd3a 100644
--- a/src/interfaces/libpq/fe-auth.h
+++ b/src/interfaces/libpq/fe-auth.h
@@ -26,4 +26,7 @@ extern char *pg_fe_getauthname(PQExpBuffer errorMessage);
extern const pg_sasl_mech pg_scram_mech;
extern char *pg_fe_scram_build_secret(const char *password);
+/* Mechanisms in fe-auth-oauth.c */
+extern const pg_sasl_mech pg_oauth_mech;
+
#endif /* FE_AUTH_H */
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 10d007582c..1d4bca9194 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -345,6 +345,23 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"Target-Session-Attrs", "", 15, /* sizeof("prefer-standby") = 15 */
offsetof(struct pg_conn, target_session_attrs)},
+ /* OAuth v2 */
+ {"oauth_issuer", NULL, NULL, NULL,
+ "OAuth-Issuer", "", 40,
+ offsetof(struct pg_conn, oauth_issuer)},
+
+ {"oauth_client_id", NULL, NULL, NULL,
+ "OAuth-Client-ID", "", 40,
+ offsetof(struct pg_conn, oauth_client_id)},
+
+ {"oauth_client_secret", NULL, NULL, NULL,
+ "OAuth-Client-Secret", "", 40,
+ offsetof(struct pg_conn, oauth_client_secret)},
+
+ {"oauth_scope", NULL, NULL, NULL,
+ "OAuth-Scope", "", 15,
+ offsetof(struct pg_conn, oauth_scope)},
+
/* Terminating entry --- MUST BE LAST */
{NULL, NULL, NULL, NULL,
NULL, NULL, 0}
@@ -607,6 +624,7 @@ pqDropServerData(PGconn *conn)
conn->write_err_msg = NULL;
conn->be_pid = 0;
conn->be_key = 0;
+ /* conn->oauth_want_retry = false; TODO */
}
@@ -3356,6 +3374,16 @@ keep_going: /* We will come back to here until there is
/* Check to see if we should mention pgpassfile */
pgpassfileWarning(conn);
+#ifdef USE_OAUTH
+ if (conn->sasl == &pg_oauth_mech
+ && conn->oauth_want_retry)
+ {
+ /* TODO: only allow retry once */
+ need_new_connection = true;
+ goto keep_going;
+ }
+#endif
+
#ifdef ENABLE_GSS
/*
@@ -4130,6 +4158,16 @@ freePGconn(PGconn *conn)
free(conn->rowBuf);
if (conn->target_session_attrs)
free(conn->target_session_attrs);
+ if (conn->oauth_issuer)
+ free(conn->oauth_issuer);
+ if (conn->oauth_discovery_uri)
+ free(conn->oauth_discovery_uri);
+ if (conn->oauth_client_id)
+ free(conn->oauth_client_id);
+ if (conn->oauth_client_secret)
+ free(conn->oauth_client_secret);
+ if (conn->oauth_scope)
+ free(conn->oauth_scope);
termPQExpBuffer(&conn->errorMessage);
termPQExpBuffer(&conn->workBuffer);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 25eaa231c5..b749c6c05d 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -340,7 +340,7 @@ typedef struct pg_conn_host
} pg_conn_host;
typedef void *(*pg_sasl_init_func)(PGconn *, const char *, const char *);
-typedef void (*pg_sasl_exchange_func)(void *, char *, int, char **, int *, bool *, bool *);
+typedef void (*pg_sasl_exchange_func)(void *, bool, char *, int, char **, int *, bool *, bool *);
typedef bool (*pg_sasl_channel_bound_func)(void *);
typedef void (*pg_sasl_free_func)(void *);
@@ -406,6 +406,14 @@ struct pg_conn
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ /* OAuth v2 */
+ char *oauth_issuer; /* token issuer URL */
+ char *oauth_discovery_uri; /* URI of the issuer's discovery document */
+ char *oauth_client_id; /* client identifier */
+ char *oauth_client_secret; /* client secret */
+ char *oauth_scope; /* access token scope */
+ bool oauth_want_retry; /* should we retry on failure? */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
--
2.25.1
0005-backend-add-OAUTHBEARER-SASL-mechanism.patch (text/x-patch, 38.5K) download
From ee8e85d3416f381ba9d44f8d4a681e5006bd5b82 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 4 May 2021 16:21:11 -0700
Subject: [PATCH 5/7] backend: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) on the server side. This adds a new
auth method, oauth, to pg_hba.
Because OAuth implementations vary so wildly, and bearer token
validation is heavily dependent on the issuing party, authn/z is done by
communicating with an external program: the oauth_validator_command.
This command must do the following:
1. Receive the bearer token by reading its contents from a file
descriptor passed from the server. (The numeric value of this
descriptor may be inserted into the oauth_validator_command using the
%f specifier.)
This MUST be the first action the command performs. The server will
not begin reading stdout from the command until the token has been
read in full, so if the command tries to print anything and hits a
buffer limit, the backend will deadlock and time out.
2. Validate the bearer token. The correct way to do this depends on the
issuer, but it generally involves either cryptographic operations to
prove that the token was issued by a trusted party, or the
presentation of the bearer token to some other party so that _it_ can
perform validation.
The command MUST maintain confidentiality of the bearer token, since
in most cases it can be used just like a password. (There are ways to
cryptographically bind tokens to client certificates, but they are
way beyond the scope of this commit message.)
If the token cannot be validated, the command must exit with a
non-zero status. Further authentication/authorization is pointless if
the bearer token wasn't issued by someone you trust.
3. Authenticate the user, authorize the user, or both:
a. To authenticate the user, use the bearer token to retrieve some
trusted identifier string for the end user. The exact process for
this is, again, issuer-dependent. The command should print the
authenticated identity string to stdout, followed by a newline.
If the user cannot be authenticated, the validator should not
print anything to stdout. It should also exit with a non-zero
status, unless the token may be used to authorize the connection
through some other means (see below).
On a success, the command may then exit with a zero success code.
By default, the server will then check to make sure the identity
string matches the role that is being used (or matches a usermap
entry, if one is in use).
b. To optionally authorize the user, in combination with the HBA
option trust_validator_authz=1 (see below), the validator simply
returns a zero exit code if the client should be allowed to
connect with its presented role (which can be passed to the
command using the %r specifier), or a non-zero code otherwise.
The hard part is in determining whether the given token truly
authorizes the client to use the given role, which must
unfortunately be left as an exercise to the reader.
This obviously requires some care, as a poorly implemented token
validator may silently open the entire database to anyone with a
bearer token. But it may be a more portable approach, since OAuth
is designed as an authorization framework, not an authentication
framework. For example, the user's bearer token could carry an
"allow_superuser_access" claim, which would authorize pseudonymous
database access as any role. It's then up to the OAuth system
administrators to ensure that allow_superuser_access is doled out
only to the proper users.
c. It's possible that the user can be successfully authenticated but
isn't authorized to connect. In this case, the command may print
the authenticated ID and then fail with a non-zero exit code.
(This makes it easier to see what's going on in the Postgres
logs.)
4. Token validators may optionally log to stderr. This will be printed
verbatim into the Postgres server logs.
The oauth method supports the following HBA options (but note that two
of them are not optional, since we have no way of choosing sensible
defaults):
issuer: Required. The URL of the OAuth issuing party, which the client
must contact to receive a bearer token.
Some real-world examples as of time of writing:
- https://accounts.google.com
- https://login.microsoft.com/[tenant-id]/v2.0
scope: Required. The OAuth scope(s) required for the server to
authenticate and/or authorize the user. This is heavily
deployment-specific, but a simple example is "openid email".
map: Optional. Specify a standard PostgreSQL user map; this works
the same as with other auth methods such as peer. If a map is
not specified, the user ID returned by the token validator
must exactly match the role that's being requested (but see
trust_validator_authz, below).
trust_validator_authz:
Optional. When set to 1, this allows the token validator to
take full control of the authorization process. Standard user
mapping is skipped: if the validator command succeeds, the
client is allowed to connect under its desired role and no
further checks are done.
Unlike the client, servers support OAuth without needing to be built
against libiddawc (since the responsibility for "speaking" OAuth/OIDC
correctly is delegated entirely to the oauth_validator_command).
Several TODOs:
- port to platforms other than "modern Linux"
- overhaul the communication with oauth_validator_command, which is
currently a bad hack on OpenPipeStream()
- implement more sanity checks on the OAUTHBEARER message format and
tokens sent by the client
- implement more helpful handling of HBA misconfigurations
- properly interpolate JSON when generating error responses
- use logdetail during auth failures
- deal with role names that can't be safely passed to system() without
shell-escaping
- allow passing the configured issuer to the oauth_validator_command, to
deal with multi-issuer setups
- ...and more.
---
src/backend/libpq/Makefile | 1 +
src/backend/libpq/auth-oauth.c | 797 +++++++++++++++++++++++++++++++++
src/backend/libpq/auth-scram.c | 2 +
src/backend/libpq/auth.c | 43 +-
src/backend/libpq/hba.c | 29 +-
src/backend/utils/misc/guc.c | 12 +
src/include/libpq/auth.h | 1 +
src/include/libpq/hba.h | 8 +-
src/include/libpq/oauth.h | 24 +
src/include/libpq/sasl.h | 26 ++
10 files changed, 915 insertions(+), 28 deletions(-)
create mode 100644 src/backend/libpq/auth-oauth.c
create mode 100644 src/include/libpq/oauth.h
diff --git a/src/backend/libpq/Makefile b/src/backend/libpq/Makefile
index 8d1d16b0fc..40f2c50c3c 100644
--- a/src/backend/libpq/Makefile
+++ b/src/backend/libpq/Makefile
@@ -15,6 +15,7 @@ include $(top_builddir)/src/Makefile.global
# be-fsstubs is here for historical reasons, probably belongs elsewhere
OBJS = \
+ auth-oauth.o \
auth-scram.o \
auth.o \
be-fsstubs.o \
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
new file mode 100644
index 0000000000..b2b9d56e7c
--- /dev/null
+++ b/src/backend/libpq/auth-oauth.c
@@ -0,0 +1,797 @@
+/*-------------------------------------------------------------------------
+ *
+ * auth-oauth.c
+ * Server-side implementation of the SASL OAUTHBEARER mechanism.
+ *
+ * See the following RFC for more details:
+ * - RFC 7628: https://tools.ietf.org/html/rfc7628
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/backend/libpq/auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <unistd.h>
+#include <fcntl.h>
+
+#include "common/oauth-common.h"
+#include "lib/stringinfo.h"
+#include "libpq/auth.h"
+#include "libpq/hba.h"
+#include "libpq/oauth.h"
+#include "libpq/sasl.h"
+#include "storage/fd.h"
+
+/* GUC */
+char *oauth_validator_command;
+
+static void oauth_get_mechanisms(Port *port, StringInfo buf);
+static void *oauth_init(Port *port, const char *selected_mech, const char *shadow_pass);
+static int oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, char **logdetail);
+
+/* Mechanism declaration */
+const pg_be_sasl_mech pg_be_oauth_mech = {
+ oauth_get_mechanisms,
+ oauth_init,
+ oauth_exchange,
+
+ PG_MAX_AUTH_TOKEN_LENGTH,
+};
+
+
+typedef enum
+{
+ OAUTH_STATE_INIT = 0,
+ OAUTH_STATE_ERROR,
+ OAUTH_STATE_FINISHED,
+} oauth_state;
+
+struct oauth_ctx
+{
+ oauth_state state;
+ Port *port;
+ const char *issuer;
+ const char *scope;
+};
+
+static char *sanitize_char(char c);
+static char *parse_kvpairs_for_auth(char **input);
+static void generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen);
+static bool validate(Port *port, const char *auth, char **logdetail);
+static bool run_validator_command(Port *port, const char *token);
+static bool check_exit(FILE **fh, const char *command);
+static bool unset_cloexec(int fd);
+static bool username_ok_for_shell(const char *username);
+
+#define KVSEP 0x01
+#define AUTH_KEY "auth"
+#define BEARER_SCHEME "Bearer "
+
+static void
+oauth_get_mechanisms(Port *port, StringInfo buf)
+{
+ /* Only OAUTHBEARER is supported. */
+ appendStringInfoString(buf, OAUTHBEARER_NAME);
+ appendStringInfoChar(buf, '\0');
+}
+
+static void *
+oauth_init(Port *port, const char *selected_mech, const char *shadow_pass)
+{
+ struct oauth_ctx *ctx;
+
+ if (strcmp(selected_mech, OAUTHBEARER_NAME))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("client selected an invalid SASL authentication mechanism")));
+
+ ctx = palloc0(sizeof(*ctx));
+
+ ctx->state = OAUTH_STATE_INIT;
+ ctx->port = port;
+
+ Assert(port->hba);
+ ctx->issuer = port->hba->oauth_issuer;
+ ctx->scope = port->hba->oauth_scope;
+
+ return ctx;
+}
+
+static int
+oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, char **logdetail)
+{
+ char *p;
+ char cbind_flag;
+ char *auth;
+
+ struct oauth_ctx *ctx = opaq;
+
+ *output = NULL;
+ *outputlen = -1;
+
+ /*
+ * If the client didn't include an "Initial Client Response" in the
+ * SASLInitialResponse message, send an empty challenge, to which the
+ * client will respond with the same data that usually comes in the
+ * Initial Client Response.
+ */
+ if (input == NULL)
+ {
+ Assert(ctx->state == OAUTH_STATE_INIT);
+
+ *output = pstrdup("");
+ *outputlen = 0;
+ return SASL_EXCHANGE_CONTINUE;
+ }
+
+ /*
+ * Check that the input length agrees with the string length of the input.
+ */
+ if (inputlen == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The message is empty.")));
+ if (inputlen != strlen(input))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message length does not match input length.")));
+
+ switch (ctx->state)
+ {
+ case OAUTH_STATE_INIT:
+ /* Handle this case below. */
+ break;
+
+ case OAUTH_STATE_ERROR:
+ /*
+ * Only one response is valid for the client during authentication
+ * failure: a single kvsep.
+ */
+ if (inputlen != 1 || *input != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Client did not send a kvsep response.")));
+
+ /* The (failed) handshake is now complete. */
+ ctx->state = OAUTH_STATE_FINISHED;
+ return SASL_EXCHANGE_FAILURE;
+
+ default:
+ elog(ERROR, "invalid OAUTHBEARER exchange state");
+ return SASL_EXCHANGE_FAILURE;
+ }
+
+ /* Handle the client's initial message. */
+ p = strdup(input);
+
+ /*
+ * OAUTHBEARER does not currently define a channel binding (so there is no
+ * OAUTHBEARER-PLUS, and we do not accept a 'p' specifier). We accept a 'y'
+ * specifier purely for the remote chance that a future specification could
+ * define one; then future clients can still interoperate with this server
+ * implementation. 'n' is the expected case.
+ */
+ cbind_flag = *p;
+ switch (cbind_flag)
+ {
+ case 'p':
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The server does not support channel binding for OAuth, but the client message includes channel binding data.")));
+ break;
+
+ case 'y': /* fall through */
+ case 'n':
+ p++;
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Comma expected, but found character \"%s\".",
+ sanitize_char(*p))));
+ p++;
+ break;
+
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected channel-binding flag \"%s\".",
+ sanitize_char(cbind_flag))));
+ }
+
+ /*
+ * Forbid optional authzid (authorization identity). We don't support it.
+ */
+ if (*p == 'a')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("client uses authorization identity, but it is not supported")));
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected attribute \"%s\" in client-first-message.",
+ sanitize_char(*p))));
+ p++;
+
+ /* All remaining fields are separated by the RFC's kvsep (\x01). */
+ if (*p != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Key-value separator expected, but found character \"%s\".",
+ sanitize_char(*p))));
+ p++;
+
+ auth = parse_kvpairs_for_auth(&p);
+ if (!auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message does not contain an auth value.")));
+
+ /* We should be at the end of our message. */
+ if (*p)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains additional data after the final terminator.")));
+
+ if (!validate(ctx->port, auth, logdetail))
+ {
+ generate_error_response(ctx, output, outputlen);
+
+ ctx->state = OAUTH_STATE_ERROR;
+ return SASL_EXCHANGE_CONTINUE;
+ }
+
+ ctx->state = OAUTH_STATE_FINISHED;
+ return SASL_EXCHANGE_SUCCESS;
+}
+
+/*
+ * Convert an arbitrary byte to printable form. For error messages.
+ *
+ * If it's a printable ASCII character, print it as a single character.
+ * otherwise, print it in hex.
+ *
+ * The returned pointer points to a static buffer.
+ */
+static char *
+sanitize_char(char c)
+{
+ static char buf[5];
+
+ if (c >= 0x21 && c <= 0x7E)
+ snprintf(buf, sizeof(buf), "'%c'", c);
+ else
+ snprintf(buf, sizeof(buf), "0x%02x", (unsigned char) c);
+ return buf;
+}
+
+/*
+ * Consumes all kvpairs in an OAUTHBEARER exchange message. If the "auth" key is
+ * found, its value is returned.
+ */
+static char *
+parse_kvpairs_for_auth(char **input)
+{
+ char *pos = *input;
+ char *auth = NULL;
+
+ /*
+ * The relevant ABNF, from Sec. 3.1:
+ *
+ * kvsep = %x01
+ * key = 1*(ALPHA)
+ * value = *(VCHAR / SP / HTAB / CR / LF )
+ * kvpair = key "=" value kvsep
+ * ;;gs2-header = See RFC 5801
+ * client-resp = (gs2-header kvsep *kvpair kvsep) / kvsep
+ *
+ * By the time we reach this code, the gs2-header and initial kvsep have
+ * already been validated. We start at the beginning of the first kvpair.
+ */
+
+ while (*pos)
+ {
+ char *end;
+ char *sep;
+ char *key;
+ char *value;
+
+ /*
+ * Find the end of this kvpair. Note that input is null-terminated by
+ * the SASL code, so the strchr() is bounded.
+ */
+ end = strchr(pos, KVSEP);
+ if (!end)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an unterminated key/value pair.")));
+ *end = '\0';
+
+ if (pos == end)
+ {
+ /* Empty kvpair, signifying the end of the list. */
+ *input = pos + 1;
+ return auth;
+ }
+
+ /*
+ * Find the end of the key name.
+ *
+ * TODO further validate the key/value grammar? empty keys, bad chars...
+ */
+ sep = strchr(pos, '=');
+ if (!sep)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains a key without a value.")));
+ *sep = '\0';
+
+ /* Both key and value are now safely terminated. */
+ key = pos;
+ value = sep + 1;
+
+ if (!strcmp(key, AUTH_KEY))
+ {
+ if (auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains multiple auth values.")));
+
+ auth = value;
+ }
+ else
+ {
+ /*
+ * The RFC also defines the host and port keys, but they are not
+ * required for OAUTHBEARER and we do not use them. Also, per
+ * Sec. 3.1, any key/value pairs we don't recognize must be ignored.
+ */
+ }
+
+ /* Move to the next pair. */
+ pos = end + 1;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message did not contain a final terminator.")));
+
+ return NULL; /* unreachable */
+}
+
+static void
+generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
+{
+ StringInfoData buf;
+
+ /*
+ * The admin needs to set an issuer and scope for OAuth to work. There's not
+ * really a way to hide this from the user, either, because we can't choose
+ * a "default" issuer, so be honest in the failure message.
+ *
+ * TODO: see if there's a better place to fail, earlier than this.
+ */
+ if (!ctx->issuer || !ctx->scope)
+ ereport(FATAL,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("OAuth is not properly configured for this user"),
+ errdetail_log("The issuer and scope parameters must be set in pg_hba.conf.")));
+
+
+ initStringInfo(&buf);
+
+ /*
+ * TODO: JSON escaping
+ */
+ appendStringInfo(&buf,
+ "{ "
+ "\"status\": \"invalid_token\", "
+ "\"openid-configuration\": \"%s/.well-known/openid-configuration\","
+ "\"scope\": \"%s\" "
+ "}",
+ ctx->issuer, ctx->scope);
+
+ *output = buf.data;
+ *outputlen = buf.len;
+}
+
+static bool
+validate(Port *port, const char *auth, char **logdetail)
+{
+ static const char * const b64_set = "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-._~+/";
+
+ const char *token;
+ size_t span;
+ int ret;
+
+ /* TODO: handle logdetail when the test framework can check it */
+
+ /*
+ * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
+ * 2.1:
+ *
+ * b64token = 1*( ALPHA / DIGIT /
+ * "-" / "." / "_" / "~" / "+" / "/" ) *"="
+ * credentials = "Bearer" 1*SP b64token
+ *
+ * The "credentials" construction is what we receive in our auth value.
+ *
+ * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
+ * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
+ * compared case-insensitively. (This is not mentioned in RFC 6750, but it's
+ * pointed out in RFC 7628 Sec. 4.)
+ *
+ * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
+ */
+ if (strncasecmp(auth, BEARER_SCHEME, strlen(BEARER_SCHEME)))
+ return false;
+
+ /* Pull the bearer token out of the auth value. */
+ token = auth + strlen(BEARER_SCHEME);
+
+ /* Swallow any additional spaces. */
+ while (*token == ' ')
+ token++;
+
+ /*
+ * Before invoking the validator command, sanity-check the token format to
+ * avoid any injection attacks later in the chain. Invalid formats are
+ * technically a protocol violation, but don't reflect any information about
+ * the sensitive Bearer token back to the client; log at COMMERROR instead.
+ */
+
+ /* Tokens must not be empty. */
+ if (!*token)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Bearer token is empty.")));
+ return false;
+ }
+
+ /*
+ * Make sure the token contains only allowed characters. Tokens may end with
+ * any number of '=' characters.
+ */
+ span = strspn(token, b64_set);
+ while (token[span] == '=')
+ span++;
+
+ if (token[span] != '\0')
+ {
+ /*
+ * This error message could be more helpful by printing the problematic
+ * character(s), but that'd be a bit like printing a piece of someone's
+ * password into the logs.
+ */
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Bearer token is not in the correct format.")));
+ return false;
+ }
+
+ /* Have the validator check the token. */
+ if (!run_validator_command(port, token))
+ return false;
+
+ if (port->hba->oauth_skip_usermap)
+ {
+ /*
+ * If the validator is our authorization authority, we're done.
+ * Authentication may or may not have been performed depending on the
+ * validator implementation; all that matters is that the validator says
+ * the user can log in with the target role.
+ */
+ return true;
+ }
+
+ /* Make sure the validator authenticated the user. */
+ if (!port->authn_id)
+ {
+ /* TODO: use logdetail; reduce message duplication */
+ ereport(LOG,
+ (errmsg("OAuth bearer authentication failed for user \"%s\": validator provided no identity",
+ port->user_name)));
+ return false;
+ }
+
+ /* Finally, check the user map. */
+ ret = check_usermap(port->hba->usermap, port->user_name, port->authn_id,
+ false);
+ return (ret == STATUS_OK);
+}
+
+static bool
+run_validator_command(Port *port, const char *token)
+{
+ bool success = false;
+ int rc;
+ int pipefd[2];
+ int rfd = -1;
+ int wfd = -1;
+
+ StringInfoData command = { 0 };
+ char *p;
+ FILE *fh = NULL;
+
+ ssize_t written;
+ char *line = NULL;
+ size_t size = 0;
+ ssize_t len;
+
+ Assert(oauth_validator_command);
+
+ if (!oauth_validator_command[0])
+ {
+ ereport(COMMERROR,
+ (errmsg("oauth_validator_command is not set"),
+ errhint("To allow OAuth authenticated connections, set "
+ "oauth_validator_command in postgresql.conf.")));
+ return false;
+ }
+
+ /*
+ * Since popen() is unidirectional, open up a pipe for the other direction.
+ * Use CLOEXEC to ensure that our write end doesn't accidentally get copied
+ * into child processes, which would prevent us from closing it cleanly.
+ *
+ * XXX this is ugly. We should just read from the child process's stdout,
+ * but that's a lot more code.
+ * XXX by bypassing the popen API, we open the potential of process
+ * deadlock. Clearly document child process requirements (i.e. the child
+ * MUST read all data off of the pipe before writing anything).
+ * TODO: port to Windows using _pipe().
+ */
+ rc = pipe2(pipefd, O_CLOEXEC);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create child pipe: %m")));
+ return false;
+ }
+
+ rfd = pipefd[0];
+ wfd = pipefd[1];
+
+ /* Allow the read pipe be passed to the child. */
+ if (!unset_cloexec(rfd))
+ {
+ /* error message was already logged */
+ goto cleanup;
+ }
+
+ /*
+ * Construct the command, substituting any recognized %-specifiers:
+ *
+ * %f: the file descriptor of the input pipe
+ * %r: the role that the client wants to assume (port->user_name)
+ * %%: a literal '%'
+ */
+ initStringInfo(&command);
+
+ for (p = oauth_validator_command; *p; p++)
+ {
+ if (p[0] == '%')
+ {
+ switch (p[1])
+ {
+ case 'f':
+ appendStringInfo(&command, "%d", rfd);
+ p++;
+ break;
+ case 'r':
+ /*
+ * TODO: decide how this string should be escaped. The role
+ * is controlled by the client, so if we don't escape it,
+ * command injections are inevitable.
+ *
+ * This is probably an indication that the role name needs
+ * to be communicated to the validator process in some other
+ * way. For this proof of concept, just be incredibly strict
+ * about the characters that are allowed in user names.
+ */
+ if (!username_ok_for_shell(port->user_name))
+ goto cleanup;
+
+ appendStringInfoString(&command, port->user_name);
+ p++;
+ break;
+ case '%':
+ appendStringInfoChar(&command, '%');
+ p++;
+ break;
+ default:
+ appendStringInfoChar(&command, p[0]);
+ }
+ }
+ else
+ appendStringInfoChar(&command, p[0]);
+ }
+
+ /* Execute the command. */
+ fh = OpenPipeStream(command.data, "re");
+ /* TODO: handle failures */
+
+ /* We don't need the read end of the pipe anymore. */
+ close(rfd);
+ rfd = -1;
+
+ /* Give the command the token to validate. */
+ written = write(wfd, token, strlen(token));
+ if (written != strlen(token))
+ {
+ /* TODO must loop for short writes, EINTR et al */
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write token to child pipe: %m")));
+ goto cleanup;
+ }
+
+ close(wfd);
+ wfd = -1;
+
+ /*
+ * Read the command's response.
+ *
+ * TODO: getline() is probably too new to use, unfortunately.
+ * TODO: loop over all lines
+ */
+ if ((len = getline(&line, &size, fh)) >= 0)
+ {
+ /* TODO: fail if the authn_id doesn't end with a newline */
+ if (len > 0)
+ line[len - 1] = '\0';
+
+ set_authn_id(port, line);
+ }
+ else if (ferror(fh))
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from command \"%s\": %m",
+ command.data)));
+ goto cleanup;
+ }
+
+ /* Make sure the command exits cleanly. */
+ if (!check_exit(&fh, command.data))
+ {
+ /* error message already logged */
+ goto cleanup;
+ }
+
+ /* Done. */
+ success = true;
+
+cleanup:
+ if (line)
+ free(line);
+
+ /*
+ * In the successful case, the pipe fds are already closed. For the error
+ * case, always close out the pipe before waiting for the command, to
+ * prevent deadlock.
+ */
+ if (rfd >= 0)
+ close(rfd);
+ if (wfd >= 0)
+ close(wfd);
+
+ if (fh)
+ {
+ Assert(!success);
+ check_exit(&fh, command.data);
+ }
+
+ if (command.data)
+ pfree(command.data);
+
+ return success;
+}
+
+static bool
+check_exit(FILE **fh, const char *command)
+{
+ int rc;
+
+ rc = ClosePipeStream(*fh);
+ *fh = NULL;
+
+ if (rc == -1)
+ {
+ /* pclose() itself failed. */
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close pipe to command \"%s\": %m",
+ command)));
+ }
+ else if (rc != 0)
+ {
+ char *reason = wait_result_to_str(rc);
+
+ ereport(COMMERROR,
+ (errmsg("failed to execute command \"%s\": %s",
+ command, reason)));
+
+ pfree(reason);
+ }
+
+ return (rc == 0);
+}
+
+static bool
+unset_cloexec(int fd)
+{
+ int flags;
+ int rc;
+
+ flags = fcntl(fd, F_GETFD);
+ if (flags == -1)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not get fd flags for child pipe: %m")));
+ return false;
+ }
+
+ rc = fcntl(fd, F_SETFD, flags & ~FD_CLOEXEC);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not unset FD_CLOEXEC for child pipe: %m")));
+ return false;
+ }
+
+ return true;
+}
+
+/*
+ * XXX This should go away eventually and be replaced with either a proper
+ * escape or a different strategy for communication with the validator command.
+ */
+static bool
+username_ok_for_shell(const char *username)
+{
+ /* This set is borrowed from fe_utils' appendShellStringNoError(). */
+ static const char * const allowed = "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-_./:";
+ size_t span;
+
+ Assert(username && username[0]); /* should have already been checked */
+
+ span = strspn(username, allowed);
+ if (username[span] != '\0')
+ {
+ ereport(COMMERROR,
+ (errmsg("PostgreSQL user name contains unsafe characters and cannot be passed to the OAuth validator")));
+ return false;
+ }
+
+ return true;
+}
diff --git a/src/backend/libpq/auth-scram.c b/src/backend/libpq/auth-scram.c
index db3ca75a60..9e4482dc27 100644
--- a/src/backend/libpq/auth-scram.c
+++ b/src/backend/libpq/auth-scram.c
@@ -118,6 +118,8 @@ const pg_be_sasl_mech pg_be_scram_mech = {
scram_get_mechanisms,
scram_init,
scram_exchange,
+
+ PG_MAX_SASL_MESSAGE_LENGTH,
};
/*
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index e20740a7c5..354c7b0fc8 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -29,6 +29,7 @@
#include "libpq/auth.h"
#include "libpq/crypt.h"
#include "libpq/libpq.h"
+#include "libpq/oauth.h"
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
#include "libpq/scram.h"
@@ -49,7 +50,6 @@ static void sendAuthRequest(Port *port, AuthRequest areq, const char *extradata,
int extralen);
static void auth_failed(Port *port, int status, char *logdetail);
static char *recv_password_packet(Port *port);
-static void set_authn_id(Port *port, const char *id);
/*----------------------------------------------------------------
* SASL common authentication
@@ -215,29 +215,12 @@ static int CheckRADIUSAuth(Port *port);
static int PerformRadiusTransaction(const char *server, const char *secret, const char *portstr, const char *identifier, const char *user_name, const char *passwd);
-/*
- * Maximum accepted size of GSS and SSPI authentication tokens.
- * We also use this as a limit on ordinary password packet lengths.
- *
- * Kerberos tickets are usually quite small, but the TGTs issued by Windows
- * domain controllers include an authorization field known as the Privilege
- * Attribute Certificate (PAC), which contains the user's Windows permissions
- * (group memberships etc.). The PAC is copied into all tickets obtained on
- * the basis of this TGT (even those issued by Unix realms which the Windows
- * realm trusts), and can be several kB in size. The maximum token size
- * accepted by Windows systems is determined by the MaxAuthToken Windows
- * registry setting. Microsoft recommends that it is not set higher than
- * 65535 bytes, so that seems like a reasonable limit for us as well.
+/*----------------------------------------------------------------
+ * OAuth v2 Bearer Authentication
+ *----------------------------------------------------------------
*/
-#define PG_MAX_AUTH_TOKEN_LENGTH 65535
+static int CheckOAuthBearer(Port *port);
-/*
- * Maximum accepted size of SASL messages.
- *
- * The messages that the server or libpq generate are much smaller than this,
- * but have some headroom.
- */
-#define PG_MAX_SASL_MESSAGE_LENGTH 1024
/*----------------------------------------------------------------
* Global authentication functions
@@ -327,6 +310,9 @@ auth_failed(Port *port, int status, char *logdetail)
case uaRADIUS:
errstr = gettext_noop("RADIUS authentication failed for user \"%s\"");
break;
+ case uaOAuth:
+ errstr = gettext_noop("OAuth bearer authentication failed for user \"%s\"");
+ break;
default:
errstr = gettext_noop("authentication failed for user \"%s\": invalid authentication method");
break;
@@ -361,7 +347,7 @@ auth_failed(Port *port, int status, char *logdetail)
* lifetime of the Port, so it is safe to pass a string that is managed by an
* external library.
*/
-static void
+void
set_authn_id(Port *port, const char *id)
{
Assert(id);
@@ -646,6 +632,9 @@ ClientAuthentication(Port *port)
case uaTrust:
status = STATUS_OK;
break;
+ case uaOAuth:
+ status = CheckOAuthBearer(port);
+ break;
}
if ((status == STATUS_OK && port->hba->clientcert == clientCertFull)
@@ -973,7 +962,7 @@ SASL_exchange(const pg_be_sasl_mech *mech, Port *port, char *shadow_pass,
/* Get the actual SASL message */
initStringInfo(&buf);
- if (pq_getmessage(&buf, PG_MAX_SASL_MESSAGE_LENGTH))
+ if (pq_getmessage(&buf, mech->max_message_length))
{
/* EOF - pq_getmessage already logged error */
pfree(buf.data);
@@ -3495,3 +3484,9 @@ PerformRadiusTransaction(const char *server, const char *secret, const char *por
}
} /* while (true) */
}
+
+static int
+CheckOAuthBearer(Port *port)
+{
+ return SASL_exchange(&pg_be_oauth_mech, port, NULL, NULL);
+}
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index 3be8778d21..98147700dd 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -134,7 +134,8 @@ static const char *const UserAuthName[] =
"ldap",
"cert",
"radius",
- "peer"
+ "peer",
+ "oauth",
};
@@ -1399,6 +1400,8 @@ parse_hba_line(TokenizedLine *tok_line, int elevel)
#endif
else if (strcmp(token->string, "radius") == 0)
parsedline->auth_method = uaRADIUS;
+ else if (strcmp(token->string, "oauth") == 0)
+ parsedline->auth_method = uaOAuth;
else
{
ereport(elevel,
@@ -1713,8 +1716,9 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->auth_method != uaPeer &&
hbaline->auth_method != uaGSS &&
hbaline->auth_method != uaSSPI &&
+ hbaline->auth_method != uaOAuth &&
hbaline->auth_method != uaCert)
- INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, and cert"));
+ INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, oauth, and cert"));
hbaline->usermap = pstrdup(val);
}
else if (strcmp(name, "clientcert") == 0)
@@ -2098,6 +2102,27 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->radiusidentifiers = parsed_identifiers;
hbaline->radiusidentifiers_s = pstrdup(val);
}
+ else if (strcmp(name, "issuer") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("issuer", gettext_noop("oauth"));
+ hbaline->oauth_issuer = pstrdup(val);
+ }
+ else if (strcmp(name, "scope") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("scope", gettext_noop("oauth"));
+ hbaline->oauth_scope = pstrdup(val);
+ }
+ else if (strcmp(name, "trust_validator_authz") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("trust_validator_authz", gettext_noop("oauth"));
+ if (strcmp(val, "1") == 0)
+ hbaline->oauth_skip_usermap = true;
+ else
+ hbaline->oauth_skip_usermap = false;
+ }
else
{
ereport(elevel,
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 68b62d523d..1ef6b3c41e 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -56,6 +56,7 @@
#include "libpq/auth.h"
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
+#include "libpq/oauth.h"
#include "miscadmin.h"
#include "optimizer/cost.h"
#include "optimizer/geqo.h"
@@ -4587,6 +4588,17 @@ static struct config_string ConfigureNamesString[] =
check_backtrace_functions, assign_backtrace_functions, NULL
},
+ {
+ {"oauth_validator_command", PGC_SIGHUP, CONN_AUTH_AUTH,
+ gettext_noop("Command to validate OAuth v2 bearer tokens."),
+ NULL,
+ GUC_SUPERUSER_ONLY
+ },
+ &oauth_validator_command,
+ "",
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, NULL, NULL, NULL, NULL
diff --git a/src/include/libpq/auth.h b/src/include/libpq/auth.h
index 3610fae3ff..785cc5d16f 100644
--- a/src/include/libpq/auth.h
+++ b/src/include/libpq/auth.h
@@ -21,6 +21,7 @@ extern bool pg_krb_caseins_users;
extern char *pg_krb_realm;
extern void ClientAuthentication(Port *port);
+extern void set_authn_id(Port *port, const char *id);
/* Hook for plugins to get control in ClientAuthentication() */
typedef void (*ClientAuthentication_hook_type) (Port *, int);
diff --git a/src/include/libpq/hba.h b/src/include/libpq/hba.h
index 8d9f3821b1..441dd5623e 100644
--- a/src/include/libpq/hba.h
+++ b/src/include/libpq/hba.h
@@ -38,8 +38,9 @@ typedef enum UserAuth
uaLDAP,
uaCert,
uaRADIUS,
- uaPeer
-#define USER_AUTH_LAST uaPeer /* Must be last value of this enum */
+ uaPeer,
+ uaOAuth
+#define USER_AUTH_LAST uaOAuth /* Must be last value of this enum */
} UserAuth;
/*
@@ -120,6 +121,9 @@ typedef struct HbaLine
char *radiusidentifiers_s;
List *radiusports;
char *radiusports_s;
+ char *oauth_issuer;
+ char *oauth_scope;
+ bool oauth_skip_usermap;
} HbaLine;
typedef struct IdentLine
diff --git a/src/include/libpq/oauth.h b/src/include/libpq/oauth.h
new file mode 100644
index 0000000000..870e426af1
--- /dev/null
+++ b/src/include/libpq/oauth.h
@@ -0,0 +1,24 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth.h
+ * Interface to libpq/auth-oauth.c
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/libpq/oauth.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_OAUTH_H
+#define PG_OAUTH_H
+
+#include "libpq/libpq-be.h"
+#include "libpq/sasl.h"
+
+extern char *oauth_validator_command;
+
+/* Implementation */
+extern const pg_be_sasl_mech pg_be_oauth_mech;
+
+#endif /* PG_OAUTH_H */
diff --git a/src/include/libpq/sasl.h b/src/include/libpq/sasl.h
index 8c9c9983d4..f1341d0c54 100644
--- a/src/include/libpq/sasl.h
+++ b/src/include/libpq/sasl.h
@@ -19,6 +19,30 @@
#define SASL_EXCHANGE_SUCCESS 1
#define SASL_EXCHANGE_FAILURE 2
+/*
+ * Maximum accepted size of GSS and SSPI authentication tokens.
+ * We also use this as a limit on ordinary password packet lengths.
+ *
+ * Kerberos tickets are usually quite small, but the TGTs issued by Windows
+ * domain controllers include an authorization field known as the Privilege
+ * Attribute Certificate (PAC), which contains the user's Windows permissions
+ * (group memberships etc.). The PAC is copied into all tickets obtained on
+ * the basis of this TGT (even those issued by Unix realms which the Windows
+ * realm trusts), and can be several kB in size. The maximum token size
+ * accepted by Windows systems is determined by the MaxAuthToken Windows
+ * registry setting. Microsoft recommends that it is not set higher than
+ * 65535 bytes, so that seems like a reasonable limit for us as well.
+ */
+#define PG_MAX_AUTH_TOKEN_LENGTH 65535
+
+/*
+ * Maximum accepted size of SASL messages.
+ *
+ * The messages that the server or libpq generate are much smaller than this,
+ * but have some headroom.
+ */
+#define PG_MAX_SASL_MESSAGE_LENGTH 1024
+
/* Backend mechanism API */
typedef void (*pg_be_sasl_mechanism_func)(Port *, StringInfo);
typedef void *(*pg_be_sasl_init_func)(Port *, const char *, const char *);
@@ -29,6 +53,8 @@ typedef struct
pg_be_sasl_mechanism_func get_mechanisms;
pg_be_sasl_init_func init;
pg_be_sasl_exchange_func exchange;
+
+ int max_message_length;
} pg_be_sasl_mech;
#endif /* PG_SASL_H */
--
2.25.1
0006-Add-a-very-simple-authn_id-extension.patch (text/x-patch, 2.8K) download
From e468be7ff7d19645aeb77bef21a383960a47731e Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 18 May 2021 15:01:29 -0700
Subject: [PATCH 6/7] Add a very simple authn_id extension
...for retrieving the authn_id from the server in tests.
---
contrib/authn_id/Makefile | 19 +++++++++++++++++++
contrib/authn_id/authn_id--1.0.sql | 8 ++++++++
contrib/authn_id/authn_id.c | 28 ++++++++++++++++++++++++++++
contrib/authn_id/authn_id.control | 5 +++++
4 files changed, 60 insertions(+)
create mode 100644 contrib/authn_id/Makefile
create mode 100644 contrib/authn_id/authn_id--1.0.sql
create mode 100644 contrib/authn_id/authn_id.c
create mode 100644 contrib/authn_id/authn_id.control
diff --git a/contrib/authn_id/Makefile b/contrib/authn_id/Makefile
new file mode 100644
index 0000000000..46026358e0
--- /dev/null
+++ b/contrib/authn_id/Makefile
@@ -0,0 +1,19 @@
+# contrib/authn_id/Makefile
+
+MODULE_big = authn_id
+OBJS = authn_id.o
+
+EXTENSION = authn_id
+DATA = authn_id--1.0.sql
+PGFILEDESC = "authn_id - information about the authenticated user"
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/authn_id
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/authn_id/authn_id--1.0.sql b/contrib/authn_id/authn_id--1.0.sql
new file mode 100644
index 0000000000..af2a4d3991
--- /dev/null
+++ b/contrib/authn_id/authn_id--1.0.sql
@@ -0,0 +1,8 @@
+/* contrib/authn_id/authn_id--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION authn_id" to load this file. \quit
+
+CREATE FUNCTION authn_id() RETURNS text
+AS 'MODULE_PATHNAME', 'authn_id'
+LANGUAGE C IMMUTABLE;
diff --git a/contrib/authn_id/authn_id.c b/contrib/authn_id/authn_id.c
new file mode 100644
index 0000000000..0fecac36a8
--- /dev/null
+++ b/contrib/authn_id/authn_id.c
@@ -0,0 +1,28 @@
+/*
+ * Extension to expose the current user's authn_id.
+ *
+ * contrib/authn_id/authn_id.c
+ */
+
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "libpq/libpq-be.h"
+#include "miscadmin.h"
+#include "utils/builtins.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(authn_id);
+
+/*
+ * Returns the current user's authenticated identity.
+ */
+Datum
+authn_id(PG_FUNCTION_ARGS)
+{
+ if (!MyProcPort->authn_id)
+ PG_RETURN_NULL();
+
+ PG_RETURN_TEXT_P(cstring_to_text(MyProcPort->authn_id));
+}
diff --git a/contrib/authn_id/authn_id.control b/contrib/authn_id/authn_id.control
new file mode 100644
index 0000000000..e0f9e06bed
--- /dev/null
+++ b/contrib/authn_id/authn_id.control
@@ -0,0 +1,5 @@
+# authn_id extension
+comment = 'current user identity'
+default_version = '1.0'
+module_pathname = '$libdir/authn_id'
+relocatable = true
--
2.25.1
0007-Add-pytest-suite-for-OAuth.patch (text/x-patch, 131.2K) download
From 896da918cfcd16bcb119090914f687b3e905d865 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Fri, 4 Jun 2021 09:06:38 -0700
Subject: [PATCH 7/7] Add pytest suite for OAuth
Requires Python 3; on the first run of `make installcheck` the
dependencies will be installed into ./venv for you. See the README for
more details.
---
src/test/python/.gitignore | 2 +
src/test/python/Makefile | 33 +
src/test/python/README | 49 +
src/test/python/client/__init__.py | 0
src/test/python/client/conftest.py | 126 +++
src/test/python/client/test_client.py | 180 ++++
src/test/python/client/test_oauth.py | 936 ++++++++++++++++++
src/test/python/pq3.py | 727 ++++++++++++++
src/test/python/pytest.ini | 4 +
src/test/python/requirements.txt | 7 +
src/test/python/server/__init__.py | 0
src/test/python/server/conftest.py | 45 +
src/test/python/server/test_oauth.py | 1012 ++++++++++++++++++++
src/test/python/server/test_server.py | 21 +
src/test/python/server/validate_bearer.py | 101 ++
src/test/python/server/validate_reflect.py | 34 +
src/test/python/test_internals.py | 138 +++
src/test/python/test_pq3.py | 558 +++++++++++
src/test/python/tls.py | 195 ++++
19 files changed, 4168 insertions(+)
create mode 100644 src/test/python/.gitignore
create mode 100644 src/test/python/Makefile
create mode 100644 src/test/python/README
create mode 100644 src/test/python/client/__init__.py
create mode 100644 src/test/python/client/conftest.py
create mode 100644 src/test/python/client/test_client.py
create mode 100644 src/test/python/client/test_oauth.py
create mode 100644 src/test/python/pq3.py
create mode 100644 src/test/python/pytest.ini
create mode 100644 src/test/python/requirements.txt
create mode 100644 src/test/python/server/__init__.py
create mode 100644 src/test/python/server/conftest.py
create mode 100644 src/test/python/server/test_oauth.py
create mode 100644 src/test/python/server/test_server.py
create mode 100755 src/test/python/server/validate_bearer.py
create mode 100755 src/test/python/server/validate_reflect.py
create mode 100644 src/test/python/test_internals.py
create mode 100644 src/test/python/test_pq3.py
create mode 100644 src/test/python/tls.py
diff --git a/src/test/python/.gitignore b/src/test/python/.gitignore
new file mode 100644
index 0000000000..0e8f027b2e
--- /dev/null
+++ b/src/test/python/.gitignore
@@ -0,0 +1,2 @@
+__pycache__/
+/venv/
diff --git a/src/test/python/Makefile b/src/test/python/Makefile
new file mode 100644
index 0000000000..515a995106
--- /dev/null
+++ b/src/test/python/Makefile
@@ -0,0 +1,33 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+# Only Python 3 is supported, but if it's named something different on your
+# system you can override it with the PYTHON3 variable.
+PYTHON3 := python3
+
+# All dependencies are placed into this directory. The default is .gitignored
+# for you, but you can override it if you'd like.
+VENV := ./venv
+
+override VBIN := $(VENV)/bin
+override PIP := $(VBIN)/pip
+override PYTEST := $(VBIN)/py.test
+override ISORT := $(VBIN)/isort
+override BLACK := $(VBIN)/black
+
+.PHONY: installcheck indent
+
+installcheck: $(PYTEST)
+ $(PYTEST) -v -rs
+
+indent: $(ISORT) $(BLACK)
+ $(ISORT) --profile black *.py client/*.py server/*.py
+ $(BLACK) *.py client/*.py server/*.py
+
+$(PYTEST) $(ISORT) $(BLACK): requirements.txt | $(PIP)
+ $(PIP) install -r $<
+
+$(PIP):
+ $(PYTHON3) -m venv $(VENV)
diff --git a/src/test/python/README b/src/test/python/README
new file mode 100644
index 0000000000..ceae364e81
--- /dev/null
+++ b/src/test/python/README
@@ -0,0 +1,49 @@
+A test suite for exercising both the libpq client and the server backend at the
+protocol level, based on pytest and Construct.
+
+The test suite currently assumes that the standard PG* environment variables
+point to the database under test and are sufficient to log in a superuser on
+that system. In other words, a bare `psql` needs to Just Work before the test
+suite can do its thing. For a newly built dev cluster, typically all that I need
+to do is a
+
+ export PGDATABASE=postgres
+
+but you can adjust as needed for your setup.
+
+## Requirements
+
+A supported version (3.6+) of Python.
+
+The first run of
+
+ make installcheck
+
+will install a local virtual environment and all needed dependencies.
+
+## Hacking
+
+The code style is enforced by a _very_ opinionated autoformatter. Running the
+
+ make indent
+
+recipe will invoke it for you automatically. Don't fight the tool; part of the
+zen is in knowing that if the formatter makes your code ugly, there's probably a
+cleaner way to write your code.
+
+## Advanced Usage
+
+The Makefile is there for convenience, but you don't have to use it. Activate
+the virtualenv to be able to use pytest directly:
+
+ $ source venv/bin/activate
+ $ py.test -k oauth
+ ...
+ $ py.test ./server/test_server.py
+ ...
+ $ deactivate # puts the PATH et al back the way it was before
+
+To make quick smoke tests possible, slow tests have been marked explicitly. You
+can skip them by saying e.g.
+
+ $ py.test -m 'not slow'
diff --git a/src/test/python/client/__init__.py b/src/test/python/client/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/src/test/python/client/conftest.py b/src/test/python/client/conftest.py
new file mode 100644
index 0000000000..f38da7a138
--- /dev/null
+++ b/src/test/python/client/conftest.py
@@ -0,0 +1,126 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import socket
+import sys
+import threading
+
+import psycopg2
+import pytest
+
+import pq3
+
+BLOCKING_TIMEOUT = 2 # the number of seconds to wait for blocking calls
+
+
+@pytest.fixture
+def server_socket(unused_tcp_port_factory):
+ """
+ Returns a listening socket bound to an ephemeral port.
+ """
+ with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+ s.bind(("127.0.0.1", unused_tcp_port_factory()))
+ s.listen(1)
+ s.settimeout(BLOCKING_TIMEOUT)
+ yield s
+
+
+class ClientHandshake(threading.Thread):
+ """
+ A thread that connects to a local Postgres server using psycopg2. Once the
+ opening handshake completes, the connection will be immediately closed.
+ """
+
+ def __init__(self, *, port, **kwargs):
+ super().__init__()
+
+ kwargs["port"] = port
+ self._kwargs = kwargs
+
+ self.exception = None
+
+ def run(self):
+ try:
+ conn = psycopg2.connect(host="127.0.0.1", **self._kwargs)
+ conn.close()
+ except Exception as e:
+ self.exception = e
+
+ def check_completed(self, timeout=BLOCKING_TIMEOUT):
+ """
+ Joins the client thread. Raises an exception if the thread could not be
+ joined, or if it threw an exception itself. (The exception will be
+ cleared, so future calls to check_completed will succeed.)
+ """
+ self.join(timeout)
+
+ if self.is_alive():
+ raise TimeoutError("client thread did not handshake within the timeout")
+ elif self.exception:
+ e = self.exception
+ self.exception = None
+ raise e
+
+
+@pytest.fixture
+def accept(server_socket):
+ """
+ Returns a factory function that, when called, returns a pair (sock, client)
+ where sock is a server socket that has accepted a connection from client,
+ and client is an instance of ClientHandshake. Clients will complete their
+ handshakes and cleanly disconnect.
+
+ The default connstring options may be extended or overridden by passing
+ arbitrary keyword arguments. Keep in mind that you generally should not
+ override the host or port, since they point to the local test server.
+
+ For situations where a client needs to connect more than once to complete a
+ handshake, the accept function may be called more than once. (The client
+ returned for subsequent calls will always be the same client that was
+ returned for the first call.)
+
+ Tests must either complete the handshake so that the client thread can be
+ automatically joined during teardown, or else call client.check_completed()
+ and manually handle any expected errors.
+ """
+ _, port = server_socket.getsockname()
+
+ client = None
+ default_opts = dict(
+ port=port,
+ user=pq3.pguser(),
+ sslmode="disable",
+ )
+
+ def factory(**kwargs):
+ nonlocal client
+
+ if client is None:
+ opts = dict(default_opts)
+ opts.update(kwargs)
+
+ # The server_socket is already listening, so the client thread can
+ # be safely started; it'll block on the connection until we accept.
+ client = ClientHandshake(**opts)
+ client.start()
+
+ sock, _ = server_socket.accept()
+ return sock, client
+
+ yield factory
+ client.check_completed()
+
+
+@pytest.fixture
+def conn(accept):
+ """
+ Returns an accepted, wrapped pq3 connection to a psycopg2 client. The socket
+ will be closed when the test finishes, and the client will be checked for a
+ cleanly completed handshake.
+ """
+ sock, client = accept()
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ yield conn
diff --git a/src/test/python/client/test_client.py b/src/test/python/client/test_client.py
new file mode 100644
index 0000000000..c4c946fda4
--- /dev/null
+++ b/src/test/python/client/test_client.py
@@ -0,0 +1,180 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import sys
+
+import psycopg2
+import pytest
+from cryptography.hazmat.primitives import hashes, hmac
+
+import pq3
+
+
+def finish_handshake(conn):
+ """
+ Sends the AuthenticationOK message and the standard opening salvo of server
+ messages, then asserts that the client immediately sends a Terminate message
+ to close the connection cleanly.
+ """
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.OK)
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"client_encoding", value=b"UTF-8")
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"DateStyle", value=b"ISO, MDY")
+ pq3.send(conn, pq3.types.ReadyForQuery, status=b"I")
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.Terminate
+
+
+def test_handshake(conn):
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ finish_handshake(conn)
+
+
+def test_aborted_connection(accept):
+ """
+ Make sure the client correctly reports an early close during handshakes.
+ """
+ sock, client = accept()
+ sock.close()
+
+ expected = "server closed the connection unexpectedly"
+ with pytest.raises(psycopg2.OperationalError, match=expected):
+ client.check_completed()
+
+
+#
+# SCRAM-SHA-256 (see RFC 5802: https://tools.ietf.org/html/rfc5802)
+#
+
+
+@pytest.fixture
+def password():
+ """
+ Returns a password for use by both client and server.
+ """
+ # TODO: parameterize this with passwords that require SASLprep.
+ return "secret"
+
+
+@pytest.fixture
+def pwconn(accept, password):
+ """
+ Like the conn fixture, but uses a password in the connection.
+ """
+ sock, client = accept(password=password)
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ yield conn
+
+
+def sha256(data):
+ """The H(str) function from Section 2.2."""
+ digest = hashes.Hash(hashes.SHA256())
+ digest.update(data)
+ return digest.finalize()
+
+
+def hmac_256(key, data):
+ """The HMAC(key, str) function from Section 2.2."""
+ h = hmac.HMAC(key, hashes.SHA256())
+ h.update(data)
+ return h.finalize()
+
+
+def xor(a, b):
+ """The XOR operation from Section 2.2."""
+ res = bytearray(a)
+ for i, byte in enumerate(b):
+ res[i] ^= byte
+ return bytes(res)
+
+
+def h_i(data, salt, i):
+ """The Hi(str, salt, i) function from Section 2.2."""
+ assert i > 0
+
+ acc = hmac_256(data, salt + b"\x00\x00\x00\x01")
+ last = acc
+ i -= 1
+
+ while i:
+ u = hmac_256(data, last)
+ acc = xor(acc, u)
+
+ last = u
+ i -= 1
+
+ return acc
+
+
+def test_scram(pwconn, password):
+ startup = pq3.recv1(pwconn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ pwconn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"SCRAM-SHA-256", b""],
+ )
+
+ # Get the client-first-message.
+ pkt = pq3.recv1(pwconn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ initial = pq3.SASLInitialResponse.parse(pkt.payload)
+ assert initial.name == b"SCRAM-SHA-256"
+
+ c_bind, authzid, c_name, c_nonce = initial.data.split(b",")
+ assert c_bind == b"n" # no channel bindings on a plaintext connection
+ assert authzid == b"" # we don't support authzid currently
+ assert c_name == b"n=" # libpq doesn't honor the GS2 username
+ assert c_nonce.startswith(b"r=")
+
+ # Send the server-first-message.
+ salt = b"12345"
+ iterations = 2
+
+ s_nonce = c_nonce + b"somenonce"
+ s_salt = b"s=" + base64.b64encode(salt)
+ s_iterations = b"i=%d" % iterations
+
+ msg = b",".join([s_nonce, s_salt, s_iterations])
+ pq3.send(pwconn, pq3.types.AuthnRequest, type=pq3.authn.SASLContinue, body=msg)
+
+ # Get the client-final-message.
+ pkt = pq3.recv1(pwconn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ c_bind_final, c_nonce_final, c_proof = pkt.payload.split(b",")
+ assert c_bind_final == b"c=" + base64.b64encode(c_bind + b"," + authzid + b",")
+ assert c_nonce_final == s_nonce
+
+ # Calculate what the client proof should be.
+ salted_password = h_i(password.encode("ascii"), salt, iterations)
+ client_key = hmac_256(salted_password, b"Client Key")
+ stored_key = sha256(client_key)
+
+ auth_message = b",".join(
+ [c_name, c_nonce, s_nonce, s_salt, s_iterations, c_bind_final, c_nonce_final]
+ )
+ client_signature = hmac_256(stored_key, auth_message)
+ client_proof = xor(client_key, client_signature)
+
+ expected = b"p=" + base64.b64encode(client_proof)
+ assert c_proof == expected
+
+ # Send the correct server signature.
+ server_key = hmac_256(salted_password, b"Server Key")
+ server_signature = hmac_256(server_key, auth_message)
+
+ s_verify = b"v=" + base64.b64encode(server_signature)
+ pq3.send(pwconn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal, body=s_verify)
+
+ # Done!
+ finish_handshake(pwconn)
diff --git a/src/test/python/client/test_oauth.py b/src/test/python/client/test_oauth.py
new file mode 100644
index 0000000000..a754a9c0b6
--- /dev/null
+++ b/src/test/python/client/test_oauth.py
@@ -0,0 +1,936 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import http.server
+import json
+import secrets
+import sys
+import threading
+import time
+import urllib.parse
+
+import psycopg2
+import pytest
+
+import pq3
+
+from .conftest import BLOCKING_TIMEOUT
+
+
+def finish_handshake(conn):
+ """
+ Sends the AuthenticationOK message and the standard opening salvo of server
+ messages, then asserts that the client immediately sends a Terminate message
+ to close the connection cleanly.
+ """
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.OK)
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"client_encoding", value=b"UTF-8")
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"DateStyle", value=b"ISO, MDY")
+ pq3.send(conn, pq3.types.ReadyForQuery, status=b"I")
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.Terminate
+
+
+#
+# OAUTHBEARER (see RFC 7628: https://tools.ietf.org/html/rfc7628)
+#
+
+
+def start_oauth_handshake(conn):
+ """
+ Negotiates an OAUTHBEARER SASL challenge. Returns the client's initial
+ response data.
+ """
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn, pq3.types.AuthnRequest, type=pq3.authn.SASL, body=[b"OAUTHBEARER", b""]
+ )
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ initial = pq3.SASLInitialResponse.parse(pkt.payload)
+ assert initial.name == b"OAUTHBEARER"
+
+ return initial.data
+
+
+def get_auth_value(initial):
+ """
+ Finds the auth value (e.g. "Bearer somedata..." in the client's initial SASL
+ response.
+ """
+ kvpairs = initial.split(b"\x01")
+ assert kvpairs[0] == b"n,," # no channel binding or authzid
+ assert kvpairs[2] == b"" # ends with an empty kvpair
+ assert kvpairs[3] == b"" # ...and there's nothing after it
+ assert len(kvpairs) == 4
+
+ key, value = kvpairs[1].split(b"=", 2)
+ assert key == b"auth"
+
+ return value
+
+
+def xtest_oauth_success(conn): # TODO
+ initial = start_oauth_handshake(conn)
+
+ auth = get_auth_value(initial)
+ assert auth.startswith(b"Bearer ")
+
+ # Accept the token. TODO actually validate
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+class OpenIDProvider(threading.Thread):
+ """
+ A thread that runs a mock OpenID provider server.
+ """
+
+ def __init__(self, *, port):
+ super().__init__()
+
+ self.exception = None
+
+ addr = ("", port)
+ self.server = self._Server(addr, self._Handler)
+
+ # TODO: allow HTTPS only, somehow
+ oauth = self._OAuthState()
+ oauth.host = f"localhost:{port}"
+ oauth.issuer = f"http://localhost:{port}"
+
+ # The following endpoints are required to be advertised by providers,
+ # even though our chosen client implementation does not actually make
+ # use of them.
+ oauth.register_endpoint(
+ "authorization_endpoint", "POST", "/authorize", self._authorization_handler
+ )
+ oauth.register_endpoint("jwks_uri", "GET", "/keys", self._jwks_handler)
+
+ self.server.oauth = oauth
+
+ def run(self):
+ try:
+ self.server.serve_forever()
+ except Exception as e:
+ self.exception = e
+
+ def stop(self, timeout=BLOCKING_TIMEOUT):
+ """
+ Shuts down the server and joins its thread. Raises an exception if the
+ thread could not be joined, or if it threw an exception itself. Must
+ only be called once, after start().
+ """
+ self.server.shutdown()
+ self.join(timeout)
+
+ if self.is_alive():
+ raise TimeoutError("client thread did not handshake within the timeout")
+ elif self.exception:
+ e = self.exception
+ raise e
+
+ class _OAuthState(object):
+ def __init__(self):
+ self.endpoint_paths = {}
+ self._endpoints = {}
+
+ def register_endpoint(self, name, method, path, func):
+ if method not in self._endpoints:
+ self._endpoints[method] = {}
+
+ self._endpoints[method][path] = func
+ self.endpoint_paths[name] = path
+
+ def endpoint(self, method, path):
+ if method not in self._endpoints:
+ return None
+
+ return self._endpoints[method].get(path)
+
+ class _Server(http.server.HTTPServer):
+ def handle_error(self, request, addr):
+ self.shutdown_request(request)
+ raise
+
+ @staticmethod
+ def _jwks_handler(headers, params):
+ return 200, {"keys": []}
+
+ @staticmethod
+ def _authorization_handler(headers, params):
+ # We don't actually want this to be called during these tests -- we
+ # should be using the device authorization endpoint instead.
+ assert (
+ False
+ ), "authorization handler called instead of device authorization handler"
+
+ class _Handler(http.server.BaseHTTPRequestHandler):
+ timeout = BLOCKING_TIMEOUT
+
+ def _discovery_handler(self, headers, params):
+ oauth = self.server.oauth
+
+ doc = {
+ "issuer": oauth.issuer,
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ }
+
+ for name, path in oauth.endpoint_paths.items():
+ doc[name] = oauth.issuer + path
+
+ return 200, doc
+
+ def _handle(self, *, params=None, handler=None):
+ oauth = self.server.oauth
+ assert self.headers["Host"] == oauth.host
+
+ if handler is None:
+ handler = oauth.endpoint(self.command, self.path)
+ assert (
+ handler is not None
+ ), f"no registered endpoint for {self.command} {self.path}"
+
+ code, resp = handler(self.headers, params)
+
+ self.send_response(code)
+ self.send_header("Content-Type", "application/json")
+ self.end_headers()
+
+ resp = json.dumps(resp)
+ resp = resp.encode("utf-8")
+ self.wfile.write(resp)
+
+ self.close_connection = True
+
+ def do_GET(self):
+ if self.path == "/.well-known/openid-configuration":
+ self._handle(handler=self._discovery_handler)
+ return
+
+ self._handle()
+
+ def _request_body(self):
+ length = self.headers["Content-Length"]
+
+ # Handle only an explicit content-length.
+ assert length is not None
+ length = int(length)
+
+ return self.rfile.read(length).decode("utf-8")
+
+ def do_POST(self):
+ assert self.headers["Content-Type"] == "application/x-www-form-urlencoded"
+
+ body = self._request_body()
+ params = urllib.parse.parse_qs(body)
+
+ self._handle(params=params)
+
+
+@pytest.fixture
+def openid_provider(unused_tcp_port_factory):
+ """
+ A fixture that returns the OAuth state of a running OpenID provider server. The
+ server will be stopped when the fixture is torn down.
+ """
+ thread = OpenIDProvider(port=unused_tcp_port_factory())
+ thread.start()
+
+ try:
+ yield thread.server.oauth
+ finally:
+ thread.stop()
+
+
+@pytest.mark.parametrize("secret", [None, "", "hunter2"])
+@pytest.mark.parametrize("scope", [None, "", "openid email"])
+@pytest.mark.parametrize("retries", [0, 1])
+def test_oauth_with_explicit_issuer(
+ capfd, accept, openid_provider, retries, scope, secret
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ oauth_client_secret=secret,
+ oauth_scope=scope,
+ )
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+ verification_url = "https://example.com/device"
+
+ access_token = secrets.token_urlsafe()
+
+ def check_client_authn(headers, params):
+ if not secret:
+ assert params["client_id"] == [client_id]
+ return
+
+ # Require the client to use Basic authn; request-body credentials are
+ # NOT RECOMMENDED (RFC 6749, Sec. 2.3.1).
+ assert "Authorization" in headers
+
+ method, creds = headers["Authorization"].split()
+ assert method == "Basic"
+
+ expected = f"{client_id}:{secret}"
+ assert base64.b64decode(creds) == expected.encode("ascii")
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ check_client_authn(headers, params)
+
+ if scope:
+ assert params["scope"] == [scope]
+ else:
+ assert "scope" not in params
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": verification_url,
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ attempts = 0
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ check_client_authn(headers, params)
+
+ assert params["grant_type"] == ["urn:ietf:params:oauth:grant-type:device_code"]
+ assert params["device_code"] == [device_code]
+
+ now = time.monotonic()
+
+ with retry_lock:
+ nonlocal attempts
+
+ # If the test wants to force the client to retry, return an
+ # authorization_pending response and decrement the retry count.
+ if attempts < retries:
+ attempts += 1
+ return 400, {"error": "authorization_pending"}
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+ if retries:
+ # Finally, make sure that the client prompted the user with the expected
+ # authorization URL and user code.
+ expected = f"Visit {verification_url} and enter the code: {user_code}"
+ _, stderr = capfd.readouterr()
+ assert expected in stderr
+
+
+def test_oauth_requires_client_id(accept, openid_provider):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ # Do not set a client ID; this should cause a client error after the
+ # server asks for OAUTHBEARER and the client tries to contact the
+ # issuer.
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake.
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"OAUTHBEARER", b""],
+ )
+
+ # The client should disconnect at this point.
+ assert not conn.read()
+
+ expected_error = "no oauth_client_id is set"
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.slow
+@pytest.mark.parametrize("error_code", ["authorization_pending", "slow_down"])
+@pytest.mark.parametrize("retries", [1, 2])
+def test_oauth_retry_interval(accept, openid_provider, retries, error_code):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id="some-id",
+ )
+
+ expected_retry_interval = 1
+ access_token = secrets.token_urlsafe()
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ resp = {
+ "device_code": "my-device-code",
+ "user_code": "my-user-code",
+ "interval": expected_retry_interval,
+ "verification_uri": "https://example.com",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ attempts = 0
+ last_retry = None
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ now = time.monotonic()
+
+ with retry_lock:
+ nonlocal attempts, last_retry, expected_retry_interval
+
+ # Make sure the retry interval is being respected by the client.
+ if last_retry is not None:
+ interval = now - last_retry
+ assert interval >= expected_retry_interval
+
+ last_retry = now
+
+ # If the test wants to force the client to retry, return the desired
+ # error response and decrement the retry count.
+ if attempts < retries:
+ attempts += 1
+
+ # A slow_down code requires the client to additionally increase
+ # its interval by five seconds.
+ if error_code == "slow_down":
+ expected_retry_interval += 5
+
+ return 400, {"error": error_code}
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+@pytest.mark.parametrize(
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
+ {
+ "error": "invalid_client",
+ "error_description": "client authentication failed",
+ },
+ r"client authentication failed \(invalid_client\)",
+ id="authentication failure with description",
+ ),
+ pytest.param(
+ {"error": "invalid_request"},
+ r"\(invalid_request\)",
+ id="invalid request without description",
+ ),
+ pytest.param(
+ {},
+ r"failed to obtain device authorization",
+ id="broken error response",
+ ),
+ ],
+)
+def test_oauth_device_authorization_failures(
+ accept, openid_provider, failure_mode, error_pattern
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ )
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ return 400, failure_mode
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert False, "token endpoint was invoked unexpectedly"
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"OAUTHBEARER", b""],
+ )
+
+ # The client should not continue the connection due to the hardcoded
+ # provider failure; we disconnect here.
+
+ # Now make sure the client correctly failed.
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
+ {
+ "error": "expired_token",
+ "error_description": "the device code has expired",
+ },
+ r"the device code has expired \(expired_token\)",
+ id="expired token with description",
+ ),
+ pytest.param(
+ {"error": "access_denied"},
+ r"\(access_denied\)",
+ id="access denied without description",
+ ),
+ pytest.param(
+ {},
+ r"OAuth token retrieval failed",
+ id="broken error response",
+ ),
+ ],
+)
+@pytest.mark.parametrize("retries", [0, 1])
+def test_oauth_token_failures(
+ accept, openid_provider, retries, failure_mode, error_pattern
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ )
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ assert params["client_id"] == [client_id]
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": "https://example.com/device",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ with retry_lock:
+ nonlocal retries
+
+ # If the test wants to force the client to retry, return an
+ # authorization_pending response and decrement the retry count.
+ if retries > 0:
+ retries -= 1
+ return 400, {"error": "authorization_pending"}
+
+ return 400, failure_mode
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"OAUTHBEARER", b""],
+ )
+
+ # The client should not continue the connection due to the hardcoded
+ # provider failure; we disconnect here.
+
+ # Now make sure the client correctly failed.
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize("scope", [None, "openid email"])
+@pytest.mark.parametrize(
+ "base_response",
+ [
+ {"status": "invalid_token"},
+ {"extra_object": {"key": "value"}, "status": "invalid_token"},
+ {"extra_object": {"status": 1}, "status": "invalid_token"},
+ ],
+)
+def test_oauth_discovery(accept, openid_provider, base_response, scope):
+ sock, client = accept(oauth_client_id=secrets.token_hex())
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+ verification_url = "https://example.com/device"
+
+ access_token = secrets.token_urlsafe()
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ if scope:
+ assert params["scope"] == [scope]
+ else:
+ assert "scope" not in params
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": verification_url,
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert params["grant_type"] == ["urn:ietf:params:oauth:grant-type:device_code"]
+ assert params["device_code"] == [device_code]
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # For discovery, the client should send an empty auth header. See
+ # RFC 7628, Sec. 4.3.
+ auth = get_auth_value(initial)
+ assert auth == b""
+
+ # We will fail the first SASL exchange. First return a link to the
+ # discovery document, pointing to the test provider server.
+ resp = dict(base_response)
+
+ discovery_uri = f"{openid_provider.issuer}/.well-known/openid-configuration"
+ resp["openid-configuration"] = discovery_uri
+
+ if scope:
+ resp["scope"] = scope
+
+ resp = json.dumps(resp)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASLContinue,
+ body=resp.encode("ascii"),
+ )
+
+ # Per RFC, the client is required to send a dummy ^A response.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+ assert pkt.payload == b"\x01"
+
+ # Now fail the SASL exchange.
+ pq3.send(
+ conn,
+ pq3.types.ErrorResponse,
+ fields=[
+ b"SFATAL",
+ b"C28000",
+ b"Mdoesn't matter",
+ b"",
+ ],
+ )
+
+ # The client will connect to us a second time, using the parameters we sent
+ # it.
+ sock, _ = accept()
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+@pytest.mark.parametrize(
+ "response,expected_error",
+ [
+ pytest.param(
+ "abcde",
+ 'Token "abcde" is invalid',
+ id="bad JSON: invalid syntax",
+ ),
+ pytest.param(
+ '"abcde"',
+ "top-level element must be an object",
+ id="bad JSON: top-level element is a string",
+ ),
+ pytest.param(
+ "[]",
+ "top-level element must be an object",
+ id="bad JSON: top-level element is an array",
+ ),
+ pytest.param(
+ "{}",
+ "server sent error response without a status",
+ id="bad JSON: no status member",
+ ),
+ pytest.param(
+ '{ "status": null }',
+ 'field "status" must be a string',
+ id="bad JSON: null status member",
+ ),
+ pytest.param(
+ '{ "status": 0 }',
+ 'field "status" must be a string',
+ id="bad JSON: int status member",
+ ),
+ pytest.param(
+ '{ "status": [ "bad" ] }',
+ 'field "status" must be a string',
+ id="bad JSON: array status member",
+ ),
+ pytest.param(
+ '{ "status": { "bad": "bad" } }',
+ 'field "status" must be a string',
+ id="bad JSON: object status member",
+ ),
+ pytest.param(
+ '{ "nested": { "status": "bad" } }',
+ "server sent error response without a status",
+ id="bad JSON: nested status",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token" ',
+ "The input string ended unexpectedly",
+ id="bad JSON: unterminated object",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token" } { }',
+ 'Expected end of input, but found "{"',
+ id="bad JSON: trailing data",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "openid-configuration": 1 }',
+ 'field "openid-configuration" must be a string',
+ id="bad JSON: int openid-configuration member",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "openid-configuration": 1 }',
+ 'field "openid-configuration" must be a string',
+ id="bad JSON: int openid-configuration member",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "scope": 1 }',
+ 'field "scope" must be a string',
+ id="bad JSON: int scope member",
+ ),
+ ],
+)
+def test_oauth_discovery_server_error(accept, response, expected_error):
+ sock, client = accept(oauth_client_id=secrets.token_hex())
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # Fail the SASL exchange with an invalid JSON response.
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASLContinue,
+ body=response.encode("utf-8"),
+ )
+
+ # The client should disconnect, so the socket is closed here. (If
+ # the client doesn't disconnect, it will report a different error
+ # below and the test will fail.)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "sasl_err,resp_type,resp_payload,expected_error",
+ [
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.ErrorResponse,
+ dict(
+ fields=[b"SFATAL", b"C28000", b"Mexpected error message", b""],
+ ),
+ "expected error message",
+ id="standard server error: invalid_request",
+ ),
+ pytest.param(
+ {"status": "invalid_token"},
+ pq3.types.ErrorResponse,
+ dict(
+ fields=[b"SFATAL", b"C28000", b"Mexpected error message", b""],
+ ),
+ "expected error message",
+ id="standard server error: invalid_token without discovery URI",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASLContinue, body=b""),
+ "server sent additional OAuth data",
+ id="broken server: additional challenge after error",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASLFinal),
+ "server sent additional OAuth data",
+ id="broken server: SASL success after error",
+ ),
+ ],
+)
+def test_oauth_server_error(accept, sasl_err, resp_type, resp_payload, expected_error):
+ sock, client = accept()
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ start_oauth_handshake(conn)
+
+ # Ignore the client data. Return an error "challenge".
+ resp = json.dumps(sasl_err)
+ resp = resp.encode("utf-8")
+
+ pq3.send(
+ conn, pq3.types.AuthnRequest, type=pq3.authn.SASLContinue, body=resp
+ )
+
+ # Per RFC, the client is required to send a dummy ^A response.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+ assert pkt.payload == b"\x01"
+
+ # Now fail the SASL exchange (in either a valid way, or an invalid
+ # one, depending on the test).
+ pq3.send(conn, resp_type, **resp_payload)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
diff --git a/src/test/python/pq3.py b/src/test/python/pq3.py
new file mode 100644
index 0000000000..3a22dad0b6
--- /dev/null
+++ b/src/test/python/pq3.py
@@ -0,0 +1,727 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import getpass
+import io
+import os
+import ssl
+import sys
+import textwrap
+
+from construct import *
+
+import tls
+
+
+def protocol(major, minor):
+ """
+ Returns the protocol version, in integer format, corresponding to the given
+ major and minor version numbers.
+ """
+ return (major << 16) | minor
+
+
+# Startup
+
+StringList = GreedyRange(NullTerminated(GreedyBytes))
+
+
+class KeyValueAdapter(Adapter):
+ """
+ Turns a key-value store into a null-terminated list of null-terminated
+ strings, as presented on the wire in the startup packet.
+ """
+
+ def _encode(self, obj, context, path):
+ if isinstance(obj, list):
+ return obj
+
+ l = []
+
+ for k, v in obj.items():
+ if isinstance(k, str):
+ k = k.encode("utf-8")
+ l.append(k)
+
+ if isinstance(v, str):
+ v = v.encode("utf-8")
+ l.append(v)
+
+ l.append(b"")
+ return l
+
+ def _decode(self, obj, context, path):
+ # TODO: turn a list back into a dict
+ return obj
+
+
+KeyValues = KeyValueAdapter(StringList)
+
+_startup_payload = Switch(
+ this.proto,
+ {
+ protocol(3, 0): KeyValues,
+ },
+ default=GreedyBytes,
+)
+
+
+def _default_protocol(this):
+ try:
+ if isinstance(this.payload, (list, dict)):
+ return protocol(3, 0)
+ except AttributeError:
+ pass # no payload passed during build
+
+ return 0
+
+
+def _startup_payload_len(this):
+ """
+ The payload field has a fixed size based on the length of the packet. But
+ if the caller hasn't supplied an explicit length at build time, we have to
+ build the payload to figure out how long it is, which requires us to know
+ the length first... This function exists solely to break the cycle.
+ """
+ assert this._building, "_startup_payload_len() cannot be called during parsing"
+
+ try:
+ payload = this.payload
+ except AttributeError:
+ return 0 # no payload
+
+ if isinstance(payload, bytes):
+ # already serialized; just use the given length
+ return len(payload)
+
+ try:
+ proto = this.proto
+ except AttributeError:
+ proto = _default_protocol(this)
+
+ data = _startup_payload.build(payload, proto=proto)
+ return len(data)
+
+
+Startup = Struct(
+ "len" / Default(Int32sb, lambda this: _startup_payload_len(this) + 8),
+ "proto" / Default(Hex(Int32sb), _default_protocol),
+ "payload" / FixedSized(this.len - 8, Default(_startup_payload, b"")),
+)
+
+# Pq3
+
+# Adapted from construct.core.EnumIntegerString
+class EnumNamedByte:
+ def __init__(self, val, name):
+ self._val = val
+ self._name = name
+
+ def __int__(self):
+ return ord(self._val)
+
+ def __str__(self):
+ return "(enum) %s %r" % (self._name, self._val)
+
+ def __repr__(self):
+ return "EnumNamedByte(%r)" % self._val
+
+ def __eq__(self, other):
+ if isinstance(other, EnumNamedByte):
+ other = other._val
+ if not isinstance(other, bytes):
+ return NotImplemented
+
+ return self._val == other
+
+ def __hash__(self):
+ return hash(self._val)
+
+
+# Adapted from construct.core.Enum
+class ByteEnum(Adapter):
+ def __init__(self, **mapping):
+ super(ByteEnum, self).__init__(Byte)
+ self.namemapping = {k: EnumNamedByte(v, k) for k, v in mapping.items()}
+ self.decmapping = {v: EnumNamedByte(v, k) for k, v in mapping.items()}
+
+ def __getattr__(self, name):
+ if name in self.namemapping:
+ return self.decmapping[self.namemapping[name]]
+ raise AttributeError
+
+ def _decode(self, obj, context, path):
+ b = bytes([obj])
+ try:
+ return self.decmapping[b]
+ except KeyError:
+ return EnumNamedByte(b, "(unknown)")
+
+ def _encode(self, obj, context, path):
+ if isinstance(obj, int):
+ return obj
+ elif isinstance(obj, bytes):
+ return ord(obj)
+ return int(obj)
+
+
+types = ByteEnum(
+ ErrorResponse=b"E",
+ ReadyForQuery=b"Z",
+ Query=b"Q",
+ EmptyQueryResponse=b"I",
+ AuthnRequest=b"R",
+ PasswordMessage=b"p",
+ BackendKeyData=b"K",
+ CommandComplete=b"C",
+ ParameterStatus=b"S",
+ DataRow=b"D",
+ Terminate=b"X",
+)
+
+
+authn = Enum(
+ Int32ub,
+ OK=0,
+ SASL=10,
+ SASLContinue=11,
+ SASLFinal=12,
+)
+
+
+_authn_body = Switch(
+ this.type,
+ {
+ authn.OK: Terminated,
+ authn.SASL: StringList,
+ },
+ default=GreedyBytes,
+)
+
+
+def _data_len(this):
+ assert this._building, "_data_len() cannot be called during parsing"
+
+ if not hasattr(this, "data") or this.data is None:
+ return -1
+
+ return len(this.data)
+
+
+# The protocol reuses the PasswordMessage for several authentication response
+# types, and there's no good way to figure out which is which without keeping
+# state for the entire stream. So this is a separate Construct that can be
+# explicitly parsed/built by code that knows it's needed.
+SASLInitialResponse = Struct(
+ "name" / NullTerminated(GreedyBytes),
+ "len" / Default(Int32sb, lambda this: _data_len(this)),
+ "data"
+ / IfThenElse(
+ # Allow tests to explicitly pass an incorrect length during testing, by
+ # not enforcing a FixedSized during build. (The len calculation above
+ # defaults to the correct size.)
+ this._building,
+ Optional(GreedyBytes),
+ If(this.len != -1, Default(FixedSized(this.len, GreedyBytes), b"")),
+ ),
+ Terminated, # make sure the entire response is consumed
+)
+
+
+_column = FocusedSeq(
+ "data",
+ "len" / Default(Int32sb, lambda this: _data_len(this)),
+ "data" / If(this.len != -1, FixedSized(this.len, GreedyBytes)),
+)
+
+
+_payload_map = {
+ types.ErrorResponse: Struct("fields" / StringList),
+ types.ReadyForQuery: Struct("status" / Bytes(1)),
+ types.Query: Struct("query" / NullTerminated(GreedyBytes)),
+ types.EmptyQueryResponse: Terminated,
+ types.AuthnRequest: Struct("type" / authn, "body" / Default(_authn_body, b"")),
+ types.BackendKeyData: Struct("pid" / Int32ub, "key" / Hex(Int32ub)),
+ types.CommandComplete: Struct("tag" / NullTerminated(GreedyBytes)),
+ types.ParameterStatus: Struct(
+ "name" / NullTerminated(GreedyBytes), "value" / NullTerminated(GreedyBytes)
+ ),
+ types.DataRow: Struct("columns" / Default(PrefixedArray(Int16sb, _column), b"")),
+ types.Terminate: Terminated,
+}
+
+
+_payload = FocusedSeq(
+ "_payload",
+ "_payload"
+ / Switch(
+ this._.type,
+ _payload_map,
+ default=GreedyBytes,
+ ),
+ Terminated, # make sure every payload consumes the entire packet
+)
+
+
+def _payload_len(this):
+ """
+ See _startup_payload_len() for an explanation.
+ """
+ assert this._building, "_payload_len() cannot be called during parsing"
+
+ try:
+ payload = this.payload
+ except AttributeError:
+ return 0 # no payload
+
+ if isinstance(payload, bytes):
+ # already serialized; just use the given length
+ return len(payload)
+
+ data = _payload.build(payload, type=this.type)
+ return len(data)
+
+
+Pq3 = Struct(
+ "type" / types,
+ "len" / Default(Int32ub, lambda this: _payload_len(this) + 4),
+ "payload" / FixedSized(this.len - 4, Default(_payload, b"")),
+)
+
+
+# Environment
+
+
+def pghost():
+ return os.environ.get("PGHOST", default="localhost")
+
+
+def pgport():
+ return int(os.environ.get("PGPORT", default=5432))
+
+
+def pguser():
+ try:
+ return os.environ["PGUSER"]
+ except KeyError:
+ return getpass.getuser()
+
+
+def pgdatabase():
+ return os.environ.get("PGDATABASE", default="postgres")
+
+
+# Connections
+
+
+def _hexdump_translation_map():
+ """
+ For hexdumps. Translates any unprintable or non-ASCII bytes into '.'.
+ """
+ input = bytearray()
+
+ for i in range(128):
+ c = chr(i)
+
+ if not c.isprintable():
+ input += bytes([i])
+
+ input += bytes(range(128, 256))
+
+ return bytes.maketrans(input, b"." * len(input))
+
+
+class _DebugStream(object):
+ """
+ Wraps a file-like object and adds hexdumps of the read and write data. Call
+ end_packet() on a _DebugStream to write the accumulated hexdumps to the
+ output stream, along with the packet that was sent.
+ """
+
+ _translation_map = _hexdump_translation_map()
+
+ def __init__(self, stream, out=sys.stdout):
+ """
+ Creates a new _DebugStream wrapping the given stream (which must have
+ been created by wrap()). All attributes not provided by the _DebugStream
+ are delegated to the wrapped stream. out is the text stream to which
+ hexdumps are written.
+ """
+ self.raw = stream
+ self._out = out
+ self._rbuf = io.BytesIO()
+ self._wbuf = io.BytesIO()
+
+ def __getattr__(self, name):
+ return getattr(self.raw, name)
+
+ def __setattr__(self, name, value):
+ if name in ("raw", "_out", "_rbuf", "_wbuf"):
+ return object.__setattr__(self, name, value)
+
+ setattr(self.raw, name, value)
+
+ def read(self, *args, **kwargs):
+ buf = self.raw.read(*args, **kwargs)
+
+ self._rbuf.write(buf)
+ return buf
+
+ def write(self, b):
+ self._wbuf.write(b)
+ return self.raw.write(b)
+
+ def recv(self, *args):
+ buf = self.raw.recv(*args)
+
+ self._rbuf.write(buf)
+ return buf
+
+ def _flush(self, buf, prefix):
+ width = 16
+ hexwidth = width * 3 - 1
+
+ count = 0
+ buf.seek(0)
+
+ while True:
+ line = buf.read(16)
+
+ if not line:
+ if count:
+ self._out.write("\n") # separate the output block with a newline
+ return
+
+ self._out.write("%s %04X:\t" % (prefix, count))
+ self._out.write("%*s\t" % (-hexwidth, line.hex(" ")))
+ self._out.write(line.translate(self._translation_map).decode("ascii"))
+ self._out.write("\n")
+
+ count += 16
+
+ def print_debug(self, obj, *, prefix=""):
+ contents = ""
+ if obj is not None:
+ contents = str(obj)
+
+ for line in contents.splitlines():
+ self._out.write("%s%s\n" % (prefix, line))
+
+ self._out.write("\n")
+
+ def flush_debug(self, *, prefix=""):
+ self._flush(self._rbuf, prefix + "<")
+ self._rbuf = io.BytesIO()
+
+ self._flush(self._wbuf, prefix + ">")
+ self._wbuf = io.BytesIO()
+
+ def end_packet(self, pkt, *, read=False, prefix="", indent=" "):
+ """
+ Marks the end of a logical "packet" of data. A string representation of
+ pkt will be printed, and the debug buffers will be flushed with an
+ indent. All lines can be optionally prefixed.
+
+ If read is True, the packet representation is written after the debug
+ buffers; otherwise the default of False (meaning write) causes the
+ packet representation to be dumped first. This is meant to capture the
+ logical flow of layer translation.
+ """
+ write = not read
+
+ if write:
+ self.print_debug(pkt, prefix=prefix + "> ")
+
+ self.flush_debug(prefix=prefix + indent)
+
+ if read:
+ self.print_debug(pkt, prefix=prefix + "< ")
+
+
+@contextlib.contextmanager
+def wrap(socket, *, debug_stream=None):
+ """
+ Transforms a raw socket into a connection that can be used for Construct
+ building and parsing. The return value is a context manager and can be used
+ in a with statement.
+ """
+ # It is critical that buffering be disabled here, so that we can still
+ # manipulate the raw socket without desyncing the stream.
+ with socket.makefile("rwb", buffering=0) as sfile:
+ # Expose the original socket's recv() on the SocketIO object we return.
+ def recv(self, *args):
+ return socket.recv(*args)
+
+ sfile.recv = recv.__get__(sfile)
+
+ conn = sfile
+ if debug_stream:
+ conn = _DebugStream(conn, debug_stream)
+
+ try:
+ yield conn
+ finally:
+ if debug_stream:
+ conn.flush_debug(prefix="? ")
+
+
+def _send(stream, cls, obj):
+ debugging = hasattr(stream, "flush_debug")
+ out = io.BytesIO()
+
+ # Ideally we would build directly to the passed stream, but because we need
+ # to reparse the generated output for the debugging case, build to an
+ # intermediate BytesIO and send it instead.
+ cls.build_stream(obj, out)
+ buf = out.getvalue()
+
+ stream.write(buf)
+ if debugging:
+ pkt = cls.parse(buf)
+ stream.end_packet(pkt)
+
+ stream.flush()
+
+
+def send(stream, packet_type, payload_data=None, **payloadkw):
+ """
+ Sends a packet on the given pq3 connection. type is the pq3.types member
+ that should be assigned to the packet. If payload_data is given, it will be
+ used as the packet payload; otherwise the key/value pairs in payloadkw will
+ be the payload contents.
+ """
+ data = payloadkw
+
+ if payload_data is not None:
+ if payloadkw:
+ raise ValueError(
+ "payload_data and payload keywords may not be used simultaneously"
+ )
+
+ data = payload_data
+
+ _send(stream, Pq3, dict(type=packet_type, payload=data))
+
+
+def send_startup(stream, proto=None, **kwargs):
+ """
+ Sends a startup packet on the given pq3 connection. In most cases you should
+ use the handshake functions instead, which will do this for you.
+
+ By default, a protocol version 3 packet will be sent. This can be overridden
+ with the proto parameter.
+ """
+ pkt = {}
+
+ if proto is not None:
+ pkt["proto"] = proto
+ if kwargs:
+ pkt["payload"] = kwargs
+
+ _send(stream, Startup, pkt)
+
+
+def recv1(stream, *, cls=Pq3):
+ """
+ Receives a single pq3 packet from the given stream and returns it.
+ """
+ resp = cls.parse_stream(stream)
+
+ debugging = hasattr(stream, "flush_debug")
+ if debugging:
+ stream.end_packet(resp, read=True)
+
+ return resp
+
+
+def handshake(stream, **kwargs):
+ """
+ Performs a libpq v3 startup handshake. kwargs should contain the key/value
+ parameters to send to the server in the startup packet.
+ """
+ # Send our startup parameters.
+ send_startup(stream, **kwargs)
+
+ # Receive and dump packets until the server indicates it's ready for our
+ # first query.
+ while True:
+ resp = recv1(stream)
+ if resp is None:
+ raise RuntimeError("server closed connection during handshake")
+
+ if resp.type == types.ReadyForQuery:
+ return
+ elif resp.type == types.ErrorResponse:
+ raise RuntimeError(
+ f"received error response from peer: {resp.payload.fields!r}"
+ )
+
+
+# TLS
+
+
+class _TLSStream(object):
+ """
+ A file-like object that performs TLS encryption/decryption on a wrapped
+ stream. Differs from ssl.SSLSocket in that we have full visibility and
+ control over the TLS layer.
+ """
+
+ def __init__(self, stream, context):
+ self._stream = stream
+ self._debugging = hasattr(stream, "flush_debug")
+
+ self._in = ssl.MemoryBIO()
+ self._out = ssl.MemoryBIO()
+ self._ssl = context.wrap_bio(self._in, self._out)
+
+ def handshake(self):
+ try:
+ self._pump(lambda: self._ssl.do_handshake())
+ finally:
+ self._flush_debug(prefix="? ")
+
+ def read(self, *args):
+ return self._pump(lambda: self._ssl.read(*args))
+
+ def write(self, *args):
+ return self._pump(lambda: self._ssl.write(*args))
+
+ def _decode(self, buf):
+ """
+ Attempts to decode a buffer of TLS data into a packet representation
+ that can be printed.
+
+ TODO: handle buffers (and record fragments) that don't align with packet
+ boundaries.
+ """
+ end = len(buf)
+ bio = io.BytesIO(buf)
+
+ ret = io.StringIO()
+
+ while bio.tell() < end:
+ record = tls.Plaintext.parse_stream(bio)
+
+ if ret.tell() > 0:
+ ret.write("\n")
+ ret.write("[Record] ")
+ ret.write(str(record))
+ ret.write("\n")
+
+ if record.type == tls.ContentType.handshake:
+ record_cls = tls.Handshake
+ else:
+ continue
+
+ innerlen = len(record.fragment)
+ inner = io.BytesIO(record.fragment)
+
+ while inner.tell() < innerlen:
+ msg = record_cls.parse_stream(inner)
+
+ indented = "[Message] " + str(msg)
+ indented = textwrap.indent(indented, " ")
+
+ ret.write("\n")
+ ret.write(indented)
+ ret.write("\n")
+
+ return ret.getvalue()
+
+ def flush(self):
+ if not self._out.pending:
+ self._stream.flush()
+ return
+
+ buf = self._out.read()
+ self._stream.write(buf)
+
+ if self._debugging:
+ pkt = self._decode(buf)
+ self._stream.end_packet(pkt, prefix=" ")
+
+ self._stream.flush()
+
+ def _pump(self, operation):
+ while True:
+ try:
+ return operation()
+ except (ssl.SSLWantReadError, ssl.SSLWantWriteError) as e:
+ want = e
+ self._read_write(want)
+
+ def _recv(self, maxsize):
+ buf = self._stream.recv(4096)
+ if not buf:
+ self._in.write_eof()
+ return
+
+ self._in.write(buf)
+
+ if not self._debugging:
+ return
+
+ pkt = self._decode(buf)
+ self._stream.end_packet(pkt, read=True, prefix=" ")
+
+ def _read_write(self, want):
+ # XXX This needs work. So many corner cases yet to handle. For one,
+ # doing blocking writes in flush may lead to distributed deadlock if the
+ # peer is already blocking on its writes.
+
+ if isinstance(want, ssl.SSLWantWriteError):
+ assert self._out.pending, "SSL backend wants write without data"
+
+ self.flush()
+
+ if isinstance(want, ssl.SSLWantReadError):
+ self._recv(4096)
+
+ def _flush_debug(self, prefix):
+ if not self._debugging:
+ return
+
+ self._stream.flush_debug(prefix=prefix)
+
+
+@contextlib.contextmanager
+def tls_handshake(stream, context):
+ """
+ Performs a TLS handshake over the given stream (which must have been created
+ via a call to wrap()), and returns a new stream which transparently tunnels
+ data over the TLS connection.
+
+ If the passed stream has debugging enabled, the returned stream will also
+ have debugging, using the same output IO.
+ """
+ debugging = hasattr(stream, "flush_debug")
+
+ # Send our startup parameters.
+ send_startup(stream, proto=protocol(1234, 5679))
+
+ # Look at the SSL response.
+ resp = stream.read(1)
+ if debugging:
+ stream.flush_debug(prefix=" ")
+
+ if resp == b"N":
+ raise RuntimeError("server does not support SSLRequest")
+ if resp != b"S":
+ raise RuntimeError(f"unexpected response of type {resp!r} during TLS startup")
+
+ tls = _TLSStream(stream, context)
+ tls.handshake()
+
+ if debugging:
+ tls = _DebugStream(tls, stream._out)
+
+ try:
+ yield tls
+ # TODO: teardown/unwrap the connection?
+ finally:
+ if debugging:
+ tls.flush_debug(prefix="? ")
diff --git a/src/test/python/pytest.ini b/src/test/python/pytest.ini
new file mode 100644
index 0000000000..ab7a6e7fb9
--- /dev/null
+++ b/src/test/python/pytest.ini
@@ -0,0 +1,4 @@
+[pytest]
+
+markers =
+ slow: mark test as slow
diff --git a/src/test/python/requirements.txt b/src/test/python/requirements.txt
new file mode 100644
index 0000000000..32f105ea84
--- /dev/null
+++ b/src/test/python/requirements.txt
@@ -0,0 +1,7 @@
+black
+cryptography~=3.4.6
+construct~=2.10.61
+isort~=5.6
+psycopg2~=2.8.6
+pytest~=6.1
+pytest-asyncio~=0.14.0
diff --git a/src/test/python/server/__init__.py b/src/test/python/server/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/src/test/python/server/conftest.py b/src/test/python/server/conftest.py
new file mode 100644
index 0000000000..ba7342a453
--- /dev/null
+++ b/src/test/python/server/conftest.py
@@ -0,0 +1,45 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import socket
+import sys
+
+import pytest
+
+import pq3
+
+
+@pytest.fixture
+def connect():
+ """
+ A factory fixture that, when called, returns a socket connected to a
+ Postgres server, wrapped in a pq3 connection. The calling test will be
+ skipped automatically if a server is not running at PGHOST:PGPORT, so it's
+ best to connect as soon as possible after the test case begins, to avoid
+ doing unnecessary work.
+ """
+ # Set up an ExitStack to handle safe cleanup of all of the moving pieces.
+ with contextlib.ExitStack() as stack:
+
+ def conn_factory():
+ addr = (pq3.pghost(), pq3.pgport())
+
+ try:
+ sock = socket.create_connection(addr, timeout=2)
+ except ConnectionError as e:
+ pytest.skip(f"unable to connect to {addr}: {e}")
+
+ # Have ExitStack close our socket.
+ stack.enter_context(sock)
+
+ # Wrap the connection in a pq3 layer and have ExitStack clean it up
+ # too.
+ wrap_ctx = pq3.wrap(sock, debug_stream=sys.stdout)
+ conn = stack.enter_context(wrap_ctx)
+
+ return conn
+
+ yield conn_factory
diff --git a/src/test/python/server/test_oauth.py b/src/test/python/server/test_oauth.py
new file mode 100644
index 0000000000..355ef8e4bd
--- /dev/null
+++ b/src/test/python/server/test_oauth.py
@@ -0,0 +1,1012 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import contextlib
+import json
+import os
+import pathlib
+import secrets
+import shlex
+import shutil
+import socket
+import struct
+from multiprocessing import shared_memory
+
+import psycopg2
+import pytest
+from psycopg2 import sql
+
+import pq3
+
+MAX_SASL_MESSAGE_LENGTH = 65535
+
+INVALID_AUTHORIZATION_ERRCODE = b"28000"
+PROTOCOL_VIOLATION_ERRCODE = b"08P01"
+FEATURE_NOT_SUPPORTED_ERRCODE = b"0A000"
+
+SHARED_MEM_NAME = "oauth-pytest"
+MAX_TOKEN_SIZE = 4096
+MAX_UINT16 = 2 ** 16 - 1
+
+
+def skip_if_no_postgres():
+ """
+ Used by the oauth_ctx fixture to skip this test module if no Postgres server
+ is running.
+
+ This logic is nearly duplicated with the conn fixture. Ideally oauth_ctx
+ would depend on that, but a module-scope fixture can't depend on a
+ test-scope fixture, and we haven't reached the rule of three yet.
+ """
+ addr = (pq3.pghost(), pq3.pgport())
+
+ try:
+ with socket.create_connection(addr, timeout=2):
+ pass
+ except ConnectionError as e:
+ pytest.skip(f"unable to connect to {addr}: {e}")
+
+
+@contextlib.contextmanager
+def prepend_file(path, lines):
+ """
+ A context manager that prepends a file on disk with the desired lines of
+ text. When the context manager is exited, the file will be restored to its
+ original contents.
+ """
+ # First make a backup of the original file.
+ bak = path + ".bak"
+ shutil.copy2(path, bak)
+
+ try:
+ # Write the new lines, followed by the original file content.
+ with open(path, "w") as new, open(bak, "r") as orig:
+ new.writelines(lines)
+ shutil.copyfileobj(orig, new)
+
+ # Return control to the calling code.
+ yield
+
+ finally:
+ # Put the backup back into place.
+ os.replace(bak, path)
+
+
+@pytest.fixture(scope="module")
+def oauth_ctx():
+ """
+ Creates a database and user that use the oauth auth method. The context
+ object contains the dbname and user attributes as strings to be used during
+ connection, as well as the issuer and scope that have been set in the HBA
+ configuration.
+
+ This fixture assumes that the standard PG* environment variables point to a
+ server running on a local machine, and that the PGUSER has rights to create
+ databases and roles.
+ """
+ skip_if_no_postgres() # don't bother running these tests without a server
+
+ id = secrets.token_hex(4)
+
+ class Context:
+ dbname = "oauth_test_" + id
+
+ user = "oauth_user_" + id
+ map_user = "oauth_map_user_" + id
+ authz_user = "oauth_authz_user_" + id
+
+ issuer = "https://example.com/" + id
+ scope = "openid " + id
+
+ ctx = Context()
+ hba_lines = (
+ f'host {ctx.dbname} {ctx.map_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" map=oauth\n',
+ f'host {ctx.dbname} {ctx.authz_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" trust_validator_authz=1\n',
+ f'host {ctx.dbname} all samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}"\n',
+ )
+ ident_lines = (r"oauth /^(.*)@example\.com$ \1",)
+
+ conn = psycopg2.connect("")
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+
+ # Create our roles and database.
+ user = sql.Identifier(ctx.user)
+ map_user = sql.Identifier(ctx.map_user)
+ authz_user = sql.Identifier(ctx.authz_user)
+ dbname = sql.Identifier(ctx.dbname)
+
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(map_user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(authz_user))
+ c.execute(sql.SQL("CREATE DATABASE {};").format(dbname))
+
+ # Make this test script the server's oauth_validator.
+ path = pathlib.Path(__file__).parent / "validate_bearer.py"
+ path = str(path.absolute())
+
+ cmd = f"{shlex.quote(path)} {SHARED_MEM_NAME} <&%f"
+ c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (cmd,))
+
+ # Replace pg_hba and pg_ident.
+ c.execute("SHOW hba_file;")
+ hba = c.fetchone()[0]
+
+ c.execute("SHOW ident_file;")
+ ident = c.fetchone()[0]
+
+ with prepend_file(hba, hba_lines), prepend_file(ident, ident_lines):
+ c.execute("SELECT pg_reload_conf();")
+
+ # Use the new database and user.
+ yield ctx
+
+ # Put things back the way they were.
+ c.execute("SELECT pg_reload_conf();")
+
+ c.execute("ALTER SYSTEM RESET oauth_validator_command;")
+ c.execute(sql.SQL("DROP DATABASE {};").format(dbname))
+ c.execute(sql.SQL("DROP ROLE {};").format(authz_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(map_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(user))
+
+
+@pytest.fixture()
+def conn(oauth_ctx, connect):
+ """
+ A convenience wrapper for connect(). The main purpose of this fixture is to
+ make sure oauth_ctx runs its setup code before the connection is made.
+ """
+ return connect()
+
+
+@pytest.fixture(scope="module", autouse=True)
+def authn_id_extension(oauth_ctx):
+ """
+ Performs a `CREATE EXTENSION authn_id` in the test database. This fixture is
+ autoused, so tests don't need to rely on it.
+ """
+ conn = psycopg2.connect(database=oauth_ctx.dbname)
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+ c.execute("CREATE EXTENSION authn_id;")
+
+
+@pytest.fixture(scope="session")
+def shared_mem():
+ """
+ Yields a shared memory segment that can be used for communication between
+ the bearer_token fixture and ./validate_bearer.py.
+ """
+ size = MAX_TOKEN_SIZE + 2 # two byte length prefix
+ mem = shared_memory.SharedMemory(SHARED_MEM_NAME, create=True, size=size)
+
+ try:
+ with contextlib.closing(mem):
+ yield mem
+ finally:
+ mem.unlink()
+
+
+@pytest.fixture()
+def bearer_token(shared_mem):
+ """
+ Returns a factory function that, when called, will store a Bearer token in
+ shared_mem. If token is None (the default), a new token will be generated
+ using secrets.token_urlsafe() and returned; otherwise the passed token will
+ be used as-is.
+
+ When token is None, the generated token size in bytes may be specified as an
+ argument; if unset, a small 16-byte token will be generated. The token size
+ may not exceed MAX_TOKEN_SIZE in any case.
+
+ The return value is the token, converted to a bytes object.
+
+ As a special case for testing failure modes, accept_any may be set to True.
+ This signals to the validator command that any bearer token should be
+ accepted. The returned token in this case may be used or discarded as needed
+ by the test.
+ """
+
+ def set_token(token=None, *, size=16, accept_any=False):
+ if token is not None:
+ size = len(token)
+
+ if size > MAX_TOKEN_SIZE:
+ raise ValueError(f"token size {size} exceeds maximum size {MAX_TOKEN_SIZE}")
+
+ if token is None:
+ if size % 4:
+ raise ValueError(f"requested token size {size} is not a multiple of 4")
+
+ token = secrets.token_urlsafe(size // 4 * 3)
+ assert len(token) == size
+
+ try:
+ token = token.encode("ascii")
+ except AttributeError:
+ pass # already encoded
+
+ if accept_any:
+ # Two-byte magic value.
+ shared_mem.buf[:2] = struct.pack("H", MAX_UINT16)
+ else:
+ # Two-byte length prefix, then the token data.
+ shared_mem.buf[:2] = struct.pack("H", len(token))
+ shared_mem.buf[2 : size + 2] = token
+
+ return token
+
+ return set_token
+
+
+def begin_oauth_handshake(conn, oauth_ctx, *, user=None):
+ if user is None:
+ user = oauth_ctx.authz_user
+
+ pq3.send_startup(conn, user=user, database=oauth_ctx.dbname)
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.AuthnRequest
+
+ # The server should advertise exactly one mechanism.
+ assert resp.payload.type == pq3.authn.SASL
+ assert resp.payload.body == [b"OAUTHBEARER", b""]
+
+
+def send_initial_response(conn, *, auth=None, bearer=None):
+ """
+ Sends the OAUTHBEARER initial response on the connection, using the given
+ bearer token. Alternatively to a bearer token, the initial response's auth
+ field may be explicitly specified to test corner cases.
+ """
+ if bearer is not None and auth is not None:
+ raise ValueError("exactly one of the auth and bearer kwargs must be set")
+
+ if bearer is not None:
+ auth = b"Bearer " + bearer
+
+ if auth is None:
+ raise ValueError("exactly one of the auth and bearer kwargs must be set")
+
+ initial = pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"n,,\x01auth=" + auth + b"\x01\x01",
+ )
+ )
+ pq3.send(conn, pq3.types.PasswordMessage, initial)
+
+
+def expect_handshake_success(conn):
+ """
+ Validates that the server responds with an AuthnOK message, and then drains
+ the connection until a ReadyForQuery message is received.
+ """
+ resp = pq3.recv1(conn)
+
+ assert resp.type == pq3.types.AuthnRequest
+ assert resp.payload.type == pq3.authn.OK
+ assert not resp.payload.body
+
+ receive_until(conn, pq3.types.ReadyForQuery)
+
+
+def expect_handshake_failure(conn, oauth_ctx):
+ """
+ Performs the OAUTHBEARER SASL failure "handshake" and validates the server's
+ side of the conversation, including the final ErrorResponse.
+ """
+
+ # We expect a discovery "challenge" back from the server before the authn
+ # failure message.
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.AuthnRequest
+
+ req = resp.payload
+ assert req.type == pq3.authn.SASLContinue
+
+ body = json.loads(req.body)
+ assert body["status"] == "invalid_token"
+ assert body["scope"] == oauth_ctx.scope
+
+ expected_config = oauth_ctx.issuer + "/.well-known/openid-configuration"
+ assert body["openid-configuration"] == expected_config
+
+ # Send the dummy response to complete the failed handshake.
+ pq3.send(conn, pq3.types.PasswordMessage, b"\x01")
+ resp = pq3.recv1(conn)
+
+ err = ExpectedError(INVALID_AUTHORIZATION_ERRCODE, "bearer authentication failed")
+ err.match(resp)
+
+
+def receive_until(conn, type):
+ """
+ receive_until pulls packets off the pq3 connection until a packet with the
+ desired type is found, or an error response is received.
+ """
+ while True:
+ pkt = pq3.recv1(conn)
+
+ if pkt.type == type:
+ return pkt
+ elif pkt.type == pq3.types.ErrorResponse:
+ raise RuntimeError(
+ f"received error response from peer: {pkt.payload.fields!r}"
+ )
+
+
+@pytest.mark.parametrize("token_len", [16, 1024, 4096])
+@pytest.mark.parametrize(
+ "auth_prefix",
+ [
+ b"Bearer ",
+ b"bearer ",
+ b"Bearer ",
+ ],
+)
+def test_oauth(conn, oauth_ctx, bearer_token, auth_prefix, token_len):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Generate our bearer token with the desired length.
+ token = bearer_token(size=token_len)
+ auth = auth_prefix + token
+
+ send_initial_response(conn, auth=auth)
+ expect_handshake_success(conn)
+
+ # Make sure that the server has not set an authenticated ID.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT authn_id();")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ assert row.columns == [None]
+
+
+@pytest.mark.parametrize(
+ "token_value",
+ [
+ "abcdzA==",
+ "123456M=",
+ "x-._~+/x",
+ ],
+)
+def test_oauth_bearer_corner_cases(conn, oauth_ctx, bearer_token, token_value):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ send_initial_response(conn, bearer=bearer_token(token_value))
+
+ expect_handshake_success(conn)
+
+
+@pytest.mark.parametrize(
+ "user,authn_id,should_succeed",
+ [
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: ctx.user,
+ True,
+ id="validator authn: succeeds when authn_id == username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: None,
+ False,
+ id="validator authn: fails when authn_id is not set",
+ ),
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: ctx.authz_user,
+ False,
+ id="validator authn: fails when authn_id != username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: ctx.map_user + "@example.com",
+ True,
+ id="validator with map: succeeds when authn_id matches map",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: None,
+ False,
+ id="validator with map: fails when authn_id is not set",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: ctx.map_user + "@example.net",
+ False,
+ id="validator with map: fails when authn_id doesn't match map",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: None,
+ True,
+ id="validator authz: succeeds with no authn_id",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "",
+ True,
+ id="validator authz: succeeds with empty authn_id",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "postgres",
+ True,
+ id="validator authz: succeeds with basic username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "me@example.com",
+ True,
+ id="validator authz: succeeds with email address",
+ ),
+ ],
+)
+def test_oauth_authn_id(conn, oauth_ctx, bearer_token, user, authn_id, should_succeed):
+ token = None
+
+ authn_id = authn_id(oauth_ctx)
+ if authn_id is not None:
+ authn_id = authn_id.encode("ascii")
+
+ # As a hack to get the validator to reflect arbitrary output from this
+ # test, encode the desired output as a base64 token. The validator will
+ # key on the leading "output=" to differentiate this from the random
+ # tokens generated by secrets.token_urlsafe().
+ output = b"output=" + authn_id + b"\n"
+ token = base64.urlsafe_b64encode(output)
+
+ token = bearer_token(token)
+ username = user(oauth_ctx)
+
+ begin_oauth_handshake(conn, oauth_ctx, user=username)
+ send_initial_response(conn, bearer=token)
+
+ if not should_succeed:
+ expect_handshake_failure(conn, oauth_ctx)
+ return
+
+ expect_handshake_success(conn)
+
+ # Check the reported authn_id.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT authn_id();")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ assert row.columns == [authn_id]
+
+
+class ExpectedError(object):
+ def __init__(self, code, msg=None, detail=None):
+ self.code = code
+ self.msg = msg
+ self.detail = detail
+
+ # Protect against the footgun of an accidental empty string, which will
+ # "match" anything. If you don't want to match message or detail, just
+ # don't pass them.
+ if self.msg == "":
+ raise ValueError("msg must be non-empty or None")
+ if self.detail == "":
+ raise ValueError("detail must be non-empty or None")
+
+ def _getfield(self, resp, type):
+ """
+ Searches an ErrorResponse for a single field of the given type (e.g.
+ "M", "C", "D") and returns its value. Asserts if it doesn't find exactly
+ one field.
+ """
+ prefix = type.encode("ascii")
+ fields = [f for f in resp.payload.fields if f.startswith(prefix)]
+
+ assert len(fields) == 1
+ return fields[0][1:] # strip off the type byte
+
+ def match(self, resp):
+ """
+ Checks that the given response matches the expected code, message, and
+ detail (if given). The error code must match exactly. The expected
+ message and detail must be contained within the actual strings.
+ """
+ assert resp.type == pq3.types.ErrorResponse
+
+ code = self._getfield(resp, "C")
+ assert code == self.code
+
+ if self.msg:
+ msg = self._getfield(resp, "M")
+ expected = self.msg.encode("utf-8")
+ assert expected in msg
+
+ if self.detail:
+ detail = self._getfield(resp, "D")
+ expected = self.detail.encode("utf-8")
+ assert expected in detail
+
+
+def test_oauth_rejected_bearer(conn, oauth_ctx, bearer_token):
+ # Generate a new bearer token, which we will proceed not to use.
+ _ = bearer_token()
+
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send a bearer token that doesn't match what the validator expects. It
+ # should fail the connection.
+ send_initial_response(conn, bearer=b"xxxxxx")
+
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.parametrize(
+ "bad_bearer",
+ [
+ b"Bearer ",
+ b"Bearer a===b",
+ b"Bearer hello!",
+ b"Bearer me@example.com",
+ b'OAuth realm="Example"',
+ b"",
+ ],
+)
+def test_oauth_invalid_bearer(conn, oauth_ctx, bearer_token, bad_bearer):
+ # Tell the validator to accept any token. This ensures that the invalid
+ # bearer tokens are rejected before the validation step.
+ _ = bearer_token(accept_any=True)
+
+ begin_oauth_handshake(conn, oauth_ctx)
+ send_initial_response(conn, auth=bad_bearer)
+
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.slow
+@pytest.mark.parametrize(
+ "resp_type,resp,err",
+ [
+ pytest.param(
+ None,
+ None,
+ None,
+ marks=pytest.mark.slow,
+ id="no response (expect timeout)",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"hello",
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not send a kvsep response",
+ ),
+ id="bad dummy response",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"\x01\x01",
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not send a kvsep response",
+ ),
+ id="multiple kvseps",
+ ),
+ pytest.param(
+ pq3.types.Query,
+ dict(query=b""),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "expected SASL response"),
+ id="bad response message type",
+ ),
+ ],
+)
+def test_oauth_bad_response_to_error_challenge(conn, oauth_ctx, resp_type, resp, err):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send an empty auth initial response, which will force an authn failure.
+ send_initial_response(conn, auth=b"")
+
+ # We expect a discovery "challenge" back from the server before the authn
+ # failure message.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.AuthnRequest
+
+ req = pkt.payload
+ assert req.type == pq3.authn.SASLContinue
+
+ body = json.loads(req.body)
+ assert body["status"] == "invalid_token"
+
+ if resp_type is None:
+ # Do not send the dummy response. We should time out and not get a
+ # response from the server.
+ with pytest.raises(socket.timeout):
+ conn.read(1)
+
+ # Done with the test.
+ return
+
+ # Send the bad response.
+ pq3.send(conn, resp_type, resp)
+
+ # Make sure the server fails the connection correctly.
+ pkt = pq3.recv1(conn)
+ err.match(pkt)
+
+
+@pytest.mark.parametrize(
+ "type,payload,err",
+ [
+ pytest.param(
+ pq3.types.ErrorResponse,
+ dict(fields=[b""]),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "expected SASL response"),
+ id="error response in initial message",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"x" * (MAX_SASL_MESSAGE_LENGTH + 1),
+ ExpectedError(
+ INVALID_AUTHORIZATION_ERRCODE, "bearer authentication failed"
+ ),
+ id="overlong initial response data",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"SCRAM-SHA-256")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE, "invalid SASL authentication mechanism"
+ ),
+ id="bad SASL mechanism selection",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", len=2, data=b"x")),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "insufficient data"),
+ id="SASL data underflow",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", len=0, data=b"x")),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "invalid message format"),
+ id="SASL data overflow",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "message is empty",
+ ),
+ id="empty",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"n,,\x01auth=\x01\x01\0")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "length does not match input length",
+ ),
+ id="contains null byte",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected channel-binding flag", # XXX this is a bit strange
+ ),
+ id="initial error response",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"p=tls-server-end-point,,\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "server does not support channel binding",
+ ),
+ id="uses channel binding",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"x,,\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected channel-binding flag",
+ ),
+ id="invalid channel binding specifier",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Comma expected",
+ ),
+ id="bad GS2 header: missing channel binding terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,a")),
+ ExpectedError(
+ FEATURE_NOT_SUPPORTED_ERRCODE,
+ "client uses authorization identity",
+ ),
+ id="bad GS2 header: authzid in use",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,b,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected attribute",
+ ),
+ id="bad GS2 header: extra attribute",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ 'Unexpected attribute "0x00"', # XXX this is a bit strange
+ ),
+ id="bad GS2 header: missing authzid terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Key-value separator expected",
+ ),
+ id="missing initial kvsep",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Key-value separator expected",
+ ),
+ id="missing initial kvsep",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "does not contain an auth value",
+ ),
+ id="missing auth value: empty key-value list",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01host=example.com\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "does not contain an auth value",
+ ),
+ id="missing auth value: other keys present",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01host=example.com")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "unterminated key/value pair",
+ ),
+ id="missing value terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not contain a final terminator",
+ ),
+ id="missing list terminator: empty list",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01auth=Bearer 0\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not contain a final terminator",
+ ),
+ id="missing list terminator: with auth value",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01auth=Bearer 0\x01\x01blah")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "additional data after the final terminator",
+ ),
+ id="additional key after terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01key\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "key without a value",
+ ),
+ id="key without value",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"y,,\x01auth=Bearer 0\x01auth=Bearer 1\x01\x01",
+ )
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "contains multiple auth values",
+ ),
+ id="multiple auth values",
+ ),
+ ],
+)
+def test_oauth_bad_initial_response(conn, oauth_ctx, type, payload, err):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # The server expects a SASL response; give it something else instead.
+ if not isinstance(payload, dict):
+ payload = dict(payload_data=payload)
+ pq3.send(conn, type, **payload)
+
+ resp = pq3.recv1(conn)
+ err.match(resp)
+
+
+def test_oauth_empty_initial_response(conn, oauth_ctx, bearer_token):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send an initial response without data.
+ initial = pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER"))
+ pq3.send(conn, pq3.types.PasswordMessage, initial)
+
+ # The server should respond with an empty challenge so we can send the data
+ # it wants.
+ pkt = pq3.recv1(conn)
+
+ assert pkt.type == pq3.types.AuthnRequest
+ assert pkt.payload.type == pq3.authn.SASLContinue
+ assert not pkt.payload.body
+
+ # Now send the initial data.
+ data = b"n,,\x01auth=Bearer " + bearer_token() + b"\x01\x01"
+ pq3.send(conn, pq3.types.PasswordMessage, data)
+
+ # Server should now complete the handshake.
+ expect_handshake_success(conn)
+
+
+@pytest.fixture()
+def set_validator():
+ """
+ A per-test fixture that allows a test to override the setting of
+ oauth_validator_command for the cluster. The setting will be reverted during
+ teardown.
+
+ Passing None will perform an ALTER SYSTEM RESET.
+ """
+ conn = psycopg2.connect("")
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+
+ # Save the previous value.
+ c.execute("SHOW oauth_validator_command;")
+ prev_cmd = c.fetchone()[0]
+
+ def setter(cmd):
+ c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (cmd,))
+ c.execute("SELECT pg_reload_conf();")
+
+ yield setter
+
+ # Restore the previous value.
+ c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (prev_cmd,))
+ c.execute("SELECT pg_reload_conf();")
+
+
+def test_oauth_no_validator(oauth_ctx, set_validator, connect, bearer_token):
+ # Clear out our validator command, then establish a new connection.
+ set_validator("")
+ conn = connect()
+
+ begin_oauth_handshake(conn, oauth_ctx)
+ send_initial_response(conn, bearer=bearer_token())
+
+ # The server should fail the connection.
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+def test_oauth_validator_role(oauth_ctx, set_validator, connect):
+ # Switch the validator implementation. This validator will reflect the
+ # PGUSER as the authenticated identity.
+ path = pathlib.Path(__file__).parent / "validate_reflect.py"
+ path = str(path.absolute())
+
+ set_validator(f"{shlex.quote(path)} '%r' <&%f")
+ conn = connect()
+
+ # Log in. Note that the reflection validator ignores the bearer token.
+ begin_oauth_handshake(conn, oauth_ctx, user=oauth_ctx.user)
+ send_initial_response(conn, bearer=b"dontcare")
+ expect_handshake_success(conn)
+
+ # Check the user identity.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT authn_id();")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ expected = oauth_ctx.user.encode("utf-8")
+ assert row.columns == [expected]
+
+
+def test_oauth_role_with_shell_unsafe_characters(oauth_ctx, set_validator, connect):
+ """
+ XXX This test pins undesirable behavior. We should be able to handle any
+ valid Postgres role name.
+ """
+ # Switch the validator implementation. This validator will reflect the
+ # PGUSER as the authenticated identity.
+ path = pathlib.Path(__file__).parent / "validate_reflect.py"
+ path = str(path.absolute())
+
+ set_validator(f"{shlex.quote(path)} '%r' <&%f")
+ conn = connect()
+
+ unsafe_username = "hello'there"
+ begin_oauth_handshake(conn, oauth_ctx, user=unsafe_username)
+
+ # The server should reject the handshake.
+ send_initial_response(conn, bearer=b"dontcare")
+ expect_handshake_failure(conn, oauth_ctx)
diff --git a/src/test/python/server/test_server.py b/src/test/python/server/test_server.py
new file mode 100644
index 0000000000..02126dba79
--- /dev/null
+++ b/src/test/python/server/test_server.py
@@ -0,0 +1,21 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import pq3
+
+
+def test_handshake(connect):
+ """Basic sanity check."""
+ conn = connect()
+
+ pq3.handshake(conn, user=pq3.pguser(), database=pq3.pgdatabase())
+
+ pq3.send(conn, pq3.types.Query, query=b"")
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.EmptyQueryResponse
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.ReadyForQuery
diff --git a/src/test/python/server/validate_bearer.py b/src/test/python/server/validate_bearer.py
new file mode 100755
index 0000000000..2cc73ff154
--- /dev/null
+++ b/src/test/python/server/validate_bearer.py
@@ -0,0 +1,101 @@
+#! /usr/bin/env python3
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+# DO NOT USE THIS OAUTH VALIDATOR IN PRODUCTION. It doesn't actually validate
+# anything, and it logs the bearer token data, which is sensitive.
+#
+# This executable is used as an oauth_validator_command in concert with
+# test_oauth.py. Memory is shared and communicated from that test module's
+# bearer_token() fixture.
+#
+# This script must run under the Postgres server environment; keep the
+# dependency list fairly standard.
+
+import base64
+import binascii
+import contextlib
+import struct
+import sys
+from multiprocessing import shared_memory
+
+MAX_UINT16 = 2 ** 16 - 1
+
+
+def remove_shm_from_resource_tracker():
+ """
+ Monkey-patch multiprocessing.resource_tracker so SharedMemory won't be
+ tracked. Pulled from this thread, where there are more details:
+
+ https://bugs.python.org/issue38119
+
+ TL;DR: all clients of shared memory segments automatically destroy them on
+ process exit, which makes shared memory segments much less useful. This
+ monkeypatch removes that behavior so that we can defer to the test to manage
+ the segment lifetime.
+
+ Ideally a future Python patch will pull in this fix and then the entire
+ function can go away.
+ """
+ from multiprocessing import resource_tracker
+
+ def fix_register(name, rtype):
+ if rtype == "shared_memory":
+ return
+ return resource_tracker._resource_tracker.register(self, name, rtype)
+
+ resource_tracker.register = fix_register
+
+ def fix_unregister(name, rtype):
+ if rtype == "shared_memory":
+ return
+ return resource_tracker._resource_tracker.unregister(self, name, rtype)
+
+ resource_tracker.unregister = fix_unregister
+
+ if "shared_memory" in resource_tracker._CLEANUP_FUNCS:
+ del resource_tracker._CLEANUP_FUNCS["shared_memory"]
+
+
+def main(args):
+ remove_shm_from_resource_tracker() # XXX remove some day
+
+ # Get the expected token from the currently running test.
+ shared_mem_name = args[0]
+
+ mem = shared_memory.SharedMemory(shared_mem_name)
+ with contextlib.closing(mem):
+ # First two bytes are the token length.
+ size = struct.unpack("H", mem.buf[:2])[0]
+
+ if size == MAX_UINT16:
+ # Special case: the test wants us to accept any token.
+ sys.stderr.write("accepting token without validation\n")
+ return
+
+ # The remainder of the buffer contains the expected token.
+ assert size <= (mem.size - 2)
+ expected_token = mem.buf[2 : size + 2].tobytes()
+
+ mem.buf[:] = b"\0" * mem.size # scribble over the token
+
+ token = sys.stdin.buffer.read()
+ if token != expected_token:
+ sys.exit(f"failed to match Bearer token ({token!r} != {expected_token!r})")
+
+ # See if the test wants us to print anything. If so, it will have encoded
+ # the desired output in the token with an "output=" prefix.
+ try:
+ # altchars="-_" corresponds to the urlsafe alphabet.
+ data = base64.b64decode(token, altchars="-_", validate=True)
+
+ if data.startswith(b"output="):
+ sys.stdout.buffer.write(data[7:])
+
+ except binascii.Error:
+ pass
+
+
+if __name__ == "__main__":
+ main(sys.argv[1:])
diff --git a/src/test/python/server/validate_reflect.py b/src/test/python/server/validate_reflect.py
new file mode 100755
index 0000000000..24c3a7e715
--- /dev/null
+++ b/src/test/python/server/validate_reflect.py
@@ -0,0 +1,34 @@
+#! /usr/bin/env python3
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+# DO NOT USE THIS OAUTH VALIDATOR IN PRODUCTION. It ignores the bearer token
+# entirely and automatically logs the user in.
+#
+# This executable is used as an oauth_validator_command in concert with
+# test_oauth.py. It expects the user's desired role name as an argument; the
+# actual token will be discarded and the user will be logged in with the role
+# name as the authenticated identity.
+#
+# This script must run under the Postgres server environment; keep the
+# dependency list fairly standard.
+
+import sys
+
+
+def main(args):
+ # We have to read the entire token as our first action to unblock the
+ # server, but we won't actually use it.
+ _ = sys.stdin.buffer.read()
+
+ if len(args) != 1:
+ sys.exit("usage: ./validate_reflect.py ROLE")
+
+ # Log the user in as the provided role.
+ role = args[0]
+ print(role)
+
+
+if __name__ == "__main__":
+ main(sys.argv[1:])
diff --git a/src/test/python/test_internals.py b/src/test/python/test_internals.py
new file mode 100644
index 0000000000..dee4855fc0
--- /dev/null
+++ b/src/test/python/test_internals.py
@@ -0,0 +1,138 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import io
+
+from pq3 import _DebugStream
+
+
+def test_DebugStream_read():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+
+ stream = _DebugStream(under, out)
+
+ res = stream.read(5)
+ assert res == b"abcde"
+
+ res = stream.read(16)
+ assert res == b"fghijklmnopqrstu"
+
+ stream.flush_debug()
+
+ res = stream.read()
+ assert res == b"vwxyz"
+
+ stream.flush_debug()
+
+ expected = (
+ "< 0000:\t61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 70\tabcdefghijklmnop\n"
+ "< 0010:\t71 72 73 74 75 \tqrstu\n"
+ "\n"
+ "< 0000:\t76 77 78 79 7a \tvwxyz\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_write():
+ under = io.BytesIO()
+ out = io.StringIO()
+
+ stream = _DebugStream(under, out)
+
+ stream.write(b"\x00\x01\x02")
+ stream.flush()
+
+ assert under.getvalue() == b"\x00\x01\x02"
+
+ stream.write(b"\xc0\xc1\xc2")
+ stream.flush()
+
+ assert under.getvalue() == b"\x00\x01\x02\xc0\xc1\xc2"
+
+ stream.flush_debug()
+
+ expected = "> 0000:\t00 01 02 c0 c1 c2 \t......\n\n"
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_read_write():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+ stream = _DebugStream(under, out)
+
+ res = stream.read(5)
+ assert res == b"abcde"
+
+ stream.write(b"xxxxx")
+ stream.flush()
+
+ assert under.getvalue() == b"abcdexxxxxklmnopqrstuvwxyz"
+
+ res = stream.read(5)
+ assert res == b"klmno"
+
+ stream.write(b"xxxxx")
+ stream.flush()
+
+ assert under.getvalue() == b"abcdexxxxxklmnoxxxxxuvwxyz"
+
+ stream.flush_debug()
+
+ expected = (
+ "< 0000:\t61 62 63 64 65 6b 6c 6d 6e 6f \tabcdeklmno\n"
+ "\n"
+ "> 0000:\t78 78 78 78 78 78 78 78 78 78 \txxxxxxxxxx\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_end_packet():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+ stream = _DebugStream(under, out)
+
+ stream.read(5)
+ stream.end_packet("read description", read=True, indent=" ")
+
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("write description", indent=" ")
+
+ stream.read(5)
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("read/write combo for read", read=True, indent=" ")
+
+ stream.read(5)
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("read/write combo for write", indent=" ")
+
+ expected = (
+ " < 0000:\t61 62 63 64 65 \tabcde\n"
+ "\n"
+ "< read description\n"
+ "\n"
+ "> write description\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ " < 0000:\t6b 6c 6d 6e 6f \tklmno\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ "< read/write combo for read\n"
+ "\n"
+ "> read/write combo for write\n"
+ "\n"
+ " < 0000:\t75 76 77 78 79 \tuvwxy\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
diff --git a/src/test/python/test_pq3.py b/src/test/python/test_pq3.py
new file mode 100644
index 0000000000..e0c0e0568d
--- /dev/null
+++ b/src/test/python/test_pq3.py
@@ -0,0 +1,558 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import getpass
+import io
+import struct
+import sys
+
+import pytest
+from construct import Container, PaddingError, StreamError, TerminatedError
+
+import pq3
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"\x00\x00\x00\x10\x00\x04\x00\x00abcdefgh",
+ Container(len=16, proto=0x40000, payload=b"abcdefgh"),
+ b"",
+ id="8-byte payload",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x08\x00\x04\x00\x00",
+ Container(len=8, proto=0x40000, payload=b""),
+ b"",
+ id="no payload",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x09\x00\x04\x00\x00abcde",
+ Container(len=9, proto=0x40000, payload=b"a"),
+ b"bcde",
+ id="1-byte payload and extra padding",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x0B\x00\x03\x00\x00hi\x00",
+ Container(len=11, proto=pq3.protocol(3, 0), payload=[b"hi"]),
+ b"",
+ id="implied parameter list when using proto version 3.0",
+ ),
+ ],
+)
+def test_Startup_parse(raw, expected, extra):
+ with io.BytesIO(raw) as stream:
+ actual = pq3.Startup.parse_stream(stream)
+
+ assert actual == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "packet,expected_bytes",
+ [
+ pytest.param(
+ dict(),
+ b"\x00\x00\x00\x08\x00\x00\x00\x00",
+ id="nothing set",
+ ),
+ pytest.param(
+ dict(len=10, proto=0x12345678),
+ b"\x00\x00\x00\x0A\x12\x34\x56\x78\x00\x00",
+ id="len and proto set explicitly",
+ ),
+ pytest.param(
+ dict(proto=0x12345678),
+ b"\x00\x00\x00\x08\x12\x34\x56\x78",
+ id="implied len with no payload",
+ ),
+ pytest.param(
+ dict(proto=0x12345678, payload=b"abcd"),
+ b"\x00\x00\x00\x0C\x12\x34\x56\x78abcd",
+ id="implied len with payload",
+ ),
+ pytest.param(
+ dict(payload=[b""]),
+ b"\x00\x00\x00\x09\x00\x03\x00\x00\x00",
+ id="implied proto version 3 when sending parameters",
+ ),
+ pytest.param(
+ dict(payload=[b"hi", b""]),
+ b"\x00\x00\x00\x0C\x00\x03\x00\x00hi\x00\x00",
+ id="implied proto version 3 and len when sending more than one parameter",
+ ),
+ pytest.param(
+ dict(payload=dict(user="jsmith", database="postgres")),
+ b"\x00\x00\x00\x27\x00\x03\x00\x00user\x00jsmith\x00database\x00postgres\x00\x00",
+ id="auto-serialization of dict parameters",
+ ),
+ ],
+)
+def test_Startup_build(packet, expected_bytes):
+ actual = pq3.Startup.build(packet)
+ assert actual == expected_bytes
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"*\x00\x00\x00\x08abcd",
+ dict(type=b"*", len=8, payload=b"abcd"),
+ b"",
+ id="4-byte payload",
+ ),
+ pytest.param(
+ b"*\x00\x00\x00\x04",
+ dict(type=b"*", len=4, payload=b""),
+ b"",
+ id="no payload",
+ ),
+ pytest.param(
+ b"*\x00\x00\x00\x05xabcd",
+ dict(type=b"*", len=5, payload=b"x"),
+ b"abcd",
+ id="1-byte payload with extra padding",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x08\x00\x00\x00\x00",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=8,
+ payload=dict(type=pq3.authn.OK, body=None),
+ ),
+ b"",
+ id="AuthenticationOk",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x12\x00\x00\x00\x0AEXTERNAL\x00\x00",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=18,
+ payload=dict(type=pq3.authn.SASL, body=[b"EXTERNAL", b""]),
+ ),
+ b"",
+ id="AuthenticationSASL",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0B12345",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=13,
+ payload=dict(type=pq3.authn.SASLContinue, body=b"12345"),
+ ),
+ b"",
+ id="AuthenticationSASLContinue",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0C12345",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=13,
+ payload=dict(type=pq3.authn.SASLFinal, body=b"12345"),
+ ),
+ b"",
+ id="AuthenticationSASLFinal",
+ ),
+ pytest.param(
+ b"p\x00\x00\x00\x0Bhunter2",
+ dict(
+ type=pq3.types.PasswordMessage,
+ len=11,
+ payload=b"hunter2",
+ ),
+ b"",
+ id="PasswordMessage",
+ ),
+ pytest.param(
+ b"K\x00\x00\x00\x0C\x00\x00\x00\x00\x12\x34\x56\x78",
+ dict(
+ type=pq3.types.BackendKeyData,
+ len=12,
+ payload=dict(pid=0, key=0x12345678),
+ ),
+ b"",
+ id="BackendKeyData",
+ ),
+ pytest.param(
+ b"C\x00\x00\x00\x08SET\x00",
+ dict(
+ type=pq3.types.CommandComplete,
+ len=8,
+ payload=dict(tag=b"SET"),
+ ),
+ b"",
+ id="CommandComplete",
+ ),
+ pytest.param(
+ b"E\x00\x00\x00\x11Mbad!\x00Mdog!\x00\x00",
+ dict(type=b"E", len=17, payload=dict(fields=[b"Mbad!", b"Mdog!", b""])),
+ b"",
+ id="ErrorResponse",
+ ),
+ pytest.param(
+ b"S\x00\x00\x00\x08a\x00b\x00",
+ dict(
+ type=pq3.types.ParameterStatus,
+ len=8,
+ payload=dict(name=b"a", value=b"b"),
+ ),
+ b"",
+ id="ParameterStatus",
+ ),
+ pytest.param(
+ b"Z\x00\x00\x00\x05x",
+ dict(type=b"Z", len=5, payload=dict(status=b"x")),
+ b"",
+ id="ReadyForQuery",
+ ),
+ pytest.param(
+ b"Q\x00\x00\x00\x06!\x00",
+ dict(type=pq3.types.Query, len=6, payload=dict(query=b"!")),
+ b"",
+ id="Query",
+ ),
+ pytest.param(
+ b"D\x00\x00\x00\x0B\x00\x01\x00\x00\x00\x01!",
+ dict(type=pq3.types.DataRow, len=11, payload=dict(columns=[b"!"])),
+ b"",
+ id="DataRow",
+ ),
+ pytest.param(
+ b"D\x00\x00\x00\x06\x00\x00extra",
+ dict(type=pq3.types.DataRow, len=6, payload=dict(columns=[])),
+ b"extra",
+ id="DataRow with extra data",
+ ),
+ pytest.param(
+ b"I\x00\x00\x00\x04",
+ dict(type=pq3.types.EmptyQueryResponse, len=4, payload=None),
+ b"",
+ id="EmptyQueryResponse",
+ ),
+ pytest.param(
+ b"I\x00\x00\x00\x04\xFF",
+ dict(type=b"I", len=4, payload=None),
+ b"\xFF",
+ id="EmptyQueryResponse with extra bytes",
+ ),
+ pytest.param(
+ b"X\x00\x00\x00\x04",
+ dict(type=pq3.types.Terminate, len=4, payload=None),
+ b"",
+ id="Terminate",
+ ),
+ ],
+)
+def test_Pq3_parse(raw, expected, extra):
+ with io.BytesIO(raw) as stream:
+ actual = pq3.Pq3.parse_stream(stream)
+
+ assert actual == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(type=b"*", len=5),
+ b"*\x00\x00\x00\x05\x00",
+ id="type and len set explicitly",
+ ),
+ pytest.param(
+ dict(type=b"*"),
+ b"*\x00\x00\x00\x04",
+ id="implied len with no payload",
+ ),
+ pytest.param(
+ dict(type=b"*", payload=b"1234"),
+ b"*\x00\x00\x00\x081234",
+ id="implied len with payload",
+ ),
+ pytest.param(
+ dict(type=pq3.types.AuthnRequest, payload=dict(type=pq3.authn.OK)),
+ b"R\x00\x00\x00\x08\x00\x00\x00\x00",
+ id="implied len/type for AuthenticationOK",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(
+ type=pq3.authn.SASL,
+ body=[b"SCRAM-SHA-256-PLUS", b"SCRAM-SHA-256", b""],
+ ),
+ ),
+ b"R\x00\x00\x00\x2A\x00\x00\x00\x0ASCRAM-SHA-256-PLUS\x00SCRAM-SHA-256\x00\x00",
+ id="implied len/type for AuthenticationSASL",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(type=pq3.authn.SASLContinue, body=b"12345"),
+ ),
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0B12345",
+ id="implied len/type for AuthenticationSASLContinue",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(type=pq3.authn.SASLFinal, body=b"12345"),
+ ),
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0C12345",
+ id="implied len/type for AuthenticationSASLFinal",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.PasswordMessage,
+ payload=b"hunter2",
+ ),
+ b"p\x00\x00\x00\x0Bhunter2",
+ id="implied len/type for PasswordMessage",
+ ),
+ pytest.param(
+ dict(type=pq3.types.BackendKeyData, payload=dict(pid=1, key=7)),
+ b"K\x00\x00\x00\x0C\x00\x00\x00\x01\x00\x00\x00\x07",
+ id="implied len/type for BackendKeyData",
+ ),
+ pytest.param(
+ dict(type=pq3.types.CommandComplete, payload=dict(tag=b"SET")),
+ b"C\x00\x00\x00\x08SET\x00",
+ id="implied len/type for CommandComplete",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ErrorResponse, payload=dict(fields=[b"error", b""])),
+ b"E\x00\x00\x00\x0Berror\x00\x00",
+ id="implied len/type for ErrorResponse",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ParameterStatus, payload=dict(name=b"a", value=b"b")),
+ b"S\x00\x00\x00\x08a\x00b\x00",
+ id="implied len/type for ParameterStatus",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ReadyForQuery, payload=dict(status=b"I")),
+ b"Z\x00\x00\x00\x05I",
+ id="implied len/type for ReadyForQuery",
+ ),
+ pytest.param(
+ dict(type=pq3.types.Query, payload=dict(query=b"SELECT 1;")),
+ b"Q\x00\x00\x00\x0eSELECT 1;\x00",
+ id="implied len/type for Query",
+ ),
+ pytest.param(
+ dict(type=pq3.types.DataRow, payload=dict(columns=[b"abcd"])),
+ b"D\x00\x00\x00\x0E\x00\x01\x00\x00\x00\x04abcd",
+ id="implied len/type for DataRow",
+ ),
+ pytest.param(
+ dict(type=pq3.types.EmptyQueryResponse),
+ b"I\x00\x00\x00\x04",
+ id="implied len for EmptyQueryResponse",
+ ),
+ pytest.param(
+ dict(type=pq3.types.Terminate),
+ b"X\x00\x00\x00\x04",
+ id="implied len for Terminate",
+ ),
+ ],
+)
+def test_Pq3_build(fields, expected):
+ actual = pq3.Pq3.build(fields)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"\x00\x00",
+ dict(columns=[]),
+ b"",
+ id="no columns",
+ ),
+ pytest.param(
+ b"\x00\x01\x00\x00\x00\x04abcd",
+ dict(columns=[b"abcd"]),
+ b"",
+ id="one column",
+ ),
+ pytest.param(
+ b"\x00\x02\x00\x00\x00\x04abcd\x00\x00\x00\x01x",
+ dict(columns=[b"abcd", b"x"]),
+ b"",
+ id="multiple columns",
+ ),
+ pytest.param(
+ b"\x00\x02\x00\x00\x00\x00\x00\x00\x00\x01x",
+ dict(columns=[b"", b"x"]),
+ b"",
+ id="empty column value",
+ ),
+ pytest.param(
+ b"\x00\x02\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF",
+ dict(columns=[None, None]),
+ b"",
+ id="null columns",
+ ),
+ ],
+)
+def test_DataRow_parse(raw, expected, extra):
+ pkt = b"D" + struct.pack("!i", len(raw) + 4) + raw
+ with io.BytesIO(pkt) as stream:
+ actual = pq3.Pq3.parse_stream(stream)
+
+ assert actual.type == pq3.types.DataRow
+ assert actual.payload == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(),
+ b"\x00\x00",
+ id="no columns",
+ ),
+ pytest.param(
+ dict(columns=[None, None]),
+ b"\x00\x02\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF",
+ id="null columns",
+ ),
+ ],
+)
+def test_DataRow_build(fields, expected):
+ actual = pq3.Pq3.build(dict(type=pq3.types.DataRow, payload=fields))
+
+ expected = b"D" + struct.pack("!i", len(expected) + 4) + expected
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "raw,expected,exception",
+ [
+ pytest.param(
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ dict(name=b"EXTERNAL", len=-1, data=None),
+ None,
+ id="no initial response",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\x02me",
+ dict(name=b"EXTERNAL", len=2, data=b"me"),
+ None,
+ id="initial response",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\x02meextra",
+ None,
+ TerminatedError,
+ id="extra data",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\xFFme",
+ None,
+ StreamError,
+ id="underflow",
+ ),
+ ],
+)
+def test_SASLInitialResponse_parse(raw, expected, exception):
+ ctx = contextlib.nullcontext()
+ if exception:
+ ctx = pytest.raises(exception)
+
+ with ctx:
+ actual = pq3.SASLInitialResponse.parse(raw)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(name=b"EXTERNAL"),
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ id="no initial response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=None),
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ id="no initial response (explicit None)",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=b""),
+ b"EXTERNAL\x00\x00\x00\x00\x00",
+ id="empty response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=b"me@example.com"),
+ b"EXTERNAL\x00\x00\x00\x00\x0Eme@example.com",
+ id="initial response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", len=2, data=b"me@example.com"),
+ b"EXTERNAL\x00\x00\x00\x00\x02me@example.com",
+ id="data overflow",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", len=14, data=b"me"),
+ b"EXTERNAL\x00\x00\x00\x00\x0Eme",
+ id="data underflow",
+ ),
+ ],
+)
+def test_SASLInitialResponse_build(fields, expected):
+ actual = pq3.SASLInitialResponse.build(fields)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "version,expected_bytes",
+ [
+ pytest.param((3, 0), b"\x00\x03\x00\x00", id="version 3"),
+ pytest.param((1234, 5679), b"\x04\xd2\x16\x2f", id="SSLRequest"),
+ ],
+)
+def test_protocol(version, expected_bytes):
+ # Make sure the integer returned by protocol is correctly serialized on the
+ # wire.
+ assert struct.pack("!i", pq3.protocol(*version)) == expected_bytes
+
+
+@pytest.mark.parametrize(
+ "envvar,func,expected",
+ [
+ ("PGHOST", pq3.pghost, "localhost"),
+ ("PGPORT", pq3.pgport, 5432),
+ ("PGUSER", pq3.pguser, getpass.getuser()),
+ ("PGDATABASE", pq3.pgdatabase, "postgres"),
+ ],
+)
+def test_env_defaults(monkeypatch, envvar, func, expected):
+ monkeypatch.delenv(envvar, raising=False)
+
+ actual = func()
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "envvars,func,expected",
+ [
+ (dict(PGHOST="otherhost"), pq3.pghost, "otherhost"),
+ (dict(PGPORT="6789"), pq3.pgport, 6789),
+ (dict(PGUSER="postgres"), pq3.pguser, "postgres"),
+ (dict(PGDATABASE="template1"), pq3.pgdatabase, "template1"),
+ ],
+)
+def test_env(monkeypatch, envvars, func, expected):
+ for k, v in envvars.items():
+ monkeypatch.setenv(k, v)
+
+ actual = func()
+ assert actual == expected
diff --git a/src/test/python/tls.py b/src/test/python/tls.py
new file mode 100644
index 0000000000..075c02c1ca
--- /dev/null
+++ b/src/test/python/tls.py
@@ -0,0 +1,195 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+from construct import *
+
+#
+# TLS 1.3
+#
+# Most of the types below are transcribed from RFC 8446:
+#
+# https://tools.ietf.org/html/rfc8446
+#
+
+
+def _Vector(size_field, element):
+ return Prefixed(size_field, GreedyRange(element))
+
+
+# Alerts
+
+AlertLevel = Enum(
+ Byte,
+ warning=1,
+ fatal=2,
+)
+
+AlertDescription = Enum(
+ Byte,
+ close_notify=0,
+ unexpected_message=10,
+ bad_record_mac=20,
+ decryption_failed_RESERVED=21,
+ record_overflow=22,
+ decompression_failure=30,
+ handshake_failure=40,
+ no_certificate_RESERVED=41,
+ bad_certificate=42,
+ unsupported_certificate=43,
+ certificate_revoked=44,
+ certificate_expired=45,
+ certificate_unknown=46,
+ illegal_parameter=47,
+ unknown_ca=48,
+ access_denied=49,
+ decode_error=50,
+ decrypt_error=51,
+ export_restriction_RESERVED=60,
+ protocol_version=70,
+ insufficient_security=71,
+ internal_error=80,
+ user_canceled=90,
+ no_renegotiation=100,
+ unsupported_extension=110,
+)
+
+Alert = Struct(
+ "level" / AlertLevel,
+ "description" / AlertDescription,
+)
+
+
+# Extensions
+
+ExtensionType = Enum(
+ Int16ub,
+ server_name=0,
+ max_fragment_length=1,
+ status_request=5,
+ supported_groups=10,
+ signature_algorithms=13,
+ use_srtp=14,
+ heartbeat=15,
+ application_layer_protocol_negotiation=16,
+ signed_certificate_timestamp=18,
+ client_certificate_type=19,
+ server_certificate_type=20,
+ padding=21,
+ pre_shared_key=41,
+ early_data=42,
+ supported_versions=43,
+ cookie=44,
+ psk_key_exchange_modes=45,
+ certificate_authorities=47,
+ oid_filters=48,
+ post_handshake_auth=49,
+ signature_algorithms_cert=50,
+ key_share=51,
+)
+
+Extension = Struct(
+ "extension_type" / ExtensionType,
+ "extension_data" / Prefixed(Int16ub, GreedyBytes),
+)
+
+
+# ClientHello
+
+
+class _CipherSuiteAdapter(Adapter):
+ class _hextuple(tuple):
+ def __repr__(self):
+ return f"(0x{self[0]:02X}, 0x{self[1]:02X})"
+
+ def _encode(self, obj, context, path):
+ return bytes(obj)
+
+ def _decode(self, obj, context, path):
+ assert len(obj) == 2
+ return self._hextuple(obj)
+
+
+ProtocolVersion = Hex(Int16ub)
+
+Random = Hex(Bytes(32))
+
+CipherSuite = _CipherSuiteAdapter(Byte[2])
+
+ClientHello = Struct(
+ "legacy_version" / ProtocolVersion,
+ "random" / Random,
+ "legacy_session_id" / Prefixed(Byte, Hex(GreedyBytes)),
+ "cipher_suites" / _Vector(Int16ub, CipherSuite),
+ "legacy_compression_methods" / Prefixed(Byte, GreedyBytes),
+ "extensions" / _Vector(Int16ub, Extension),
+)
+
+# ServerHello
+
+ServerHello = Struct(
+ "legacy_version" / ProtocolVersion,
+ "random" / Random,
+ "legacy_session_id_echo" / Prefixed(Byte, Hex(GreedyBytes)),
+ "cipher_suite" / CipherSuite,
+ "legacy_compression_method" / Hex(Byte),
+ "extensions" / _Vector(Int16ub, Extension),
+)
+
+# Handshake
+
+HandshakeType = Enum(
+ Byte,
+ client_hello=1,
+ server_hello=2,
+ new_session_ticket=4,
+ end_of_early_data=5,
+ encrypted_extensions=8,
+ certificate=11,
+ certificate_request=13,
+ certificate_verify=15,
+ finished=20,
+ key_update=24,
+ message_hash=254,
+)
+
+Handshake = Struct(
+ "msg_type" / HandshakeType,
+ "length" / Int24ub,
+ "payload"
+ / Switch(
+ this.msg_type,
+ {
+ HandshakeType.client_hello: ClientHello,
+ HandshakeType.server_hello: ServerHello,
+ # HandshakeType.end_of_early_data: EndOfEarlyData,
+ # HandshakeType.encrypted_extensions: EncryptedExtensions,
+ # HandshakeType.certificate_request: CertificateRequest,
+ # HandshakeType.certificate: Certificate,
+ # HandshakeType.certificate_verify: CertificateVerify,
+ # HandshakeType.finished: Finished,
+ # HandshakeType.new_session_ticket: NewSessionTicket,
+ # HandshakeType.key_update: KeyUpdate,
+ },
+ default=FixedSized(this.length, GreedyBytes),
+ ),
+)
+
+# Records
+
+ContentType = Enum(
+ Byte,
+ invalid=0,
+ change_cipher_spec=20,
+ alert=21,
+ handshake=22,
+ application_data=23,
+)
+
+Plaintext = Struct(
+ "type" / ContentType,
+ "legacy_record_version" / ProtocolVersion,
+ "length" / Int16ub,
+ "fragment" / FixedSized(this.length, GreedyBytes),
+)
--
2.25.1
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2021-06-18 04:07 Michael Paquier <michael@paquier.xyz>
parent: Jacob Champion <pchampion@vmware.com>
1 sibling, 1 reply; 424+ messages in thread
From: Michael Paquier @ 2021-06-18 04:07 UTC (permalink / raw)
To: Jacob Champion <pchampion@vmware.com>; +Cc: pgsql-hackers
On Tue, Jun 08, 2021 at 04:37:46PM +0000, Jacob Champion wrote:
> 1. Prep
>
> 0001 decouples the SASL code from the SCRAM implementation.
> 0002 makes it possible to use common/jsonapi from the frontend.
> 0003 lets the json_errdetail() result be freed, to avoid leaks.
>
> The first three patches are, hopefully, generally useful outside of
> this implementation, and I'll plan to register them in the next
> commitfest. The middle two patches are the "interesting" pieces, and
> I've split them into client and server for ease of understanding,
> though neither is particularly useful without the other.
Beginning with the beginning, could you spawn two threads for the
jsonapi rework and the SASL/SCRAM business? I agree that these look
independently useful. Glad to see someone improving the code with
SASL and SCRAM which are too inter-dependent now. I saw in the RFCs
dedicated to OAUTH the need for the JSON part as well.
+# define check_stack_depth()
+# ifdef JSONAPI_NO_LOG
+# define json_log_and_abort(...) \
+ do { fprintf(stderr, __VA_ARGS__); exit(1); } while(0)
+# else
In patch 0002, this is the wrong approach. libpq will not be able to
feed on such reports, and you cannot use any of the APIs from the
palloc() family either as these just fail on OOM. libpq should be
able to know about the error, and would fill in the error back to the
application. This abstraction is not necessary on HEAD as
pg_verifybackup is fine with this level of reporting. My rough guess
is that we will need to split the existing jsonapi.c into two files,
one that can be used in shared libraries and a second that handles the
errors.
+ /* TODO: SASL_EXCHANGE_FAILURE with output is forbidden in SASL */
if (result == SASL_EXCHANGE_SUCCESS)
sendAuthRequest(port,
AUTH_REQ_SASL_FIN,
output,
outputlen);
Perhaps that's an issue we need to worry on its own? I didn't recall
this part..
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2021-06-18 08:31 Heikki Linnakangas <hlinnaka@iki.fi>
parent: Jacob Champion <pchampion@vmware.com>
1 sibling, 1 reply; 424+ messages in thread
From: Heikki Linnakangas @ 2021-06-18 08:31 UTC (permalink / raw)
To: Jacob Champion <pchampion@vmware.com>; pgsql-hackers
On 08/06/2021 19:37, Jacob Champion wrote:
> We've been working on ways to expand the list of third-party auth
> methods that Postgres provides. Some example use cases might be "I want
> to let anyone with a Google account read this table" or "let anyone who
> belongs to this GitHub organization connect as a superuser".
Cool!
> The iddawc dependency for client-side OAuth was extremely helpful to
> develop this proof of concept quickly, but I don't think it would be an
> appropriate component to build a real feature on. It's extremely
> heavyweight -- it incorporates a huge stack of dependencies, including
> a logging framework and a web server, to implement features we would
> probably never use -- and it's fairly difficult to debug in practice.
> If a device authorization flow were the only thing that libpq needed to
> support natively, I think we should just depend on a widely used HTTP
> client, like libcurl or neon, and implement the minimum spec directly
> against the existing test suite.
You could punt and let the application implement that stuff. I'm
imagining that the application code would look something like this:
conn = PQconnectStartParams(...);
for (;;)
{
status = PQconnectPoll(conn)
switch (status)
{
case CONNECTION_SASL_TOKEN_REQUIRED:
/* open a browser for the user, get token */
token = open_browser()
PQauthResponse(token);
break;
...
}
}
It would be nice to have a simple default implementation, though, for
psql and all the other client applications that come with PostgreSQL itself.
> If you've read this far, thank you for your interest, and I hope you
> enjoy playing with it!
A few small things caught my eye in the backend oauth_exchange function:
> + /* Handle the client's initial message. */
> + p = strdup(input);
this strdup() should be pstrdup().
In the same function, there are a bunch of reports like this:
> ereport(ERROR,
> + (errcode(ERRCODE_PROTOCOL_VIOLATION),
> + errmsg("malformed OAUTHBEARER message"),
> + errdetail("Comma expected, but found character \"%s\".",
> + sanitize_char(*p))));
I don't think the double quotes are needed here, because sanitize_char
will return quotes if it's a single character. So it would end up
looking like this: ... found character "'x'".
- Heikki
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2021-06-22 23:22 Jacob Champion <pchampion@vmware.com>
parent: Heikki Linnakangas <hlinnaka@iki.fi>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2021-06-22 23:22 UTC (permalink / raw)
To: hlinnaka@iki.fi <hlinnaka@iki.fi>; pgsql-hackers
On Fri, 2021-06-18 at 11:31 +0300, Heikki Linnakangas wrote:
> On 08/06/2021 19:37, Jacob Champion wrote:
> > We've been working on ways to expand the list of third-party auth
> > methods that Postgres provides. Some example use cases might be "I want
> > to let anyone with a Google account read this table" or "let anyone who
> > belongs to this GitHub organization connect as a superuser".
>
> Cool!
Glad you think so! :D
> > The iddawc dependency for client-side OAuth was extremely helpful to
> > develop this proof of concept quickly, but I don't think it would be an
> > appropriate component to build a real feature on. It's extremely
> > heavyweight -- it incorporates a huge stack of dependencies, including
> > a logging framework and a web server, to implement features we would
> > probably never use -- and it's fairly difficult to debug in practice.
> > If a device authorization flow were the only thing that libpq needed to
> > support natively, I think we should just depend on a widely used HTTP
> > client, like libcurl or neon, and implement the minimum spec directly
> > against the existing test suite.
>
> You could punt and let the application implement that stuff. I'm
> imagining that the application code would look something like this:
>
> conn = PQconnectStartParams(...);
> for (;;)
> {
> status = PQconnectPoll(conn)
> switch (status)
> {
> case CONNECTION_SASL_TOKEN_REQUIRED:
> /* open a browser for the user, get token */
> token = open_browser()
> PQauthResponse(token);
> break;
> ...
> }
> }
I was toying with the idea of having a callback for libpq clients,
where they could take full control of the OAuth flow if they wanted to.
Doing it inline with PQconnectPoll seems like it would work too. It has
a couple of drawbacks that I can see:
- If a client isn't currently using a poll loop, they'd have to switch
to one to be able to use OAuth connections. Not a difficult change, but
considering all the other hurdles to making this work, I'm hoping to
minimize the hoop-jumping.
- A client would still have to receive a bunch of OAuth parameters from
some new libpq API in order to construct the correct URL to visit, so
the overall complexity for implementers might be higher than if we just
passed those params directly in a callback.
> It would be nice to have a simple default implementation, though, for
> psql and all the other client applications that come with PostgreSQL itself.
I agree. I think having a bare-bones implementation in libpq itself
would make initial adoption *much* easier, and then if specific
applications wanted to have richer control over an authorization flow,
then they could implement that themselves with the aforementioned
callback.
The Device Authorization flow was the most minimal working
implementation I could find, since it doesn't require a web browser on
the system, just the ability to print a prompt to the console. But if
anyone knows of a better flow for this use case, I'm all ears.
> > If you've read this far, thank you for your interest, and I hope you
> > enjoy playing with it!
>
> A few small things caught my eye in the backend oauth_exchange function:
>
> > + /* Handle the client's initial message. */
> > + p = strdup(input);
>
> this strdup() should be pstrdup().
Thanks, I'll fix that in the next re-roll.
> In the same function, there are a bunch of reports like this:
>
> > ereport(ERROR,
> > + (errcode(ERRCODE_PROTOCOL_VIOLATION),
> > + errmsg("malformed OAUTHBEARER message"),
> > + errdetail("Comma expected, but found character \"%s\".",
> > + sanitize_char(*p))));
>
> I don't think the double quotes are needed here, because sanitize_char
> will return quotes if it's a single character. So it would end up
> looking like this: ... found character "'x'".
I'll fix this too. Thanks!
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2021-06-22 23:26 Jacob Champion <pchampion@vmware.com>
parent: Michael Paquier <michael@paquier.xyz>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2021-06-22 23:26 UTC (permalink / raw)
To: michael@paquier.xyz <michael@paquier.xyz>; +Cc: pgsql-hackers
On Fri, 2021-06-18 at 13:07 +0900, Michael Paquier wrote:
> On Tue, Jun 08, 2021 at 04:37:46PM +0000, Jacob Champion wrote:
> > 1. Prep
> >
> > 0001 decouples the SASL code from the SCRAM implementation.
> > 0002 makes it possible to use common/jsonapi from the frontend.
> > 0003 lets the json_errdetail() result be freed, to avoid leaks.
> >
> > The first three patches are, hopefully, generally useful outside of
> > this implementation, and I'll plan to register them in the next
> > commitfest. The middle two patches are the "interesting" pieces, and
> > I've split them into client and server for ease of understanding,
> > though neither is particularly useful without the other.
>
> Beginning with the beginning, could you spawn two threads for the
> jsonapi rework and the SASL/SCRAM business?
Done [1, 2]. I've copied your comments into those threads with my
responses, and I'll have them registered in commitfest shortly.
Thanks!
--Jacob
[1] https://www.postgresql.org/message-id/3d2a6f5d50e741117d6baf83eb67ebf1a8a35a11.camel%40vmware.com
[2] https://www.postgresql.org/message-id/a250d475ba1c0cc0efb7dfec8e538fcc77cdcb8e.camel%40vmware.com
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2021-06-23 06:10 Michael Paquier <michael@paquier.xyz>
parent: Jacob Champion <pchampion@vmware.com>
0 siblings, 0 replies; 424+ messages in thread
From: Michael Paquier @ 2021-06-23 06:10 UTC (permalink / raw)
To: Jacob Champion <pchampion@vmware.com>; +Cc: pgsql-hackers
On Tue, Jun 22, 2021 at 11:26:03PM +0000, Jacob Champion wrote:
> Done [1, 2]. I've copied your comments into those threads with my
> responses, and I'll have them registered in commitfest shortly.
Thanks!
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2021-08-25 18:41 Jacob Champion <pchampion@vmware.com>
parent: Jacob Champion <pchampion@vmware.com>
0 siblings, 3 replies; 424+ messages in thread
From: Jacob Champion @ 2021-08-25 18:41 UTC (permalink / raw)
To: hlinnaka@iki.fi <hlinnaka@iki.fi>; pgsql-hackers; +Cc: michael@paquier.xyz <michael@paquier.xyz>
On Tue, 2021-06-22 at 23:22 +0000, Jacob Champion wrote:
> On Fri, 2021-06-18 at 11:31 +0300, Heikki Linnakangas wrote:
> >
> > A few small things caught my eye in the backend oauth_exchange function:
> >
> > > + /* Handle the client's initial message. */
> > > + p = strdup(input);
> >
> > this strdup() should be pstrdup().
>
> Thanks, I'll fix that in the next re-roll.
>
> > In the same function, there are a bunch of reports like this:
> >
> > > ereport(ERROR,
> > > + (errcode(ERRCODE_PROTOCOL_VIOLATION),
> > > + errmsg("malformed OAUTHBEARER message"),
> > > + errdetail("Comma expected, but found character \"%s\".",
> > > + sanitize_char(*p))));
> >
> > I don't think the double quotes are needed here, because sanitize_char
> > will return quotes if it's a single character. So it would end up
> > looking like this: ... found character "'x'".
>
> I'll fix this too. Thanks!
v2, attached, incorporates Heikki's suggested fixes and also rebases on
top of latest HEAD, which had the SASL refactoring changes committed
last month.
The biggest change from the last patchset is 0001, an attempt at
enabling jsonapi in the frontend without the use of palloc(), based on
suggestions by Michael and Tom from last commitfest. I've also made
some improvements to the pytest suite. No major changes to the OAuth
implementation yet.
--Jacob
Attachments: download all (5 files, .tar.gz)
v2-0001-common-jsonapi-support-FRONTEND-clients.patch (text/x-patch, 20.4K) download
From 8c4b82940efb7e0f0f33ac915d5f7969a36e3644 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Mon, 3 May 2021 15:38:26 -0700
Subject: [PATCH v2 1/5] common/jsonapi: support FRONTEND clients
Based on a patch by Michael Paquier.
For frontend code, use PQExpBuffer instead of StringInfo. This requires
us to track allocation failures so that we can return JSON_OUT_OF_MEMORY
as needed. json_errdetail() now allocates its error message inside
memory owned by the JsonLexContext, so clients don't need to worry about
freeing it.
For convenience, the backend now has destroyJsonLexContext() to mirror
other create/destroy APIs. The frontend has init/term versions of the
API to handle stack-allocated JsonLexContexts.
We can now partially revert b44669b2ca, now that json_errdetail() works
correctly.
---
src/backend/utils/adt/jsonfuncs.c | 4 +-
src/bin/pg_verifybackup/parse_manifest.c | 13 +-
src/bin/pg_verifybackup/t/005_bad_manifest.pl | 2 +-
src/common/Makefile | 2 +-
src/common/jsonapi.c | 290 +++++++++++++-----
src/include/common/jsonapi.h | 47 ++-
6 files changed, 270 insertions(+), 88 deletions(-)
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index 5fd54b64b5..fa39751188 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -723,9 +723,7 @@ json_object_keys(PG_FUNCTION_ARGS)
pg_parse_json_or_ereport(lex, sem);
/* keys are now in state->result */
- pfree(lex->strval->data);
- pfree(lex->strval);
- pfree(lex);
+ destroyJsonLexContext(lex);
pfree(sem);
MemoryContextSwitchTo(oldcontext);
diff --git a/src/bin/pg_verifybackup/parse_manifest.c b/src/bin/pg_verifybackup/parse_manifest.c
index c7ccc78c70..6cedb7435f 100644
--- a/src/bin/pg_verifybackup/parse_manifest.c
+++ b/src/bin/pg_verifybackup/parse_manifest.c
@@ -119,7 +119,7 @@ void
json_parse_manifest(JsonManifestParseContext *context, char *buffer,
size_t size)
{
- JsonLexContext *lex;
+ JsonLexContext lex = {0};
JsonParseErrorType json_error;
JsonSemAction sem;
JsonManifestParseState parse;
@@ -129,8 +129,8 @@ json_parse_manifest(JsonManifestParseContext *context, char *buffer,
parse.state = JM_EXPECT_TOPLEVEL_START;
parse.saw_version_field = false;
- /* Create a JSON lexing context. */
- lex = makeJsonLexContextCstringLen(buffer, size, PG_UTF8, true);
+ /* Initialize a JSON lexing context. */
+ initJsonLexContextCstringLen(&lex, buffer, size, PG_UTF8, true);
/* Set up semantic actions. */
sem.semstate = &parse;
@@ -145,14 +145,17 @@ json_parse_manifest(JsonManifestParseContext *context, char *buffer,
sem.scalar = json_manifest_scalar;
/* Run the actual JSON parser. */
- json_error = pg_parse_json(lex, &sem);
+ json_error = pg_parse_json(&lex, &sem);
if (json_error != JSON_SUCCESS)
- json_manifest_parse_failure(context, "parsing failed");
+ json_manifest_parse_failure(context, json_errdetail(json_error, &lex));
if (parse.state != JM_EXPECT_EOF)
json_manifest_parse_failure(context, "manifest ended unexpectedly");
/* Verify the manifest checksum. */
verify_manifest_checksum(&parse, buffer, size);
+
+ /* Clean up. */
+ termJsonLexContext(&lex);
}
/*
diff --git a/src/bin/pg_verifybackup/t/005_bad_manifest.pl b/src/bin/pg_verifybackup/t/005_bad_manifest.pl
index 4f5b8f5a49..9f8a100a71 100644
--- a/src/bin/pg_verifybackup/t/005_bad_manifest.pl
+++ b/src/bin/pg_verifybackup/t/005_bad_manifest.pl
@@ -16,7 +16,7 @@ my $tempdir = TestLib::tempdir;
test_bad_manifest(
'input string ended unexpectedly',
- qr/could not parse backup manifest: parsing failed/,
+ qr/could not parse backup manifest: The input string ended unexpectedly/,
<<EOM);
{
EOM
diff --git a/src/common/Makefile b/src/common/Makefile
index 880722fcf5..5ecb09a8c4 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -40,7 +40,7 @@ override CPPFLAGS += -DVAL_LDFLAGS_EX="\"$(LDFLAGS_EX)\""
override CPPFLAGS += -DVAL_LDFLAGS_SL="\"$(LDFLAGS_SL)\""
override CPPFLAGS += -DVAL_LIBS="\"$(LIBS)\""
-override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common $(CPPFLAGS)
+override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common -I$(libpq_srcdir) $(CPPFLAGS)
LIBS += $(PTHREAD_LIBS)
# If you add objects here, see also src/tools/msvc/Mkvcbuild.pm
diff --git a/src/common/jsonapi.c b/src/common/jsonapi.c
index 5504072b4f..3a9620f739 100644
--- a/src/common/jsonapi.c
+++ b/src/common/jsonapi.c
@@ -20,10 +20,39 @@
#include "common/jsonapi.h"
#include "mb/pg_wchar.h"
-#ifndef FRONTEND
+#ifdef FRONTEND
+#include "pqexpbuffer.h"
+#else
+#include "lib/stringinfo.h"
#include "miscadmin.h"
#endif
+/*
+ * In backend, we will use palloc/pfree along with StringInfo. In frontend, use
+ * malloc and PQExpBuffer, and return JSON_OUT_OF_MEMORY on out-of-memory.
+ */
+#ifdef FRONTEND
+
+#define STRDUP(s) strdup(s)
+#define ALLOC(size) malloc(size)
+
+#define appendStrVal appendPQExpBuffer
+#define appendStrValChar appendPQExpBufferChar
+#define createStrVal createPQExpBuffer
+#define resetStrVal resetPQExpBuffer
+
+#else /* !FRONTEND */
+
+#define STRDUP(s) pstrdup(s)
+#define ALLOC(size) palloc(size)
+
+#define appendStrVal appendStringInfo
+#define appendStrValChar appendStringInfoChar
+#define createStrVal makeStringInfo
+#define resetStrVal resetStringInfo
+
+#endif
+
/*
* The context of the parser is maintained by the recursive descent
* mechanism, but is passed explicitly to the error reporting routine
@@ -132,10 +161,12 @@ IsValidJsonNumber(const char *str, int len)
return (!numeric_error) && (total_len == dummy_lex.input_length);
}
+#ifndef FRONTEND
+
/*
* makeJsonLexContextCstringLen
*
- * lex constructor, with or without StringInfo object for de-escaped lexemes.
+ * lex constructor, with or without a string object for de-escaped lexemes.
*
* Without is better as it makes the processing faster, so only make one
* if really required.
@@ -145,13 +176,66 @@ makeJsonLexContextCstringLen(char *json, int len, int encoding, bool need_escape
{
JsonLexContext *lex = palloc0(sizeof(JsonLexContext));
+ initJsonLexContextCstringLen(lex, json, len, encoding, need_escapes);
+
+ return lex;
+}
+
+void
+destroyJsonLexContext(JsonLexContext *lex)
+{
+ termJsonLexContext(lex);
+ pfree(lex);
+}
+
+#endif /* !FRONTEND */
+
+void
+initJsonLexContextCstringLen(JsonLexContext *lex, char *json, int len, int encoding, bool need_escapes)
+{
lex->input = lex->token_terminator = lex->line_start = json;
lex->line_number = 1;
lex->input_length = len;
lex->input_encoding = encoding;
- if (need_escapes)
- lex->strval = makeStringInfo();
- return lex;
+ lex->parse_strval = need_escapes;
+ if (lex->parse_strval)
+ {
+ /*
+ * This call can fail in FRONTEND code. We defer error handling to time
+ * of use (json_lex_string()) since there's no way to signal failure
+ * here, and we might not need to parse any strings anyway.
+ */
+ lex->strval = createStrVal();
+ }
+ lex->errormsg = NULL;
+}
+
+void
+termJsonLexContext(JsonLexContext *lex)
+{
+ static const JsonLexContext empty = {0};
+
+ if (lex->strval)
+ {
+#ifdef FRONTEND
+ destroyPQExpBuffer(lex->strval);
+#else
+ pfree(lex->strval->data);
+ pfree(lex->strval);
+#endif
+ }
+
+ if (lex->errormsg)
+ {
+#ifdef FRONTEND
+ destroyPQExpBuffer(lex->errormsg);
+#else
+ pfree(lex->errormsg->data);
+ pfree(lex->errormsg);
+#endif
+ }
+
+ *lex = empty;
}
/*
@@ -217,7 +301,7 @@ json_count_array_elements(JsonLexContext *lex, int *elements)
* etc, so doing this with a copy makes that safe.
*/
memcpy(©lex, lex, sizeof(JsonLexContext));
- copylex.strval = NULL; /* not interested in values here */
+ copylex.parse_strval = false; /* not interested in values here */
copylex.lex_level++;
count = 0;
@@ -279,14 +363,21 @@ parse_scalar(JsonLexContext *lex, JsonSemAction *sem)
/* extract the de-escaped string value, or the raw lexeme */
if (lex_peek(lex) == JSON_TOKEN_STRING)
{
- if (lex->strval != NULL)
- val = pstrdup(lex->strval->data);
+ if (lex->parse_strval)
+ {
+ val = STRDUP(lex->strval->data);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
}
else
{
int len = (lex->token_terminator - lex->token_start);
- val = palloc(len + 1);
+ val = ALLOC(len + 1);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+
memcpy(val, lex->token_start, len);
val[len] = '\0';
}
@@ -320,8 +411,12 @@ parse_object_field(JsonLexContext *lex, JsonSemAction *sem)
if (lex_peek(lex) != JSON_TOKEN_STRING)
return report_parse_error(JSON_PARSE_STRING, lex);
- if ((ostart != NULL || oend != NULL) && lex->strval != NULL)
- fname = pstrdup(lex->strval->data);
+ if ((ostart != NULL || oend != NULL) && lex->parse_strval)
+ {
+ fname = STRDUP(lex->strval->data);
+ if (fname == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
result = json_lex(lex);
if (result != JSON_SUCCESS)
return result;
@@ -368,6 +463,10 @@ parse_object(JsonLexContext *lex, JsonSemAction *sem)
JsonParseErrorType result;
#ifndef FRONTEND
+ /*
+ * TODO: clients need some way to put a bound on stack growth. Parse level
+ * limits maybe?
+ */
check_stack_depth();
#endif
@@ -676,8 +775,15 @@ json_lex_string(JsonLexContext *lex)
int len;
int hi_surrogate = -1;
- if (lex->strval != NULL)
- resetStringInfo(lex->strval);
+ if (lex->parse_strval)
+ {
+#ifdef FRONTEND
+ /* make sure initialization succeeded */
+ if (lex->strval == NULL)
+ return JSON_OUT_OF_MEMORY;
+#endif
+ resetStrVal(lex->strval);
+ }
Assert(lex->input_length > 0);
s = lex->token_start;
@@ -737,7 +843,7 @@ json_lex_string(JsonLexContext *lex)
return JSON_UNICODE_ESCAPE_FORMAT;
}
}
- if (lex->strval != NULL)
+ if (lex->parse_strval)
{
/*
* Combine surrogate pairs.
@@ -797,19 +903,19 @@ json_lex_string(JsonLexContext *lex)
unicode_to_utf8(ch, (unsigned char *) utf8str);
utf8len = pg_utf_mblen((unsigned char *) utf8str);
- appendBinaryStringInfo(lex->strval, utf8str, utf8len);
+ appendBinaryPQExpBuffer(lex->strval, utf8str, utf8len);
}
else if (ch <= 0x007f)
{
/* The ASCII range is the same in all encodings */
- appendStringInfoChar(lex->strval, (char) ch);
+ appendPQExpBufferChar(lex->strval, (char) ch);
}
else
return JSON_UNICODE_HIGH_ESCAPE;
#endif /* FRONTEND */
}
}
- else if (lex->strval != NULL)
+ else if (lex->parse_strval)
{
if (hi_surrogate != -1)
return JSON_UNICODE_LOW_SURROGATE;
@@ -819,22 +925,22 @@ json_lex_string(JsonLexContext *lex)
case '"':
case '\\':
case '/':
- appendStringInfoChar(lex->strval, *s);
+ appendStrValChar(lex->strval, *s);
break;
case 'b':
- appendStringInfoChar(lex->strval, '\b');
+ appendStrValChar(lex->strval, '\b');
break;
case 'f':
- appendStringInfoChar(lex->strval, '\f');
+ appendStrValChar(lex->strval, '\f');
break;
case 'n':
- appendStringInfoChar(lex->strval, '\n');
+ appendStrValChar(lex->strval, '\n');
break;
case 'r':
- appendStringInfoChar(lex->strval, '\r');
+ appendStrValChar(lex->strval, '\r');
break;
case 't':
- appendStringInfoChar(lex->strval, '\t');
+ appendStrValChar(lex->strval, '\t');
break;
default:
/* Not a valid string escape, so signal error. */
@@ -858,12 +964,12 @@ json_lex_string(JsonLexContext *lex)
}
}
- else if (lex->strval != NULL)
+ else if (lex->parse_strval)
{
if (hi_surrogate != -1)
return JSON_UNICODE_LOW_SURROGATE;
- appendStringInfoChar(lex->strval, *s);
+ appendStrValChar(lex->strval, *s);
}
}
@@ -871,6 +977,11 @@ json_lex_string(JsonLexContext *lex)
if (hi_surrogate != -1)
return JSON_UNICODE_LOW_SURROGATE;
+#ifdef FRONTEND
+ if (lex->parse_strval && PQExpBufferBroken(lex->strval))
+ return JSON_OUT_OF_MEMORY;
+#endif
+
/* Hooray, we found the end of the string! */
lex->prev_token_terminator = lex->token_terminator;
lex->token_terminator = s + 1;
@@ -1043,72 +1154,93 @@ report_parse_error(JsonParseContext ctx, JsonLexContext *lex)
return JSON_SUCCESS; /* silence stupider compilers */
}
-
-#ifndef FRONTEND
-/*
- * Extract the current token from a lexing context, for error reporting.
- */
-static char *
-extract_token(JsonLexContext *lex)
-{
- int toklen = lex->token_terminator - lex->token_start;
- char *token = palloc(toklen + 1);
-
- memcpy(token, lex->token_start, toklen);
- token[toklen] = '\0';
- return token;
-}
-
/*
* Construct a detail message for a JSON error.
*
- * Note that the error message generated by this routine may not be
- * palloc'd, making it unsafe for frontend code as there is no way to
- * know if this can be safery pfree'd or not.
+ * The returned allocation is either static or owned by the JsonLexContext and
+ * should not be freed.
*/
char *
json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
{
+ int toklen = lex->token_terminator - lex->token_start;
+
+ if (error == JSON_OUT_OF_MEMORY)
+ {
+ /* Short circuit. Allocating anything for this case is unhelpful. */
+ return _("out of memory");
+ }
+
+ if (lex->errormsg)
+ resetStrVal(lex->errormsg);
+ else
+ lex->errormsg = createStrVal();
+
switch (error)
{
case JSON_SUCCESS:
/* fall through to the error code after switch */
break;
case JSON_ESCAPING_INVALID:
- return psprintf(_("Escape sequence \"\\%s\" is invalid."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Escape sequence \"\\%.*s\" is invalid."),
+ toklen, lex->token_start);
+ break;
case JSON_ESCAPING_REQUIRED:
- return psprintf(_("Character with value 0x%02x must be escaped."),
- (unsigned char) *(lex->token_terminator));
+ appendStrVal(lex->errormsg,
+ _("Character with value 0x%02x must be escaped."),
+ (unsigned char) *(lex->token_terminator));
+ break;
case JSON_EXPECTED_END:
- return psprintf(_("Expected end of input, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected end of input, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_ARRAY_FIRST:
- return psprintf(_("Expected array element or \"]\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected array element or \"]\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_ARRAY_NEXT:
- return psprintf(_("Expected \",\" or \"]\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \",\" or \"]\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_COLON:
- return psprintf(_("Expected \":\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \":\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_JSON:
- return psprintf(_("Expected JSON value, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected JSON value, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_MORE:
return _("The input string ended unexpectedly.");
case JSON_EXPECTED_OBJECT_FIRST:
- return psprintf(_("Expected string or \"}\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected string or \"}\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_OBJECT_NEXT:
- return psprintf(_("Expected \",\" or \"}\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \",\" or \"}\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_STRING:
- return psprintf(_("Expected string, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected string, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_INVALID_TOKEN:
- return psprintf(_("Token \"%s\" is invalid."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Token \"%.*s\" is invalid."),
+ toklen, lex->token_start);
+ break;
+ case JSON_OUT_OF_MEMORY:
+ /* should have been handled above; use the error path */
+ break;
case JSON_UNICODE_CODE_POINT_ZERO:
return _("\\u0000 cannot be converted to text.");
case JSON_UNICODE_ESCAPE_FORMAT:
@@ -1122,12 +1254,22 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
return _("Unicode low surrogate must follow a high surrogate.");
}
- /*
- * We don't use a default: case, so that the compiler will warn about
- * unhandled enum values. But this needs to be here anyway to cover the
- * possibility of an incorrect input.
- */
- elog(ERROR, "unexpected json parse error type: %d", (int) error);
- return NULL;
-}
+ /* Note that lex->errormsg can be NULL in FRONTEND code. */
+ if (lex->errormsg && !lex->errormsg->data[0])
+ {
+ /*
+ * We don't use a default: case, so that the compiler will warn about
+ * unhandled enum values. But this needs to be here anyway to cover the
+ * possibility of an incorrect input.
+ */
+ appendStrVal(lex->errormsg,
+ "unexpected json parse error type: %d", (int) error);
+ }
+
+#ifdef FRONTEND
+ if (PQExpBufferBroken(lex->errormsg))
+ return _("out of memory while constructing error description");
#endif
+
+ return lex->errormsg->data;
+}
diff --git a/src/include/common/jsonapi.h b/src/include/common/jsonapi.h
index ec3dfce9c3..dc71ab2cd3 100644
--- a/src/include/common/jsonapi.h
+++ b/src/include/common/jsonapi.h
@@ -14,8 +14,6 @@
#ifndef JSONAPI_H
#define JSONAPI_H
-#include "lib/stringinfo.h"
-
typedef enum
{
JSON_TOKEN_INVALID,
@@ -48,6 +46,7 @@ typedef enum
JSON_EXPECTED_OBJECT_NEXT,
JSON_EXPECTED_STRING,
JSON_INVALID_TOKEN,
+ JSON_OUT_OF_MEMORY,
JSON_UNICODE_CODE_POINT_ZERO,
JSON_UNICODE_ESCAPE_FORMAT,
JSON_UNICODE_HIGH_ESCAPE,
@@ -55,6 +54,17 @@ typedef enum
JSON_UNICODE_LOW_SURROGATE
} JsonParseErrorType;
+/*
+ * Don't depend on the internal type header for strval; if callers need access
+ * then they can include the appropriate header themselves.
+ */
+#ifdef FRONTEND
+#define StrValType PQExpBufferData
+#else
+#define StrValType StringInfoData
+#endif
+
+typedef struct StrValType StrValType;
/*
* All the fields in this structure should be treated as read-only.
@@ -81,7 +91,9 @@ typedef struct JsonLexContext
int lex_level;
int line_number; /* line number, starting from 1 */
char *line_start; /* where that line starts within input */
- StringInfo strval;
+ bool parse_strval;
+ StrValType *strval; /* only used if parse_strval == true */
+ StrValType *errormsg;
} JsonLexContext;
typedef void (*json_struct_action) (void *state);
@@ -141,9 +153,10 @@ extern JsonSemAction nullSemAction;
*/
extern JsonParseErrorType json_count_array_elements(JsonLexContext *lex,
int *elements);
+#ifndef FRONTEND
/*
- * constructor for JsonLexContext, with or without strval element.
+ * allocating constructor for JsonLexContext, with or without strval element.
* If supplied, the strval element will contain a de-escaped version of
* the lexeme. However, doing this imposes a performance penalty, so
* it should be avoided if the de-escaped lexeme is not required.
@@ -153,6 +166,32 @@ extern JsonLexContext *makeJsonLexContextCstringLen(char *json,
int encoding,
bool need_escapes);
+/*
+ * Counterpart to makeJsonLexContextCstringLen(): clears and deallocates lex.
+ * The context pointer should not be used after this call.
+ */
+extern void destroyJsonLexContext(JsonLexContext *lex);
+
+#endif /* !FRONTEND */
+
+/*
+ * stack constructor for JsonLexContext, with or without strval element.
+ * If supplied, the strval element will contain a de-escaped version of
+ * the lexeme. However, doing this imposes a performance penalty, so
+ * it should be avoided if the de-escaped lexeme is not required.
+ */
+extern void initJsonLexContextCstringLen(JsonLexContext *lex,
+ char *json,
+ int len,
+ int encoding,
+ bool need_escapes);
+
+/*
+ * Counterpart to initJsonLexContextCstringLen(): clears the contents of lex,
+ * but does not deallocate lex itself.
+ */
+extern void termJsonLexContext(JsonLexContext *lex);
+
/* lex one token */
extern JsonParseErrorType json_lex(JsonLexContext *lex);
--
2.25.1
v2-0002-libpq-add-OAUTHBEARER-SASL-mechanism.patch (text/x-patch, 35.7K) download
From 52ac4bd25ca19735eb2bd863e8b1549ccbe6560a Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 13 Apr 2021 10:27:27 -0700
Subject: [PATCH v2 2/5] libpq: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) and OAuth 2.0 Device Authorization
Grants (RFC 8628) on the client side. When speaking to a OAuth-enabled
server, it looks a bit like this:
$ psql 'host=example.org oauth_client_id=f02c6361-0635-...'
Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG
The OAuth issuer must support device authorization. No other OAuth flows
are currently implemented.
The client implementation requires libiddawc and its development
headers. Configure --with-oauth (and --with-includes/--with-libraries to
point at the iddawc installation, if it's in a custom location).
Several TODOs:
- don't retry forever if the server won't accept our token
- perform several sanity checks on the OAuth issuer's responses
- handle cases where the client has been set up with an issuer and
scope, but the Postgres server wants to use something different
- improve error debuggability during the OAuth handshake
- ...and more.
---
configure | 100 ++++
configure.ac | 19 +
src/Makefile.global.in | 1 +
src/include/common/oauth-common.h | 19 +
src/include/pg_config.h.in | 6 +
src/interfaces/libpq/Makefile | 7 +-
src/interfaces/libpq/fe-auth-oauth.c | 745 +++++++++++++++++++++++++++
src/interfaces/libpq/fe-auth-sasl.h | 5 +-
src/interfaces/libpq/fe-auth-scram.c | 6 +-
src/interfaces/libpq/fe-auth.c | 42 +-
src/interfaces/libpq/fe-auth.h | 3 +
src/interfaces/libpq/fe-connect.c | 38 ++
src/interfaces/libpq/libpq-int.h | 8 +
13 files changed, 980 insertions(+), 19 deletions(-)
create mode 100644 src/include/common/oauth-common.h
create mode 100644 src/interfaces/libpq/fe-auth-oauth.c
diff --git a/configure b/configure
index 7542fe30a1..2ddbe9a1d9 100755
--- a/configure
+++ b/configure
@@ -713,6 +713,7 @@ with_uuid
with_readline
with_systemd
with_selinux
+with_oauth
with_ldap
with_krb_srvnam
krb_srvtab
@@ -856,6 +857,7 @@ with_krb_srvnam
with_pam
with_bsd_auth
with_ldap
+with_oauth
with_bonjour
with_selinux
with_systemd
@@ -1562,6 +1564,7 @@ Optional Packages:
--with-pam build with PAM support
--with-bsd-auth build with BSD Authentication support
--with-ldap build with LDAP support
+ --with-oauth build with OAuth 2.0 support
--with-bonjour build with Bonjour support
--with-selinux build with SELinux support
--with-systemd build with systemd support
@@ -8144,6 +8147,42 @@ $as_echo "$with_ldap" >&6; }
+#
+# OAuth 2.0
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with OAuth support" >&5
+$as_echo_n "checking whether to build with OAuth support... " >&6; }
+
+
+
+# Check whether --with-oauth was given.
+if test "${with_oauth+set}" = set; then :
+ withval=$with_oauth;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_OAUTH 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-oauth option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_oauth=no
+
+fi
+
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_oauth" >&5
+$as_echo "$with_oauth" >&6; }
+
+
+
#
# Bonjour
#
@@ -13084,6 +13123,56 @@ fi
+if test "$with_oauth" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for i_init_session in -liddawc" >&5
+$as_echo_n "checking for i_init_session in -liddawc... " >&6; }
+if ${ac_cv_lib_iddawc_i_init_session+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-liddawc $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char i_init_session ();
+int
+main ()
+{
+return i_init_session ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_iddawc_i_init_session=yes
+else
+ ac_cv_lib_iddawc_i_init_session=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_iddawc_i_init_session" >&5
+$as_echo "$ac_cv_lib_iddawc_i_init_session" >&6; }
+if test "x$ac_cv_lib_iddawc_i_init_session" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBIDDAWC 1
+_ACEOF
+
+ LIBS="-liddawc $LIBS"
+
+else
+ as_fn_error $? "library 'iddawc' is required for OAuth support" "$LINENO" 5
+fi
+
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for security_compute_create_name in -lselinux" >&5
@@ -13978,6 +14067,17 @@ fi
done
+fi
+
+if test "$with_oauth" != no; then
+ ac_fn_c_check_header_mongrel "$LINENO" "iddawc.h" "ac_cv_header_iddawc_h" "$ac_includes_default"
+if test "x$ac_cv_header_iddawc_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <iddawc.h> is required for OAuth" "$LINENO" 5
+fi
+
+
fi
if test "$PORTNAME" = "win32" ; then
diff --git a/configure.ac b/configure.ac
index ed3cdb9a8e..22026476d9 100644
--- a/configure.ac
+++ b/configure.ac
@@ -851,6 +851,17 @@ AC_MSG_RESULT([$with_ldap])
AC_SUBST(with_ldap)
+#
+# OAuth 2.0
+#
+AC_MSG_CHECKING([whether to build with OAuth support])
+PGAC_ARG_BOOL(with, oauth, no,
+ [build with OAuth 2.0 support],
+ [AC_DEFINE([USE_OAUTH], 1, [Define to 1 to build with OAuth 2.0 support. (--with-oauth)])])
+AC_MSG_RESULT([$with_oauth])
+AC_SUBST(with_oauth)
+
+
#
# Bonjour
#
@@ -1321,6 +1332,10 @@ fi
AC_SUBST(LDAP_LIBS_FE)
AC_SUBST(LDAP_LIBS_BE)
+if test "$with_oauth" = yes ; then
+ AC_CHECK_LIB(iddawc, i_init_session, [], [AC_MSG_ERROR([library 'iddawc' is required for OAuth support])])
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
AC_CHECK_LIB(selinux, security_compute_create_name, [],
@@ -1531,6 +1546,10 @@ elif test "$with_uuid" = ossp ; then
[AC_MSG_ERROR([header file <ossp/uuid.h> or <uuid.h> is required for OSSP UUID])])])
fi
+if test "$with_oauth" != no; then
+ AC_CHECK_HEADER(iddawc.h, [], [AC_MSG_ERROR([header file <iddawc.h> is required for OAuth])])
+fi
+
if test "$PORTNAME" = "win32" ; then
AC_CHECK_HEADERS(crtdefs.h)
fi
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 6e2f224cc4..d67912711e 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -193,6 +193,7 @@ with_ldap = @with_ldap@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
+with_oauth = @with_oauth@
with_system_tzdata = @with_system_tzdata@
with_uuid = @with_uuid@
with_zlib = @with_zlib@
diff --git a/src/include/common/oauth-common.h b/src/include/common/oauth-common.h
new file mode 100644
index 0000000000..3fa95ac7e8
--- /dev/null
+++ b/src/include/common/oauth-common.h
@@ -0,0 +1,19 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth-common.h
+ * Declarations for helper functions used for OAuth/OIDC authentication
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/common/oauth-common.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef OAUTH_COMMON_H
+#define OAUTH_COMMON_H
+
+/* Name of SASL mechanism per IANA */
+#define OAUTHBEARER_NAME "OAUTHBEARER"
+
+#endif /* OAUTH_COMMON_H */
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 15ffdd895a..f82ab38536 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -331,6 +331,9 @@
/* Define to 1 if you have the `crypto' library (-lcrypto). */
#undef HAVE_LIBCRYPTO
+/* Define to 1 if you have the `iddawc' library (-liddawc). */
+#undef HAVE_LIBIDDAWC
+
/* Define to 1 if you have the `ldap' library (-lldap). */
#undef HAVE_LIBLDAP
@@ -926,6 +929,9 @@
/* Define to select named POSIX semaphores. */
#undef USE_NAMED_POSIX_SEMAPHORES
+/* Define to 1 to build with OAuth 2.0 support. (--with-oauth) */
+#undef USE_OAUTH
+
/* Define to 1 to build with OpenSSL support. (--with-ssl=openssl) */
#undef USE_OPENSSL
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 7cbdeb589b..3cdf19294b 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -62,6 +62,11 @@ OBJS += \
fe-secure-gssapi.o
endif
+ifeq ($(with_oauth),yes)
+OBJS += \
+ fe-auth-oauth.o
+endif
+
ifeq ($(PORTNAME), cygwin)
override shlib = cyg$(NAME)$(DLSUFFIX)
endif
@@ -83,7 +88,7 @@ endif
# that are built correctly for use in a shlib.
SHLIB_LINK_INTERNAL = -lpgcommon_shlib -lpgport_shlib
ifneq ($(PORTNAME), win32)
-SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
+SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -liddawc -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
else
SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi32 -lssl -lsocket -lnsl -lresolv -lintl -lm $(PTHREAD_LIBS), $(LIBS)) $(LDAP_LIBS_FE)
endif
diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
new file mode 100644
index 0000000000..91d2c69f16
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth.c
@@ -0,0 +1,745 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth.c
+ * The front-end (client) implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include <iddawc.h>
+
+#include "postgres_fe.h"
+
+#include "common/base64.h"
+#include "common/hmac.h"
+#include "common/jsonapi.h"
+#include "common/oauth-common.h"
+#include "fe-auth.h"
+#include "mb/pg_wchar.h"
+
+/* The exported OAuth callback mechanism. */
+static void *oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism);
+static void oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen,
+ bool *done, bool *success);
+static bool oauth_channel_bound(void *opaq);
+static void oauth_free(void *opaq);
+
+const pg_fe_sasl_mech pg_oauth_mech = {
+ oauth_init,
+ oauth_exchange,
+ oauth_channel_bound,
+ oauth_free,
+};
+
+typedef enum
+{
+ FE_OAUTH_INIT,
+ FE_OAUTH_BEARER_SENT,
+ FE_OAUTH_SERVER_ERROR,
+} fe_oauth_state_enum;
+
+typedef struct
+{
+ fe_oauth_state_enum state;
+
+ PGconn *conn;
+} fe_oauth_state;
+
+static void *
+oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism)
+{
+ fe_oauth_state *state;
+
+ /*
+ * We only support one SASL mechanism here; anything else is programmer
+ * error.
+ */
+ Assert(sasl_mechanism != NULL);
+ Assert(!strcmp(sasl_mechanism, OAUTHBEARER_NAME));
+
+ state = malloc(sizeof(*state));
+ if (!state)
+ return NULL;
+
+ state->state = FE_OAUTH_INIT;
+ state->conn = conn;
+
+ return state;
+}
+
+static const char *
+iddawc_error_string(int errcode)
+{
+ switch (errcode)
+ {
+ case I_OK:
+ return "I_OK";
+
+ case I_ERROR:
+ return "I_ERROR";
+
+ case I_ERROR_PARAM:
+ return "I_ERROR_PARAM";
+
+ case I_ERROR_MEMORY:
+ return "I_ERROR_MEMORY";
+
+ case I_ERROR_UNAUTHORIZED:
+ return "I_ERROR_UNAUTHORIZED";
+
+ case I_ERROR_SERVER:
+ return "I_ERROR_SERVER";
+ }
+
+ return "<unknown>";
+}
+
+static void
+iddawc_error(PGconn *conn, int errcode, const char *msg)
+{
+ appendPQExpBufferStr(&conn->errorMessage, libpq_gettext(msg));
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext(" (iddawc error %s)\n"),
+ iddawc_error_string(errcode));
+}
+
+static void
+iddawc_request_error(PGconn *conn, struct _i_session *i, int err, const char *msg)
+{
+ const char *error_code;
+ const char *desc;
+
+ appendPQExpBuffer(&conn->errorMessage, "%s: ", libpq_gettext(msg));
+
+ error_code = i_get_str_parameter(i, I_OPT_ERROR);
+ if (!error_code)
+ {
+ /*
+ * The server didn't give us any useful information, so just print the
+ * error code.
+ */
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("(iddawc error %s)\n"),
+ iddawc_error_string(err));
+ return;
+ }
+
+ /* If the server gave a string description, print that too. */
+ desc = i_get_str_parameter(i, I_OPT_ERROR_DESCRIPTION);
+ if (desc)
+ appendPQExpBuffer(&conn->errorMessage, "%s ", desc);
+
+ appendPQExpBuffer(&conn->errorMessage, "(%s)\n", error_code);
+}
+
+static char *
+get_auth_token(PGconn *conn)
+{
+ PQExpBuffer token_buf = NULL;
+ struct _i_session session;
+ int err;
+ int auth_method;
+ bool user_prompted = false;
+ const char *verification_uri;
+ const char *user_code;
+ const char *access_token;
+ const char *token_type;
+ char *token = NULL;
+
+ if (!conn->oauth_discovery_uri)
+ return strdup(""); /* ask the server for one */
+
+ i_init_session(&session);
+
+ if (!conn->oauth_client_id)
+ {
+ /* We can't talk to a server without a client identifier. */
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("no oauth_client_id is set for the connection"));
+ goto cleanup;
+ }
+
+ token_buf = createPQExpBuffer();
+
+ if (!token_buf)
+ goto cleanup;
+
+ err = i_set_str_parameter(&session, I_OPT_OPENID_CONFIG_ENDPOINT, conn->oauth_discovery_uri);
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set OpenID config endpoint");
+ goto cleanup;
+ }
+
+ err = i_get_openid_config(&session);
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to fetch OpenID discovery document");
+ goto cleanup;
+ }
+
+ if (!i_get_str_parameter(&session, I_OPT_TOKEN_ENDPOINT))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer has no token endpoint"));
+ goto cleanup;
+ }
+
+ if (!i_get_str_parameter(&session, I_OPT_DEVICE_AUTHORIZATION_ENDPOINT))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer does not support device authorization"));
+ goto cleanup;
+ }
+
+ err = i_set_response_type(&session, I_RESPONSE_TYPE_DEVICE_CODE);
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set device code response type");
+ goto cleanup;
+ }
+
+ auth_method = I_TOKEN_AUTH_METHOD_NONE;
+ if (conn->oauth_client_secret && *conn->oauth_client_secret)
+ auth_method = I_TOKEN_AUTH_METHOD_SECRET_BASIC;
+
+ err = i_set_parameter_list(&session,
+ I_OPT_CLIENT_ID, conn->oauth_client_id,
+ I_OPT_CLIENT_SECRET, conn->oauth_client_secret,
+ I_OPT_TOKEN_METHOD, auth_method,
+ I_OPT_SCOPE, conn->oauth_scope,
+ I_OPT_NONE
+ );
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set client identifier");
+ goto cleanup;
+ }
+
+ err = i_run_device_auth_request(&session);
+ if (err)
+ {
+ iddawc_request_error(conn, &session, err,
+ "failed to obtain device authorization");
+ goto cleanup;
+ }
+
+ verification_uri = i_get_str_parameter(&session, I_OPT_DEVICE_AUTH_VERIFICATION_URI);
+ if (!verification_uri)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer did not provide a verification URI"));
+ goto cleanup;
+ }
+
+ user_code = i_get_str_parameter(&session, I_OPT_DEVICE_AUTH_USER_CODE);
+ if (!user_code)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer did not provide a user code"));
+ goto cleanup;
+ }
+
+ /*
+ * Poll the token endpoint until either the user logs in and authorizes the
+ * use of a token, or a hard failure occurs. We perform one ping _before_
+ * prompting the user, so that we don't make them do the work of logging in
+ * only to find that the token endpoint is completely unreachable.
+ */
+ err = i_run_token_request(&session);
+ while (err)
+ {
+ const char *error_code;
+ uint interval;
+
+ error_code = i_get_str_parameter(&session, I_OPT_ERROR);
+
+ /*
+ * authorization_pending and slow_down are the only acceptable errors;
+ * anything else and we bail.
+ */
+ if (!error_code || (strcmp(error_code, "authorization_pending")
+ && strcmp(error_code, "slow_down")))
+ {
+ iddawc_request_error(conn, &session, err,
+ "OAuth token retrieval failed");
+ goto cleanup;
+ }
+
+ if (!user_prompted)
+ {
+ /*
+ * Now that we know the token endpoint isn't broken, give the user
+ * the login instructions.
+ */
+ pqInternalNotice(&conn->noticeHooks,
+ "Visit %s and enter the code: %s",
+ verification_uri, user_code);
+
+ user_prompted = true;
+ }
+
+ /*
+ * We are required to wait between polls; the server tells us how long.
+ * TODO: if interval's not set, we need to default to five seconds
+ * TODO: sanity check the interval
+ */
+ interval = i_get_int_parameter(&session, I_OPT_DEVICE_AUTH_INTERVAL);
+
+ /*
+ * A slow_down error requires us to permanently increase our retry
+ * interval by five seconds. RFC 8628, Sec. 3.5.
+ */
+ if (!strcmp(error_code, "slow_down"))
+ {
+ interval += 5;
+ i_set_int_parameter(&session, I_OPT_DEVICE_AUTH_INTERVAL, interval);
+ }
+
+ sleep(interval);
+
+ /*
+ * XXX Reset the error code before every call, because iddawc won't do
+ * that for us. This matters if the server first sends a "pending" error
+ * code, then later hard-fails without sending an error code to
+ * overwrite the first one.
+ *
+ * That we have to do this at all seems like a bug in iddawc.
+ */
+ i_set_str_parameter(&session, I_OPT_ERROR, NULL);
+
+ err = i_run_token_request(&session);
+ }
+
+ access_token = i_get_str_parameter(&session, I_OPT_ACCESS_TOKEN);
+ token_type = i_get_str_parameter(&session, I_OPT_TOKEN_TYPE);
+
+ if (!access_token || !token_type || strcasecmp(token_type, "Bearer"))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer did not provide a bearer token"));
+ goto cleanup;
+ }
+
+ appendPQExpBufferStr(token_buf, "Bearer ");
+ appendPQExpBufferStr(token_buf, access_token);
+
+ if (PQExpBufferBroken(token_buf))
+ goto cleanup;
+
+ token = strdup(token_buf->data);
+
+cleanup:
+ if (token_buf)
+ destroyPQExpBuffer(token_buf);
+ i_clean_session(&session);
+
+ return token;
+}
+
+#define kvsep "\x01"
+
+static char *
+client_initial_response(PGconn *conn)
+{
+ static const char * const resp_format = "n,," kvsep "auth=%s" kvsep kvsep;
+
+ PQExpBuffer token_buf;
+ PQExpBuffer discovery_buf = NULL;
+ char *token = NULL;
+ char *response = NULL;
+
+ token_buf = createPQExpBuffer();
+ if (!token_buf)
+ goto cleanup;
+
+ /*
+ * If we don't yet have a discovery URI, but the user gave us an explicit
+ * issuer, use the .well-known discovery URI for that issuer.
+ */
+ if (!conn->oauth_discovery_uri && conn->oauth_issuer)
+ {
+ discovery_buf = createPQExpBuffer();
+ if (!discovery_buf)
+ goto cleanup;
+
+ appendPQExpBufferStr(discovery_buf, conn->oauth_issuer);
+ appendPQExpBufferStr(discovery_buf, "/.well-known/openid-configuration");
+
+ if (PQExpBufferBroken(discovery_buf))
+ goto cleanup;
+
+ conn->oauth_discovery_uri = strdup(discovery_buf->data);
+ }
+
+ token = get_auth_token(conn);
+ if (!token)
+ goto cleanup;
+
+ appendPQExpBuffer(token_buf, resp_format, token);
+ if (PQExpBufferBroken(token_buf))
+ goto cleanup;
+
+ response = strdup(token_buf->data);
+
+cleanup:
+ if (token)
+ free(token);
+ if (discovery_buf)
+ destroyPQExpBuffer(discovery_buf);
+ if (token_buf)
+ destroyPQExpBuffer(token_buf);
+
+ return response;
+}
+
+#define ERROR_STATUS_FIELD "status"
+#define ERROR_SCOPE_FIELD "scope"
+#define ERROR_OPENID_CONFIGURATION_FIELD "openid-configuration"
+
+struct json_ctx
+{
+ char *errmsg; /* any non-NULL value stops all processing */
+ PQExpBufferData errbuf; /* backing memory for errmsg */
+ int nested; /* nesting level (zero is the top) */
+
+ const char *target_field_name; /* points to a static allocation */
+ char **target_field; /* see below */
+
+ /* target_field, if set, points to one of the following: */
+ char *status;
+ char *scope;
+ char *discovery_uri;
+};
+
+#define oauth_json_has_error(ctx) \
+ (PQExpBufferDataBroken((ctx)->errbuf) || (ctx)->errmsg)
+
+#define oauth_json_set_error(ctx, ...) \
+ do { \
+ appendPQExpBuffer(&(ctx)->errbuf, __VA_ARGS__); \
+ (ctx)->errmsg = (ctx)->errbuf.data; \
+ } while (0)
+
+static void
+oauth_json_object_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (oauth_json_has_error(ctx))
+ return; /* short-circuit */
+
+ if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ ++ctx->nested;
+}
+
+static void
+oauth_json_object_end(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (oauth_json_has_error(ctx))
+ return; /* short-circuit */
+
+ --ctx->nested;
+}
+
+static void
+oauth_json_object_field_start(void *state, char *name, bool isnull)
+{
+ struct json_ctx *ctx = state;
+
+ if (oauth_json_has_error(ctx))
+ {
+ /* short-circuit */
+ free(name);
+ return;
+ }
+
+ if (ctx->nested == 1)
+ {
+ if (!strcmp(name, ERROR_STATUS_FIELD))
+ {
+ ctx->target_field_name = ERROR_STATUS_FIELD;
+ ctx->target_field = &ctx->status;
+ }
+ else if (!strcmp(name, ERROR_SCOPE_FIELD))
+ {
+ ctx->target_field_name = ERROR_SCOPE_FIELD;
+ ctx->target_field = &ctx->scope;
+ }
+ else if (!strcmp(name, ERROR_OPENID_CONFIGURATION_FIELD))
+ {
+ ctx->target_field_name = ERROR_OPENID_CONFIGURATION_FIELD;
+ ctx->target_field = &ctx->discovery_uri;
+ }
+ }
+
+ free(name);
+}
+
+static void
+oauth_json_array_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (oauth_json_has_error(ctx))
+ return; /* short-circuit */
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+}
+
+static void
+oauth_json_scalar(void *state, char *token, JsonTokenType type)
+{
+ struct json_ctx *ctx = state;
+
+ if (oauth_json_has_error(ctx))
+ {
+ /* short-circuit */
+ free(token);
+ return;
+ }
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ if (type == JSON_TOKEN_STRING)
+ {
+ *ctx->target_field = token;
+
+ ctx->target_field = NULL;
+ ctx->target_field_name = NULL;
+
+ return; /* don't free the token we're using */
+ }
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ free(token);
+}
+
+static bool
+handle_oauth_sasl_error(PGconn *conn, char *msg, int msglen)
+{
+ JsonLexContext lex = {0};
+ JsonSemAction sem = {0};
+ JsonParseErrorType err;
+ struct json_ctx ctx = {0};
+ char *errmsg = NULL;
+
+ /* Sanity check. */
+ if (strlen(msg) != msglen)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server's error message contained an embedded NULL"));
+ return false;
+ }
+
+ initJsonLexContextCstringLen(&lex, msg, msglen, PG_UTF8, true);
+
+ initPQExpBuffer(&ctx.errbuf);
+ sem.semstate = &ctx;
+
+ sem.object_start = oauth_json_object_start;
+ sem.object_end = oauth_json_object_end;
+ sem.object_field_start = oauth_json_object_field_start;
+ sem.array_start = oauth_json_array_start;
+ sem.scalar = oauth_json_scalar;
+
+ err = pg_parse_json(&lex, &sem);
+
+ if (err != JSON_SUCCESS)
+ {
+ errmsg = json_errdetail(err, &lex);
+ }
+ else if (PQExpBufferDataBroken(ctx.errbuf))
+ {
+ errmsg = libpq_gettext("out of memory");
+ }
+ else if (ctx.errmsg)
+ {
+ errmsg = ctx.errmsg;
+ }
+
+ if (errmsg)
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("failed to parse server's error response: %s"),
+ errmsg);
+
+ /* Don't need the error buffer or the JSON lexer anymore. */
+ termPQExpBuffer(&ctx.errbuf);
+ termJsonLexContext(&lex);
+
+ if (errmsg)
+ return false;
+
+ /* TODO: what if these override what the user already specified? */
+ if (ctx.discovery_uri)
+ {
+ if (conn->oauth_discovery_uri)
+ free(conn->oauth_discovery_uri);
+
+ conn->oauth_discovery_uri = ctx.discovery_uri;
+ }
+
+ if (ctx.scope)
+ {
+ if (conn->oauth_scope)
+ free(conn->oauth_scope);
+
+ conn->oauth_scope = ctx.scope;
+ }
+ /* TODO: missing error scope should clear any existing connection scope */
+
+ if (!ctx.status)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("server sent error response without a status"));
+ return false;
+ }
+
+ if (!strcmp(ctx.status, "invalid_token"))
+ {
+ /*
+ * invalid_token is the only error code we'll automatically retry for,
+ * but only if we have enough information to do so.
+ */
+ if (conn->oauth_discovery_uri)
+ conn->oauth_want_retry = true;
+ }
+ /* TODO: include status in hard failure message */
+
+ return true;
+}
+
+static void
+oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen,
+ bool *done, bool *success)
+{
+ fe_oauth_state *state = opaq;
+ PGconn *conn = state->conn;
+
+ *done = false;
+ *success = false;
+ *output = NULL;
+ *outputlen = 0;
+
+ switch (state->state)
+ {
+ case FE_OAUTH_INIT:
+ Assert(inputlen == -1);
+
+ *output = client_initial_response(conn);
+ if (!*output)
+ goto error;
+
+ *outputlen = strlen(*output);
+ state->state = FE_OAUTH_BEARER_SENT;
+
+ break;
+
+ case FE_OAUTH_BEARER_SENT:
+ if (final)
+ {
+ /* TODO: ensure there is no message content here. */
+ *done = true;
+ *success = true;
+
+ break;
+ }
+
+ /*
+ * Error message sent by the server.
+ */
+ if (!handle_oauth_sasl_error(conn, input, inputlen))
+ goto error;
+
+ /*
+ * Respond with the required dummy message (RFC 7628, sec. 3.2.3).
+ */
+ *output = strdup(kvsep);
+ *outputlen = strlen(*output); /* == 1 */
+
+ state->state = FE_OAUTH_SERVER_ERROR;
+ break;
+
+ case FE_OAUTH_SERVER_ERROR:
+ /*
+ * After an error, the server should send an error response to fail
+ * the SASL handshake, which is handled in higher layers.
+ *
+ * If we get here, the server either sent *another* challenge which
+ * isn't defined in the RFC, or completed the handshake successfully
+ * after telling us it was going to fail. Neither is acceptable.
+ */
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server sent additional OAuth data after error\n"));
+ goto error;
+
+ default:
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("invalid OAuth exchange state\n"));
+ goto error;
+ }
+
+ return;
+
+error:
+ *done = true;
+ *success = false;
+}
+
+static bool
+oauth_channel_bound(void *opaq)
+{
+ /* This mechanism does not support channel binding. */
+ return false;
+}
+
+static void
+oauth_free(void *opaq)
+{
+ fe_oauth_state *state = opaq;
+
+ free(state);
+}
diff --git a/src/interfaces/libpq/fe-auth-sasl.h b/src/interfaces/libpq/fe-auth-sasl.h
index 3d7ee576f2..0920102908 100644
--- a/src/interfaces/libpq/fe-auth-sasl.h
+++ b/src/interfaces/libpq/fe-auth-sasl.h
@@ -65,6 +65,8 @@ typedef struct pg_fe_sasl_mech
*
* state: The opaque mechanism state returned by init()
*
+ * final: true if the server has sent a final exchange outcome
+ *
* input: The challenge data sent by the server, or NULL when
* generating a client-first initial response (that is, when
* the server expects the client to send a message to start
@@ -92,7 +94,8 @@ typedef struct pg_fe_sasl_mech
* Ignored if *done is false.
*--------
*/
- void (*exchange) (void *state, char *input, int inputlen,
+ void (*exchange) (void *state, bool final,
+ char *input, int inputlen,
char **output, int *outputlen,
bool *done, bool *success);
diff --git a/src/interfaces/libpq/fe-auth-scram.c b/src/interfaces/libpq/fe-auth-scram.c
index 4337e89ce9..489cbeda50 100644
--- a/src/interfaces/libpq/fe-auth-scram.c
+++ b/src/interfaces/libpq/fe-auth-scram.c
@@ -24,7 +24,8 @@
/* The exported SCRAM callback mechanism. */
static void *scram_init(PGconn *conn, const char *password,
const char *sasl_mechanism);
-static void scram_exchange(void *opaq, char *input, int inputlen,
+static void scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen,
bool *done, bool *success);
static bool scram_channel_bound(void *opaq);
@@ -205,7 +206,8 @@ scram_free(void *opaq)
* Exchange a SCRAM message with backend.
*/
static void
-scram_exchange(void *opaq, char *input, int inputlen,
+scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen,
bool *done, bool *success)
{
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index 3421ed4685..0b5b91962a 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -39,6 +39,7 @@
#endif
#include "common/md5.h"
+#include "common/oauth-common.h"
#include "common/scram-common.h"
#include "fe-auth.h"
#include "fe-auth-sasl.h"
@@ -423,7 +424,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
bool success;
const char *selected_mechanism;
PQExpBufferData mechanism_buf;
- char *password;
+ char *password = NULL;
initPQExpBuffer(&mechanism_buf);
@@ -445,8 +446,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
/*
* Parse the list of SASL authentication mechanisms in the
* AuthenticationSASL message, and select the best mechanism that we
- * support. SCRAM-SHA-256-PLUS and SCRAM-SHA-256 are the only ones
- * supported at the moment, listed by order of decreasing importance.
+ * support. Mechanisms are listed by order of decreasing importance.
*/
selected_mechanism = NULL;
for (;;)
@@ -486,6 +486,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
selected_mechanism = SCRAM_SHA_256_PLUS_NAME;
conn->sasl = &pg_scram_mech;
+ conn->password_needed = true;
}
#else
/*
@@ -523,7 +524,17 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
selected_mechanism = SCRAM_SHA_256_NAME;
conn->sasl = &pg_scram_mech;
+ conn->password_needed = true;
}
+#ifdef USE_OAUTH
+ else if (strcmp(mechanism_buf.data, OAUTHBEARER_NAME) == 0 &&
+ !selected_mechanism)
+ {
+ selected_mechanism = OAUTHBEARER_NAME;
+ conn->sasl = &pg_oauth_mech;
+ conn->password_needed = false;
+ }
+#endif
}
if (!selected_mechanism)
@@ -548,18 +559,19 @@ pg_SASL_init(PGconn *conn, int payloadlen)
/*
* First, select the password to use for the exchange, complaining if
- * there isn't one. Currently, all supported SASL mechanisms require a
- * password, so we can just go ahead here without further distinction.
+ * there isn't one and the SASL mechanism needs it.
*/
- conn->password_needed = true;
- password = conn->connhost[conn->whichhost].password;
- if (password == NULL)
- password = conn->pgpass;
- if (password == NULL || password[0] == '\0')
+ if (conn->password_needed)
{
- appendPQExpBufferStr(&conn->errorMessage,
- PQnoPasswordSupplied);
- goto error;
+ password = conn->connhost[conn->whichhost].password;
+ if (password == NULL)
+ password = conn->pgpass;
+ if (password == NULL || password[0] == '\0')
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ PQnoPasswordSupplied);
+ goto error;
+ }
}
Assert(conn->sasl);
@@ -577,7 +589,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
goto oom_error;
/* Get the mechanism-specific Initial Client Response, if any */
- conn->sasl->exchange(conn->sasl_state,
+ conn->sasl->exchange(conn->sasl_state, false,
NULL, -1,
&initialresponse, &initialresponselen,
&done, &success);
@@ -658,7 +670,7 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
/* For safety and convenience, ensure the buffer is NULL-terminated. */
challenge[payloadlen] = '\0';
- conn->sasl->exchange(conn->sasl_state,
+ conn->sasl->exchange(conn->sasl_state, final,
challenge, payloadlen,
&output, &outputlen,
&done, &success);
diff --git a/src/interfaces/libpq/fe-auth.h b/src/interfaces/libpq/fe-auth.h
index 63927480ee..03bea124a6 100644
--- a/src/interfaces/libpq/fe-auth.h
+++ b/src/interfaces/libpq/fe-auth.h
@@ -26,4 +26,7 @@ extern char *pg_fe_getauthname(PQExpBuffer errorMessage);
extern const pg_fe_sasl_mech pg_scram_mech;
extern char *pg_fe_scram_build_secret(const char *password);
+/* Mechanisms in fe-auth-oauth.c */
+extern const pg_fe_sasl_mech pg_oauth_mech;
+
#endif /* FE_AUTH_H */
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 49eec3e835..ba9c097060 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -344,6 +344,23 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"Target-Session-Attrs", "", 15, /* sizeof("prefer-standby") = 15 */
offsetof(struct pg_conn, target_session_attrs)},
+ /* OAuth v2 */
+ {"oauth_issuer", NULL, NULL, NULL,
+ "OAuth-Issuer", "", 40,
+ offsetof(struct pg_conn, oauth_issuer)},
+
+ {"oauth_client_id", NULL, NULL, NULL,
+ "OAuth-Client-ID", "", 40,
+ offsetof(struct pg_conn, oauth_client_id)},
+
+ {"oauth_client_secret", NULL, NULL, NULL,
+ "OAuth-Client-Secret", "", 40,
+ offsetof(struct pg_conn, oauth_client_secret)},
+
+ {"oauth_scope", NULL, NULL, NULL,
+ "OAuth-Scope", "", 15,
+ offsetof(struct pg_conn, oauth_scope)},
+
/* Terminating entry --- MUST BE LAST */
{NULL, NULL, NULL, NULL,
NULL, NULL, 0}
@@ -606,6 +623,7 @@ pqDropServerData(PGconn *conn)
conn->write_err_msg = NULL;
conn->be_pid = 0;
conn->be_key = 0;
+ /* conn->oauth_want_retry = false; TODO */
}
@@ -3355,6 +3373,16 @@ keep_going: /* We will come back to here until there is
/* Check to see if we should mention pgpassfile */
pgpassfileWarning(conn);
+#ifdef USE_OAUTH
+ if (conn->sasl == &pg_oauth_mech
+ && conn->oauth_want_retry)
+ {
+ /* TODO: only allow retry once */
+ need_new_connection = true;
+ goto keep_going;
+ }
+#endif
+
#ifdef ENABLE_GSS
/*
@@ -4129,6 +4157,16 @@ freePGconn(PGconn *conn)
free(conn->rowBuf);
if (conn->target_session_attrs)
free(conn->target_session_attrs);
+ if (conn->oauth_issuer)
+ free(conn->oauth_issuer);
+ if (conn->oauth_discovery_uri)
+ free(conn->oauth_discovery_uri);
+ if (conn->oauth_client_id)
+ free(conn->oauth_client_id);
+ if (conn->oauth_client_secret)
+ free(conn->oauth_client_secret);
+ if (conn->oauth_scope)
+ free(conn->oauth_scope);
termPQExpBuffer(&conn->errorMessage);
termPQExpBuffer(&conn->workBuffer);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 490458adef..3d20482550 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -394,6 +394,14 @@ struct pg_conn
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ /* OAuth v2 */
+ char *oauth_issuer; /* token issuer URL */
+ char *oauth_discovery_uri; /* URI of the issuer's discovery document */
+ char *oauth_client_id; /* client identifier */
+ char *oauth_client_secret; /* client secret */
+ char *oauth_scope; /* access token scope */
+ bool oauth_want_retry; /* should we retry on failure? */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
--
2.25.1
v2-0003-backend-add-OAUTHBEARER-SASL-mechanism.patch (text/x-patch, 38.6K) download
From d09a00b4d52a5ed578ee5cd7623108ebdd12f202 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 4 May 2021 16:21:11 -0700
Subject: [PATCH v2 3/5] backend: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) on the server side. This adds a new
auth method, oauth, to pg_hba.
Because OAuth implementations vary so wildly, and bearer token
validation is heavily dependent on the issuing party, authn/z is done by
communicating with an external program: the oauth_validator_command.
This command must do the following:
1. Receive the bearer token by reading its contents from a file
descriptor passed from the server. (The numeric value of this
descriptor may be inserted into the oauth_validator_command using the
%f specifier.)
This MUST be the first action the command performs. The server will
not begin reading stdout from the command until the token has been
read in full, so if the command tries to print anything and hits a
buffer limit, the backend will deadlock and time out.
2. Validate the bearer token. The correct way to do this depends on the
issuer, but it generally involves either cryptographic operations to
prove that the token was issued by a trusted party, or the
presentation of the bearer token to some other party so that _it_ can
perform validation.
The command MUST maintain confidentiality of the bearer token, since
in most cases it can be used just like a password. (There are ways to
cryptographically bind tokens to client certificates, but they are
way beyond the scope of this commit message.)
If the token cannot be validated, the command must exit with a
non-zero status. Further authentication/authorization is pointless if
the bearer token wasn't issued by someone you trust.
3. Authenticate the user, authorize the user, or both:
a. To authenticate the user, use the bearer token to retrieve some
trusted identifier string for the end user. The exact process for
this is, again, issuer-dependent. The command should print the
authenticated identity string to stdout, followed by a newline.
If the user cannot be authenticated, the validator should not
print anything to stdout. It should also exit with a non-zero
status, unless the token may be used to authorize the connection
through some other means (see below).
On a success, the command may then exit with a zero success code.
By default, the server will then check to make sure the identity
string matches the role that is being used (or matches a usermap
entry, if one is in use).
b. To optionally authorize the user, in combination with the HBA
option trust_validator_authz=1 (see below), the validator simply
returns a zero exit code if the client should be allowed to
connect with its presented role (which can be passed to the
command using the %r specifier), or a non-zero code otherwise.
The hard part is in determining whether the given token truly
authorizes the client to use the given role, which must
unfortunately be left as an exercise to the reader.
This obviously requires some care, as a poorly implemented token
validator may silently open the entire database to anyone with a
bearer token. But it may be a more portable approach, since OAuth
is designed as an authorization framework, not an authentication
framework. For example, the user's bearer token could carry an
"allow_superuser_access" claim, which would authorize pseudonymous
database access as any role. It's then up to the OAuth system
administrators to ensure that allow_superuser_access is doled out
only to the proper users.
c. It's possible that the user can be successfully authenticated but
isn't authorized to connect. In this case, the command may print
the authenticated ID and then fail with a non-zero exit code.
(This makes it easier to see what's going on in the Postgres
logs.)
4. Token validators may optionally log to stderr. This will be printed
verbatim into the Postgres server logs.
The oauth method supports the following HBA options (but note that two
of them are not optional, since we have no way of choosing sensible
defaults):
issuer: Required. The URL of the OAuth issuing party, which the client
must contact to receive a bearer token.
Some real-world examples as of time of writing:
- https://accounts.google.com
- https://login.microsoft.com/[tenant-id]/v2.0
scope: Required. The OAuth scope(s) required for the server to
authenticate and/or authorize the user. This is heavily
deployment-specific, but a simple example is "openid email".
map: Optional. Specify a standard PostgreSQL user map; this works
the same as with other auth methods such as peer. If a map is
not specified, the user ID returned by the token validator
must exactly match the role that's being requested (but see
trust_validator_authz, below).
trust_validator_authz:
Optional. When set to 1, this allows the token validator to
take full control of the authorization process. Standard user
mapping is skipped: if the validator command succeeds, the
client is allowed to connect under its desired role and no
further checks are done.
Unlike the client, servers support OAuth without needing to be built
against libiddawc (since the responsibility for "speaking" OAuth/OIDC
correctly is delegated entirely to the oauth_validator_command).
Several TODOs:
- port to platforms other than "modern Linux"
- overhaul the communication with oauth_validator_command, which is
currently a bad hack on OpenPipeStream()
- implement more sanity checks on the OAUTHBEARER message format and
tokens sent by the client
- implement more helpful handling of HBA misconfigurations
- properly interpolate JSON when generating error responses
- use logdetail during auth failures
- deal with role names that can't be safely passed to system() without
shell-escaping
- allow passing the configured issuer to the oauth_validator_command, to
deal with multi-issuer setups
- ...and more.
---
src/backend/libpq/Makefile | 1 +
src/backend/libpq/auth-oauth.c | 797 +++++++++++++++++++++++++++++++++
src/backend/libpq/auth-sasl.c | 10 +-
src/backend/libpq/auth-scram.c | 4 +-
src/backend/libpq/auth.c | 26 +-
src/backend/libpq/hba.c | 29 +-
src/backend/utils/misc/guc.c | 12 +
src/include/libpq/auth.h | 17 +
src/include/libpq/hba.h | 8 +-
src/include/libpq/oauth.h | 24 +
src/include/libpq/sasl.h | 11 +
11 files changed, 907 insertions(+), 32 deletions(-)
create mode 100644 src/backend/libpq/auth-oauth.c
create mode 100644 src/include/libpq/oauth.h
diff --git a/src/backend/libpq/Makefile b/src/backend/libpq/Makefile
index 6d385fd6a4..98eb2a8242 100644
--- a/src/backend/libpq/Makefile
+++ b/src/backend/libpq/Makefile
@@ -15,6 +15,7 @@ include $(top_builddir)/src/Makefile.global
# be-fsstubs is here for historical reasons, probably belongs elsewhere
OBJS = \
+ auth-oauth.o \
auth-sasl.o \
auth-scram.o \
auth.o \
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
new file mode 100644
index 0000000000..c47211132c
--- /dev/null
+++ b/src/backend/libpq/auth-oauth.c
@@ -0,0 +1,797 @@
+/*-------------------------------------------------------------------------
+ *
+ * auth-oauth.c
+ * Server-side implementation of the SASL OAUTHBEARER mechanism.
+ *
+ * See the following RFC for more details:
+ * - RFC 7628: https://tools.ietf.org/html/rfc7628
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/backend/libpq/auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <unistd.h>
+#include <fcntl.h>
+
+#include "common/oauth-common.h"
+#include "lib/stringinfo.h"
+#include "libpq/auth.h"
+#include "libpq/hba.h"
+#include "libpq/oauth.h"
+#include "libpq/sasl.h"
+#include "storage/fd.h"
+
+/* GUC */
+char *oauth_validator_command;
+
+static void oauth_get_mechanisms(Port *port, StringInfo buf);
+static void *oauth_init(Port *port, const char *selected_mech, const char *shadow_pass);
+static int oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, char **logdetail);
+
+/* Mechanism declaration */
+const pg_be_sasl_mech pg_be_oauth_mech = {
+ oauth_get_mechanisms,
+ oauth_init,
+ oauth_exchange,
+
+ PG_MAX_AUTH_TOKEN_LENGTH,
+};
+
+
+typedef enum
+{
+ OAUTH_STATE_INIT = 0,
+ OAUTH_STATE_ERROR,
+ OAUTH_STATE_FINISHED,
+} oauth_state;
+
+struct oauth_ctx
+{
+ oauth_state state;
+ Port *port;
+ const char *issuer;
+ const char *scope;
+};
+
+static char *sanitize_char(char c);
+static char *parse_kvpairs_for_auth(char **input);
+static void generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen);
+static bool validate(Port *port, const char *auth, char **logdetail);
+static bool run_validator_command(Port *port, const char *token);
+static bool check_exit(FILE **fh, const char *command);
+static bool unset_cloexec(int fd);
+static bool username_ok_for_shell(const char *username);
+
+#define KVSEP 0x01
+#define AUTH_KEY "auth"
+#define BEARER_SCHEME "Bearer "
+
+static void
+oauth_get_mechanisms(Port *port, StringInfo buf)
+{
+ /* Only OAUTHBEARER is supported. */
+ appendStringInfoString(buf, OAUTHBEARER_NAME);
+ appendStringInfoChar(buf, '\0');
+}
+
+static void *
+oauth_init(Port *port, const char *selected_mech, const char *shadow_pass)
+{
+ struct oauth_ctx *ctx;
+
+ if (strcmp(selected_mech, OAUTHBEARER_NAME))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("client selected an invalid SASL authentication mechanism")));
+
+ ctx = palloc0(sizeof(*ctx));
+
+ ctx->state = OAUTH_STATE_INIT;
+ ctx->port = port;
+
+ Assert(port->hba);
+ ctx->issuer = port->hba->oauth_issuer;
+ ctx->scope = port->hba->oauth_scope;
+
+ return ctx;
+}
+
+static int
+oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, char **logdetail)
+{
+ char *p;
+ char cbind_flag;
+ char *auth;
+
+ struct oauth_ctx *ctx = opaq;
+
+ *output = NULL;
+ *outputlen = -1;
+
+ /*
+ * If the client didn't include an "Initial Client Response" in the
+ * SASLInitialResponse message, send an empty challenge, to which the
+ * client will respond with the same data that usually comes in the
+ * Initial Client Response.
+ */
+ if (input == NULL)
+ {
+ Assert(ctx->state == OAUTH_STATE_INIT);
+
+ *output = pstrdup("");
+ *outputlen = 0;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ /*
+ * Check that the input length agrees with the string length of the input.
+ */
+ if (inputlen == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The message is empty.")));
+ if (inputlen != strlen(input))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message length does not match input length.")));
+
+ switch (ctx->state)
+ {
+ case OAUTH_STATE_INIT:
+ /* Handle this case below. */
+ break;
+
+ case OAUTH_STATE_ERROR:
+ /*
+ * Only one response is valid for the client during authentication
+ * failure: a single kvsep.
+ */
+ if (inputlen != 1 || *input != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Client did not send a kvsep response.")));
+
+ /* The (failed) handshake is now complete. */
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_FAILURE;
+
+ default:
+ elog(ERROR, "invalid OAUTHBEARER exchange state");
+ return PG_SASL_EXCHANGE_FAILURE;
+ }
+
+ /* Handle the client's initial message. */
+ p = pstrdup(input);
+
+ /*
+ * OAUTHBEARER does not currently define a channel binding (so there is no
+ * OAUTHBEARER-PLUS, and we do not accept a 'p' specifier). We accept a 'y'
+ * specifier purely for the remote chance that a future specification could
+ * define one; then future clients can still interoperate with this server
+ * implementation. 'n' is the expected case.
+ */
+ cbind_flag = *p;
+ switch (cbind_flag)
+ {
+ case 'p':
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The server does not support channel binding for OAuth, but the client message includes channel binding data.")));
+ break;
+
+ case 'y': /* fall through */
+ case 'n':
+ p++;
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Comma expected, but found character %s.",
+ sanitize_char(*p))));
+ p++;
+ break;
+
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected channel-binding flag %s.",
+ sanitize_char(cbind_flag))));
+ }
+
+ /*
+ * Forbid optional authzid (authorization identity). We don't support it.
+ */
+ if (*p == 'a')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("client uses authorization identity, but it is not supported")));
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected attribute %s in client-first-message.",
+ sanitize_char(*p))));
+ p++;
+
+ /* All remaining fields are separated by the RFC's kvsep (\x01). */
+ if (*p != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Key-value separator expected, but found character %s.",
+ sanitize_char(*p))));
+ p++;
+
+ auth = parse_kvpairs_for_auth(&p);
+ if (!auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message does not contain an auth value.")));
+
+ /* We should be at the end of our message. */
+ if (*p)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains additional data after the final terminator.")));
+
+ if (!validate(ctx->port, auth, logdetail))
+ {
+ generate_error_response(ctx, output, outputlen);
+
+ ctx->state = OAUTH_STATE_ERROR;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_SUCCESS;
+}
+
+/*
+ * Convert an arbitrary byte to printable form. For error messages.
+ *
+ * If it's a printable ASCII character, print it as a single character.
+ * otherwise, print it in hex.
+ *
+ * The returned pointer points to a static buffer.
+ */
+static char *
+sanitize_char(char c)
+{
+ static char buf[5];
+
+ if (c >= 0x21 && c <= 0x7E)
+ snprintf(buf, sizeof(buf), "'%c'", c);
+ else
+ snprintf(buf, sizeof(buf), "0x%02x", (unsigned char) c);
+ return buf;
+}
+
+/*
+ * Consumes all kvpairs in an OAUTHBEARER exchange message. If the "auth" key is
+ * found, its value is returned.
+ */
+static char *
+parse_kvpairs_for_auth(char **input)
+{
+ char *pos = *input;
+ char *auth = NULL;
+
+ /*
+ * The relevant ABNF, from Sec. 3.1:
+ *
+ * kvsep = %x01
+ * key = 1*(ALPHA)
+ * value = *(VCHAR / SP / HTAB / CR / LF )
+ * kvpair = key "=" value kvsep
+ * ;;gs2-header = See RFC 5801
+ * client-resp = (gs2-header kvsep *kvpair kvsep) / kvsep
+ *
+ * By the time we reach this code, the gs2-header and initial kvsep have
+ * already been validated. We start at the beginning of the first kvpair.
+ */
+
+ while (*pos)
+ {
+ char *end;
+ char *sep;
+ char *key;
+ char *value;
+
+ /*
+ * Find the end of this kvpair. Note that input is null-terminated by
+ * the SASL code, so the strchr() is bounded.
+ */
+ end = strchr(pos, KVSEP);
+ if (!end)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an unterminated key/value pair.")));
+ *end = '\0';
+
+ if (pos == end)
+ {
+ /* Empty kvpair, signifying the end of the list. */
+ *input = pos + 1;
+ return auth;
+ }
+
+ /*
+ * Find the end of the key name.
+ *
+ * TODO further validate the key/value grammar? empty keys, bad chars...
+ */
+ sep = strchr(pos, '=');
+ if (!sep)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains a key without a value.")));
+ *sep = '\0';
+
+ /* Both key and value are now safely terminated. */
+ key = pos;
+ value = sep + 1;
+
+ if (!strcmp(key, AUTH_KEY))
+ {
+ if (auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains multiple auth values.")));
+
+ auth = value;
+ }
+ else
+ {
+ /*
+ * The RFC also defines the host and port keys, but they are not
+ * required for OAUTHBEARER and we do not use them. Also, per
+ * Sec. 3.1, any key/value pairs we don't recognize must be ignored.
+ */
+ }
+
+ /* Move to the next pair. */
+ pos = end + 1;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message did not contain a final terminator.")));
+
+ return NULL; /* unreachable */
+}
+
+static void
+generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
+{
+ StringInfoData buf;
+
+ /*
+ * The admin needs to set an issuer and scope for OAuth to work. There's not
+ * really a way to hide this from the user, either, because we can't choose
+ * a "default" issuer, so be honest in the failure message.
+ *
+ * TODO: see if there's a better place to fail, earlier than this.
+ */
+ if (!ctx->issuer || !ctx->scope)
+ ereport(FATAL,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("OAuth is not properly configured for this user"),
+ errdetail_log("The issuer and scope parameters must be set in pg_hba.conf.")));
+
+
+ initStringInfo(&buf);
+
+ /*
+ * TODO: JSON escaping
+ */
+ appendStringInfo(&buf,
+ "{ "
+ "\"status\": \"invalid_token\", "
+ "\"openid-configuration\": \"%s/.well-known/openid-configuration\","
+ "\"scope\": \"%s\" "
+ "}",
+ ctx->issuer, ctx->scope);
+
+ *output = buf.data;
+ *outputlen = buf.len;
+}
+
+static bool
+validate(Port *port, const char *auth, char **logdetail)
+{
+ static const char * const b64_set = "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-._~+/";
+
+ const char *token;
+ size_t span;
+ int ret;
+
+ /* TODO: handle logdetail when the test framework can check it */
+
+ /*
+ * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
+ * 2.1:
+ *
+ * b64token = 1*( ALPHA / DIGIT /
+ * "-" / "." / "_" / "~" / "+" / "/" ) *"="
+ * credentials = "Bearer" 1*SP b64token
+ *
+ * The "credentials" construction is what we receive in our auth value.
+ *
+ * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
+ * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
+ * compared case-insensitively. (This is not mentioned in RFC 6750, but it's
+ * pointed out in RFC 7628 Sec. 4.)
+ *
+ * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
+ */
+ if (strncasecmp(auth, BEARER_SCHEME, strlen(BEARER_SCHEME)))
+ return false;
+
+ /* Pull the bearer token out of the auth value. */
+ token = auth + strlen(BEARER_SCHEME);
+
+ /* Swallow any additional spaces. */
+ while (*token == ' ')
+ token++;
+
+ /*
+ * Before invoking the validator command, sanity-check the token format to
+ * avoid any injection attacks later in the chain. Invalid formats are
+ * technically a protocol violation, but don't reflect any information about
+ * the sensitive Bearer token back to the client; log at COMMERROR instead.
+ */
+
+ /* Tokens must not be empty. */
+ if (!*token)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Bearer token is empty.")));
+ return false;
+ }
+
+ /*
+ * Make sure the token contains only allowed characters. Tokens may end with
+ * any number of '=' characters.
+ */
+ span = strspn(token, b64_set);
+ while (token[span] == '=')
+ span++;
+
+ if (token[span] != '\0')
+ {
+ /*
+ * This error message could be more helpful by printing the problematic
+ * character(s), but that'd be a bit like printing a piece of someone's
+ * password into the logs.
+ */
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Bearer token is not in the correct format.")));
+ return false;
+ }
+
+ /* Have the validator check the token. */
+ if (!run_validator_command(port, token))
+ return false;
+
+ if (port->hba->oauth_skip_usermap)
+ {
+ /*
+ * If the validator is our authorization authority, we're done.
+ * Authentication may or may not have been performed depending on the
+ * validator implementation; all that matters is that the validator says
+ * the user can log in with the target role.
+ */
+ return true;
+ }
+
+ /* Make sure the validator authenticated the user. */
+ if (!port->authn_id)
+ {
+ /* TODO: use logdetail; reduce message duplication */
+ ereport(LOG,
+ (errmsg("OAuth bearer authentication failed for user \"%s\": validator provided no identity",
+ port->user_name)));
+ return false;
+ }
+
+ /* Finally, check the user map. */
+ ret = check_usermap(port->hba->usermap, port->user_name, port->authn_id,
+ false);
+ return (ret == STATUS_OK);
+}
+
+static bool
+run_validator_command(Port *port, const char *token)
+{
+ bool success = false;
+ int rc;
+ int pipefd[2];
+ int rfd = -1;
+ int wfd = -1;
+
+ StringInfoData command = { 0 };
+ char *p;
+ FILE *fh = NULL;
+
+ ssize_t written;
+ char *line = NULL;
+ size_t size = 0;
+ ssize_t len;
+
+ Assert(oauth_validator_command);
+
+ if (!oauth_validator_command[0])
+ {
+ ereport(COMMERROR,
+ (errmsg("oauth_validator_command is not set"),
+ errhint("To allow OAuth authenticated connections, set "
+ "oauth_validator_command in postgresql.conf.")));
+ return false;
+ }
+
+ /*
+ * Since popen() is unidirectional, open up a pipe for the other direction.
+ * Use CLOEXEC to ensure that our write end doesn't accidentally get copied
+ * into child processes, which would prevent us from closing it cleanly.
+ *
+ * XXX this is ugly. We should just read from the child process's stdout,
+ * but that's a lot more code.
+ * XXX by bypassing the popen API, we open the potential of process
+ * deadlock. Clearly document child process requirements (i.e. the child
+ * MUST read all data off of the pipe before writing anything).
+ * TODO: port to Windows using _pipe().
+ */
+ rc = pipe2(pipefd, O_CLOEXEC);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create child pipe: %m")));
+ return false;
+ }
+
+ rfd = pipefd[0];
+ wfd = pipefd[1];
+
+ /* Allow the read pipe be passed to the child. */
+ if (!unset_cloexec(rfd))
+ {
+ /* error message was already logged */
+ goto cleanup;
+ }
+
+ /*
+ * Construct the command, substituting any recognized %-specifiers:
+ *
+ * %f: the file descriptor of the input pipe
+ * %r: the role that the client wants to assume (port->user_name)
+ * %%: a literal '%'
+ */
+ initStringInfo(&command);
+
+ for (p = oauth_validator_command; *p; p++)
+ {
+ if (p[0] == '%')
+ {
+ switch (p[1])
+ {
+ case 'f':
+ appendStringInfo(&command, "%d", rfd);
+ p++;
+ break;
+ case 'r':
+ /*
+ * TODO: decide how this string should be escaped. The role
+ * is controlled by the client, so if we don't escape it,
+ * command injections are inevitable.
+ *
+ * This is probably an indication that the role name needs
+ * to be communicated to the validator process in some other
+ * way. For this proof of concept, just be incredibly strict
+ * about the characters that are allowed in user names.
+ */
+ if (!username_ok_for_shell(port->user_name))
+ goto cleanup;
+
+ appendStringInfoString(&command, port->user_name);
+ p++;
+ break;
+ case '%':
+ appendStringInfoChar(&command, '%');
+ p++;
+ break;
+ default:
+ appendStringInfoChar(&command, p[0]);
+ }
+ }
+ else
+ appendStringInfoChar(&command, p[0]);
+ }
+
+ /* Execute the command. */
+ fh = OpenPipeStream(command.data, "re");
+ /* TODO: handle failures */
+
+ /* We don't need the read end of the pipe anymore. */
+ close(rfd);
+ rfd = -1;
+
+ /* Give the command the token to validate. */
+ written = write(wfd, token, strlen(token));
+ if (written != strlen(token))
+ {
+ /* TODO must loop for short writes, EINTR et al */
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write token to child pipe: %m")));
+ goto cleanup;
+ }
+
+ close(wfd);
+ wfd = -1;
+
+ /*
+ * Read the command's response.
+ *
+ * TODO: getline() is probably too new to use, unfortunately.
+ * TODO: loop over all lines
+ */
+ if ((len = getline(&line, &size, fh)) >= 0)
+ {
+ /* TODO: fail if the authn_id doesn't end with a newline */
+ if (len > 0)
+ line[len - 1] = '\0';
+
+ set_authn_id(port, line);
+ }
+ else if (ferror(fh))
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from command \"%s\": %m",
+ command.data)));
+ goto cleanup;
+ }
+
+ /* Make sure the command exits cleanly. */
+ if (!check_exit(&fh, command.data))
+ {
+ /* error message already logged */
+ goto cleanup;
+ }
+
+ /* Done. */
+ success = true;
+
+cleanup:
+ if (line)
+ free(line);
+
+ /*
+ * In the successful case, the pipe fds are already closed. For the error
+ * case, always close out the pipe before waiting for the command, to
+ * prevent deadlock.
+ */
+ if (rfd >= 0)
+ close(rfd);
+ if (wfd >= 0)
+ close(wfd);
+
+ if (fh)
+ {
+ Assert(!success);
+ check_exit(&fh, command.data);
+ }
+
+ if (command.data)
+ pfree(command.data);
+
+ return success;
+}
+
+static bool
+check_exit(FILE **fh, const char *command)
+{
+ int rc;
+
+ rc = ClosePipeStream(*fh);
+ *fh = NULL;
+
+ if (rc == -1)
+ {
+ /* pclose() itself failed. */
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close pipe to command \"%s\": %m",
+ command)));
+ }
+ else if (rc != 0)
+ {
+ char *reason = wait_result_to_str(rc);
+
+ ereport(COMMERROR,
+ (errmsg("failed to execute command \"%s\": %s",
+ command, reason)));
+
+ pfree(reason);
+ }
+
+ return (rc == 0);
+}
+
+static bool
+unset_cloexec(int fd)
+{
+ int flags;
+ int rc;
+
+ flags = fcntl(fd, F_GETFD);
+ if (flags == -1)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not get fd flags for child pipe: %m")));
+ return false;
+ }
+
+ rc = fcntl(fd, F_SETFD, flags & ~FD_CLOEXEC);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not unset FD_CLOEXEC for child pipe: %m")));
+ return false;
+ }
+
+ return true;
+}
+
+/*
+ * XXX This should go away eventually and be replaced with either a proper
+ * escape or a different strategy for communication with the validator command.
+ */
+static bool
+username_ok_for_shell(const char *username)
+{
+ /* This set is borrowed from fe_utils' appendShellStringNoError(). */
+ static const char * const allowed = "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-_./:";
+ size_t span;
+
+ Assert(username && username[0]); /* should have already been checked */
+
+ span = strspn(username, allowed);
+ if (username[span] != '\0')
+ {
+ ereport(COMMERROR,
+ (errmsg("PostgreSQL user name contains unsafe characters and cannot be passed to the OAuth validator")));
+ return false;
+ }
+
+ return true;
+}
diff --git a/src/backend/libpq/auth-sasl.c b/src/backend/libpq/auth-sasl.c
index 6cfd90fa21..f6c49a4de5 100644
--- a/src/backend/libpq/auth-sasl.c
+++ b/src/backend/libpq/auth-sasl.c
@@ -20,14 +20,6 @@
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
-/*
- * Maximum accepted size of SASL messages.
- *
- * The messages that the server or libpq generate are much smaller than this,
- * but have some headroom.
- */
-#define PG_MAX_SASL_MESSAGE_LENGTH 1024
-
/*
* Perform a SASL exchange with a libpq client, using a specific mechanism
* implementation.
@@ -103,7 +95,7 @@ CheckSASLAuth(const pg_be_sasl_mech *mech, Port *port, char *shadow_pass,
/* Get the actual SASL message */
initStringInfo(&buf);
- if (pq_getmessage(&buf, PG_MAX_SASL_MESSAGE_LENGTH))
+ if (pq_getmessage(&buf, mech->max_message_length))
{
/* EOF - pq_getmessage already logged error */
pfree(buf.data);
diff --git a/src/backend/libpq/auth-scram.c b/src/backend/libpq/auth-scram.c
index 9df8f17837..5bb0388c01 100644
--- a/src/backend/libpq/auth-scram.c
+++ b/src/backend/libpq/auth-scram.c
@@ -117,7 +117,9 @@ static int scram_exchange(void *opaq, const char *input, int inputlen,
const pg_be_sasl_mech pg_be_scram_mech = {
scram_get_mechanisms,
scram_init,
- scram_exchange
+ scram_exchange,
+
+ PG_MAX_SASL_MESSAGE_LENGTH
};
/*
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 8cc23ef7fb..fbcc2c55b4 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -29,6 +29,7 @@
#include "libpq/auth.h"
#include "libpq/crypt.h"
#include "libpq/libpq.h"
+#include "libpq/oauth.h"
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
#include "libpq/scram.h"
@@ -47,7 +48,6 @@
*/
static void auth_failed(Port *port, int status, char *logdetail);
static char *recv_password_packet(Port *port);
-static void set_authn_id(Port *port, const char *id);
/*----------------------------------------------------------------
@@ -205,22 +205,6 @@ static int CheckRADIUSAuth(Port *port);
static int PerformRadiusTransaction(const char *server, const char *secret, const char *portstr, const char *identifier, const char *user_name, const char *passwd);
-/*
- * Maximum accepted size of GSS and SSPI authentication tokens.
- * We also use this as a limit on ordinary password packet lengths.
- *
- * Kerberos tickets are usually quite small, but the TGTs issued by Windows
- * domain controllers include an authorization field known as the Privilege
- * Attribute Certificate (PAC), which contains the user's Windows permissions
- * (group memberships etc.). The PAC is copied into all tickets obtained on
- * the basis of this TGT (even those issued by Unix realms which the Windows
- * realm trusts), and can be several kB in size. The maximum token size
- * accepted by Windows systems is determined by the MaxAuthToken Windows
- * registry setting. Microsoft recommends that it is not set higher than
- * 65535 bytes, so that seems like a reasonable limit for us as well.
- */
-#define PG_MAX_AUTH_TOKEN_LENGTH 65535
-
/*----------------------------------------------------------------
* Global authentication functions
*----------------------------------------------------------------
@@ -309,6 +293,9 @@ auth_failed(Port *port, int status, char *logdetail)
case uaRADIUS:
errstr = gettext_noop("RADIUS authentication failed for user \"%s\"");
break;
+ case uaOAuth:
+ errstr = gettext_noop("OAuth bearer authentication failed for user \"%s\"");
+ break;
default:
errstr = gettext_noop("authentication failed for user \"%s\": invalid authentication method");
break;
@@ -343,7 +330,7 @@ auth_failed(Port *port, int status, char *logdetail)
* lifetime of the Port, so it is safe to pass a string that is managed by an
* external library.
*/
-static void
+void
set_authn_id(Port *port, const char *id)
{
Assert(id);
@@ -628,6 +615,9 @@ ClientAuthentication(Port *port)
case uaTrust:
status = STATUS_OK;
break;
+ case uaOAuth:
+ status = CheckSASLAuth(&pg_be_oauth_mech, port, NULL, NULL);
+ break;
}
if ((status == STATUS_OK && port->hba->clientcert == clientCertFull)
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index 3be8778d21..98147700dd 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -134,7 +134,8 @@ static const char *const UserAuthName[] =
"ldap",
"cert",
"radius",
- "peer"
+ "peer",
+ "oauth",
};
@@ -1399,6 +1400,8 @@ parse_hba_line(TokenizedLine *tok_line, int elevel)
#endif
else if (strcmp(token->string, "radius") == 0)
parsedline->auth_method = uaRADIUS;
+ else if (strcmp(token->string, "oauth") == 0)
+ parsedline->auth_method = uaOAuth;
else
{
ereport(elevel,
@@ -1713,8 +1716,9 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->auth_method != uaPeer &&
hbaline->auth_method != uaGSS &&
hbaline->auth_method != uaSSPI &&
+ hbaline->auth_method != uaOAuth &&
hbaline->auth_method != uaCert)
- INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, and cert"));
+ INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, oauth, and cert"));
hbaline->usermap = pstrdup(val);
}
else if (strcmp(name, "clientcert") == 0)
@@ -2098,6 +2102,27 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->radiusidentifiers = parsed_identifiers;
hbaline->radiusidentifiers_s = pstrdup(val);
}
+ else if (strcmp(name, "issuer") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("issuer", gettext_noop("oauth"));
+ hbaline->oauth_issuer = pstrdup(val);
+ }
+ else if (strcmp(name, "scope") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("scope", gettext_noop("oauth"));
+ hbaline->oauth_scope = pstrdup(val);
+ }
+ else if (strcmp(name, "trust_validator_authz") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("trust_validator_authz", gettext_noop("oauth"));
+ if (strcmp(val, "1") == 0)
+ hbaline->oauth_skip_usermap = true;
+ else
+ hbaline->oauth_skip_usermap = false;
+ }
else
{
ereport(elevel,
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 467b0fd6fe..2b42862f71 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -56,6 +56,7 @@
#include "libpq/auth.h"
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
+#include "libpq/oauth.h"
#include "miscadmin.h"
#include "optimizer/cost.h"
#include "optimizer/geqo.h"
@@ -4594,6 +4595,17 @@ static struct config_string ConfigureNamesString[] =
check_backtrace_functions, assign_backtrace_functions, NULL
},
+ {
+ {"oauth_validator_command", PGC_SIGHUP, CONN_AUTH_AUTH,
+ gettext_noop("Command to validate OAuth v2 bearer tokens."),
+ NULL,
+ GUC_SUPERUSER_ONLY
+ },
+ &oauth_validator_command,
+ "",
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, NULL, NULL, NULL, NULL
diff --git a/src/include/libpq/auth.h b/src/include/libpq/auth.h
index 3d6734f253..1c77dcb0c1 100644
--- a/src/include/libpq/auth.h
+++ b/src/include/libpq/auth.h
@@ -16,6 +16,22 @@
#include "libpq/libpq-be.h"
+/*
+ * Maximum accepted size of GSS and SSPI authentication tokens.
+ * We also use this as a limit on ordinary password packet lengths.
+ *
+ * Kerberos tickets are usually quite small, but the TGTs issued by Windows
+ * domain controllers include an authorization field known as the Privilege
+ * Attribute Certificate (PAC), which contains the user's Windows permissions
+ * (group memberships etc.). The PAC is copied into all tickets obtained on
+ * the basis of this TGT (even those issued by Unix realms which the Windows
+ * realm trusts), and can be several kB in size. The maximum token size
+ * accepted by Windows systems is determined by the MaxAuthToken Windows
+ * registry setting. Microsoft recommends that it is not set higher than
+ * 65535 bytes, so that seems like a reasonable limit for us as well.
+ */
+#define PG_MAX_AUTH_TOKEN_LENGTH 65535
+
extern char *pg_krb_server_keyfile;
extern bool pg_krb_caseins_users;
extern char *pg_krb_realm;
@@ -23,6 +39,7 @@ extern char *pg_krb_realm;
extern void ClientAuthentication(Port *port);
extern void sendAuthRequest(Port *port, AuthRequest areq, const char *extradata,
int extralen);
+extern void set_authn_id(Port *port, const char *id);
/* Hook for plugins to get control in ClientAuthentication() */
typedef void (*ClientAuthentication_hook_type) (Port *, int);
diff --git a/src/include/libpq/hba.h b/src/include/libpq/hba.h
index 8d9f3821b1..441dd5623e 100644
--- a/src/include/libpq/hba.h
+++ b/src/include/libpq/hba.h
@@ -38,8 +38,9 @@ typedef enum UserAuth
uaLDAP,
uaCert,
uaRADIUS,
- uaPeer
-#define USER_AUTH_LAST uaPeer /* Must be last value of this enum */
+ uaPeer,
+ uaOAuth
+#define USER_AUTH_LAST uaOAuth /* Must be last value of this enum */
} UserAuth;
/*
@@ -120,6 +121,9 @@ typedef struct HbaLine
char *radiusidentifiers_s;
List *radiusports;
char *radiusports_s;
+ char *oauth_issuer;
+ char *oauth_scope;
+ bool oauth_skip_usermap;
} HbaLine;
typedef struct IdentLine
diff --git a/src/include/libpq/oauth.h b/src/include/libpq/oauth.h
new file mode 100644
index 0000000000..870e426af1
--- /dev/null
+++ b/src/include/libpq/oauth.h
@@ -0,0 +1,24 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth.h
+ * Interface to libpq/auth-oauth.c
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/libpq/oauth.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_OAUTH_H
+#define PG_OAUTH_H
+
+#include "libpq/libpq-be.h"
+#include "libpq/sasl.h"
+
+extern char *oauth_validator_command;
+
+/* Implementation */
+extern const pg_be_sasl_mech pg_be_oauth_mech;
+
+#endif /* PG_OAUTH_H */
diff --git a/src/include/libpq/sasl.h b/src/include/libpq/sasl.h
index 4c611bab6b..c0a88430d5 100644
--- a/src/include/libpq/sasl.h
+++ b/src/include/libpq/sasl.h
@@ -26,6 +26,14 @@
#define PG_SASL_EXCHANGE_SUCCESS 1
#define PG_SASL_EXCHANGE_FAILURE 2
+/*
+ * Maximum accepted size of SASL messages.
+ *
+ * The messages that the server or libpq generate are much smaller than this,
+ * but have some headroom.
+ */
+#define PG_MAX_SASL_MESSAGE_LENGTH 1024
+
/*
* Backend SASL mechanism callbacks.
*
@@ -127,6 +135,9 @@ typedef struct pg_be_sasl_mech
const char *input, int inputlen,
char **output, int *outputlen,
char **logdetail);
+
+ /* The maximum size allowed for client SASLResponses. */
+ int max_message_length;
} pg_be_sasl_mech;
/* Common implementation for auth.c */
--
2.25.1
v2-0004-Add-a-very-simple-authn_id-extension.patch (text/x-patch, 2.8K) download
From 7c4175f9ad87141d40dd44d6c9fe9312ce8e5b88 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 18 May 2021 15:01:29 -0700
Subject: [PATCH v2 4/5] Add a very simple authn_id extension
...for retrieving the authn_id from the server in tests.
---
contrib/authn_id/Makefile | 19 +++++++++++++++++++
contrib/authn_id/authn_id--1.0.sql | 8 ++++++++
contrib/authn_id/authn_id.c | 28 ++++++++++++++++++++++++++++
contrib/authn_id/authn_id.control | 5 +++++
4 files changed, 60 insertions(+)
create mode 100644 contrib/authn_id/Makefile
create mode 100644 contrib/authn_id/authn_id--1.0.sql
create mode 100644 contrib/authn_id/authn_id.c
create mode 100644 contrib/authn_id/authn_id.control
diff --git a/contrib/authn_id/Makefile b/contrib/authn_id/Makefile
new file mode 100644
index 0000000000..46026358e0
--- /dev/null
+++ b/contrib/authn_id/Makefile
@@ -0,0 +1,19 @@
+# contrib/authn_id/Makefile
+
+MODULE_big = authn_id
+OBJS = authn_id.o
+
+EXTENSION = authn_id
+DATA = authn_id--1.0.sql
+PGFILEDESC = "authn_id - information about the authenticated user"
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/authn_id
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/authn_id/authn_id--1.0.sql b/contrib/authn_id/authn_id--1.0.sql
new file mode 100644
index 0000000000..af2a4d3991
--- /dev/null
+++ b/contrib/authn_id/authn_id--1.0.sql
@@ -0,0 +1,8 @@
+/* contrib/authn_id/authn_id--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION authn_id" to load this file. \quit
+
+CREATE FUNCTION authn_id() RETURNS text
+AS 'MODULE_PATHNAME', 'authn_id'
+LANGUAGE C IMMUTABLE;
diff --git a/contrib/authn_id/authn_id.c b/contrib/authn_id/authn_id.c
new file mode 100644
index 0000000000..0fecac36a8
--- /dev/null
+++ b/contrib/authn_id/authn_id.c
@@ -0,0 +1,28 @@
+/*
+ * Extension to expose the current user's authn_id.
+ *
+ * contrib/authn_id/authn_id.c
+ */
+
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "libpq/libpq-be.h"
+#include "miscadmin.h"
+#include "utils/builtins.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(authn_id);
+
+/*
+ * Returns the current user's authenticated identity.
+ */
+Datum
+authn_id(PG_FUNCTION_ARGS)
+{
+ if (!MyProcPort->authn_id)
+ PG_RETURN_NULL();
+
+ PG_RETURN_TEXT_P(cstring_to_text(MyProcPort->authn_id));
+}
diff --git a/contrib/authn_id/authn_id.control b/contrib/authn_id/authn_id.control
new file mode 100644
index 0000000000..e0f9e06bed
--- /dev/null
+++ b/contrib/authn_id/authn_id.control
@@ -0,0 +1,5 @@
+# authn_id extension
+comment = 'current user identity'
+default_version = '1.0'
+module_pathname = '$libdir/authn_id'
+relocatable = true
--
2.25.1
v2-0005-Add-pytest-suite-for-OAuth.patch (text/x-patch, 131.6K) download
From 0281635f35a44e0fdfd4369423f98ebe5b467ce3 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Fri, 4 Jun 2021 09:06:38 -0700
Subject: [PATCH v2 5/5] Add pytest suite for OAuth
Requires Python 3; on the first run of `make installcheck` the
dependencies will be installed into ./venv for you. See the README for
more details.
---
src/test/python/.gitignore | 2 +
src/test/python/Makefile | 38 +
src/test/python/README | 54 ++
src/test/python/client/__init__.py | 0
src/test/python/client/conftest.py | 126 +++
src/test/python/client/test_client.py | 180 ++++
src/test/python/client/test_oauth.py | 936 ++++++++++++++++++
src/test/python/pq3.py | 727 ++++++++++++++
src/test/python/pytest.ini | 4 +
src/test/python/requirements.txt | 7 +
src/test/python/server/__init__.py | 0
src/test/python/server/conftest.py | 45 +
src/test/python/server/test_oauth.py | 1012 ++++++++++++++++++++
src/test/python/server/test_server.py | 21 +
src/test/python/server/validate_bearer.py | 101 ++
src/test/python/server/validate_reflect.py | 34 +
src/test/python/test_internals.py | 138 +++
src/test/python/test_pq3.py | 558 +++++++++++
src/test/python/tls.py | 195 ++++
19 files changed, 4178 insertions(+)
create mode 100644 src/test/python/.gitignore
create mode 100644 src/test/python/Makefile
create mode 100644 src/test/python/README
create mode 100644 src/test/python/client/__init__.py
create mode 100644 src/test/python/client/conftest.py
create mode 100644 src/test/python/client/test_client.py
create mode 100644 src/test/python/client/test_oauth.py
create mode 100644 src/test/python/pq3.py
create mode 100644 src/test/python/pytest.ini
create mode 100644 src/test/python/requirements.txt
create mode 100644 src/test/python/server/__init__.py
create mode 100644 src/test/python/server/conftest.py
create mode 100644 src/test/python/server/test_oauth.py
create mode 100644 src/test/python/server/test_server.py
create mode 100755 src/test/python/server/validate_bearer.py
create mode 100755 src/test/python/server/validate_reflect.py
create mode 100644 src/test/python/test_internals.py
create mode 100644 src/test/python/test_pq3.py
create mode 100644 src/test/python/tls.py
diff --git a/src/test/python/.gitignore b/src/test/python/.gitignore
new file mode 100644
index 0000000000..0e8f027b2e
--- /dev/null
+++ b/src/test/python/.gitignore
@@ -0,0 +1,2 @@
+__pycache__/
+/venv/
diff --git a/src/test/python/Makefile b/src/test/python/Makefile
new file mode 100644
index 0000000000..b0695b6287
--- /dev/null
+++ b/src/test/python/Makefile
@@ -0,0 +1,38 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+# Only Python 3 is supported, but if it's named something different on your
+# system you can override it with the PYTHON3 variable.
+PYTHON3 := python3
+
+# All dependencies are placed into this directory. The default is .gitignored
+# for you, but you can override it if you'd like.
+VENV := ./venv
+
+override VBIN := $(VENV)/bin
+override PIP := $(VBIN)/pip
+override PYTEST := $(VBIN)/py.test
+override ISORT := $(VBIN)/isort
+override BLACK := $(VBIN)/black
+
+.PHONY: installcheck indent
+
+installcheck: $(PYTEST)
+ $(PYTEST) -v -rs
+
+indent: $(ISORT) $(BLACK)
+ $(ISORT) --profile black *.py client/*.py server/*.py
+ $(BLACK) *.py client/*.py server/*.py
+
+$(PYTEST) $(ISORT) $(BLACK) &: requirements.txt | $(PIP)
+ $(PIP) install --force-reinstall -r $<
+
+$(PIP):
+ $(PYTHON3) -m venv $(VENV)
+
+# A convenience recipe to rebuild psycopg2 against the local libpq.
+.PHONY: rebuild-psycopg2
+rebuild-psycopg2: | $(PIP)
+ $(PIP) install --force-reinstall --no-binary :all: $(shell grep psycopg2 requirements.txt)
diff --git a/src/test/python/README b/src/test/python/README
new file mode 100644
index 0000000000..0bda582c4b
--- /dev/null
+++ b/src/test/python/README
@@ -0,0 +1,54 @@
+A test suite for exercising both the libpq client and the server backend at the
+protocol level, based on pytest and Construct.
+
+The test suite currently assumes that the standard PG* environment variables
+point to the database under test and are sufficient to log in a superuser on
+that system. In other words, a bare `psql` needs to Just Work before the test
+suite can do its thing. For a newly built dev cluster, typically all that I need
+to do is a
+
+ export PGDATABASE=postgres
+
+but you can adjust as needed for your setup.
+
+## Requirements
+
+A supported version (3.6+) of Python.
+
+The first run of
+
+ make installcheck
+
+will install a local virtual environment and all needed dependencies. During
+development, if libpq changes incompatibly, you can issue
+
+ $ make rebuild-psycopg2
+
+to force a rebuild of the client library.
+
+## Hacking
+
+The code style is enforced by a _very_ opinionated autoformatter. Running the
+
+ make indent
+
+recipe will invoke it for you automatically. Don't fight the tool; part of the
+zen is in knowing that if the formatter makes your code ugly, there's probably a
+cleaner way to write your code.
+
+## Advanced Usage
+
+The Makefile is there for convenience, but you don't have to use it. Activate
+the virtualenv to be able to use pytest directly:
+
+ $ source venv/bin/activate
+ $ py.test -k oauth
+ ...
+ $ py.test ./server/test_server.py
+ ...
+ $ deactivate # puts the PATH et al back the way it was before
+
+To make quick smoke tests possible, slow tests have been marked explicitly. You
+can skip them by saying e.g.
+
+ $ py.test -m 'not slow'
diff --git a/src/test/python/client/__init__.py b/src/test/python/client/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/src/test/python/client/conftest.py b/src/test/python/client/conftest.py
new file mode 100644
index 0000000000..f38da7a138
--- /dev/null
+++ b/src/test/python/client/conftest.py
@@ -0,0 +1,126 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import socket
+import sys
+import threading
+
+import psycopg2
+import pytest
+
+import pq3
+
+BLOCKING_TIMEOUT = 2 # the number of seconds to wait for blocking calls
+
+
+@pytest.fixture
+def server_socket(unused_tcp_port_factory):
+ """
+ Returns a listening socket bound to an ephemeral port.
+ """
+ with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+ s.bind(("127.0.0.1", unused_tcp_port_factory()))
+ s.listen(1)
+ s.settimeout(BLOCKING_TIMEOUT)
+ yield s
+
+
+class ClientHandshake(threading.Thread):
+ """
+ A thread that connects to a local Postgres server using psycopg2. Once the
+ opening handshake completes, the connection will be immediately closed.
+ """
+
+ def __init__(self, *, port, **kwargs):
+ super().__init__()
+
+ kwargs["port"] = port
+ self._kwargs = kwargs
+
+ self.exception = None
+
+ def run(self):
+ try:
+ conn = psycopg2.connect(host="127.0.0.1", **self._kwargs)
+ conn.close()
+ except Exception as e:
+ self.exception = e
+
+ def check_completed(self, timeout=BLOCKING_TIMEOUT):
+ """
+ Joins the client thread. Raises an exception if the thread could not be
+ joined, or if it threw an exception itself. (The exception will be
+ cleared, so future calls to check_completed will succeed.)
+ """
+ self.join(timeout)
+
+ if self.is_alive():
+ raise TimeoutError("client thread did not handshake within the timeout")
+ elif self.exception:
+ e = self.exception
+ self.exception = None
+ raise e
+
+
+@pytest.fixture
+def accept(server_socket):
+ """
+ Returns a factory function that, when called, returns a pair (sock, client)
+ where sock is a server socket that has accepted a connection from client,
+ and client is an instance of ClientHandshake. Clients will complete their
+ handshakes and cleanly disconnect.
+
+ The default connstring options may be extended or overridden by passing
+ arbitrary keyword arguments. Keep in mind that you generally should not
+ override the host or port, since they point to the local test server.
+
+ For situations where a client needs to connect more than once to complete a
+ handshake, the accept function may be called more than once. (The client
+ returned for subsequent calls will always be the same client that was
+ returned for the first call.)
+
+ Tests must either complete the handshake so that the client thread can be
+ automatically joined during teardown, or else call client.check_completed()
+ and manually handle any expected errors.
+ """
+ _, port = server_socket.getsockname()
+
+ client = None
+ default_opts = dict(
+ port=port,
+ user=pq3.pguser(),
+ sslmode="disable",
+ )
+
+ def factory(**kwargs):
+ nonlocal client
+
+ if client is None:
+ opts = dict(default_opts)
+ opts.update(kwargs)
+
+ # The server_socket is already listening, so the client thread can
+ # be safely started; it'll block on the connection until we accept.
+ client = ClientHandshake(**opts)
+ client.start()
+
+ sock, _ = server_socket.accept()
+ return sock, client
+
+ yield factory
+ client.check_completed()
+
+
+@pytest.fixture
+def conn(accept):
+ """
+ Returns an accepted, wrapped pq3 connection to a psycopg2 client. The socket
+ will be closed when the test finishes, and the client will be checked for a
+ cleanly completed handshake.
+ """
+ sock, client = accept()
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ yield conn
diff --git a/src/test/python/client/test_client.py b/src/test/python/client/test_client.py
new file mode 100644
index 0000000000..c4c946fda4
--- /dev/null
+++ b/src/test/python/client/test_client.py
@@ -0,0 +1,180 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import sys
+
+import psycopg2
+import pytest
+from cryptography.hazmat.primitives import hashes, hmac
+
+import pq3
+
+
+def finish_handshake(conn):
+ """
+ Sends the AuthenticationOK message and the standard opening salvo of server
+ messages, then asserts that the client immediately sends a Terminate message
+ to close the connection cleanly.
+ """
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.OK)
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"client_encoding", value=b"UTF-8")
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"DateStyle", value=b"ISO, MDY")
+ pq3.send(conn, pq3.types.ReadyForQuery, status=b"I")
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.Terminate
+
+
+def test_handshake(conn):
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ finish_handshake(conn)
+
+
+def test_aborted_connection(accept):
+ """
+ Make sure the client correctly reports an early close during handshakes.
+ """
+ sock, client = accept()
+ sock.close()
+
+ expected = "server closed the connection unexpectedly"
+ with pytest.raises(psycopg2.OperationalError, match=expected):
+ client.check_completed()
+
+
+#
+# SCRAM-SHA-256 (see RFC 5802: https://tools.ietf.org/html/rfc5802)
+#
+
+
+@pytest.fixture
+def password():
+ """
+ Returns a password for use by both client and server.
+ """
+ # TODO: parameterize this with passwords that require SASLprep.
+ return "secret"
+
+
+@pytest.fixture
+def pwconn(accept, password):
+ """
+ Like the conn fixture, but uses a password in the connection.
+ """
+ sock, client = accept(password=password)
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ yield conn
+
+
+def sha256(data):
+ """The H(str) function from Section 2.2."""
+ digest = hashes.Hash(hashes.SHA256())
+ digest.update(data)
+ return digest.finalize()
+
+
+def hmac_256(key, data):
+ """The HMAC(key, str) function from Section 2.2."""
+ h = hmac.HMAC(key, hashes.SHA256())
+ h.update(data)
+ return h.finalize()
+
+
+def xor(a, b):
+ """The XOR operation from Section 2.2."""
+ res = bytearray(a)
+ for i, byte in enumerate(b):
+ res[i] ^= byte
+ return bytes(res)
+
+
+def h_i(data, salt, i):
+ """The Hi(str, salt, i) function from Section 2.2."""
+ assert i > 0
+
+ acc = hmac_256(data, salt + b"\x00\x00\x00\x01")
+ last = acc
+ i -= 1
+
+ while i:
+ u = hmac_256(data, last)
+ acc = xor(acc, u)
+
+ last = u
+ i -= 1
+
+ return acc
+
+
+def test_scram(pwconn, password):
+ startup = pq3.recv1(pwconn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ pwconn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"SCRAM-SHA-256", b""],
+ )
+
+ # Get the client-first-message.
+ pkt = pq3.recv1(pwconn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ initial = pq3.SASLInitialResponse.parse(pkt.payload)
+ assert initial.name == b"SCRAM-SHA-256"
+
+ c_bind, authzid, c_name, c_nonce = initial.data.split(b",")
+ assert c_bind == b"n" # no channel bindings on a plaintext connection
+ assert authzid == b"" # we don't support authzid currently
+ assert c_name == b"n=" # libpq doesn't honor the GS2 username
+ assert c_nonce.startswith(b"r=")
+
+ # Send the server-first-message.
+ salt = b"12345"
+ iterations = 2
+
+ s_nonce = c_nonce + b"somenonce"
+ s_salt = b"s=" + base64.b64encode(salt)
+ s_iterations = b"i=%d" % iterations
+
+ msg = b",".join([s_nonce, s_salt, s_iterations])
+ pq3.send(pwconn, pq3.types.AuthnRequest, type=pq3.authn.SASLContinue, body=msg)
+
+ # Get the client-final-message.
+ pkt = pq3.recv1(pwconn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ c_bind_final, c_nonce_final, c_proof = pkt.payload.split(b",")
+ assert c_bind_final == b"c=" + base64.b64encode(c_bind + b"," + authzid + b",")
+ assert c_nonce_final == s_nonce
+
+ # Calculate what the client proof should be.
+ salted_password = h_i(password.encode("ascii"), salt, iterations)
+ client_key = hmac_256(salted_password, b"Client Key")
+ stored_key = sha256(client_key)
+
+ auth_message = b",".join(
+ [c_name, c_nonce, s_nonce, s_salt, s_iterations, c_bind_final, c_nonce_final]
+ )
+ client_signature = hmac_256(stored_key, auth_message)
+ client_proof = xor(client_key, client_signature)
+
+ expected = b"p=" + base64.b64encode(client_proof)
+ assert c_proof == expected
+
+ # Send the correct server signature.
+ server_key = hmac_256(salted_password, b"Server Key")
+ server_signature = hmac_256(server_key, auth_message)
+
+ s_verify = b"v=" + base64.b64encode(server_signature)
+ pq3.send(pwconn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal, body=s_verify)
+
+ # Done!
+ finish_handshake(pwconn)
diff --git a/src/test/python/client/test_oauth.py b/src/test/python/client/test_oauth.py
new file mode 100644
index 0000000000..a754a9c0b6
--- /dev/null
+++ b/src/test/python/client/test_oauth.py
@@ -0,0 +1,936 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import http.server
+import json
+import secrets
+import sys
+import threading
+import time
+import urllib.parse
+
+import psycopg2
+import pytest
+
+import pq3
+
+from .conftest import BLOCKING_TIMEOUT
+
+
+def finish_handshake(conn):
+ """
+ Sends the AuthenticationOK message and the standard opening salvo of server
+ messages, then asserts that the client immediately sends a Terminate message
+ to close the connection cleanly.
+ """
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.OK)
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"client_encoding", value=b"UTF-8")
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"DateStyle", value=b"ISO, MDY")
+ pq3.send(conn, pq3.types.ReadyForQuery, status=b"I")
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.Terminate
+
+
+#
+# OAUTHBEARER (see RFC 7628: https://tools.ietf.org/html/rfc7628)
+#
+
+
+def start_oauth_handshake(conn):
+ """
+ Negotiates an OAUTHBEARER SASL challenge. Returns the client's initial
+ response data.
+ """
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn, pq3.types.AuthnRequest, type=pq3.authn.SASL, body=[b"OAUTHBEARER", b""]
+ )
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ initial = pq3.SASLInitialResponse.parse(pkt.payload)
+ assert initial.name == b"OAUTHBEARER"
+
+ return initial.data
+
+
+def get_auth_value(initial):
+ """
+ Finds the auth value (e.g. "Bearer somedata..." in the client's initial SASL
+ response.
+ """
+ kvpairs = initial.split(b"\x01")
+ assert kvpairs[0] == b"n,," # no channel binding or authzid
+ assert kvpairs[2] == b"" # ends with an empty kvpair
+ assert kvpairs[3] == b"" # ...and there's nothing after it
+ assert len(kvpairs) == 4
+
+ key, value = kvpairs[1].split(b"=", 2)
+ assert key == b"auth"
+
+ return value
+
+
+def xtest_oauth_success(conn): # TODO
+ initial = start_oauth_handshake(conn)
+
+ auth = get_auth_value(initial)
+ assert auth.startswith(b"Bearer ")
+
+ # Accept the token. TODO actually validate
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+class OpenIDProvider(threading.Thread):
+ """
+ A thread that runs a mock OpenID provider server.
+ """
+
+ def __init__(self, *, port):
+ super().__init__()
+
+ self.exception = None
+
+ addr = ("", port)
+ self.server = self._Server(addr, self._Handler)
+
+ # TODO: allow HTTPS only, somehow
+ oauth = self._OAuthState()
+ oauth.host = f"localhost:{port}"
+ oauth.issuer = f"http://localhost:{port}"
+
+ # The following endpoints are required to be advertised by providers,
+ # even though our chosen client implementation does not actually make
+ # use of them.
+ oauth.register_endpoint(
+ "authorization_endpoint", "POST", "/authorize", self._authorization_handler
+ )
+ oauth.register_endpoint("jwks_uri", "GET", "/keys", self._jwks_handler)
+
+ self.server.oauth = oauth
+
+ def run(self):
+ try:
+ self.server.serve_forever()
+ except Exception as e:
+ self.exception = e
+
+ def stop(self, timeout=BLOCKING_TIMEOUT):
+ """
+ Shuts down the server and joins its thread. Raises an exception if the
+ thread could not be joined, or if it threw an exception itself. Must
+ only be called once, after start().
+ """
+ self.server.shutdown()
+ self.join(timeout)
+
+ if self.is_alive():
+ raise TimeoutError("client thread did not handshake within the timeout")
+ elif self.exception:
+ e = self.exception
+ raise e
+
+ class _OAuthState(object):
+ def __init__(self):
+ self.endpoint_paths = {}
+ self._endpoints = {}
+
+ def register_endpoint(self, name, method, path, func):
+ if method not in self._endpoints:
+ self._endpoints[method] = {}
+
+ self._endpoints[method][path] = func
+ self.endpoint_paths[name] = path
+
+ def endpoint(self, method, path):
+ if method not in self._endpoints:
+ return None
+
+ return self._endpoints[method].get(path)
+
+ class _Server(http.server.HTTPServer):
+ def handle_error(self, request, addr):
+ self.shutdown_request(request)
+ raise
+
+ @staticmethod
+ def _jwks_handler(headers, params):
+ return 200, {"keys": []}
+
+ @staticmethod
+ def _authorization_handler(headers, params):
+ # We don't actually want this to be called during these tests -- we
+ # should be using the device authorization endpoint instead.
+ assert (
+ False
+ ), "authorization handler called instead of device authorization handler"
+
+ class _Handler(http.server.BaseHTTPRequestHandler):
+ timeout = BLOCKING_TIMEOUT
+
+ def _discovery_handler(self, headers, params):
+ oauth = self.server.oauth
+
+ doc = {
+ "issuer": oauth.issuer,
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ }
+
+ for name, path in oauth.endpoint_paths.items():
+ doc[name] = oauth.issuer + path
+
+ return 200, doc
+
+ def _handle(self, *, params=None, handler=None):
+ oauth = self.server.oauth
+ assert self.headers["Host"] == oauth.host
+
+ if handler is None:
+ handler = oauth.endpoint(self.command, self.path)
+ assert (
+ handler is not None
+ ), f"no registered endpoint for {self.command} {self.path}"
+
+ code, resp = handler(self.headers, params)
+
+ self.send_response(code)
+ self.send_header("Content-Type", "application/json")
+ self.end_headers()
+
+ resp = json.dumps(resp)
+ resp = resp.encode("utf-8")
+ self.wfile.write(resp)
+
+ self.close_connection = True
+
+ def do_GET(self):
+ if self.path == "/.well-known/openid-configuration":
+ self._handle(handler=self._discovery_handler)
+ return
+
+ self._handle()
+
+ def _request_body(self):
+ length = self.headers["Content-Length"]
+
+ # Handle only an explicit content-length.
+ assert length is not None
+ length = int(length)
+
+ return self.rfile.read(length).decode("utf-8")
+
+ def do_POST(self):
+ assert self.headers["Content-Type"] == "application/x-www-form-urlencoded"
+
+ body = self._request_body()
+ params = urllib.parse.parse_qs(body)
+
+ self._handle(params=params)
+
+
+@pytest.fixture
+def openid_provider(unused_tcp_port_factory):
+ """
+ A fixture that returns the OAuth state of a running OpenID provider server. The
+ server will be stopped when the fixture is torn down.
+ """
+ thread = OpenIDProvider(port=unused_tcp_port_factory())
+ thread.start()
+
+ try:
+ yield thread.server.oauth
+ finally:
+ thread.stop()
+
+
+@pytest.mark.parametrize("secret", [None, "", "hunter2"])
+@pytest.mark.parametrize("scope", [None, "", "openid email"])
+@pytest.mark.parametrize("retries", [0, 1])
+def test_oauth_with_explicit_issuer(
+ capfd, accept, openid_provider, retries, scope, secret
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ oauth_client_secret=secret,
+ oauth_scope=scope,
+ )
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+ verification_url = "https://example.com/device"
+
+ access_token = secrets.token_urlsafe()
+
+ def check_client_authn(headers, params):
+ if not secret:
+ assert params["client_id"] == [client_id]
+ return
+
+ # Require the client to use Basic authn; request-body credentials are
+ # NOT RECOMMENDED (RFC 6749, Sec. 2.3.1).
+ assert "Authorization" in headers
+
+ method, creds = headers["Authorization"].split()
+ assert method == "Basic"
+
+ expected = f"{client_id}:{secret}"
+ assert base64.b64decode(creds) == expected.encode("ascii")
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ check_client_authn(headers, params)
+
+ if scope:
+ assert params["scope"] == [scope]
+ else:
+ assert "scope" not in params
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": verification_url,
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ attempts = 0
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ check_client_authn(headers, params)
+
+ assert params["grant_type"] == ["urn:ietf:params:oauth:grant-type:device_code"]
+ assert params["device_code"] == [device_code]
+
+ now = time.monotonic()
+
+ with retry_lock:
+ nonlocal attempts
+
+ # If the test wants to force the client to retry, return an
+ # authorization_pending response and decrement the retry count.
+ if attempts < retries:
+ attempts += 1
+ return 400, {"error": "authorization_pending"}
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+ if retries:
+ # Finally, make sure that the client prompted the user with the expected
+ # authorization URL and user code.
+ expected = f"Visit {verification_url} and enter the code: {user_code}"
+ _, stderr = capfd.readouterr()
+ assert expected in stderr
+
+
+def test_oauth_requires_client_id(accept, openid_provider):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ # Do not set a client ID; this should cause a client error after the
+ # server asks for OAUTHBEARER and the client tries to contact the
+ # issuer.
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake.
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"OAUTHBEARER", b""],
+ )
+
+ # The client should disconnect at this point.
+ assert not conn.read()
+
+ expected_error = "no oauth_client_id is set"
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.slow
+@pytest.mark.parametrize("error_code", ["authorization_pending", "slow_down"])
+@pytest.mark.parametrize("retries", [1, 2])
+def test_oauth_retry_interval(accept, openid_provider, retries, error_code):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id="some-id",
+ )
+
+ expected_retry_interval = 1
+ access_token = secrets.token_urlsafe()
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ resp = {
+ "device_code": "my-device-code",
+ "user_code": "my-user-code",
+ "interval": expected_retry_interval,
+ "verification_uri": "https://example.com",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ attempts = 0
+ last_retry = None
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ now = time.monotonic()
+
+ with retry_lock:
+ nonlocal attempts, last_retry, expected_retry_interval
+
+ # Make sure the retry interval is being respected by the client.
+ if last_retry is not None:
+ interval = now - last_retry
+ assert interval >= expected_retry_interval
+
+ last_retry = now
+
+ # If the test wants to force the client to retry, return the desired
+ # error response and decrement the retry count.
+ if attempts < retries:
+ attempts += 1
+
+ # A slow_down code requires the client to additionally increase
+ # its interval by five seconds.
+ if error_code == "slow_down":
+ expected_retry_interval += 5
+
+ return 400, {"error": error_code}
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+@pytest.mark.parametrize(
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
+ {
+ "error": "invalid_client",
+ "error_description": "client authentication failed",
+ },
+ r"client authentication failed \(invalid_client\)",
+ id="authentication failure with description",
+ ),
+ pytest.param(
+ {"error": "invalid_request"},
+ r"\(invalid_request\)",
+ id="invalid request without description",
+ ),
+ pytest.param(
+ {},
+ r"failed to obtain device authorization",
+ id="broken error response",
+ ),
+ ],
+)
+def test_oauth_device_authorization_failures(
+ accept, openid_provider, failure_mode, error_pattern
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ )
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ return 400, failure_mode
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert False, "token endpoint was invoked unexpectedly"
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"OAUTHBEARER", b""],
+ )
+
+ # The client should not continue the connection due to the hardcoded
+ # provider failure; we disconnect here.
+
+ # Now make sure the client correctly failed.
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
+ {
+ "error": "expired_token",
+ "error_description": "the device code has expired",
+ },
+ r"the device code has expired \(expired_token\)",
+ id="expired token with description",
+ ),
+ pytest.param(
+ {"error": "access_denied"},
+ r"\(access_denied\)",
+ id="access denied without description",
+ ),
+ pytest.param(
+ {},
+ r"OAuth token retrieval failed",
+ id="broken error response",
+ ),
+ ],
+)
+@pytest.mark.parametrize("retries", [0, 1])
+def test_oauth_token_failures(
+ accept, openid_provider, retries, failure_mode, error_pattern
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ )
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ assert params["client_id"] == [client_id]
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": "https://example.com/device",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ with retry_lock:
+ nonlocal retries
+
+ # If the test wants to force the client to retry, return an
+ # authorization_pending response and decrement the retry count.
+ if retries > 0:
+ retries -= 1
+ return 400, {"error": "authorization_pending"}
+
+ return 400, failure_mode
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"OAUTHBEARER", b""],
+ )
+
+ # The client should not continue the connection due to the hardcoded
+ # provider failure; we disconnect here.
+
+ # Now make sure the client correctly failed.
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize("scope", [None, "openid email"])
+@pytest.mark.parametrize(
+ "base_response",
+ [
+ {"status": "invalid_token"},
+ {"extra_object": {"key": "value"}, "status": "invalid_token"},
+ {"extra_object": {"status": 1}, "status": "invalid_token"},
+ ],
+)
+def test_oauth_discovery(accept, openid_provider, base_response, scope):
+ sock, client = accept(oauth_client_id=secrets.token_hex())
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+ verification_url = "https://example.com/device"
+
+ access_token = secrets.token_urlsafe()
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ if scope:
+ assert params["scope"] == [scope]
+ else:
+ assert "scope" not in params
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": verification_url,
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert params["grant_type"] == ["urn:ietf:params:oauth:grant-type:device_code"]
+ assert params["device_code"] == [device_code]
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # For discovery, the client should send an empty auth header. See
+ # RFC 7628, Sec. 4.3.
+ auth = get_auth_value(initial)
+ assert auth == b""
+
+ # We will fail the first SASL exchange. First return a link to the
+ # discovery document, pointing to the test provider server.
+ resp = dict(base_response)
+
+ discovery_uri = f"{openid_provider.issuer}/.well-known/openid-configuration"
+ resp["openid-configuration"] = discovery_uri
+
+ if scope:
+ resp["scope"] = scope
+
+ resp = json.dumps(resp)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASLContinue,
+ body=resp.encode("ascii"),
+ )
+
+ # Per RFC, the client is required to send a dummy ^A response.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+ assert pkt.payload == b"\x01"
+
+ # Now fail the SASL exchange.
+ pq3.send(
+ conn,
+ pq3.types.ErrorResponse,
+ fields=[
+ b"SFATAL",
+ b"C28000",
+ b"Mdoesn't matter",
+ b"",
+ ],
+ )
+
+ # The client will connect to us a second time, using the parameters we sent
+ # it.
+ sock, _ = accept()
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+@pytest.mark.parametrize(
+ "response,expected_error",
+ [
+ pytest.param(
+ "abcde",
+ 'Token "abcde" is invalid',
+ id="bad JSON: invalid syntax",
+ ),
+ pytest.param(
+ '"abcde"',
+ "top-level element must be an object",
+ id="bad JSON: top-level element is a string",
+ ),
+ pytest.param(
+ "[]",
+ "top-level element must be an object",
+ id="bad JSON: top-level element is an array",
+ ),
+ pytest.param(
+ "{}",
+ "server sent error response without a status",
+ id="bad JSON: no status member",
+ ),
+ pytest.param(
+ '{ "status": null }',
+ 'field "status" must be a string',
+ id="bad JSON: null status member",
+ ),
+ pytest.param(
+ '{ "status": 0 }',
+ 'field "status" must be a string',
+ id="bad JSON: int status member",
+ ),
+ pytest.param(
+ '{ "status": [ "bad" ] }',
+ 'field "status" must be a string',
+ id="bad JSON: array status member",
+ ),
+ pytest.param(
+ '{ "status": { "bad": "bad" } }',
+ 'field "status" must be a string',
+ id="bad JSON: object status member",
+ ),
+ pytest.param(
+ '{ "nested": { "status": "bad" } }',
+ "server sent error response without a status",
+ id="bad JSON: nested status",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token" ',
+ "The input string ended unexpectedly",
+ id="bad JSON: unterminated object",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token" } { }',
+ 'Expected end of input, but found "{"',
+ id="bad JSON: trailing data",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "openid-configuration": 1 }',
+ 'field "openid-configuration" must be a string',
+ id="bad JSON: int openid-configuration member",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "openid-configuration": 1 }',
+ 'field "openid-configuration" must be a string',
+ id="bad JSON: int openid-configuration member",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "scope": 1 }',
+ 'field "scope" must be a string',
+ id="bad JSON: int scope member",
+ ),
+ ],
+)
+def test_oauth_discovery_server_error(accept, response, expected_error):
+ sock, client = accept(oauth_client_id=secrets.token_hex())
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # Fail the SASL exchange with an invalid JSON response.
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASLContinue,
+ body=response.encode("utf-8"),
+ )
+
+ # The client should disconnect, so the socket is closed here. (If
+ # the client doesn't disconnect, it will report a different error
+ # below and the test will fail.)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "sasl_err,resp_type,resp_payload,expected_error",
+ [
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.ErrorResponse,
+ dict(
+ fields=[b"SFATAL", b"C28000", b"Mexpected error message", b""],
+ ),
+ "expected error message",
+ id="standard server error: invalid_request",
+ ),
+ pytest.param(
+ {"status": "invalid_token"},
+ pq3.types.ErrorResponse,
+ dict(
+ fields=[b"SFATAL", b"C28000", b"Mexpected error message", b""],
+ ),
+ "expected error message",
+ id="standard server error: invalid_token without discovery URI",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASLContinue, body=b""),
+ "server sent additional OAuth data",
+ id="broken server: additional challenge after error",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASLFinal),
+ "server sent additional OAuth data",
+ id="broken server: SASL success after error",
+ ),
+ ],
+)
+def test_oauth_server_error(accept, sasl_err, resp_type, resp_payload, expected_error):
+ sock, client = accept()
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ start_oauth_handshake(conn)
+
+ # Ignore the client data. Return an error "challenge".
+ resp = json.dumps(sasl_err)
+ resp = resp.encode("utf-8")
+
+ pq3.send(
+ conn, pq3.types.AuthnRequest, type=pq3.authn.SASLContinue, body=resp
+ )
+
+ # Per RFC, the client is required to send a dummy ^A response.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+ assert pkt.payload == b"\x01"
+
+ # Now fail the SASL exchange (in either a valid way, or an invalid
+ # one, depending on the test).
+ pq3.send(conn, resp_type, **resp_payload)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
diff --git a/src/test/python/pq3.py b/src/test/python/pq3.py
new file mode 100644
index 0000000000..3a22dad0b6
--- /dev/null
+++ b/src/test/python/pq3.py
@@ -0,0 +1,727 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import getpass
+import io
+import os
+import ssl
+import sys
+import textwrap
+
+from construct import *
+
+import tls
+
+
+def protocol(major, minor):
+ """
+ Returns the protocol version, in integer format, corresponding to the given
+ major and minor version numbers.
+ """
+ return (major << 16) | minor
+
+
+# Startup
+
+StringList = GreedyRange(NullTerminated(GreedyBytes))
+
+
+class KeyValueAdapter(Adapter):
+ """
+ Turns a key-value store into a null-terminated list of null-terminated
+ strings, as presented on the wire in the startup packet.
+ """
+
+ def _encode(self, obj, context, path):
+ if isinstance(obj, list):
+ return obj
+
+ l = []
+
+ for k, v in obj.items():
+ if isinstance(k, str):
+ k = k.encode("utf-8")
+ l.append(k)
+
+ if isinstance(v, str):
+ v = v.encode("utf-8")
+ l.append(v)
+
+ l.append(b"")
+ return l
+
+ def _decode(self, obj, context, path):
+ # TODO: turn a list back into a dict
+ return obj
+
+
+KeyValues = KeyValueAdapter(StringList)
+
+_startup_payload = Switch(
+ this.proto,
+ {
+ protocol(3, 0): KeyValues,
+ },
+ default=GreedyBytes,
+)
+
+
+def _default_protocol(this):
+ try:
+ if isinstance(this.payload, (list, dict)):
+ return protocol(3, 0)
+ except AttributeError:
+ pass # no payload passed during build
+
+ return 0
+
+
+def _startup_payload_len(this):
+ """
+ The payload field has a fixed size based on the length of the packet. But
+ if the caller hasn't supplied an explicit length at build time, we have to
+ build the payload to figure out how long it is, which requires us to know
+ the length first... This function exists solely to break the cycle.
+ """
+ assert this._building, "_startup_payload_len() cannot be called during parsing"
+
+ try:
+ payload = this.payload
+ except AttributeError:
+ return 0 # no payload
+
+ if isinstance(payload, bytes):
+ # already serialized; just use the given length
+ return len(payload)
+
+ try:
+ proto = this.proto
+ except AttributeError:
+ proto = _default_protocol(this)
+
+ data = _startup_payload.build(payload, proto=proto)
+ return len(data)
+
+
+Startup = Struct(
+ "len" / Default(Int32sb, lambda this: _startup_payload_len(this) + 8),
+ "proto" / Default(Hex(Int32sb), _default_protocol),
+ "payload" / FixedSized(this.len - 8, Default(_startup_payload, b"")),
+)
+
+# Pq3
+
+# Adapted from construct.core.EnumIntegerString
+class EnumNamedByte:
+ def __init__(self, val, name):
+ self._val = val
+ self._name = name
+
+ def __int__(self):
+ return ord(self._val)
+
+ def __str__(self):
+ return "(enum) %s %r" % (self._name, self._val)
+
+ def __repr__(self):
+ return "EnumNamedByte(%r)" % self._val
+
+ def __eq__(self, other):
+ if isinstance(other, EnumNamedByte):
+ other = other._val
+ if not isinstance(other, bytes):
+ return NotImplemented
+
+ return self._val == other
+
+ def __hash__(self):
+ return hash(self._val)
+
+
+# Adapted from construct.core.Enum
+class ByteEnum(Adapter):
+ def __init__(self, **mapping):
+ super(ByteEnum, self).__init__(Byte)
+ self.namemapping = {k: EnumNamedByte(v, k) for k, v in mapping.items()}
+ self.decmapping = {v: EnumNamedByte(v, k) for k, v in mapping.items()}
+
+ def __getattr__(self, name):
+ if name in self.namemapping:
+ return self.decmapping[self.namemapping[name]]
+ raise AttributeError
+
+ def _decode(self, obj, context, path):
+ b = bytes([obj])
+ try:
+ return self.decmapping[b]
+ except KeyError:
+ return EnumNamedByte(b, "(unknown)")
+
+ def _encode(self, obj, context, path):
+ if isinstance(obj, int):
+ return obj
+ elif isinstance(obj, bytes):
+ return ord(obj)
+ return int(obj)
+
+
+types = ByteEnum(
+ ErrorResponse=b"E",
+ ReadyForQuery=b"Z",
+ Query=b"Q",
+ EmptyQueryResponse=b"I",
+ AuthnRequest=b"R",
+ PasswordMessage=b"p",
+ BackendKeyData=b"K",
+ CommandComplete=b"C",
+ ParameterStatus=b"S",
+ DataRow=b"D",
+ Terminate=b"X",
+)
+
+
+authn = Enum(
+ Int32ub,
+ OK=0,
+ SASL=10,
+ SASLContinue=11,
+ SASLFinal=12,
+)
+
+
+_authn_body = Switch(
+ this.type,
+ {
+ authn.OK: Terminated,
+ authn.SASL: StringList,
+ },
+ default=GreedyBytes,
+)
+
+
+def _data_len(this):
+ assert this._building, "_data_len() cannot be called during parsing"
+
+ if not hasattr(this, "data") or this.data is None:
+ return -1
+
+ return len(this.data)
+
+
+# The protocol reuses the PasswordMessage for several authentication response
+# types, and there's no good way to figure out which is which without keeping
+# state for the entire stream. So this is a separate Construct that can be
+# explicitly parsed/built by code that knows it's needed.
+SASLInitialResponse = Struct(
+ "name" / NullTerminated(GreedyBytes),
+ "len" / Default(Int32sb, lambda this: _data_len(this)),
+ "data"
+ / IfThenElse(
+ # Allow tests to explicitly pass an incorrect length during testing, by
+ # not enforcing a FixedSized during build. (The len calculation above
+ # defaults to the correct size.)
+ this._building,
+ Optional(GreedyBytes),
+ If(this.len != -1, Default(FixedSized(this.len, GreedyBytes), b"")),
+ ),
+ Terminated, # make sure the entire response is consumed
+)
+
+
+_column = FocusedSeq(
+ "data",
+ "len" / Default(Int32sb, lambda this: _data_len(this)),
+ "data" / If(this.len != -1, FixedSized(this.len, GreedyBytes)),
+)
+
+
+_payload_map = {
+ types.ErrorResponse: Struct("fields" / StringList),
+ types.ReadyForQuery: Struct("status" / Bytes(1)),
+ types.Query: Struct("query" / NullTerminated(GreedyBytes)),
+ types.EmptyQueryResponse: Terminated,
+ types.AuthnRequest: Struct("type" / authn, "body" / Default(_authn_body, b"")),
+ types.BackendKeyData: Struct("pid" / Int32ub, "key" / Hex(Int32ub)),
+ types.CommandComplete: Struct("tag" / NullTerminated(GreedyBytes)),
+ types.ParameterStatus: Struct(
+ "name" / NullTerminated(GreedyBytes), "value" / NullTerminated(GreedyBytes)
+ ),
+ types.DataRow: Struct("columns" / Default(PrefixedArray(Int16sb, _column), b"")),
+ types.Terminate: Terminated,
+}
+
+
+_payload = FocusedSeq(
+ "_payload",
+ "_payload"
+ / Switch(
+ this._.type,
+ _payload_map,
+ default=GreedyBytes,
+ ),
+ Terminated, # make sure every payload consumes the entire packet
+)
+
+
+def _payload_len(this):
+ """
+ See _startup_payload_len() for an explanation.
+ """
+ assert this._building, "_payload_len() cannot be called during parsing"
+
+ try:
+ payload = this.payload
+ except AttributeError:
+ return 0 # no payload
+
+ if isinstance(payload, bytes):
+ # already serialized; just use the given length
+ return len(payload)
+
+ data = _payload.build(payload, type=this.type)
+ return len(data)
+
+
+Pq3 = Struct(
+ "type" / types,
+ "len" / Default(Int32ub, lambda this: _payload_len(this) + 4),
+ "payload" / FixedSized(this.len - 4, Default(_payload, b"")),
+)
+
+
+# Environment
+
+
+def pghost():
+ return os.environ.get("PGHOST", default="localhost")
+
+
+def pgport():
+ return int(os.environ.get("PGPORT", default=5432))
+
+
+def pguser():
+ try:
+ return os.environ["PGUSER"]
+ except KeyError:
+ return getpass.getuser()
+
+
+def pgdatabase():
+ return os.environ.get("PGDATABASE", default="postgres")
+
+
+# Connections
+
+
+def _hexdump_translation_map():
+ """
+ For hexdumps. Translates any unprintable or non-ASCII bytes into '.'.
+ """
+ input = bytearray()
+
+ for i in range(128):
+ c = chr(i)
+
+ if not c.isprintable():
+ input += bytes([i])
+
+ input += bytes(range(128, 256))
+
+ return bytes.maketrans(input, b"." * len(input))
+
+
+class _DebugStream(object):
+ """
+ Wraps a file-like object and adds hexdumps of the read and write data. Call
+ end_packet() on a _DebugStream to write the accumulated hexdumps to the
+ output stream, along with the packet that was sent.
+ """
+
+ _translation_map = _hexdump_translation_map()
+
+ def __init__(self, stream, out=sys.stdout):
+ """
+ Creates a new _DebugStream wrapping the given stream (which must have
+ been created by wrap()). All attributes not provided by the _DebugStream
+ are delegated to the wrapped stream. out is the text stream to which
+ hexdumps are written.
+ """
+ self.raw = stream
+ self._out = out
+ self._rbuf = io.BytesIO()
+ self._wbuf = io.BytesIO()
+
+ def __getattr__(self, name):
+ return getattr(self.raw, name)
+
+ def __setattr__(self, name, value):
+ if name in ("raw", "_out", "_rbuf", "_wbuf"):
+ return object.__setattr__(self, name, value)
+
+ setattr(self.raw, name, value)
+
+ def read(self, *args, **kwargs):
+ buf = self.raw.read(*args, **kwargs)
+
+ self._rbuf.write(buf)
+ return buf
+
+ def write(self, b):
+ self._wbuf.write(b)
+ return self.raw.write(b)
+
+ def recv(self, *args):
+ buf = self.raw.recv(*args)
+
+ self._rbuf.write(buf)
+ return buf
+
+ def _flush(self, buf, prefix):
+ width = 16
+ hexwidth = width * 3 - 1
+
+ count = 0
+ buf.seek(0)
+
+ while True:
+ line = buf.read(16)
+
+ if not line:
+ if count:
+ self._out.write("\n") # separate the output block with a newline
+ return
+
+ self._out.write("%s %04X:\t" % (prefix, count))
+ self._out.write("%*s\t" % (-hexwidth, line.hex(" ")))
+ self._out.write(line.translate(self._translation_map).decode("ascii"))
+ self._out.write("\n")
+
+ count += 16
+
+ def print_debug(self, obj, *, prefix=""):
+ contents = ""
+ if obj is not None:
+ contents = str(obj)
+
+ for line in contents.splitlines():
+ self._out.write("%s%s\n" % (prefix, line))
+
+ self._out.write("\n")
+
+ def flush_debug(self, *, prefix=""):
+ self._flush(self._rbuf, prefix + "<")
+ self._rbuf = io.BytesIO()
+
+ self._flush(self._wbuf, prefix + ">")
+ self._wbuf = io.BytesIO()
+
+ def end_packet(self, pkt, *, read=False, prefix="", indent=" "):
+ """
+ Marks the end of a logical "packet" of data. A string representation of
+ pkt will be printed, and the debug buffers will be flushed with an
+ indent. All lines can be optionally prefixed.
+
+ If read is True, the packet representation is written after the debug
+ buffers; otherwise the default of False (meaning write) causes the
+ packet representation to be dumped first. This is meant to capture the
+ logical flow of layer translation.
+ """
+ write = not read
+
+ if write:
+ self.print_debug(pkt, prefix=prefix + "> ")
+
+ self.flush_debug(prefix=prefix + indent)
+
+ if read:
+ self.print_debug(pkt, prefix=prefix + "< ")
+
+
+@contextlib.contextmanager
+def wrap(socket, *, debug_stream=None):
+ """
+ Transforms a raw socket into a connection that can be used for Construct
+ building and parsing. The return value is a context manager and can be used
+ in a with statement.
+ """
+ # It is critical that buffering be disabled here, so that we can still
+ # manipulate the raw socket without desyncing the stream.
+ with socket.makefile("rwb", buffering=0) as sfile:
+ # Expose the original socket's recv() on the SocketIO object we return.
+ def recv(self, *args):
+ return socket.recv(*args)
+
+ sfile.recv = recv.__get__(sfile)
+
+ conn = sfile
+ if debug_stream:
+ conn = _DebugStream(conn, debug_stream)
+
+ try:
+ yield conn
+ finally:
+ if debug_stream:
+ conn.flush_debug(prefix="? ")
+
+
+def _send(stream, cls, obj):
+ debugging = hasattr(stream, "flush_debug")
+ out = io.BytesIO()
+
+ # Ideally we would build directly to the passed stream, but because we need
+ # to reparse the generated output for the debugging case, build to an
+ # intermediate BytesIO and send it instead.
+ cls.build_stream(obj, out)
+ buf = out.getvalue()
+
+ stream.write(buf)
+ if debugging:
+ pkt = cls.parse(buf)
+ stream.end_packet(pkt)
+
+ stream.flush()
+
+
+def send(stream, packet_type, payload_data=None, **payloadkw):
+ """
+ Sends a packet on the given pq3 connection. type is the pq3.types member
+ that should be assigned to the packet. If payload_data is given, it will be
+ used as the packet payload; otherwise the key/value pairs in payloadkw will
+ be the payload contents.
+ """
+ data = payloadkw
+
+ if payload_data is not None:
+ if payloadkw:
+ raise ValueError(
+ "payload_data and payload keywords may not be used simultaneously"
+ )
+
+ data = payload_data
+
+ _send(stream, Pq3, dict(type=packet_type, payload=data))
+
+
+def send_startup(stream, proto=None, **kwargs):
+ """
+ Sends a startup packet on the given pq3 connection. In most cases you should
+ use the handshake functions instead, which will do this for you.
+
+ By default, a protocol version 3 packet will be sent. This can be overridden
+ with the proto parameter.
+ """
+ pkt = {}
+
+ if proto is not None:
+ pkt["proto"] = proto
+ if kwargs:
+ pkt["payload"] = kwargs
+
+ _send(stream, Startup, pkt)
+
+
+def recv1(stream, *, cls=Pq3):
+ """
+ Receives a single pq3 packet from the given stream and returns it.
+ """
+ resp = cls.parse_stream(stream)
+
+ debugging = hasattr(stream, "flush_debug")
+ if debugging:
+ stream.end_packet(resp, read=True)
+
+ return resp
+
+
+def handshake(stream, **kwargs):
+ """
+ Performs a libpq v3 startup handshake. kwargs should contain the key/value
+ parameters to send to the server in the startup packet.
+ """
+ # Send our startup parameters.
+ send_startup(stream, **kwargs)
+
+ # Receive and dump packets until the server indicates it's ready for our
+ # first query.
+ while True:
+ resp = recv1(stream)
+ if resp is None:
+ raise RuntimeError("server closed connection during handshake")
+
+ if resp.type == types.ReadyForQuery:
+ return
+ elif resp.type == types.ErrorResponse:
+ raise RuntimeError(
+ f"received error response from peer: {resp.payload.fields!r}"
+ )
+
+
+# TLS
+
+
+class _TLSStream(object):
+ """
+ A file-like object that performs TLS encryption/decryption on a wrapped
+ stream. Differs from ssl.SSLSocket in that we have full visibility and
+ control over the TLS layer.
+ """
+
+ def __init__(self, stream, context):
+ self._stream = stream
+ self._debugging = hasattr(stream, "flush_debug")
+
+ self._in = ssl.MemoryBIO()
+ self._out = ssl.MemoryBIO()
+ self._ssl = context.wrap_bio(self._in, self._out)
+
+ def handshake(self):
+ try:
+ self._pump(lambda: self._ssl.do_handshake())
+ finally:
+ self._flush_debug(prefix="? ")
+
+ def read(self, *args):
+ return self._pump(lambda: self._ssl.read(*args))
+
+ def write(self, *args):
+ return self._pump(lambda: self._ssl.write(*args))
+
+ def _decode(self, buf):
+ """
+ Attempts to decode a buffer of TLS data into a packet representation
+ that can be printed.
+
+ TODO: handle buffers (and record fragments) that don't align with packet
+ boundaries.
+ """
+ end = len(buf)
+ bio = io.BytesIO(buf)
+
+ ret = io.StringIO()
+
+ while bio.tell() < end:
+ record = tls.Plaintext.parse_stream(bio)
+
+ if ret.tell() > 0:
+ ret.write("\n")
+ ret.write("[Record] ")
+ ret.write(str(record))
+ ret.write("\n")
+
+ if record.type == tls.ContentType.handshake:
+ record_cls = tls.Handshake
+ else:
+ continue
+
+ innerlen = len(record.fragment)
+ inner = io.BytesIO(record.fragment)
+
+ while inner.tell() < innerlen:
+ msg = record_cls.parse_stream(inner)
+
+ indented = "[Message] " + str(msg)
+ indented = textwrap.indent(indented, " ")
+
+ ret.write("\n")
+ ret.write(indented)
+ ret.write("\n")
+
+ return ret.getvalue()
+
+ def flush(self):
+ if not self._out.pending:
+ self._stream.flush()
+ return
+
+ buf = self._out.read()
+ self._stream.write(buf)
+
+ if self._debugging:
+ pkt = self._decode(buf)
+ self._stream.end_packet(pkt, prefix=" ")
+
+ self._stream.flush()
+
+ def _pump(self, operation):
+ while True:
+ try:
+ return operation()
+ except (ssl.SSLWantReadError, ssl.SSLWantWriteError) as e:
+ want = e
+ self._read_write(want)
+
+ def _recv(self, maxsize):
+ buf = self._stream.recv(4096)
+ if not buf:
+ self._in.write_eof()
+ return
+
+ self._in.write(buf)
+
+ if not self._debugging:
+ return
+
+ pkt = self._decode(buf)
+ self._stream.end_packet(pkt, read=True, prefix=" ")
+
+ def _read_write(self, want):
+ # XXX This needs work. So many corner cases yet to handle. For one,
+ # doing blocking writes in flush may lead to distributed deadlock if the
+ # peer is already blocking on its writes.
+
+ if isinstance(want, ssl.SSLWantWriteError):
+ assert self._out.pending, "SSL backend wants write without data"
+
+ self.flush()
+
+ if isinstance(want, ssl.SSLWantReadError):
+ self._recv(4096)
+
+ def _flush_debug(self, prefix):
+ if not self._debugging:
+ return
+
+ self._stream.flush_debug(prefix=prefix)
+
+
+@contextlib.contextmanager
+def tls_handshake(stream, context):
+ """
+ Performs a TLS handshake over the given stream (which must have been created
+ via a call to wrap()), and returns a new stream which transparently tunnels
+ data over the TLS connection.
+
+ If the passed stream has debugging enabled, the returned stream will also
+ have debugging, using the same output IO.
+ """
+ debugging = hasattr(stream, "flush_debug")
+
+ # Send our startup parameters.
+ send_startup(stream, proto=protocol(1234, 5679))
+
+ # Look at the SSL response.
+ resp = stream.read(1)
+ if debugging:
+ stream.flush_debug(prefix=" ")
+
+ if resp == b"N":
+ raise RuntimeError("server does not support SSLRequest")
+ if resp != b"S":
+ raise RuntimeError(f"unexpected response of type {resp!r} during TLS startup")
+
+ tls = _TLSStream(stream, context)
+ tls.handshake()
+
+ if debugging:
+ tls = _DebugStream(tls, stream._out)
+
+ try:
+ yield tls
+ # TODO: teardown/unwrap the connection?
+ finally:
+ if debugging:
+ tls.flush_debug(prefix="? ")
diff --git a/src/test/python/pytest.ini b/src/test/python/pytest.ini
new file mode 100644
index 0000000000..ab7a6e7fb9
--- /dev/null
+++ b/src/test/python/pytest.ini
@@ -0,0 +1,4 @@
+[pytest]
+
+markers =
+ slow: mark test as slow
diff --git a/src/test/python/requirements.txt b/src/test/python/requirements.txt
new file mode 100644
index 0000000000..32f105ea84
--- /dev/null
+++ b/src/test/python/requirements.txt
@@ -0,0 +1,7 @@
+black
+cryptography~=3.4.6
+construct~=2.10.61
+isort~=5.6
+psycopg2~=2.8.6
+pytest~=6.1
+pytest-asyncio~=0.14.0
diff --git a/src/test/python/server/__init__.py b/src/test/python/server/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/src/test/python/server/conftest.py b/src/test/python/server/conftest.py
new file mode 100644
index 0000000000..ba7342a453
--- /dev/null
+++ b/src/test/python/server/conftest.py
@@ -0,0 +1,45 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import socket
+import sys
+
+import pytest
+
+import pq3
+
+
+@pytest.fixture
+def connect():
+ """
+ A factory fixture that, when called, returns a socket connected to a
+ Postgres server, wrapped in a pq3 connection. The calling test will be
+ skipped automatically if a server is not running at PGHOST:PGPORT, so it's
+ best to connect as soon as possible after the test case begins, to avoid
+ doing unnecessary work.
+ """
+ # Set up an ExitStack to handle safe cleanup of all of the moving pieces.
+ with contextlib.ExitStack() as stack:
+
+ def conn_factory():
+ addr = (pq3.pghost(), pq3.pgport())
+
+ try:
+ sock = socket.create_connection(addr, timeout=2)
+ except ConnectionError as e:
+ pytest.skip(f"unable to connect to {addr}: {e}")
+
+ # Have ExitStack close our socket.
+ stack.enter_context(sock)
+
+ # Wrap the connection in a pq3 layer and have ExitStack clean it up
+ # too.
+ wrap_ctx = pq3.wrap(sock, debug_stream=sys.stdout)
+ conn = stack.enter_context(wrap_ctx)
+
+ return conn
+
+ yield conn_factory
diff --git a/src/test/python/server/test_oauth.py b/src/test/python/server/test_oauth.py
new file mode 100644
index 0000000000..cb5ca7fa23
--- /dev/null
+++ b/src/test/python/server/test_oauth.py
@@ -0,0 +1,1012 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import contextlib
+import json
+import os
+import pathlib
+import secrets
+import shlex
+import shutil
+import socket
+import struct
+from multiprocessing import shared_memory
+
+import psycopg2
+import pytest
+from psycopg2 import sql
+
+import pq3
+
+MAX_SASL_MESSAGE_LENGTH = 65535
+
+INVALID_AUTHORIZATION_ERRCODE = b"28000"
+PROTOCOL_VIOLATION_ERRCODE = b"08P01"
+FEATURE_NOT_SUPPORTED_ERRCODE = b"0A000"
+
+SHARED_MEM_NAME = "oauth-pytest"
+MAX_TOKEN_SIZE = 4096
+MAX_UINT16 = 2 ** 16 - 1
+
+
+def skip_if_no_postgres():
+ """
+ Used by the oauth_ctx fixture to skip this test module if no Postgres server
+ is running.
+
+ This logic is nearly duplicated with the conn fixture. Ideally oauth_ctx
+ would depend on that, but a module-scope fixture can't depend on a
+ test-scope fixture, and we haven't reached the rule of three yet.
+ """
+ addr = (pq3.pghost(), pq3.pgport())
+
+ try:
+ with socket.create_connection(addr, timeout=2):
+ pass
+ except ConnectionError as e:
+ pytest.skip(f"unable to connect to {addr}: {e}")
+
+
+@contextlib.contextmanager
+def prepend_file(path, lines):
+ """
+ A context manager that prepends a file on disk with the desired lines of
+ text. When the context manager is exited, the file will be restored to its
+ original contents.
+ """
+ # First make a backup of the original file.
+ bak = path + ".bak"
+ shutil.copy2(path, bak)
+
+ try:
+ # Write the new lines, followed by the original file content.
+ with open(path, "w") as new, open(bak, "r") as orig:
+ new.writelines(lines)
+ shutil.copyfileobj(orig, new)
+
+ # Return control to the calling code.
+ yield
+
+ finally:
+ # Put the backup back into place.
+ os.replace(bak, path)
+
+
+@pytest.fixture(scope="module")
+def oauth_ctx():
+ """
+ Creates a database and user that use the oauth auth method. The context
+ object contains the dbname and user attributes as strings to be used during
+ connection, as well as the issuer and scope that have been set in the HBA
+ configuration.
+
+ This fixture assumes that the standard PG* environment variables point to a
+ server running on a local machine, and that the PGUSER has rights to create
+ databases and roles.
+ """
+ skip_if_no_postgres() # don't bother running these tests without a server
+
+ id = secrets.token_hex(4)
+
+ class Context:
+ dbname = "oauth_test_" + id
+
+ user = "oauth_user_" + id
+ map_user = "oauth_map_user_" + id
+ authz_user = "oauth_authz_user_" + id
+
+ issuer = "https://example.com/" + id
+ scope = "openid " + id
+
+ ctx = Context()
+ hba_lines = (
+ f'host {ctx.dbname} {ctx.map_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" map=oauth\n',
+ f'host {ctx.dbname} {ctx.authz_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" trust_validator_authz=1\n',
+ f'host {ctx.dbname} all samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}"\n',
+ )
+ ident_lines = (r"oauth /^(.*)@example\.com$ \1",)
+
+ conn = psycopg2.connect("")
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+
+ # Create our roles and database.
+ user = sql.Identifier(ctx.user)
+ map_user = sql.Identifier(ctx.map_user)
+ authz_user = sql.Identifier(ctx.authz_user)
+ dbname = sql.Identifier(ctx.dbname)
+
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(map_user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(authz_user))
+ c.execute(sql.SQL("CREATE DATABASE {};").format(dbname))
+
+ # Make this test script the server's oauth_validator.
+ path = pathlib.Path(__file__).parent / "validate_bearer.py"
+ path = str(path.absolute())
+
+ cmd = f"{shlex.quote(path)} {SHARED_MEM_NAME} <&%f"
+ c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (cmd,))
+
+ # Replace pg_hba and pg_ident.
+ c.execute("SHOW hba_file;")
+ hba = c.fetchone()[0]
+
+ c.execute("SHOW ident_file;")
+ ident = c.fetchone()[0]
+
+ with prepend_file(hba, hba_lines), prepend_file(ident, ident_lines):
+ c.execute("SELECT pg_reload_conf();")
+
+ # Use the new database and user.
+ yield ctx
+
+ # Put things back the way they were.
+ c.execute("SELECT pg_reload_conf();")
+
+ c.execute("ALTER SYSTEM RESET oauth_validator_command;")
+ c.execute(sql.SQL("DROP DATABASE {};").format(dbname))
+ c.execute(sql.SQL("DROP ROLE {};").format(authz_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(map_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(user))
+
+
+@pytest.fixture()
+def conn(oauth_ctx, connect):
+ """
+ A convenience wrapper for connect(). The main purpose of this fixture is to
+ make sure oauth_ctx runs its setup code before the connection is made.
+ """
+ return connect()
+
+
+@pytest.fixture(scope="module", autouse=True)
+def authn_id_extension(oauth_ctx):
+ """
+ Performs a `CREATE EXTENSION authn_id` in the test database. This fixture is
+ autoused, so tests don't need to rely on it.
+ """
+ conn = psycopg2.connect(database=oauth_ctx.dbname)
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+ c.execute("CREATE EXTENSION authn_id;")
+
+
+@pytest.fixture(scope="session")
+def shared_mem():
+ """
+ Yields a shared memory segment that can be used for communication between
+ the bearer_token fixture and ./validate_bearer.py.
+ """
+ size = MAX_TOKEN_SIZE + 2 # two byte length prefix
+ mem = shared_memory.SharedMemory(SHARED_MEM_NAME, create=True, size=size)
+
+ try:
+ with contextlib.closing(mem):
+ yield mem
+ finally:
+ mem.unlink()
+
+
+@pytest.fixture()
+def bearer_token(shared_mem):
+ """
+ Returns a factory function that, when called, will store a Bearer token in
+ shared_mem. If token is None (the default), a new token will be generated
+ using secrets.token_urlsafe() and returned; otherwise the passed token will
+ be used as-is.
+
+ When token is None, the generated token size in bytes may be specified as an
+ argument; if unset, a small 16-byte token will be generated. The token size
+ may not exceed MAX_TOKEN_SIZE in any case.
+
+ The return value is the token, converted to a bytes object.
+
+ As a special case for testing failure modes, accept_any may be set to True.
+ This signals to the validator command that any bearer token should be
+ accepted. The returned token in this case may be used or discarded as needed
+ by the test.
+ """
+
+ def set_token(token=None, *, size=16, accept_any=False):
+ if token is not None:
+ size = len(token)
+
+ if size > MAX_TOKEN_SIZE:
+ raise ValueError(f"token size {size} exceeds maximum size {MAX_TOKEN_SIZE}")
+
+ if token is None:
+ if size % 4:
+ raise ValueError(f"requested token size {size} is not a multiple of 4")
+
+ token = secrets.token_urlsafe(size // 4 * 3)
+ assert len(token) == size
+
+ try:
+ token = token.encode("ascii")
+ except AttributeError:
+ pass # already encoded
+
+ if accept_any:
+ # Two-byte magic value.
+ shared_mem.buf[:2] = struct.pack("H", MAX_UINT16)
+ else:
+ # Two-byte length prefix, then the token data.
+ shared_mem.buf[:2] = struct.pack("H", len(token))
+ shared_mem.buf[2 : size + 2] = token
+
+ return token
+
+ return set_token
+
+
+def begin_oauth_handshake(conn, oauth_ctx, *, user=None):
+ if user is None:
+ user = oauth_ctx.authz_user
+
+ pq3.send_startup(conn, user=user, database=oauth_ctx.dbname)
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.AuthnRequest
+
+ # The server should advertise exactly one mechanism.
+ assert resp.payload.type == pq3.authn.SASL
+ assert resp.payload.body == [b"OAUTHBEARER", b""]
+
+
+def send_initial_response(conn, *, auth=None, bearer=None):
+ """
+ Sends the OAUTHBEARER initial response on the connection, using the given
+ bearer token. Alternatively to a bearer token, the initial response's auth
+ field may be explicitly specified to test corner cases.
+ """
+ if bearer is not None and auth is not None:
+ raise ValueError("exactly one of the auth and bearer kwargs must be set")
+
+ if bearer is not None:
+ auth = b"Bearer " + bearer
+
+ if auth is None:
+ raise ValueError("exactly one of the auth and bearer kwargs must be set")
+
+ initial = pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"n,,\x01auth=" + auth + b"\x01\x01",
+ )
+ )
+ pq3.send(conn, pq3.types.PasswordMessage, initial)
+
+
+def expect_handshake_success(conn):
+ """
+ Validates that the server responds with an AuthnOK message, and then drains
+ the connection until a ReadyForQuery message is received.
+ """
+ resp = pq3.recv1(conn)
+
+ assert resp.type == pq3.types.AuthnRequest
+ assert resp.payload.type == pq3.authn.OK
+ assert not resp.payload.body
+
+ receive_until(conn, pq3.types.ReadyForQuery)
+
+
+def expect_handshake_failure(conn, oauth_ctx):
+ """
+ Performs the OAUTHBEARER SASL failure "handshake" and validates the server's
+ side of the conversation, including the final ErrorResponse.
+ """
+
+ # We expect a discovery "challenge" back from the server before the authn
+ # failure message.
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.AuthnRequest
+
+ req = resp.payload
+ assert req.type == pq3.authn.SASLContinue
+
+ body = json.loads(req.body)
+ assert body["status"] == "invalid_token"
+ assert body["scope"] == oauth_ctx.scope
+
+ expected_config = oauth_ctx.issuer + "/.well-known/openid-configuration"
+ assert body["openid-configuration"] == expected_config
+
+ # Send the dummy response to complete the failed handshake.
+ pq3.send(conn, pq3.types.PasswordMessage, b"\x01")
+ resp = pq3.recv1(conn)
+
+ err = ExpectedError(INVALID_AUTHORIZATION_ERRCODE, "bearer authentication failed")
+ err.match(resp)
+
+
+def receive_until(conn, type):
+ """
+ receive_until pulls packets off the pq3 connection until a packet with the
+ desired type is found, or an error response is received.
+ """
+ while True:
+ pkt = pq3.recv1(conn)
+
+ if pkt.type == type:
+ return pkt
+ elif pkt.type == pq3.types.ErrorResponse:
+ raise RuntimeError(
+ f"received error response from peer: {pkt.payload.fields!r}"
+ )
+
+
+@pytest.mark.parametrize("token_len", [16, 1024, 4096])
+@pytest.mark.parametrize(
+ "auth_prefix",
+ [
+ b"Bearer ",
+ b"bearer ",
+ b"Bearer ",
+ ],
+)
+def test_oauth(conn, oauth_ctx, bearer_token, auth_prefix, token_len):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Generate our bearer token with the desired length.
+ token = bearer_token(size=token_len)
+ auth = auth_prefix + token
+
+ send_initial_response(conn, auth=auth)
+ expect_handshake_success(conn)
+
+ # Make sure that the server has not set an authenticated ID.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT authn_id();")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ assert row.columns == [None]
+
+
+@pytest.mark.parametrize(
+ "token_value",
+ [
+ "abcdzA==",
+ "123456M=",
+ "x-._~+/x",
+ ],
+)
+def test_oauth_bearer_corner_cases(conn, oauth_ctx, bearer_token, token_value):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ send_initial_response(conn, bearer=bearer_token(token_value))
+
+ expect_handshake_success(conn)
+
+
+@pytest.mark.parametrize(
+ "user,authn_id,should_succeed",
+ [
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: ctx.user,
+ True,
+ id="validator authn: succeeds when authn_id == username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: None,
+ False,
+ id="validator authn: fails when authn_id is not set",
+ ),
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: ctx.authz_user,
+ False,
+ id="validator authn: fails when authn_id != username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: ctx.map_user + "@example.com",
+ True,
+ id="validator with map: succeeds when authn_id matches map",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: None,
+ False,
+ id="validator with map: fails when authn_id is not set",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: ctx.map_user + "@example.net",
+ False,
+ id="validator with map: fails when authn_id doesn't match map",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: None,
+ True,
+ id="validator authz: succeeds with no authn_id",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "",
+ True,
+ id="validator authz: succeeds with empty authn_id",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "postgres",
+ True,
+ id="validator authz: succeeds with basic username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "me@example.com",
+ True,
+ id="validator authz: succeeds with email address",
+ ),
+ ],
+)
+def test_oauth_authn_id(conn, oauth_ctx, bearer_token, user, authn_id, should_succeed):
+ token = None
+
+ authn_id = authn_id(oauth_ctx)
+ if authn_id is not None:
+ authn_id = authn_id.encode("ascii")
+
+ # As a hack to get the validator to reflect arbitrary output from this
+ # test, encode the desired output as a base64 token. The validator will
+ # key on the leading "output=" to differentiate this from the random
+ # tokens generated by secrets.token_urlsafe().
+ output = b"output=" + authn_id + b"\n"
+ token = base64.urlsafe_b64encode(output)
+
+ token = bearer_token(token)
+ username = user(oauth_ctx)
+
+ begin_oauth_handshake(conn, oauth_ctx, user=username)
+ send_initial_response(conn, bearer=token)
+
+ if not should_succeed:
+ expect_handshake_failure(conn, oauth_ctx)
+ return
+
+ expect_handshake_success(conn)
+
+ # Check the reported authn_id.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT authn_id();")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ assert row.columns == [authn_id]
+
+
+class ExpectedError(object):
+ def __init__(self, code, msg=None, detail=None):
+ self.code = code
+ self.msg = msg
+ self.detail = detail
+
+ # Protect against the footgun of an accidental empty string, which will
+ # "match" anything. If you don't want to match message or detail, just
+ # don't pass them.
+ if self.msg == "":
+ raise ValueError("msg must be non-empty or None")
+ if self.detail == "":
+ raise ValueError("detail must be non-empty or None")
+
+ def _getfield(self, resp, type):
+ """
+ Searches an ErrorResponse for a single field of the given type (e.g.
+ "M", "C", "D") and returns its value. Asserts if it doesn't find exactly
+ one field.
+ """
+ prefix = type.encode("ascii")
+ fields = [f for f in resp.payload.fields if f.startswith(prefix)]
+
+ assert len(fields) == 1
+ return fields[0][1:] # strip off the type byte
+
+ def match(self, resp):
+ """
+ Checks that the given response matches the expected code, message, and
+ detail (if given). The error code must match exactly. The expected
+ message and detail must be contained within the actual strings.
+ """
+ assert resp.type == pq3.types.ErrorResponse
+
+ code = self._getfield(resp, "C")
+ assert code == self.code
+
+ if self.msg:
+ msg = self._getfield(resp, "M")
+ expected = self.msg.encode("utf-8")
+ assert expected in msg
+
+ if self.detail:
+ detail = self._getfield(resp, "D")
+ expected = self.detail.encode("utf-8")
+ assert expected in detail
+
+
+def test_oauth_rejected_bearer(conn, oauth_ctx, bearer_token):
+ # Generate a new bearer token, which we will proceed not to use.
+ _ = bearer_token()
+
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send a bearer token that doesn't match what the validator expects. It
+ # should fail the connection.
+ send_initial_response(conn, bearer=b"xxxxxx")
+
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.parametrize(
+ "bad_bearer",
+ [
+ b"Bearer ",
+ b"Bearer a===b",
+ b"Bearer hello!",
+ b"Bearer me@example.com",
+ b'OAuth realm="Example"',
+ b"",
+ ],
+)
+def test_oauth_invalid_bearer(conn, oauth_ctx, bearer_token, bad_bearer):
+ # Tell the validator to accept any token. This ensures that the invalid
+ # bearer tokens are rejected before the validation step.
+ _ = bearer_token(accept_any=True)
+
+ begin_oauth_handshake(conn, oauth_ctx)
+ send_initial_response(conn, auth=bad_bearer)
+
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.slow
+@pytest.mark.parametrize(
+ "resp_type,resp,err",
+ [
+ pytest.param(
+ None,
+ None,
+ None,
+ marks=pytest.mark.slow,
+ id="no response (expect timeout)",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"hello",
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not send a kvsep response",
+ ),
+ id="bad dummy response",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"\x01\x01",
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not send a kvsep response",
+ ),
+ id="multiple kvseps",
+ ),
+ pytest.param(
+ pq3.types.Query,
+ dict(query=b""),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "expected SASL response"),
+ id="bad response message type",
+ ),
+ ],
+)
+def test_oauth_bad_response_to_error_challenge(conn, oauth_ctx, resp_type, resp, err):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send an empty auth initial response, which will force an authn failure.
+ send_initial_response(conn, auth=b"")
+
+ # We expect a discovery "challenge" back from the server before the authn
+ # failure message.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.AuthnRequest
+
+ req = pkt.payload
+ assert req.type == pq3.authn.SASLContinue
+
+ body = json.loads(req.body)
+ assert body["status"] == "invalid_token"
+
+ if resp_type is None:
+ # Do not send the dummy response. We should time out and not get a
+ # response from the server.
+ with pytest.raises(socket.timeout):
+ conn.read(1)
+
+ # Done with the test.
+ return
+
+ # Send the bad response.
+ pq3.send(conn, resp_type, resp)
+
+ # Make sure the server fails the connection correctly.
+ pkt = pq3.recv1(conn)
+ err.match(pkt)
+
+
+@pytest.mark.parametrize(
+ "type,payload,err",
+ [
+ pytest.param(
+ pq3.types.ErrorResponse,
+ dict(fields=[b""]),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "expected SASL response"),
+ id="error response in initial message",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"x" * (MAX_SASL_MESSAGE_LENGTH + 1),
+ ExpectedError(
+ INVALID_AUTHORIZATION_ERRCODE, "bearer authentication failed"
+ ),
+ id="overlong initial response data",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"SCRAM-SHA-256")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE, "invalid SASL authentication mechanism"
+ ),
+ id="bad SASL mechanism selection",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", len=2, data=b"x")),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "insufficient data"),
+ id="SASL data underflow",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", len=0, data=b"x")),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "invalid message format"),
+ id="SASL data overflow",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "message is empty",
+ ),
+ id="empty",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"n,,\x01auth=\x01\x01\0")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "length does not match input length",
+ ),
+ id="contains null byte",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected channel-binding flag", # XXX this is a bit strange
+ ),
+ id="initial error response",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"p=tls-server-end-point,,\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "server does not support channel binding",
+ ),
+ id="uses channel binding",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"x,,\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected channel-binding flag",
+ ),
+ id="invalid channel binding specifier",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Comma expected",
+ ),
+ id="bad GS2 header: missing channel binding terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,a")),
+ ExpectedError(
+ FEATURE_NOT_SUPPORTED_ERRCODE,
+ "client uses authorization identity",
+ ),
+ id="bad GS2 header: authzid in use",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,b,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected attribute",
+ ),
+ id="bad GS2 header: extra attribute",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected attribute 0x00", # XXX this is a bit strange
+ ),
+ id="bad GS2 header: missing authzid terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Key-value separator expected",
+ ),
+ id="missing initial kvsep",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Key-value separator expected",
+ ),
+ id="missing initial kvsep",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "does not contain an auth value",
+ ),
+ id="missing auth value: empty key-value list",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01host=example.com\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "does not contain an auth value",
+ ),
+ id="missing auth value: other keys present",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01host=example.com")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "unterminated key/value pair",
+ ),
+ id="missing value terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not contain a final terminator",
+ ),
+ id="missing list terminator: empty list",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01auth=Bearer 0\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not contain a final terminator",
+ ),
+ id="missing list terminator: with auth value",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01auth=Bearer 0\x01\x01blah")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "additional data after the final terminator",
+ ),
+ id="additional key after terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01key\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "key without a value",
+ ),
+ id="key without value",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"y,,\x01auth=Bearer 0\x01auth=Bearer 1\x01\x01",
+ )
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "contains multiple auth values",
+ ),
+ id="multiple auth values",
+ ),
+ ],
+)
+def test_oauth_bad_initial_response(conn, oauth_ctx, type, payload, err):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # The server expects a SASL response; give it something else instead.
+ if not isinstance(payload, dict):
+ payload = dict(payload_data=payload)
+ pq3.send(conn, type, **payload)
+
+ resp = pq3.recv1(conn)
+ err.match(resp)
+
+
+def test_oauth_empty_initial_response(conn, oauth_ctx, bearer_token):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send an initial response without data.
+ initial = pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER"))
+ pq3.send(conn, pq3.types.PasswordMessage, initial)
+
+ # The server should respond with an empty challenge so we can send the data
+ # it wants.
+ pkt = pq3.recv1(conn)
+
+ assert pkt.type == pq3.types.AuthnRequest
+ assert pkt.payload.type == pq3.authn.SASLContinue
+ assert not pkt.payload.body
+
+ # Now send the initial data.
+ data = b"n,,\x01auth=Bearer " + bearer_token() + b"\x01\x01"
+ pq3.send(conn, pq3.types.PasswordMessage, data)
+
+ # Server should now complete the handshake.
+ expect_handshake_success(conn)
+
+
+@pytest.fixture()
+def set_validator():
+ """
+ A per-test fixture that allows a test to override the setting of
+ oauth_validator_command for the cluster. The setting will be reverted during
+ teardown.
+
+ Passing None will perform an ALTER SYSTEM RESET.
+ """
+ conn = psycopg2.connect("")
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+
+ # Save the previous value.
+ c.execute("SHOW oauth_validator_command;")
+ prev_cmd = c.fetchone()[0]
+
+ def setter(cmd):
+ c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (cmd,))
+ c.execute("SELECT pg_reload_conf();")
+
+ yield setter
+
+ # Restore the previous value.
+ c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (prev_cmd,))
+ c.execute("SELECT pg_reload_conf();")
+
+
+def test_oauth_no_validator(oauth_ctx, set_validator, connect, bearer_token):
+ # Clear out our validator command, then establish a new connection.
+ set_validator("")
+ conn = connect()
+
+ begin_oauth_handshake(conn, oauth_ctx)
+ send_initial_response(conn, bearer=bearer_token())
+
+ # The server should fail the connection.
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+def test_oauth_validator_role(oauth_ctx, set_validator, connect):
+ # Switch the validator implementation. This validator will reflect the
+ # PGUSER as the authenticated identity.
+ path = pathlib.Path(__file__).parent / "validate_reflect.py"
+ path = str(path.absolute())
+
+ set_validator(f"{shlex.quote(path)} '%r' <&%f")
+ conn = connect()
+
+ # Log in. Note that the reflection validator ignores the bearer token.
+ begin_oauth_handshake(conn, oauth_ctx, user=oauth_ctx.user)
+ send_initial_response(conn, bearer=b"dontcare")
+ expect_handshake_success(conn)
+
+ # Check the user identity.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT authn_id();")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ expected = oauth_ctx.user.encode("utf-8")
+ assert row.columns == [expected]
+
+
+def test_oauth_role_with_shell_unsafe_characters(oauth_ctx, set_validator, connect):
+ """
+ XXX This test pins undesirable behavior. We should be able to handle any
+ valid Postgres role name.
+ """
+ # Switch the validator implementation. This validator will reflect the
+ # PGUSER as the authenticated identity.
+ path = pathlib.Path(__file__).parent / "validate_reflect.py"
+ path = str(path.absolute())
+
+ set_validator(f"{shlex.quote(path)} '%r' <&%f")
+ conn = connect()
+
+ unsafe_username = "hello'there"
+ begin_oauth_handshake(conn, oauth_ctx, user=unsafe_username)
+
+ # The server should reject the handshake.
+ send_initial_response(conn, bearer=b"dontcare")
+ expect_handshake_failure(conn, oauth_ctx)
diff --git a/src/test/python/server/test_server.py b/src/test/python/server/test_server.py
new file mode 100644
index 0000000000..02126dba79
--- /dev/null
+++ b/src/test/python/server/test_server.py
@@ -0,0 +1,21 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import pq3
+
+
+def test_handshake(connect):
+ """Basic sanity check."""
+ conn = connect()
+
+ pq3.handshake(conn, user=pq3.pguser(), database=pq3.pgdatabase())
+
+ pq3.send(conn, pq3.types.Query, query=b"")
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.EmptyQueryResponse
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.ReadyForQuery
diff --git a/src/test/python/server/validate_bearer.py b/src/test/python/server/validate_bearer.py
new file mode 100755
index 0000000000..2cc73ff154
--- /dev/null
+++ b/src/test/python/server/validate_bearer.py
@@ -0,0 +1,101 @@
+#! /usr/bin/env python3
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+# DO NOT USE THIS OAUTH VALIDATOR IN PRODUCTION. It doesn't actually validate
+# anything, and it logs the bearer token data, which is sensitive.
+#
+# This executable is used as an oauth_validator_command in concert with
+# test_oauth.py. Memory is shared and communicated from that test module's
+# bearer_token() fixture.
+#
+# This script must run under the Postgres server environment; keep the
+# dependency list fairly standard.
+
+import base64
+import binascii
+import contextlib
+import struct
+import sys
+from multiprocessing import shared_memory
+
+MAX_UINT16 = 2 ** 16 - 1
+
+
+def remove_shm_from_resource_tracker():
+ """
+ Monkey-patch multiprocessing.resource_tracker so SharedMemory won't be
+ tracked. Pulled from this thread, where there are more details:
+
+ https://bugs.python.org/issue38119
+
+ TL;DR: all clients of shared memory segments automatically destroy them on
+ process exit, which makes shared memory segments much less useful. This
+ monkeypatch removes that behavior so that we can defer to the test to manage
+ the segment lifetime.
+
+ Ideally a future Python patch will pull in this fix and then the entire
+ function can go away.
+ """
+ from multiprocessing import resource_tracker
+
+ def fix_register(name, rtype):
+ if rtype == "shared_memory":
+ return
+ return resource_tracker._resource_tracker.register(self, name, rtype)
+
+ resource_tracker.register = fix_register
+
+ def fix_unregister(name, rtype):
+ if rtype == "shared_memory":
+ return
+ return resource_tracker._resource_tracker.unregister(self, name, rtype)
+
+ resource_tracker.unregister = fix_unregister
+
+ if "shared_memory" in resource_tracker._CLEANUP_FUNCS:
+ del resource_tracker._CLEANUP_FUNCS["shared_memory"]
+
+
+def main(args):
+ remove_shm_from_resource_tracker() # XXX remove some day
+
+ # Get the expected token from the currently running test.
+ shared_mem_name = args[0]
+
+ mem = shared_memory.SharedMemory(shared_mem_name)
+ with contextlib.closing(mem):
+ # First two bytes are the token length.
+ size = struct.unpack("H", mem.buf[:2])[0]
+
+ if size == MAX_UINT16:
+ # Special case: the test wants us to accept any token.
+ sys.stderr.write("accepting token without validation\n")
+ return
+
+ # The remainder of the buffer contains the expected token.
+ assert size <= (mem.size - 2)
+ expected_token = mem.buf[2 : size + 2].tobytes()
+
+ mem.buf[:] = b"\0" * mem.size # scribble over the token
+
+ token = sys.stdin.buffer.read()
+ if token != expected_token:
+ sys.exit(f"failed to match Bearer token ({token!r} != {expected_token!r})")
+
+ # See if the test wants us to print anything. If so, it will have encoded
+ # the desired output in the token with an "output=" prefix.
+ try:
+ # altchars="-_" corresponds to the urlsafe alphabet.
+ data = base64.b64decode(token, altchars="-_", validate=True)
+
+ if data.startswith(b"output="):
+ sys.stdout.buffer.write(data[7:])
+
+ except binascii.Error:
+ pass
+
+
+if __name__ == "__main__":
+ main(sys.argv[1:])
diff --git a/src/test/python/server/validate_reflect.py b/src/test/python/server/validate_reflect.py
new file mode 100755
index 0000000000..24c3a7e715
--- /dev/null
+++ b/src/test/python/server/validate_reflect.py
@@ -0,0 +1,34 @@
+#! /usr/bin/env python3
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+# DO NOT USE THIS OAUTH VALIDATOR IN PRODUCTION. It ignores the bearer token
+# entirely and automatically logs the user in.
+#
+# This executable is used as an oauth_validator_command in concert with
+# test_oauth.py. It expects the user's desired role name as an argument; the
+# actual token will be discarded and the user will be logged in with the role
+# name as the authenticated identity.
+#
+# This script must run under the Postgres server environment; keep the
+# dependency list fairly standard.
+
+import sys
+
+
+def main(args):
+ # We have to read the entire token as our first action to unblock the
+ # server, but we won't actually use it.
+ _ = sys.stdin.buffer.read()
+
+ if len(args) != 1:
+ sys.exit("usage: ./validate_reflect.py ROLE")
+
+ # Log the user in as the provided role.
+ role = args[0]
+ print(role)
+
+
+if __name__ == "__main__":
+ main(sys.argv[1:])
diff --git a/src/test/python/test_internals.py b/src/test/python/test_internals.py
new file mode 100644
index 0000000000..dee4855fc0
--- /dev/null
+++ b/src/test/python/test_internals.py
@@ -0,0 +1,138 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import io
+
+from pq3 import _DebugStream
+
+
+def test_DebugStream_read():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+
+ stream = _DebugStream(under, out)
+
+ res = stream.read(5)
+ assert res == b"abcde"
+
+ res = stream.read(16)
+ assert res == b"fghijklmnopqrstu"
+
+ stream.flush_debug()
+
+ res = stream.read()
+ assert res == b"vwxyz"
+
+ stream.flush_debug()
+
+ expected = (
+ "< 0000:\t61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 70\tabcdefghijklmnop\n"
+ "< 0010:\t71 72 73 74 75 \tqrstu\n"
+ "\n"
+ "< 0000:\t76 77 78 79 7a \tvwxyz\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_write():
+ under = io.BytesIO()
+ out = io.StringIO()
+
+ stream = _DebugStream(under, out)
+
+ stream.write(b"\x00\x01\x02")
+ stream.flush()
+
+ assert under.getvalue() == b"\x00\x01\x02"
+
+ stream.write(b"\xc0\xc1\xc2")
+ stream.flush()
+
+ assert under.getvalue() == b"\x00\x01\x02\xc0\xc1\xc2"
+
+ stream.flush_debug()
+
+ expected = "> 0000:\t00 01 02 c0 c1 c2 \t......\n\n"
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_read_write():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+ stream = _DebugStream(under, out)
+
+ res = stream.read(5)
+ assert res == b"abcde"
+
+ stream.write(b"xxxxx")
+ stream.flush()
+
+ assert under.getvalue() == b"abcdexxxxxklmnopqrstuvwxyz"
+
+ res = stream.read(5)
+ assert res == b"klmno"
+
+ stream.write(b"xxxxx")
+ stream.flush()
+
+ assert under.getvalue() == b"abcdexxxxxklmnoxxxxxuvwxyz"
+
+ stream.flush_debug()
+
+ expected = (
+ "< 0000:\t61 62 63 64 65 6b 6c 6d 6e 6f \tabcdeklmno\n"
+ "\n"
+ "> 0000:\t78 78 78 78 78 78 78 78 78 78 \txxxxxxxxxx\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_end_packet():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+ stream = _DebugStream(under, out)
+
+ stream.read(5)
+ stream.end_packet("read description", read=True, indent=" ")
+
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("write description", indent=" ")
+
+ stream.read(5)
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("read/write combo for read", read=True, indent=" ")
+
+ stream.read(5)
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("read/write combo for write", indent=" ")
+
+ expected = (
+ " < 0000:\t61 62 63 64 65 \tabcde\n"
+ "\n"
+ "< read description\n"
+ "\n"
+ "> write description\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ " < 0000:\t6b 6c 6d 6e 6f \tklmno\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ "< read/write combo for read\n"
+ "\n"
+ "> read/write combo for write\n"
+ "\n"
+ " < 0000:\t75 76 77 78 79 \tuvwxy\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
diff --git a/src/test/python/test_pq3.py b/src/test/python/test_pq3.py
new file mode 100644
index 0000000000..e0c0e0568d
--- /dev/null
+++ b/src/test/python/test_pq3.py
@@ -0,0 +1,558 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import getpass
+import io
+import struct
+import sys
+
+import pytest
+from construct import Container, PaddingError, StreamError, TerminatedError
+
+import pq3
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"\x00\x00\x00\x10\x00\x04\x00\x00abcdefgh",
+ Container(len=16, proto=0x40000, payload=b"abcdefgh"),
+ b"",
+ id="8-byte payload",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x08\x00\x04\x00\x00",
+ Container(len=8, proto=0x40000, payload=b""),
+ b"",
+ id="no payload",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x09\x00\x04\x00\x00abcde",
+ Container(len=9, proto=0x40000, payload=b"a"),
+ b"bcde",
+ id="1-byte payload and extra padding",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x0B\x00\x03\x00\x00hi\x00",
+ Container(len=11, proto=pq3.protocol(3, 0), payload=[b"hi"]),
+ b"",
+ id="implied parameter list when using proto version 3.0",
+ ),
+ ],
+)
+def test_Startup_parse(raw, expected, extra):
+ with io.BytesIO(raw) as stream:
+ actual = pq3.Startup.parse_stream(stream)
+
+ assert actual == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "packet,expected_bytes",
+ [
+ pytest.param(
+ dict(),
+ b"\x00\x00\x00\x08\x00\x00\x00\x00",
+ id="nothing set",
+ ),
+ pytest.param(
+ dict(len=10, proto=0x12345678),
+ b"\x00\x00\x00\x0A\x12\x34\x56\x78\x00\x00",
+ id="len and proto set explicitly",
+ ),
+ pytest.param(
+ dict(proto=0x12345678),
+ b"\x00\x00\x00\x08\x12\x34\x56\x78",
+ id="implied len with no payload",
+ ),
+ pytest.param(
+ dict(proto=0x12345678, payload=b"abcd"),
+ b"\x00\x00\x00\x0C\x12\x34\x56\x78abcd",
+ id="implied len with payload",
+ ),
+ pytest.param(
+ dict(payload=[b""]),
+ b"\x00\x00\x00\x09\x00\x03\x00\x00\x00",
+ id="implied proto version 3 when sending parameters",
+ ),
+ pytest.param(
+ dict(payload=[b"hi", b""]),
+ b"\x00\x00\x00\x0C\x00\x03\x00\x00hi\x00\x00",
+ id="implied proto version 3 and len when sending more than one parameter",
+ ),
+ pytest.param(
+ dict(payload=dict(user="jsmith", database="postgres")),
+ b"\x00\x00\x00\x27\x00\x03\x00\x00user\x00jsmith\x00database\x00postgres\x00\x00",
+ id="auto-serialization of dict parameters",
+ ),
+ ],
+)
+def test_Startup_build(packet, expected_bytes):
+ actual = pq3.Startup.build(packet)
+ assert actual == expected_bytes
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"*\x00\x00\x00\x08abcd",
+ dict(type=b"*", len=8, payload=b"abcd"),
+ b"",
+ id="4-byte payload",
+ ),
+ pytest.param(
+ b"*\x00\x00\x00\x04",
+ dict(type=b"*", len=4, payload=b""),
+ b"",
+ id="no payload",
+ ),
+ pytest.param(
+ b"*\x00\x00\x00\x05xabcd",
+ dict(type=b"*", len=5, payload=b"x"),
+ b"abcd",
+ id="1-byte payload with extra padding",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x08\x00\x00\x00\x00",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=8,
+ payload=dict(type=pq3.authn.OK, body=None),
+ ),
+ b"",
+ id="AuthenticationOk",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x12\x00\x00\x00\x0AEXTERNAL\x00\x00",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=18,
+ payload=dict(type=pq3.authn.SASL, body=[b"EXTERNAL", b""]),
+ ),
+ b"",
+ id="AuthenticationSASL",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0B12345",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=13,
+ payload=dict(type=pq3.authn.SASLContinue, body=b"12345"),
+ ),
+ b"",
+ id="AuthenticationSASLContinue",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0C12345",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=13,
+ payload=dict(type=pq3.authn.SASLFinal, body=b"12345"),
+ ),
+ b"",
+ id="AuthenticationSASLFinal",
+ ),
+ pytest.param(
+ b"p\x00\x00\x00\x0Bhunter2",
+ dict(
+ type=pq3.types.PasswordMessage,
+ len=11,
+ payload=b"hunter2",
+ ),
+ b"",
+ id="PasswordMessage",
+ ),
+ pytest.param(
+ b"K\x00\x00\x00\x0C\x00\x00\x00\x00\x12\x34\x56\x78",
+ dict(
+ type=pq3.types.BackendKeyData,
+ len=12,
+ payload=dict(pid=0, key=0x12345678),
+ ),
+ b"",
+ id="BackendKeyData",
+ ),
+ pytest.param(
+ b"C\x00\x00\x00\x08SET\x00",
+ dict(
+ type=pq3.types.CommandComplete,
+ len=8,
+ payload=dict(tag=b"SET"),
+ ),
+ b"",
+ id="CommandComplete",
+ ),
+ pytest.param(
+ b"E\x00\x00\x00\x11Mbad!\x00Mdog!\x00\x00",
+ dict(type=b"E", len=17, payload=dict(fields=[b"Mbad!", b"Mdog!", b""])),
+ b"",
+ id="ErrorResponse",
+ ),
+ pytest.param(
+ b"S\x00\x00\x00\x08a\x00b\x00",
+ dict(
+ type=pq3.types.ParameterStatus,
+ len=8,
+ payload=dict(name=b"a", value=b"b"),
+ ),
+ b"",
+ id="ParameterStatus",
+ ),
+ pytest.param(
+ b"Z\x00\x00\x00\x05x",
+ dict(type=b"Z", len=5, payload=dict(status=b"x")),
+ b"",
+ id="ReadyForQuery",
+ ),
+ pytest.param(
+ b"Q\x00\x00\x00\x06!\x00",
+ dict(type=pq3.types.Query, len=6, payload=dict(query=b"!")),
+ b"",
+ id="Query",
+ ),
+ pytest.param(
+ b"D\x00\x00\x00\x0B\x00\x01\x00\x00\x00\x01!",
+ dict(type=pq3.types.DataRow, len=11, payload=dict(columns=[b"!"])),
+ b"",
+ id="DataRow",
+ ),
+ pytest.param(
+ b"D\x00\x00\x00\x06\x00\x00extra",
+ dict(type=pq3.types.DataRow, len=6, payload=dict(columns=[])),
+ b"extra",
+ id="DataRow with extra data",
+ ),
+ pytest.param(
+ b"I\x00\x00\x00\x04",
+ dict(type=pq3.types.EmptyQueryResponse, len=4, payload=None),
+ b"",
+ id="EmptyQueryResponse",
+ ),
+ pytest.param(
+ b"I\x00\x00\x00\x04\xFF",
+ dict(type=b"I", len=4, payload=None),
+ b"\xFF",
+ id="EmptyQueryResponse with extra bytes",
+ ),
+ pytest.param(
+ b"X\x00\x00\x00\x04",
+ dict(type=pq3.types.Terminate, len=4, payload=None),
+ b"",
+ id="Terminate",
+ ),
+ ],
+)
+def test_Pq3_parse(raw, expected, extra):
+ with io.BytesIO(raw) as stream:
+ actual = pq3.Pq3.parse_stream(stream)
+
+ assert actual == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(type=b"*", len=5),
+ b"*\x00\x00\x00\x05\x00",
+ id="type and len set explicitly",
+ ),
+ pytest.param(
+ dict(type=b"*"),
+ b"*\x00\x00\x00\x04",
+ id="implied len with no payload",
+ ),
+ pytest.param(
+ dict(type=b"*", payload=b"1234"),
+ b"*\x00\x00\x00\x081234",
+ id="implied len with payload",
+ ),
+ pytest.param(
+ dict(type=pq3.types.AuthnRequest, payload=dict(type=pq3.authn.OK)),
+ b"R\x00\x00\x00\x08\x00\x00\x00\x00",
+ id="implied len/type for AuthenticationOK",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(
+ type=pq3.authn.SASL,
+ body=[b"SCRAM-SHA-256-PLUS", b"SCRAM-SHA-256", b""],
+ ),
+ ),
+ b"R\x00\x00\x00\x2A\x00\x00\x00\x0ASCRAM-SHA-256-PLUS\x00SCRAM-SHA-256\x00\x00",
+ id="implied len/type for AuthenticationSASL",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(type=pq3.authn.SASLContinue, body=b"12345"),
+ ),
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0B12345",
+ id="implied len/type for AuthenticationSASLContinue",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(type=pq3.authn.SASLFinal, body=b"12345"),
+ ),
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0C12345",
+ id="implied len/type for AuthenticationSASLFinal",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.PasswordMessage,
+ payload=b"hunter2",
+ ),
+ b"p\x00\x00\x00\x0Bhunter2",
+ id="implied len/type for PasswordMessage",
+ ),
+ pytest.param(
+ dict(type=pq3.types.BackendKeyData, payload=dict(pid=1, key=7)),
+ b"K\x00\x00\x00\x0C\x00\x00\x00\x01\x00\x00\x00\x07",
+ id="implied len/type for BackendKeyData",
+ ),
+ pytest.param(
+ dict(type=pq3.types.CommandComplete, payload=dict(tag=b"SET")),
+ b"C\x00\x00\x00\x08SET\x00",
+ id="implied len/type for CommandComplete",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ErrorResponse, payload=dict(fields=[b"error", b""])),
+ b"E\x00\x00\x00\x0Berror\x00\x00",
+ id="implied len/type for ErrorResponse",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ParameterStatus, payload=dict(name=b"a", value=b"b")),
+ b"S\x00\x00\x00\x08a\x00b\x00",
+ id="implied len/type for ParameterStatus",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ReadyForQuery, payload=dict(status=b"I")),
+ b"Z\x00\x00\x00\x05I",
+ id="implied len/type for ReadyForQuery",
+ ),
+ pytest.param(
+ dict(type=pq3.types.Query, payload=dict(query=b"SELECT 1;")),
+ b"Q\x00\x00\x00\x0eSELECT 1;\x00",
+ id="implied len/type for Query",
+ ),
+ pytest.param(
+ dict(type=pq3.types.DataRow, payload=dict(columns=[b"abcd"])),
+ b"D\x00\x00\x00\x0E\x00\x01\x00\x00\x00\x04abcd",
+ id="implied len/type for DataRow",
+ ),
+ pytest.param(
+ dict(type=pq3.types.EmptyQueryResponse),
+ b"I\x00\x00\x00\x04",
+ id="implied len for EmptyQueryResponse",
+ ),
+ pytest.param(
+ dict(type=pq3.types.Terminate),
+ b"X\x00\x00\x00\x04",
+ id="implied len for Terminate",
+ ),
+ ],
+)
+def test_Pq3_build(fields, expected):
+ actual = pq3.Pq3.build(fields)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"\x00\x00",
+ dict(columns=[]),
+ b"",
+ id="no columns",
+ ),
+ pytest.param(
+ b"\x00\x01\x00\x00\x00\x04abcd",
+ dict(columns=[b"abcd"]),
+ b"",
+ id="one column",
+ ),
+ pytest.param(
+ b"\x00\x02\x00\x00\x00\x04abcd\x00\x00\x00\x01x",
+ dict(columns=[b"abcd", b"x"]),
+ b"",
+ id="multiple columns",
+ ),
+ pytest.param(
+ b"\x00\x02\x00\x00\x00\x00\x00\x00\x00\x01x",
+ dict(columns=[b"", b"x"]),
+ b"",
+ id="empty column value",
+ ),
+ pytest.param(
+ b"\x00\x02\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF",
+ dict(columns=[None, None]),
+ b"",
+ id="null columns",
+ ),
+ ],
+)
+def test_DataRow_parse(raw, expected, extra):
+ pkt = b"D" + struct.pack("!i", len(raw) + 4) + raw
+ with io.BytesIO(pkt) as stream:
+ actual = pq3.Pq3.parse_stream(stream)
+
+ assert actual.type == pq3.types.DataRow
+ assert actual.payload == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(),
+ b"\x00\x00",
+ id="no columns",
+ ),
+ pytest.param(
+ dict(columns=[None, None]),
+ b"\x00\x02\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF",
+ id="null columns",
+ ),
+ ],
+)
+def test_DataRow_build(fields, expected):
+ actual = pq3.Pq3.build(dict(type=pq3.types.DataRow, payload=fields))
+
+ expected = b"D" + struct.pack("!i", len(expected) + 4) + expected
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "raw,expected,exception",
+ [
+ pytest.param(
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ dict(name=b"EXTERNAL", len=-1, data=None),
+ None,
+ id="no initial response",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\x02me",
+ dict(name=b"EXTERNAL", len=2, data=b"me"),
+ None,
+ id="initial response",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\x02meextra",
+ None,
+ TerminatedError,
+ id="extra data",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\xFFme",
+ None,
+ StreamError,
+ id="underflow",
+ ),
+ ],
+)
+def test_SASLInitialResponse_parse(raw, expected, exception):
+ ctx = contextlib.nullcontext()
+ if exception:
+ ctx = pytest.raises(exception)
+
+ with ctx:
+ actual = pq3.SASLInitialResponse.parse(raw)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(name=b"EXTERNAL"),
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ id="no initial response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=None),
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ id="no initial response (explicit None)",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=b""),
+ b"EXTERNAL\x00\x00\x00\x00\x00",
+ id="empty response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=b"me@example.com"),
+ b"EXTERNAL\x00\x00\x00\x00\x0Eme@example.com",
+ id="initial response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", len=2, data=b"me@example.com"),
+ b"EXTERNAL\x00\x00\x00\x00\x02me@example.com",
+ id="data overflow",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", len=14, data=b"me"),
+ b"EXTERNAL\x00\x00\x00\x00\x0Eme",
+ id="data underflow",
+ ),
+ ],
+)
+def test_SASLInitialResponse_build(fields, expected):
+ actual = pq3.SASLInitialResponse.build(fields)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "version,expected_bytes",
+ [
+ pytest.param((3, 0), b"\x00\x03\x00\x00", id="version 3"),
+ pytest.param((1234, 5679), b"\x04\xd2\x16\x2f", id="SSLRequest"),
+ ],
+)
+def test_protocol(version, expected_bytes):
+ # Make sure the integer returned by protocol is correctly serialized on the
+ # wire.
+ assert struct.pack("!i", pq3.protocol(*version)) == expected_bytes
+
+
+@pytest.mark.parametrize(
+ "envvar,func,expected",
+ [
+ ("PGHOST", pq3.pghost, "localhost"),
+ ("PGPORT", pq3.pgport, 5432),
+ ("PGUSER", pq3.pguser, getpass.getuser()),
+ ("PGDATABASE", pq3.pgdatabase, "postgres"),
+ ],
+)
+def test_env_defaults(monkeypatch, envvar, func, expected):
+ monkeypatch.delenv(envvar, raising=False)
+
+ actual = func()
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "envvars,func,expected",
+ [
+ (dict(PGHOST="otherhost"), pq3.pghost, "otherhost"),
+ (dict(PGPORT="6789"), pq3.pgport, 6789),
+ (dict(PGUSER="postgres"), pq3.pguser, "postgres"),
+ (dict(PGDATABASE="template1"), pq3.pgdatabase, "template1"),
+ ],
+)
+def test_env(monkeypatch, envvars, func, expected):
+ for k, v in envvars.items():
+ monkeypatch.setenv(k, v)
+
+ actual = func()
+ assert actual == expected
diff --git a/src/test/python/tls.py b/src/test/python/tls.py
new file mode 100644
index 0000000000..075c02c1ca
--- /dev/null
+++ b/src/test/python/tls.py
@@ -0,0 +1,195 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+from construct import *
+
+#
+# TLS 1.3
+#
+# Most of the types below are transcribed from RFC 8446:
+#
+# https://tools.ietf.org/html/rfc8446
+#
+
+
+def _Vector(size_field, element):
+ return Prefixed(size_field, GreedyRange(element))
+
+
+# Alerts
+
+AlertLevel = Enum(
+ Byte,
+ warning=1,
+ fatal=2,
+)
+
+AlertDescription = Enum(
+ Byte,
+ close_notify=0,
+ unexpected_message=10,
+ bad_record_mac=20,
+ decryption_failed_RESERVED=21,
+ record_overflow=22,
+ decompression_failure=30,
+ handshake_failure=40,
+ no_certificate_RESERVED=41,
+ bad_certificate=42,
+ unsupported_certificate=43,
+ certificate_revoked=44,
+ certificate_expired=45,
+ certificate_unknown=46,
+ illegal_parameter=47,
+ unknown_ca=48,
+ access_denied=49,
+ decode_error=50,
+ decrypt_error=51,
+ export_restriction_RESERVED=60,
+ protocol_version=70,
+ insufficient_security=71,
+ internal_error=80,
+ user_canceled=90,
+ no_renegotiation=100,
+ unsupported_extension=110,
+)
+
+Alert = Struct(
+ "level" / AlertLevel,
+ "description" / AlertDescription,
+)
+
+
+# Extensions
+
+ExtensionType = Enum(
+ Int16ub,
+ server_name=0,
+ max_fragment_length=1,
+ status_request=5,
+ supported_groups=10,
+ signature_algorithms=13,
+ use_srtp=14,
+ heartbeat=15,
+ application_layer_protocol_negotiation=16,
+ signed_certificate_timestamp=18,
+ client_certificate_type=19,
+ server_certificate_type=20,
+ padding=21,
+ pre_shared_key=41,
+ early_data=42,
+ supported_versions=43,
+ cookie=44,
+ psk_key_exchange_modes=45,
+ certificate_authorities=47,
+ oid_filters=48,
+ post_handshake_auth=49,
+ signature_algorithms_cert=50,
+ key_share=51,
+)
+
+Extension = Struct(
+ "extension_type" / ExtensionType,
+ "extension_data" / Prefixed(Int16ub, GreedyBytes),
+)
+
+
+# ClientHello
+
+
+class _CipherSuiteAdapter(Adapter):
+ class _hextuple(tuple):
+ def __repr__(self):
+ return f"(0x{self[0]:02X}, 0x{self[1]:02X})"
+
+ def _encode(self, obj, context, path):
+ return bytes(obj)
+
+ def _decode(self, obj, context, path):
+ assert len(obj) == 2
+ return self._hextuple(obj)
+
+
+ProtocolVersion = Hex(Int16ub)
+
+Random = Hex(Bytes(32))
+
+CipherSuite = _CipherSuiteAdapter(Byte[2])
+
+ClientHello = Struct(
+ "legacy_version" / ProtocolVersion,
+ "random" / Random,
+ "legacy_session_id" / Prefixed(Byte, Hex(GreedyBytes)),
+ "cipher_suites" / _Vector(Int16ub, CipherSuite),
+ "legacy_compression_methods" / Prefixed(Byte, GreedyBytes),
+ "extensions" / _Vector(Int16ub, Extension),
+)
+
+# ServerHello
+
+ServerHello = Struct(
+ "legacy_version" / ProtocolVersion,
+ "random" / Random,
+ "legacy_session_id_echo" / Prefixed(Byte, Hex(GreedyBytes)),
+ "cipher_suite" / CipherSuite,
+ "legacy_compression_method" / Hex(Byte),
+ "extensions" / _Vector(Int16ub, Extension),
+)
+
+# Handshake
+
+HandshakeType = Enum(
+ Byte,
+ client_hello=1,
+ server_hello=2,
+ new_session_ticket=4,
+ end_of_early_data=5,
+ encrypted_extensions=8,
+ certificate=11,
+ certificate_request=13,
+ certificate_verify=15,
+ finished=20,
+ key_update=24,
+ message_hash=254,
+)
+
+Handshake = Struct(
+ "msg_type" / HandshakeType,
+ "length" / Int24ub,
+ "payload"
+ / Switch(
+ this.msg_type,
+ {
+ HandshakeType.client_hello: ClientHello,
+ HandshakeType.server_hello: ServerHello,
+ # HandshakeType.end_of_early_data: EndOfEarlyData,
+ # HandshakeType.encrypted_extensions: EncryptedExtensions,
+ # HandshakeType.certificate_request: CertificateRequest,
+ # HandshakeType.certificate: Certificate,
+ # HandshakeType.certificate_verify: CertificateVerify,
+ # HandshakeType.finished: Finished,
+ # HandshakeType.new_session_ticket: NewSessionTicket,
+ # HandshakeType.key_update: KeyUpdate,
+ },
+ default=FixedSized(this.length, GreedyBytes),
+ ),
+)
+
+# Records
+
+ContentType = Enum(
+ Byte,
+ invalid=0,
+ change_cipher_spec=20,
+ alert=21,
+ handshake=22,
+ application_data=23,
+)
+
+Plaintext = Struct(
+ "type" / ContentType,
+ "legacy_record_version" / ProtocolVersion,
+ "length" / Int16ub,
+ "fragment" / FixedSized(this.length, GreedyBytes),
+)
--
2.25.1
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2021-08-25 22:25 Zhihong Yu <zyu@yugabyte.com>
parent: Jacob Champion <pchampion@vmware.com>
2 siblings, 1 reply; 424+ messages in thread
From: Zhihong Yu @ 2021-08-25 22:25 UTC (permalink / raw)
To: Jacob Champion <pchampion@vmware.com>; +Cc: hlinnaka@iki.fi <hlinnaka@iki.fi>; pgsql-hackers; michael@paquier.xyz <michael@paquier.xyz>
On Wed, Aug 25, 2021 at 11:42 AM Jacob Champion <pchampion@vmware.com>
wrote:
> On Tue, 2021-06-22 at 23:22 +0000, Jacob Champion wrote:
> > On Fri, 2021-06-18 at 11:31 +0300, Heikki Linnakangas wrote:
> > >
> > > A few small things caught my eye in the backend oauth_exchange
> function:
> > >
> > > > + /* Handle the client's initial message. */
> > > > + p = strdup(input);
> > >
> > > this strdup() should be pstrdup().
> >
> > Thanks, I'll fix that in the next re-roll.
> >
> > > In the same function, there are a bunch of reports like this:
> > >
> > > > ereport(ERROR,
> > > > + (errcode(ERRCODE_PROTOCOL_VIOLATION),
> > > > + errmsg("malformed OAUTHBEARER message"),
> > > > + errdetail("Comma expected, but found
> character \"%s\".",
> > > > + sanitize_char(*p))));
> > >
> > > I don't think the double quotes are needed here, because sanitize_char
> > > will return quotes if it's a single character. So it would end up
> > > looking like this: ... found character "'x'".
> >
> > I'll fix this too. Thanks!
>
> v2, attached, incorporates Heikki's suggested fixes and also rebases on
> top of latest HEAD, which had the SASL refactoring changes committed
> last month.
>
> The biggest change from the last patchset is 0001, an attempt at
> enabling jsonapi in the frontend without the use of palloc(), based on
> suggestions by Michael and Tom from last commitfest. I've also made
> some improvements to the pytest suite. No major changes to the OAuth
> implementation yet.
>
> --Jacob
>
Hi,
For v2-0001-common-jsonapi-support-FRONTEND-clients.patch :
+ /* Clean up. */
+ termJsonLexContext(&lex);
At the end of termJsonLexContext(), empty is copied to lex. For stack
based JsonLexContext, the copy seems unnecessary.
Maybe introduce a boolean parameter for termJsonLexContext() to signal that
the copy can be omitted ?
+#ifdef FRONTEND
+ /* make sure initialization succeeded */
+ if (lex->strval == NULL)
+ return JSON_OUT_OF_MEMORY;
Should PQExpBufferBroken(lex->strval) be used for the check ?
Thanks
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2021-08-25 23:24 Zhihong Yu <zyu@yugabyte.com>
parent: Zhihong Yu <zyu@yugabyte.com>
0 siblings, 1 reply; 424+ messages in thread
From: Zhihong Yu @ 2021-08-25 23:24 UTC (permalink / raw)
To: Jacob Champion <pchampion@vmware.com>; +Cc: hlinnaka@iki.fi <hlinnaka@iki.fi>; pgsql-hackers; michael@paquier.xyz <michael@paquier.xyz>
On Wed, Aug 25, 2021 at 3:25 PM Zhihong Yu <zyu@yugabyte.com> wrote:
>
>
> On Wed, Aug 25, 2021 at 11:42 AM Jacob Champion <pchampion@vmware.com>
> wrote:
>
>> On Tue, 2021-06-22 at 23:22 +0000, Jacob Champion wrote:
>> > On Fri, 2021-06-18 at 11:31 +0300, Heikki Linnakangas wrote:
>> > >
>> > > A few small things caught my eye in the backend oauth_exchange
>> function:
>> > >
>> > > > + /* Handle the client's initial message. */
>> > > > + p = strdup(input);
>> > >
>> > > this strdup() should be pstrdup().
>> >
>> > Thanks, I'll fix that in the next re-roll.
>> >
>> > > In the same function, there are a bunch of reports like this:
>> > >
>> > > > ereport(ERROR,
>> > > > + (errcode(ERRCODE_PROTOCOL_VIOLATION),
>> > > > + errmsg("malformed OAUTHBEARER message"),
>> > > > + errdetail("Comma expected, but found
>> character \"%s\".",
>> > > > + sanitize_char(*p))));
>> > >
>> > > I don't think the double quotes are needed here, because
>> sanitize_char
>> > > will return quotes if it's a single character. So it would end up
>> > > looking like this: ... found character "'x'".
>> >
>> > I'll fix this too. Thanks!
>>
>> v2, attached, incorporates Heikki's suggested fixes and also rebases on
>> top of latest HEAD, which had the SASL refactoring changes committed
>> last month.
>>
>> The biggest change from the last patchset is 0001, an attempt at
>> enabling jsonapi in the frontend without the use of palloc(), based on
>> suggestions by Michael and Tom from last commitfest. I've also made
>> some improvements to the pytest suite. No major changes to the OAuth
>> implementation yet.
>>
>> --Jacob
>>
> Hi,
> For v2-0001-common-jsonapi-support-FRONTEND-clients.patch :
>
> + /* Clean up. */
> + termJsonLexContext(&lex);
>
> At the end of termJsonLexContext(), empty is copied to lex. For stack
> based JsonLexContext, the copy seems unnecessary.
> Maybe introduce a boolean parameter for termJsonLexContext() to signal
> that the copy can be omitted ?
>
> +#ifdef FRONTEND
> + /* make sure initialization succeeded */
> + if (lex->strval == NULL)
> + return JSON_OUT_OF_MEMORY;
>
> Should PQExpBufferBroken(lex->strval) be used for the check ?
>
> Thanks
>
Hi,
For v2-0002-libpq-add-OAUTHBEARER-SASL-mechanism.patch :
+ i_init_session(&session);
+
+ if (!conn->oauth_client_id)
+ {
+ /* We can't talk to a server without a client identifier. */
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("no oauth_client_id is set for
the connection"));
+ goto cleanup;
Can conn->oauth_client_id check be performed ahead of i_init_session() ?
That way, ```goto cleanup``` can be replaced with return.
+ if (!error_code || (strcmp(error_code, "authorization_pending")
+ && strcmp(error_code, "slow_down")))
What if, in the future, there is error code different from the above two
which doesn't represent "OAuth token retrieval failed" scenario ?
For client_initial_response(),
+ token_buf = createPQExpBuffer();
+ if (!token_buf)
+ goto cleanup;
If token_buf is NULL, there doesn't seem to be anything to free. We can
return directly.
Cheers
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2021-08-26 16:13 Jacob Champion <pchampion@vmware.com>
parent: Zhihong Yu <zyu@yugabyte.com>
0 siblings, 2 replies; 424+ messages in thread
From: Jacob Champion @ 2021-08-26 16:13 UTC (permalink / raw)
To: zyu@yugabyte.com <zyu@yugabyte.com>; +Cc: hlinnaka@iki.fi <hlinnaka@iki.fi>; pgsql-hackers; michael@paquier.xyz <michael@paquier.xyz>
On Wed, 2021-08-25 at 15:25 -0700, Zhihong Yu wrote:
>
> Hi,
> For v2-0001-common-jsonapi-support-FRONTEND-clients.patch :
>
> + /* Clean up. */
> + termJsonLexContext(&lex);
>
> At the end of termJsonLexContext(), empty is copied to lex. For stack
> based JsonLexContext, the copy seems unnecessary.
> Maybe introduce a boolean parameter for termJsonLexContext() to
> signal that the copy can be omitted ?
Do you mean heap-based? i.e. destroyJsonLexContext() does an
unnecessary copy before free? Yeah, in that case it's not super useful,
but I think I'd want some evidence that the performance hit matters
before optimizing it.
Are there any other internal APIs that take a boolean parameter like
that? If not, I think we'd probably just want to remove the copy
entirely if it's a problem.
> +#ifdef FRONTEND
> + /* make sure initialization succeeded */
> + if (lex->strval == NULL)
> + return JSON_OUT_OF_MEMORY;
>
> Should PQExpBufferBroken(lex->strval) be used for the check ?
It should be okay to continue if the strval is broken but non-NULL,
since it's about to be reset. That has the fringe benefit of allowing
the function to go as far as possible without failing, though that's
probably a pretty weak justification.
In practice, do you think that the probability of success is low enough
that we should just short-circuit and be done with it?
On Wed, 2021-08-25 at 16:24 -0700, Zhihong Yu wrote:
>
> For v2-0002-libpq-add-OAUTHBEARER-SASL-mechanism.patch :
>
> + i_init_session(&session);
> +
> + if (!conn->oauth_client_id)
> + {
> + /* We can't talk to a server without a client identifier. */
> + appendPQExpBufferStr(&conn->errorMessage,
> + libpq_gettext("no oauth_client_id is set for the connection"));
> + goto cleanup;
>
> Can conn->oauth_client_id check be performed ahead
> of i_init_session() ? That way, ```goto cleanup``` can be replaced
> with return.
Yeah, I think that makes sense. FYI, this is probably one of the
functions that will be rewritten completely once iddawc is removed.
> + if (!error_code || (strcmp(error_code, "authorization_pending")
> + && strcmp(error_code, "slow_down")))
>
> What if, in the future, there is error code different from the above
> two which doesn't represent "OAuth token retrieval failed" scenario ?
We'd have to update our code; that would be a breaking change to the
Device Authorization spec. Here's what it says today [1]:
The "authorization_pending" and "slow_down" error codes define
particularly unique behavior, as they indicate that the OAuth client
should continue to poll the token endpoint by repeating the token
request (implementing the precise behavior defined above). If the
client receives an error response with any other error code, it MUST
stop polling and SHOULD react accordingly, for example, by displaying
an error to the user.
> For client_initial_response(),
>
> + token_buf = createPQExpBuffer();
> + if (!token_buf)
> + goto cleanup;
>
> If token_buf is NULL, there doesn't seem to be anything to free. We
> can return directly.
That's true today, but implementations have a habit of changing. I
personally prefer not to introduce too many exit points from a function
that's already using goto. In my experience, that makes future
maintenance harder.
Thanks for the reviews! Have you been able to give the patchset a try
with an OAuth deployment?
--Jacob
[1] https://datatracker.ietf.org/doc/html/rfc8628#section-3.5
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2021-08-26 16:20 Zhihong Yu <zyu@yugabyte.com>
parent: Jacob Champion <pchampion@vmware.com>
1 sibling, 0 replies; 424+ messages in thread
From: Zhihong Yu @ 2021-08-26 16:20 UTC (permalink / raw)
To: Jacob Champion <pchampion@vmware.com>; +Cc: hlinnaka@iki.fi <hlinnaka@iki.fi>; pgsql-hackers; michael@paquier.xyz <michael@paquier.xyz>
On Thu, Aug 26, 2021 at 9:13 AM Jacob Champion <pchampion@vmware.com> wrote:
> On Wed, 2021-08-25 at 15:25 -0700, Zhihong Yu wrote:
> >
> > Hi,
> > For v2-0001-common-jsonapi-support-FRONTEND-clients.patch :
> >
> > + /* Clean up. */
> > + termJsonLexContext(&lex);
> >
> > At the end of termJsonLexContext(), empty is copied to lex. For stack
> > based JsonLexContext, the copy seems unnecessary.
> > Maybe introduce a boolean parameter for termJsonLexContext() to
> > signal that the copy can be omitted ?
>
> Do you mean heap-based? i.e. destroyJsonLexContext() does an
> unnecessary copy before free? Yeah, in that case it's not super useful,
> but I think I'd want some evidence that the performance hit matters
> before optimizing it.
>
> Are there any other internal APIs that take a boolean parameter like
> that? If not, I think we'd probably just want to remove the copy
> entirely if it's a problem.
>
> > +#ifdef FRONTEND
> > + /* make sure initialization succeeded */
> > + if (lex->strval == NULL)
> > + return JSON_OUT_OF_MEMORY;
> >
> > Should PQExpBufferBroken(lex->strval) be used for the check ?
>
> It should be okay to continue if the strval is broken but non-NULL,
> since it's about to be reset. That has the fringe benefit of allowing
> the function to go as far as possible without failing, though that's
> probably a pretty weak justification.
>
> In practice, do you think that the probability of success is low enough
> that we should just short-circuit and be done with it?
>
> On Wed, 2021-08-25 at 16:24 -0700, Zhihong Yu wrote:
> >
> > For v2-0002-libpq-add-OAUTHBEARER-SASL-mechanism.patch :
> >
> > + i_init_session(&session);
> > +
> > + if (!conn->oauth_client_id)
> > + {
> > + /* We can't talk to a server without a client identifier. */
> > + appendPQExpBufferStr(&conn->errorMessage,
> > + libpq_gettext("no oauth_client_id is set
> for the connection"));
> > + goto cleanup;
> >
> > Can conn->oauth_client_id check be performed ahead
> > of i_init_session() ? That way, ```goto cleanup``` can be replaced
> > with return.
>
> Yeah, I think that makes sense. FYI, this is probably one of the
> functions that will be rewritten completely once iddawc is removed.
>
> > + if (!error_code || (strcmp(error_code, "authorization_pending")
> > + && strcmp(error_code, "slow_down")))
> >
> > What if, in the future, there is error code different from the above
> > two which doesn't represent "OAuth token retrieval failed" scenario ?
>
> We'd have to update our code; that would be a breaking change to the
> Device Authorization spec. Here's what it says today [1]:
>
> The "authorization_pending" and "slow_down" error codes define
> particularly unique behavior, as they indicate that the OAuth client
> should continue to poll the token endpoint by repeating the token
> request (implementing the precise behavior defined above). If the
> client receives an error response with any other error code, it MUST
> stop polling and SHOULD react accordingly, for example, by displaying
> an error to the user.
>
> > For client_initial_response(),
> >
> > + token_buf = createPQExpBuffer();
> > + if (!token_buf)
> > + goto cleanup;
> >
> > If token_buf is NULL, there doesn't seem to be anything to free. We
> > can return directly.
>
> That's true today, but implementations have a habit of changing. I
> personally prefer not to introduce too many exit points from a function
> that's already using goto. In my experience, that makes future
> maintenance harder.
>
> Thanks for the reviews! Have you been able to give the patchset a try
> with an OAuth deployment?
>
> --Jacob
>
> [1] https://datatracker.ietf.org/doc/html/rfc8628#section-3.5
Hi,
bq. destroyJsonLexContext() does an unnecessary copy before free? Yeah, in
that case it's not super useful,
but I think I'd want some evidence that the performance hit matters before
optimizing it.
Yes I agree.
bq. In practice, do you think that the probability of success is low enough
that we should just short-circuit and be done with it?
Haven't had a chance to try your patches out yet.
I will leave this to people who are more familiar with OAuth
implementation(s).
bq. I personally prefer not to introduce too many exit points from a
function that's already using goto.
Fair enough.
Cheers
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2021-08-27 02:32 Michael Paquier <michael@paquier.xyz>
parent: Jacob Champion <pchampion@vmware.com>
1 sibling, 1 reply; 424+ messages in thread
From: Michael Paquier @ 2021-08-27 02:32 UTC (permalink / raw)
To: Jacob Champion <pchampion@vmware.com>; +Cc: zyu@yugabyte.com <zyu@yugabyte.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; pgsql-hackers
On Thu, Aug 26, 2021 at 04:13:08PM +0000, Jacob Champion wrote:
> Do you mean heap-based? i.e. destroyJsonLexContext() does an
> unnecessary copy before free? Yeah, in that case it's not super useful,
> but I think I'd want some evidence that the performance hit matters
> before optimizing it.
As an authentication code path, the impact is minimal and my take on
that would be to keep the code simple. Now if you'd really wish to
stress that without relying on the backend, one simple way is to use
pgbench -C -n with a mostly-empty script (one meta-command) coupled
with some profiling.
--
Michael
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2021-08-31 20:48 Jacob Champion <pchampion@vmware.com>
parent: Michael Paquier <michael@paquier.xyz>
0 siblings, 0 replies; 424+ messages in thread
From: Jacob Champion @ 2021-08-31 20:48 UTC (permalink / raw)
To: michael@paquier.xyz <michael@paquier.xyz>; +Cc: zyu@yugabyte.com <zyu@yugabyte.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; pgsql-hackers
On Fri, 2021-08-27 at 11:32 +0900, Michael Paquier wrote:
> Now if you'd really wish to
> stress that without relying on the backend, one simple way is to use
> pgbench -C -n with a mostly-empty script (one meta-command) coupled
> with some profiling.
Ah, thanks! I'll add that to the toolbox.
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-03-04 19:13 Jacob Champion <pchampion@vmware.com>
parent: Jacob Champion <pchampion@vmware.com>
2 siblings, 2 replies; 424+ messages in thread
From: Jacob Champion @ 2022-03-04 19:13 UTC (permalink / raw)
To: pgsql-hackers; +Cc: hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>
Hi all,
v3 rebases this patchset over the top of Samay's pluggable auth
provider API [1], included here as patches 0001-3. The final patch in
the set ports the server implementation from a core feature to a
contrib module; to switch between the two approaches, simply leave out
that final patch.
There are still some backend changes that must be made to get this
working, as pointed out in 0009, and obviously libpq support still
requires code changes.
--Jacob
[1] https://www.postgresql.org/message-id/flat/CAJxrbyxTRn5P8J-p%2BwHLwFahK5y56PhK28VOb55jqMO05Y-DJw%40m...
Attachments: download all (9 files, .tar.gz)
v3-0001-Add-support-for-custom-authentication-methods.patch (text/x-patch, 11.3K) download
From 206060ed1b31fcec48fb6ee05d61b135ec98cecf Mon Sep 17 00:00:00 2001
From: Samay Sharma <smilingsamay@gmail.com>
Date: Tue, 15 Feb 2022 22:23:29 -0800
Subject: [PATCH v3 1/9] Add support for custom authentication methods
Currently, PostgreSQL supports only a set of pre-defined authentication
methods. This patch adds support for 2 hooks which allow users to add
their custom authentication methods by defining a check function and an
error function. Users can then use these methods by using a new "custom"
keyword in pg_hba.conf and specifying the authentication provider they
want to use.
---
src/backend/libpq/auth.c | 89 ++++++++++++++++++++++++++++++----------
src/backend/libpq/hba.c | 44 ++++++++++++++++++++
src/include/libpq/auth.h | 27 ++++++++++++
src/include/libpq/hba.h | 4 ++
4 files changed, 143 insertions(+), 21 deletions(-)
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index efc53f3135..3533b0bc50 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -47,8 +47,6 @@
*----------------------------------------------------------------
*/
static void auth_failed(Port *port, int status, const char *logdetail);
-static char *recv_password_packet(Port *port);
-static void set_authn_id(Port *port, const char *id);
/*----------------------------------------------------------------
@@ -206,23 +204,6 @@ static int pg_SSPI_make_upn(char *accountname,
static int CheckRADIUSAuth(Port *port);
static int PerformRadiusTransaction(const char *server, const char *secret, const char *portstr, const char *identifier, const char *user_name, const char *passwd);
-
-/*
- * Maximum accepted size of GSS and SSPI authentication tokens.
- * We also use this as a limit on ordinary password packet lengths.
- *
- * Kerberos tickets are usually quite small, but the TGTs issued by Windows
- * domain controllers include an authorization field known as the Privilege
- * Attribute Certificate (PAC), which contains the user's Windows permissions
- * (group memberships etc.). The PAC is copied into all tickets obtained on
- * the basis of this TGT (even those issued by Unix realms which the Windows
- * realm trusts), and can be several kB in size. The maximum token size
- * accepted by Windows systems is determined by the MaxAuthToken Windows
- * registry setting. Microsoft recommends that it is not set higher than
- * 65535 bytes, so that seems like a reasonable limit for us as well.
- */
-#define PG_MAX_AUTH_TOKEN_LENGTH 65535
-
/*----------------------------------------------------------------
* Global authentication functions
*----------------------------------------------------------------
@@ -235,6 +216,16 @@ static int PerformRadiusTransaction(const char *server, const char *secret, cons
*/
ClientAuthentication_hook_type ClientAuthentication_hook = NULL;
+/*
+ * These hooks allow plugins to get control of the client authentication check
+ * and error reporting logic. This allows users to write extensions to
+ * implement authentication using any protocol of their choice. To acquire these
+ * hooks, plugins need to call the RegisterAuthProvider() function.
+ */
+static CustomAuthenticationCheck_hook_type CustomAuthenticationCheck_hook = NULL;
+static CustomAuthenticationError_hook_type CustomAuthenticationError_hook = NULL;
+char *custom_provider_name = NULL;
+
/*
* Tell the user the authentication failed, but not (much about) why.
*
@@ -311,6 +302,12 @@ auth_failed(Port *port, int status, const char *logdetail)
case uaRADIUS:
errstr = gettext_noop("RADIUS authentication failed for user \"%s\"");
break;
+ case uaCustom:
+ if (CustomAuthenticationError_hook)
+ errstr = CustomAuthenticationError_hook(port);
+ else
+ errstr = gettext_noop("Custom authentication failed for user \"%s\"");
+ break;
default:
errstr = gettext_noop("authentication failed for user \"%s\": invalid authentication method");
break;
@@ -345,7 +342,7 @@ auth_failed(Port *port, int status, const char *logdetail)
* lifetime of the Port, so it is safe to pass a string that is managed by an
* external library.
*/
-static void
+void
set_authn_id(Port *port, const char *id)
{
Assert(id);
@@ -630,6 +627,10 @@ ClientAuthentication(Port *port)
case uaTrust:
status = STATUS_OK;
break;
+ case uaCustom:
+ if (CustomAuthenticationCheck_hook)
+ status = CustomAuthenticationCheck_hook(port);
+ break;
}
if ((status == STATUS_OK && port->hba->clientcert == clientCertFull)
@@ -689,7 +690,7 @@ sendAuthRequest(Port *port, AuthRequest areq, const char *extradata, int extrale
*
* Returns NULL if couldn't get password, else palloc'd string.
*/
-static char *
+char *
recv_password_packet(Port *port)
{
StringInfoData buf;
@@ -3343,3 +3344,49 @@ PerformRadiusTransaction(const char *server, const char *secret, const char *por
}
} /* while (true) */
}
+
+/*----------------------------------------------------------------
+ * Custom authentication
+ *----------------------------------------------------------------
+ */
+
+/*
+ * RegisterAuthProvider registers a custom authentication provider to be
+ * used for authentication. Currently, we allow only one authentication
+ * provider to be registered for use at a time.
+ *
+ * This function should be called in _PG_init() by any extension looking to
+ * add a custom authentication method.
+ */
+void RegisterAuthProvider(const char *provider_name,
+ CustomAuthenticationCheck_hook_type AuthenticationCheckFunction,
+ CustomAuthenticationError_hook_type AuthenticationErrorFunction)
+{
+ if (provider_name == NULL)
+ {
+ ereport(ERROR,
+ (errmsg("cannot register authentication provider without name")));
+ }
+
+ if (AuthenticationCheckFunction == NULL)
+ {
+ ereport(ERROR,
+ (errmsg("cannot register authentication provider without a check function")));
+ }
+
+ if (custom_provider_name)
+ {
+ ereport(ERROR,
+ (errmsg("cannot register authentication provider %s", provider_name),
+ errdetail("Only one authentication provider allowed. Provider %s is already registered.",
+ custom_provider_name)));
+ }
+
+ /*
+ * Allocate in top memory context as we need to read this whenever
+ * we parse pg_hba.conf
+ */
+ custom_provider_name = MemoryContextStrdup(TopMemoryContext,provider_name);
+ CustomAuthenticationCheck_hook = AuthenticationCheckFunction;
+ CustomAuthenticationError_hook = AuthenticationErrorFunction;
+}
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index d84a40b726..ebae992964 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -134,6 +134,7 @@ static const char *const UserAuthName[] =
"ldap",
"cert",
"radius",
+ "custom",
"peer"
};
@@ -1399,6 +1400,8 @@ parse_hba_line(TokenizedLine *tok_line, int elevel)
#endif
else if (strcmp(token->string, "radius") == 0)
parsedline->auth_method = uaRADIUS;
+ else if (strcmp(token->string, "custom") == 0)
+ parsedline->auth_method = uaCustom;
else
{
ereport(elevel,
@@ -1691,6 +1694,14 @@ parse_hba_line(TokenizedLine *tok_line, int elevel)
parsedline->clientcert = clientCertFull;
}
+ /*
+ * Ensure that the provider name is specified for custom authentication method.
+ */
+ if (parsedline->auth_method == uaCustom)
+ {
+ MANDATORY_AUTH_ARG(parsedline->custom_provider, "provider", "custom");
+ }
+
return parsedline;
}
@@ -2102,6 +2113,32 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->radiusidentifiers = parsed_identifiers;
hbaline->radiusidentifiers_s = pstrdup(val);
}
+ else if (strcmp(name, "provider") == 0)
+ {
+ REQUIRE_AUTH_OPTION(uaCustom, "provider", "custom");
+
+ /*
+ * Verify that the provider mentioned is same as the one loaded
+ * via shared_preload_libraries.
+ */
+
+ if (custom_provider_name == NULL || strcmp(val,custom_provider_name) != 0)
+ {
+ ereport(elevel,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ errmsg("cannot use authentication provider %s",val),
+ errhint("Load authentication provider via shared_preload_libraries."),
+ errcontext("line %d of configuration file \"%s\"",
+ line_num, HbaFileName)));
+ *err_msg = psprintf("cannot use authentication provider %s", val);
+
+ return false;
+ }
+ else
+ {
+ hbaline->custom_provider = pstrdup(val);
+ }
+ }
else
{
ereport(elevel,
@@ -2442,6 +2479,13 @@ gethba_options(HbaLine *hba)
CStringGetTextDatum(psprintf("radiusports=%s", hba->radiusports_s));
}
+ if (hba->auth_method == uaCustom)
+ {
+ if (hba->custom_provider)
+ options[noptions++] =
+ CStringGetTextDatum(psprintf("provider=%s",hba->custom_provider));
+ }
+
/* If you add more options, consider increasing MAX_HBA_OPTIONS. */
Assert(noptions <= MAX_HBA_OPTIONS);
diff --git a/src/include/libpq/auth.h b/src/include/libpq/auth.h
index 6d7ee1acb9..1d10cccc1b 100644
--- a/src/include/libpq/auth.h
+++ b/src/include/libpq/auth.h
@@ -23,9 +23,36 @@ extern char *pg_krb_realm;
extern void ClientAuthentication(Port *port);
extern void sendAuthRequest(Port *port, AuthRequest areq, const char *extradata,
int extralen);
+extern void set_authn_id(Port *port, const char *id);
+extern char *recv_password_packet(Port *port);
/* Hook for plugins to get control in ClientAuthentication() */
+typedef int (*CustomAuthenticationCheck_hook_type) (Port *);
typedef void (*ClientAuthentication_hook_type) (Port *, int);
extern PGDLLIMPORT ClientAuthentication_hook_type ClientAuthentication_hook;
+/* Hook for plugins to report error messages in auth_failed() */
+typedef const char * (*CustomAuthenticationError_hook_type) (Port *);
+
+extern void RegisterAuthProvider
+ (const char *provider_name,
+ CustomAuthenticationCheck_hook_type CustomAuthenticationCheck_hook,
+ CustomAuthenticationError_hook_type CustomAuthenticationError_hook);
+
+/*
+ * Maximum accepted size of GSS and SSPI authentication tokens.
+ * We also use this as a limit on ordinary password packet lengths.
+ *
+ * Kerberos tickets are usually quite small, but the TGTs issued by Windows
+ * domain controllers include an authorization field known as the Privilege
+ * Attribute Certificate (PAC), which contains the user's Windows permissions
+ * (group memberships etc.). The PAC is copied into all tickets obtained on
+ * the basis of this TGT (even those issued by Unix realms which the Windows
+ * realm trusts), and can be several kB in size. The maximum token size
+ * accepted by Windows systems is determined by the MaxAuthToken Windows
+ * registry setting. Microsoft recommends that it is not set higher than
+ * 65535 bytes, so that seems like a reasonable limit for us as well.
+ */
+#define PG_MAX_AUTH_TOKEN_LENGTH 65535
+
#endif /* AUTH_H */
diff --git a/src/include/libpq/hba.h b/src/include/libpq/hba.h
index 8d9f3821b1..c5aef6994c 100644
--- a/src/include/libpq/hba.h
+++ b/src/include/libpq/hba.h
@@ -38,6 +38,7 @@ typedef enum UserAuth
uaLDAP,
uaCert,
uaRADIUS,
+ uaCustom,
uaPeer
#define USER_AUTH_LAST uaPeer /* Must be last value of this enum */
} UserAuth;
@@ -120,6 +121,7 @@ typedef struct HbaLine
char *radiusidentifiers_s;
List *radiusports;
char *radiusports_s;
+ char *custom_provider;
} HbaLine;
typedef struct IdentLine
@@ -144,4 +146,6 @@ extern int check_usermap(const char *usermap_name,
bool case_sensitive);
extern bool pg_isblank(const char c);
+extern char *custom_provider_name;
+
#endif /* HBA_H */
--
2.25.1
v3-0002-Add-sample-extension-to-test-custom-auth-provider.patch (text/x-patch, 3.6K) download
From 93b108334fa9fcc2d5f68d75fc320cd2714eb9da Mon Sep 17 00:00:00 2001
From: Samay Sharma <smilingsamay@gmail.com>
Date: Tue, 15 Feb 2022 22:28:40 -0800
Subject: [PATCH v3 2/9] Add sample extension to test custom auth provider
hooks
This change adds a new extension to src/test/modules to
test the custom authentication provider hooks. In this
extension, we use an array to define which users to
authenticate and what passwords to use.
---
src/test/modules/test_auth_provider/Makefile | 16 ++++
.../test_auth_provider/test_auth_provider.c | 90 +++++++++++++++++++
2 files changed, 106 insertions(+)
create mode 100644 src/test/modules/test_auth_provider/Makefile
create mode 100644 src/test/modules/test_auth_provider/test_auth_provider.c
diff --git a/src/test/modules/test_auth_provider/Makefile b/src/test/modules/test_auth_provider/Makefile
new file mode 100644
index 0000000000..17971a5c7a
--- /dev/null
+++ b/src/test/modules/test_auth_provider/Makefile
@@ -0,0 +1,16 @@
+# src/test/modules/test_auth_provider/Makefile
+
+MODULE_big = test_auth_provider
+OBJS = test_auth_provider.o
+PGFILEDESC = "test_auth_provider - provider to test auth hooks"
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/test_auth_provider
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/test_auth_provider/test_auth_provider.c b/src/test/modules/test_auth_provider/test_auth_provider.c
new file mode 100644
index 0000000000..477ef8b2c3
--- /dev/null
+++ b/src/test/modules/test_auth_provider/test_auth_provider.c
@@ -0,0 +1,90 @@
+/* -------------------------------------------------------------------------
+ *
+ * test_auth_provider.c
+ * example authentication provider plugin
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * contrib/test_auth_provider/test_auth_provider.c
+ *
+ * -------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "libpq/auth.h"
+#include "libpq/libpq.h"
+
+PG_MODULE_MAGIC;
+
+void _PG_init(void);
+
+static char *get_password_for_user(char *user_name);
+
+/*
+ * List of usernames / passwords to approve. Here we are not
+ * getting passwords from Postgres but from this list. In a more real-life
+ * extension, you can fetch valid credentials and authentication tokens /
+ * passwords from an external authentication provider.
+ */
+char credentials[3][3][50] = {
+ {"bob","alice","carol"},
+ {"bob123","alice123","carol123"}
+};
+
+static int TestAuthenticationCheck(Port *port)
+{
+ char *passwd;
+ int result = STATUS_ERROR;
+ char *real_pass;
+
+ sendAuthRequest(port, AUTH_REQ_PASSWORD, NULL, 0);
+
+ passwd = recv_password_packet(port);
+ if (passwd == NULL)
+ return STATUS_EOF;
+
+ real_pass = get_password_for_user(port->user_name);
+ if (real_pass)
+ {
+ if(strcmp(passwd, real_pass) == 0)
+ {
+ result = STATUS_OK;
+ }
+ pfree(real_pass);
+ }
+
+ pfree(passwd);
+
+ return result;
+}
+
+static char *
+get_password_for_user(char *user_name)
+{
+ char *password = NULL;
+ int i;
+ for (i=0; i<3; i++)
+ {
+ if (strcmp(user_name, credentials[0][i]) == 0)
+ {
+ password = pstrdup(credentials[1][i]);
+ }
+ }
+
+ return password;
+}
+
+static const char *TestAuthenticationError(Port *port)
+{
+ char *error_message = (char *)palloc (100);
+ sprintf(error_message, "Test authentication failed for user %s", port->user_name);
+ return error_message;
+}
+
+void
+_PG_init(void)
+{
+ RegisterAuthProvider("test", TestAuthenticationCheck, TestAuthenticationError);
+}
--
2.25.1
v3-0003-Add-tests-for-test_auth_provider-extension.patch (text/x-patch, 6.1K) download
From 3aa2535fa42b142beabdcef234d1939738f36b9a Mon Sep 17 00:00:00 2001
From: Samay Sharma <smilingsamay@gmail.com>
Date: Wed, 16 Feb 2022 12:28:36 -0800
Subject: [PATCH v3 3/9] Add tests for test_auth_provider extension
Add tap tests for test_auth_provider extension allow make check in
src/test/modules to run them.
---
src/test/modules/Makefile | 1 +
src/test/modules/test_auth_provider/Makefile | 2 +
.../test_auth_provider/t/001_custom_auth.pl | 125 ++++++++++++++++++
3 files changed, 128 insertions(+)
create mode 100644 src/test/modules/test_auth_provider/t/001_custom_auth.pl
diff --git a/src/test/modules/Makefile b/src/test/modules/Makefile
index dffc79b2d9..f56533ea13 100644
--- a/src/test/modules/Makefile
+++ b/src/test/modules/Makefile
@@ -14,6 +14,7 @@ SUBDIRS = \
plsample \
snapshot_too_old \
spgist_name_ops \
+ test_auth_provider \
test_bloomfilter \
test_ddl_deparse \
test_extensions \
diff --git a/src/test/modules/test_auth_provider/Makefile b/src/test/modules/test_auth_provider/Makefile
index 17971a5c7a..7d601cf7d5 100644
--- a/src/test/modules/test_auth_provider/Makefile
+++ b/src/test/modules/test_auth_provider/Makefile
@@ -4,6 +4,8 @@ MODULE_big = test_auth_provider
OBJS = test_auth_provider.o
PGFILEDESC = "test_auth_provider - provider to test auth hooks"
+TAP_TESTS = 1
+
ifdef USE_PGXS
PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
diff --git a/src/test/modules/test_auth_provider/t/001_custom_auth.pl b/src/test/modules/test_auth_provider/t/001_custom_auth.pl
new file mode 100644
index 0000000000..3b7472dc7f
--- /dev/null
+++ b/src/test/modules/test_auth_provider/t/001_custom_auth.pl
@@ -0,0 +1,125 @@
+
+# Copyright (c) 2021-2022, PostgreSQL Global Development Group
+
+# Set of tests for testing custom authentication hooks.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+# Delete pg_hba.conf from the given node, add a new entry to it
+# and then execute a reload to refresh it.
+sub reset_pg_hba
+{
+ my $node = shift;
+ my $hba_method = shift;
+
+ unlink($node->data_dir . '/pg_hba.conf');
+ # just for testing purposes, use a continuation line
+ $node->append_conf('pg_hba.conf', "local all all\\\n $hba_method");
+ $node->reload;
+ return;
+}
+
+# Test if you get expected results in pg_hba_file_rules error column after
+# changing pg_hba.conf and reloading it.
+sub test_hba_reload
+{
+ my ($node, $method, $expected_res) = @_;
+ my $status_string = 'failed';
+ $status_string = 'success' if ($expected_res eq 0);
+ my $testname = "pg_hba.conf reload $status_string for method $method";
+
+ reset_pg_hba($node, $method);
+
+ my ($cmdret, $stdout, $stderr) = $node->psql("postgres",
+ "select count(*) from pg_hba_file_rules where error is not null",extra_params => ['-U','bob']);
+
+ is($stdout, $expected_res, $testname);
+}
+
+# Test access for a single role, useful to wrap all tests into one. Extra
+# named parameters are passed to connect_ok/fails as-is.
+sub test_role
+{
+ local $Test::Builder::Level = $Test::Builder::Level + 1;
+
+ my ($node, $role, $method, $expected_res, %params) = @_;
+ my $status_string = 'failed';
+ $status_string = 'success' if ($expected_res eq 0);
+
+ my $connstr = "user=$role";
+ my $testname =
+ "authentication $status_string for method $method, role $role";
+
+ if ($expected_res eq 0)
+ {
+ $node->connect_ok($connstr, $testname, %params);
+ }
+ else
+ {
+ # No checks of the error message, only the status code.
+ $node->connect_fails($connstr, $testname, %params);
+ }
+}
+
+# Initialize server node
+my $node = PostgreSQL::Test::Cluster->new('server');
+$node->init;
+$node->append_conf('postgresql.conf', "log_connections = on\n");
+$node->append_conf('postgresql.conf', "shared_preload_libraries = 'test_auth_provider.so'\n");
+$node->start;
+
+$node->safe_psql('postgres', "CREATE ROLE bob SUPERUSER LOGIN;");
+$node->safe_psql('postgres', "CREATE ROLE alice LOGIN;");
+$node->safe_psql('postgres', "CREATE ROLE test LOGIN;");
+
+# Add custom auth method to pg_hba.conf
+reset_pg_hba($node, 'custom provider=test');
+
+# Test that users are able to login with correct passwords.
+$ENV{"PGPASSWORD"} = 'bob123';
+test_role($node, 'bob', 'custom', 0, log_like => [qr/connection authorized: user=bob/]);
+$ENV{"PGPASSWORD"} = 'alice123';
+test_role($node, 'alice', 'custom', 0, log_like => [qr/connection authorized: user=alice/]);
+
+# Test that bad passwords are rejected.
+$ENV{"PGPASSWORD"} = 'badpassword';
+test_role($node, 'bob', 'custom', 2, log_unlike => [qr/connection authorized:/]);
+test_role($node, 'alice', 'custom', 2, log_unlike => [qr/connection authorized:/]);
+
+# Test that users not in authentication list are rejected.
+test_role($node, 'test', 'custom', 2, log_unlike => [qr/connection authorized:/]);
+
+$ENV{"PGPASSWORD"} = 'bob123';
+
+# Tests for invalid auth options
+
+# Test that an incorrect provider name is not accepted.
+test_hba_reload($node, 'custom provider=wrong', 1);
+
+# Test that specifying provider option with different auth method is not allowed.
+test_hba_reload($node, 'trust provider=test', 1);
+
+# Test that provider name is a mandatory option for custom auth.
+test_hba_reload($node, 'custom', 1);
+
+# Test that correct provider name allows reload to succeed.
+test_hba_reload($node, 'custom provider=test', 0);
+
+# Custom auth modules require mentioning extension in shared_preload_libraries.
+
+# Remove extension from shared_preload_libraries and try to restart.
+$node->adjust_conf('postgresql.conf', 'shared_preload_libraries', "''");
+command_fails(['pg_ctl', '-w', '-D', $node->data_dir, '-l', $node->logfile, 'restart'],'restart with empty shared_preload_libraries failed');
+
+# Fix shared_preload_libraries and confirm that you can now restart.
+$node->adjust_conf('postgresql.conf', 'shared_preload_libraries', "'test_auth_provider.so'");
+command_ok(['pg_ctl', '-w', '-D', $node->data_dir, '-l', $node->logfile,'start'],'restart with correct shared_preload_libraries succeeded');
+
+# Test that we can connect again
+test_role($node, 'bob', 'custom', 0, log_like => [qr/connection authorized: user=bob/]);
+
+done_testing();
--
2.25.1
v3-0004-common-jsonapi-support-FRONTEND-clients.patch (text/x-patch, 20.4K) download
From 13bee49d8c674e921804b4e6edc363dcf33211ce Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Mon, 3 May 2021 15:38:26 -0700
Subject: [PATCH v3 4/9] common/jsonapi: support FRONTEND clients
Based on a patch by Michael Paquier.
For frontend code, use PQExpBuffer instead of StringInfo. This requires
us to track allocation failures so that we can return JSON_OUT_OF_MEMORY
as needed. json_errdetail() now allocates its error message inside
memory owned by the JsonLexContext, so clients don't need to worry about
freeing it.
For convenience, the backend now has destroyJsonLexContext() to mirror
other create/destroy APIs. The frontend has init/term versions of the
API to handle stack-allocated JsonLexContexts.
We can now partially revert b44669b2ca, now that json_errdetail() works
correctly.
---
src/backend/utils/adt/jsonfuncs.c | 4 +-
src/bin/pg_verifybackup/parse_manifest.c | 13 +-
src/bin/pg_verifybackup/t/005_bad_manifest.pl | 2 +-
src/common/Makefile | 2 +-
src/common/jsonapi.c | 290 +++++++++++++-----
src/include/common/jsonapi.h | 47 ++-
6 files changed, 270 insertions(+), 88 deletions(-)
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index 2457061f97..f58233cda9 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -723,9 +723,7 @@ json_object_keys(PG_FUNCTION_ARGS)
pg_parse_json_or_ereport(lex, sem);
/* keys are now in state->result */
- pfree(lex->strval->data);
- pfree(lex->strval);
- pfree(lex);
+ destroyJsonLexContext(lex);
pfree(sem);
MemoryContextSwitchTo(oldcontext);
diff --git a/src/bin/pg_verifybackup/parse_manifest.c b/src/bin/pg_verifybackup/parse_manifest.c
index 6364b01282..4b38fd3963 100644
--- a/src/bin/pg_verifybackup/parse_manifest.c
+++ b/src/bin/pg_verifybackup/parse_manifest.c
@@ -119,7 +119,7 @@ void
json_parse_manifest(JsonManifestParseContext *context, char *buffer,
size_t size)
{
- JsonLexContext *lex;
+ JsonLexContext lex = {0};
JsonParseErrorType json_error;
JsonSemAction sem;
JsonManifestParseState parse;
@@ -129,8 +129,8 @@ json_parse_manifest(JsonManifestParseContext *context, char *buffer,
parse.state = JM_EXPECT_TOPLEVEL_START;
parse.saw_version_field = false;
- /* Create a JSON lexing context. */
- lex = makeJsonLexContextCstringLen(buffer, size, PG_UTF8, true);
+ /* Initialize a JSON lexing context. */
+ initJsonLexContextCstringLen(&lex, buffer, size, PG_UTF8, true);
/* Set up semantic actions. */
sem.semstate = &parse;
@@ -145,14 +145,17 @@ json_parse_manifest(JsonManifestParseContext *context, char *buffer,
sem.scalar = json_manifest_scalar;
/* Run the actual JSON parser. */
- json_error = pg_parse_json(lex, &sem);
+ json_error = pg_parse_json(&lex, &sem);
if (json_error != JSON_SUCCESS)
- json_manifest_parse_failure(context, "parsing failed");
+ json_manifest_parse_failure(context, json_errdetail(json_error, &lex));
if (parse.state != JM_EXPECT_EOF)
json_manifest_parse_failure(context, "manifest ended unexpectedly");
/* Verify the manifest checksum. */
verify_manifest_checksum(&parse, buffer, size);
+
+ /* Clean up. */
+ termJsonLexContext(&lex);
}
/*
diff --git a/src/bin/pg_verifybackup/t/005_bad_manifest.pl b/src/bin/pg_verifybackup/t/005_bad_manifest.pl
index 118beb53d7..f2692972fe 100644
--- a/src/bin/pg_verifybackup/t/005_bad_manifest.pl
+++ b/src/bin/pg_verifybackup/t/005_bad_manifest.pl
@@ -16,7 +16,7 @@ my $tempdir = PostgreSQL::Test::Utils::tempdir;
test_bad_manifest(
'input string ended unexpectedly',
- qr/could not parse backup manifest: parsing failed/,
+ qr/could not parse backup manifest: The input string ended unexpectedly/,
<<EOM);
{
EOM
diff --git a/src/common/Makefile b/src/common/Makefile
index 31c0dd366d..8e8b27546e 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -40,7 +40,7 @@ override CPPFLAGS += -DVAL_LDFLAGS_EX="\"$(LDFLAGS_EX)\""
override CPPFLAGS += -DVAL_LDFLAGS_SL="\"$(LDFLAGS_SL)\""
override CPPFLAGS += -DVAL_LIBS="\"$(LIBS)\""
-override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common $(CPPFLAGS)
+override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common -I$(libpq_srcdir) $(CPPFLAGS)
LIBS += $(PTHREAD_LIBS)
# If you add objects here, see also src/tools/msvc/Mkvcbuild.pm
diff --git a/src/common/jsonapi.c b/src/common/jsonapi.c
index 6666077a93..7fc5eaf460 100644
--- a/src/common/jsonapi.c
+++ b/src/common/jsonapi.c
@@ -20,10 +20,39 @@
#include "common/jsonapi.h"
#include "mb/pg_wchar.h"
-#ifndef FRONTEND
+#ifdef FRONTEND
+#include "pqexpbuffer.h"
+#else
+#include "lib/stringinfo.h"
#include "miscadmin.h"
#endif
+/*
+ * In backend, we will use palloc/pfree along with StringInfo. In frontend, use
+ * malloc and PQExpBuffer, and return JSON_OUT_OF_MEMORY on out-of-memory.
+ */
+#ifdef FRONTEND
+
+#define STRDUP(s) strdup(s)
+#define ALLOC(size) malloc(size)
+
+#define appendStrVal appendPQExpBuffer
+#define appendStrValChar appendPQExpBufferChar
+#define createStrVal createPQExpBuffer
+#define resetStrVal resetPQExpBuffer
+
+#else /* !FRONTEND */
+
+#define STRDUP(s) pstrdup(s)
+#define ALLOC(size) palloc(size)
+
+#define appendStrVal appendStringInfo
+#define appendStrValChar appendStringInfoChar
+#define createStrVal makeStringInfo
+#define resetStrVal resetStringInfo
+
+#endif
+
/*
* The context of the parser is maintained by the recursive descent
* mechanism, but is passed explicitly to the error reporting routine
@@ -132,10 +161,12 @@ IsValidJsonNumber(const char *str, int len)
return (!numeric_error) && (total_len == dummy_lex.input_length);
}
+#ifndef FRONTEND
+
/*
* makeJsonLexContextCstringLen
*
- * lex constructor, with or without StringInfo object for de-escaped lexemes.
+ * lex constructor, with or without a string object for de-escaped lexemes.
*
* Without is better as it makes the processing faster, so only make one
* if really required.
@@ -145,13 +176,66 @@ makeJsonLexContextCstringLen(char *json, int len, int encoding, bool need_escape
{
JsonLexContext *lex = palloc0(sizeof(JsonLexContext));
+ initJsonLexContextCstringLen(lex, json, len, encoding, need_escapes);
+
+ return lex;
+}
+
+void
+destroyJsonLexContext(JsonLexContext *lex)
+{
+ termJsonLexContext(lex);
+ pfree(lex);
+}
+
+#endif /* !FRONTEND */
+
+void
+initJsonLexContextCstringLen(JsonLexContext *lex, char *json, int len, int encoding, bool need_escapes)
+{
lex->input = lex->token_terminator = lex->line_start = json;
lex->line_number = 1;
lex->input_length = len;
lex->input_encoding = encoding;
- if (need_escapes)
- lex->strval = makeStringInfo();
- return lex;
+ lex->parse_strval = need_escapes;
+ if (lex->parse_strval)
+ {
+ /*
+ * This call can fail in FRONTEND code. We defer error handling to time
+ * of use (json_lex_string()) since there's no way to signal failure
+ * here, and we might not need to parse any strings anyway.
+ */
+ lex->strval = createStrVal();
+ }
+ lex->errormsg = NULL;
+}
+
+void
+termJsonLexContext(JsonLexContext *lex)
+{
+ static const JsonLexContext empty = {0};
+
+ if (lex->strval)
+ {
+#ifdef FRONTEND
+ destroyPQExpBuffer(lex->strval);
+#else
+ pfree(lex->strval->data);
+ pfree(lex->strval);
+#endif
+ }
+
+ if (lex->errormsg)
+ {
+#ifdef FRONTEND
+ destroyPQExpBuffer(lex->errormsg);
+#else
+ pfree(lex->errormsg->data);
+ pfree(lex->errormsg);
+#endif
+ }
+
+ *lex = empty;
}
/*
@@ -217,7 +301,7 @@ json_count_array_elements(JsonLexContext *lex, int *elements)
* etc, so doing this with a copy makes that safe.
*/
memcpy(©lex, lex, sizeof(JsonLexContext));
- copylex.strval = NULL; /* not interested in values here */
+ copylex.parse_strval = false; /* not interested in values here */
copylex.lex_level++;
count = 0;
@@ -279,14 +363,21 @@ parse_scalar(JsonLexContext *lex, JsonSemAction *sem)
/* extract the de-escaped string value, or the raw lexeme */
if (lex_peek(lex) == JSON_TOKEN_STRING)
{
- if (lex->strval != NULL)
- val = pstrdup(lex->strval->data);
+ if (lex->parse_strval)
+ {
+ val = STRDUP(lex->strval->data);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
}
else
{
int len = (lex->token_terminator - lex->token_start);
- val = palloc(len + 1);
+ val = ALLOC(len + 1);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+
memcpy(val, lex->token_start, len);
val[len] = '\0';
}
@@ -320,8 +411,12 @@ parse_object_field(JsonLexContext *lex, JsonSemAction *sem)
if (lex_peek(lex) != JSON_TOKEN_STRING)
return report_parse_error(JSON_PARSE_STRING, lex);
- if ((ostart != NULL || oend != NULL) && lex->strval != NULL)
- fname = pstrdup(lex->strval->data);
+ if ((ostart != NULL || oend != NULL) && lex->parse_strval)
+ {
+ fname = STRDUP(lex->strval->data);
+ if (fname == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
result = json_lex(lex);
if (result != JSON_SUCCESS)
return result;
@@ -368,6 +463,10 @@ parse_object(JsonLexContext *lex, JsonSemAction *sem)
JsonParseErrorType result;
#ifndef FRONTEND
+ /*
+ * TODO: clients need some way to put a bound on stack growth. Parse level
+ * limits maybe?
+ */
check_stack_depth();
#endif
@@ -676,8 +775,15 @@ json_lex_string(JsonLexContext *lex)
int len;
int hi_surrogate = -1;
- if (lex->strval != NULL)
- resetStringInfo(lex->strval);
+ if (lex->parse_strval)
+ {
+#ifdef FRONTEND
+ /* make sure initialization succeeded */
+ if (lex->strval == NULL)
+ return JSON_OUT_OF_MEMORY;
+#endif
+ resetStrVal(lex->strval);
+ }
Assert(lex->input_length > 0);
s = lex->token_start;
@@ -737,7 +843,7 @@ json_lex_string(JsonLexContext *lex)
return JSON_UNICODE_ESCAPE_FORMAT;
}
}
- if (lex->strval != NULL)
+ if (lex->parse_strval)
{
/*
* Combine surrogate pairs.
@@ -797,19 +903,19 @@ json_lex_string(JsonLexContext *lex)
unicode_to_utf8(ch, (unsigned char *) utf8str);
utf8len = pg_utf_mblen((unsigned char *) utf8str);
- appendBinaryStringInfo(lex->strval, utf8str, utf8len);
+ appendBinaryPQExpBuffer(lex->strval, utf8str, utf8len);
}
else if (ch <= 0x007f)
{
/* The ASCII range is the same in all encodings */
- appendStringInfoChar(lex->strval, (char) ch);
+ appendPQExpBufferChar(lex->strval, (char) ch);
}
else
return JSON_UNICODE_HIGH_ESCAPE;
#endif /* FRONTEND */
}
}
- else if (lex->strval != NULL)
+ else if (lex->parse_strval)
{
if (hi_surrogate != -1)
return JSON_UNICODE_LOW_SURROGATE;
@@ -819,22 +925,22 @@ json_lex_string(JsonLexContext *lex)
case '"':
case '\\':
case '/':
- appendStringInfoChar(lex->strval, *s);
+ appendStrValChar(lex->strval, *s);
break;
case 'b':
- appendStringInfoChar(lex->strval, '\b');
+ appendStrValChar(lex->strval, '\b');
break;
case 'f':
- appendStringInfoChar(lex->strval, '\f');
+ appendStrValChar(lex->strval, '\f');
break;
case 'n':
- appendStringInfoChar(lex->strval, '\n');
+ appendStrValChar(lex->strval, '\n');
break;
case 'r':
- appendStringInfoChar(lex->strval, '\r');
+ appendStrValChar(lex->strval, '\r');
break;
case 't':
- appendStringInfoChar(lex->strval, '\t');
+ appendStrValChar(lex->strval, '\t');
break;
default:
/* Not a valid string escape, so signal error. */
@@ -858,12 +964,12 @@ json_lex_string(JsonLexContext *lex)
}
}
- else if (lex->strval != NULL)
+ else if (lex->parse_strval)
{
if (hi_surrogate != -1)
return JSON_UNICODE_LOW_SURROGATE;
- appendStringInfoChar(lex->strval, *s);
+ appendStrValChar(lex->strval, *s);
}
}
@@ -871,6 +977,11 @@ json_lex_string(JsonLexContext *lex)
if (hi_surrogate != -1)
return JSON_UNICODE_LOW_SURROGATE;
+#ifdef FRONTEND
+ if (lex->parse_strval && PQExpBufferBroken(lex->strval))
+ return JSON_OUT_OF_MEMORY;
+#endif
+
/* Hooray, we found the end of the string! */
lex->prev_token_terminator = lex->token_terminator;
lex->token_terminator = s + 1;
@@ -1043,72 +1154,93 @@ report_parse_error(JsonParseContext ctx, JsonLexContext *lex)
return JSON_SUCCESS; /* silence stupider compilers */
}
-
-#ifndef FRONTEND
-/*
- * Extract the current token from a lexing context, for error reporting.
- */
-static char *
-extract_token(JsonLexContext *lex)
-{
- int toklen = lex->token_terminator - lex->token_start;
- char *token = palloc(toklen + 1);
-
- memcpy(token, lex->token_start, toklen);
- token[toklen] = '\0';
- return token;
-}
-
/*
* Construct a detail message for a JSON error.
*
- * Note that the error message generated by this routine may not be
- * palloc'd, making it unsafe for frontend code as there is no way to
- * know if this can be safery pfree'd or not.
+ * The returned allocation is either static or owned by the JsonLexContext and
+ * should not be freed.
*/
char *
json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
{
+ int toklen = lex->token_terminator - lex->token_start;
+
+ if (error == JSON_OUT_OF_MEMORY)
+ {
+ /* Short circuit. Allocating anything for this case is unhelpful. */
+ return _("out of memory");
+ }
+
+ if (lex->errormsg)
+ resetStrVal(lex->errormsg);
+ else
+ lex->errormsg = createStrVal();
+
switch (error)
{
case JSON_SUCCESS:
/* fall through to the error code after switch */
break;
case JSON_ESCAPING_INVALID:
- return psprintf(_("Escape sequence \"\\%s\" is invalid."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Escape sequence \"\\%.*s\" is invalid."),
+ toklen, lex->token_start);
+ break;
case JSON_ESCAPING_REQUIRED:
- return psprintf(_("Character with value 0x%02x must be escaped."),
- (unsigned char) *(lex->token_terminator));
+ appendStrVal(lex->errormsg,
+ _("Character with value 0x%02x must be escaped."),
+ (unsigned char) *(lex->token_terminator));
+ break;
case JSON_EXPECTED_END:
- return psprintf(_("Expected end of input, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected end of input, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_ARRAY_FIRST:
- return psprintf(_("Expected array element or \"]\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected array element or \"]\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_ARRAY_NEXT:
- return psprintf(_("Expected \",\" or \"]\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \",\" or \"]\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_COLON:
- return psprintf(_("Expected \":\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \":\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_JSON:
- return psprintf(_("Expected JSON value, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected JSON value, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_MORE:
return _("The input string ended unexpectedly.");
case JSON_EXPECTED_OBJECT_FIRST:
- return psprintf(_("Expected string or \"}\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected string or \"}\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_OBJECT_NEXT:
- return psprintf(_("Expected \",\" or \"}\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \",\" or \"}\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_STRING:
- return psprintf(_("Expected string, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected string, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_INVALID_TOKEN:
- return psprintf(_("Token \"%s\" is invalid."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Token \"%.*s\" is invalid."),
+ toklen, lex->token_start);
+ break;
+ case JSON_OUT_OF_MEMORY:
+ /* should have been handled above; use the error path */
+ break;
case JSON_UNICODE_CODE_POINT_ZERO:
return _("\\u0000 cannot be converted to text.");
case JSON_UNICODE_ESCAPE_FORMAT:
@@ -1122,12 +1254,22 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
return _("Unicode low surrogate must follow a high surrogate.");
}
- /*
- * We don't use a default: case, so that the compiler will warn about
- * unhandled enum values. But this needs to be here anyway to cover the
- * possibility of an incorrect input.
- */
- elog(ERROR, "unexpected json parse error type: %d", (int) error);
- return NULL;
-}
+ /* Note that lex->errormsg can be NULL in FRONTEND code. */
+ if (lex->errormsg && !lex->errormsg->data[0])
+ {
+ /*
+ * We don't use a default: case, so that the compiler will warn about
+ * unhandled enum values. But this needs to be here anyway to cover the
+ * possibility of an incorrect input.
+ */
+ appendStrVal(lex->errormsg,
+ "unexpected json parse error type: %d", (int) error);
+ }
+
+#ifdef FRONTEND
+ if (PQExpBufferBroken(lex->errormsg))
+ return _("out of memory while constructing error description");
#endif
+
+ return lex->errormsg->data;
+}
diff --git a/src/include/common/jsonapi.h b/src/include/common/jsonapi.h
index 52cb4a9339..d7cafc84fe 100644
--- a/src/include/common/jsonapi.h
+++ b/src/include/common/jsonapi.h
@@ -14,8 +14,6 @@
#ifndef JSONAPI_H
#define JSONAPI_H
-#include "lib/stringinfo.h"
-
typedef enum
{
JSON_TOKEN_INVALID,
@@ -48,6 +46,7 @@ typedef enum
JSON_EXPECTED_OBJECT_NEXT,
JSON_EXPECTED_STRING,
JSON_INVALID_TOKEN,
+ JSON_OUT_OF_MEMORY,
JSON_UNICODE_CODE_POINT_ZERO,
JSON_UNICODE_ESCAPE_FORMAT,
JSON_UNICODE_HIGH_ESCAPE,
@@ -55,6 +54,17 @@ typedef enum
JSON_UNICODE_LOW_SURROGATE
} JsonParseErrorType;
+/*
+ * Don't depend on the internal type header for strval; if callers need access
+ * then they can include the appropriate header themselves.
+ */
+#ifdef FRONTEND
+#define StrValType PQExpBufferData
+#else
+#define StrValType StringInfoData
+#endif
+
+typedef struct StrValType StrValType;
/*
* All the fields in this structure should be treated as read-only.
@@ -81,7 +91,9 @@ typedef struct JsonLexContext
int lex_level;
int line_number; /* line number, starting from 1 */
char *line_start; /* where that line starts within input */
- StringInfo strval;
+ bool parse_strval;
+ StrValType *strval; /* only used if parse_strval == true */
+ StrValType *errormsg;
} JsonLexContext;
typedef void (*json_struct_action) (void *state);
@@ -141,9 +153,10 @@ extern JsonSemAction nullSemAction;
*/
extern JsonParseErrorType json_count_array_elements(JsonLexContext *lex,
int *elements);
+#ifndef FRONTEND
/*
- * constructor for JsonLexContext, with or without strval element.
+ * allocating constructor for JsonLexContext, with or without strval element.
* If supplied, the strval element will contain a de-escaped version of
* the lexeme. However, doing this imposes a performance penalty, so
* it should be avoided if the de-escaped lexeme is not required.
@@ -153,6 +166,32 @@ extern JsonLexContext *makeJsonLexContextCstringLen(char *json,
int encoding,
bool need_escapes);
+/*
+ * Counterpart to makeJsonLexContextCstringLen(): clears and deallocates lex.
+ * The context pointer should not be used after this call.
+ */
+extern void destroyJsonLexContext(JsonLexContext *lex);
+
+#endif /* !FRONTEND */
+
+/*
+ * stack constructor for JsonLexContext, with or without strval element.
+ * If supplied, the strval element will contain a de-escaped version of
+ * the lexeme. However, doing this imposes a performance penalty, so
+ * it should be avoided if the de-escaped lexeme is not required.
+ */
+extern void initJsonLexContextCstringLen(JsonLexContext *lex,
+ char *json,
+ int len,
+ int encoding,
+ bool need_escapes);
+
+/*
+ * Counterpart to initJsonLexContextCstringLen(): clears the contents of lex,
+ * but does not deallocate lex itself.
+ */
+extern void termJsonLexContext(JsonLexContext *lex);
+
/* lex one token */
extern JsonParseErrorType json_lex(JsonLexContext *lex);
--
2.25.1
v3-0005-libpq-add-OAUTHBEARER-SASL-mechanism.patch (text/x-patch, 35.6K) download
From 7f6b02652cd771a93ce4269607d498f4ac574e7f Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 13 Apr 2021 10:27:27 -0700
Subject: [PATCH v3 5/9] libpq: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) and OAuth 2.0 Device Authorization
Grants (RFC 8628) on the client side. When speaking to a OAuth-enabled
server, it looks a bit like this:
$ psql 'host=example.org oauth_client_id=f02c6361-0635-...'
Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG
The OAuth issuer must support device authorization. No other OAuth flows
are currently implemented.
The client implementation requires libiddawc and its development
headers. Configure --with-oauth (and --with-includes/--with-libraries to
point at the iddawc installation, if it's in a custom location).
Several TODOs:
- don't retry forever if the server won't accept our token
- perform several sanity checks on the OAuth issuer's responses
- handle cases where the client has been set up with an issuer and
scope, but the Postgres server wants to use something different
- improve error debuggability during the OAuth handshake
- ...and more.
---
configure | 100 ++++
configure.ac | 19 +
src/Makefile.global.in | 1 +
src/include/common/oauth-common.h | 19 +
src/include/pg_config.h.in | 6 +
src/interfaces/libpq/Makefile | 7 +-
src/interfaces/libpq/fe-auth-oauth.c | 744 +++++++++++++++++++++++++++
src/interfaces/libpq/fe-auth-sasl.h | 5 +-
src/interfaces/libpq/fe-auth-scram.c | 6 +-
src/interfaces/libpq/fe-auth.c | 42 +-
src/interfaces/libpq/fe-auth.h | 3 +
src/interfaces/libpq/fe-connect.c | 38 ++
src/interfaces/libpq/libpq-int.h | 8 +
13 files changed, 979 insertions(+), 19 deletions(-)
create mode 100644 src/include/common/oauth-common.h
create mode 100644 src/interfaces/libpq/fe-auth-oauth.c
diff --git a/configure b/configure
index f3cb5c2b51..cd0c50a951 100755
--- a/configure
+++ b/configure
@@ -718,6 +718,7 @@ with_uuid
with_readline
with_systemd
with_selinux
+with_oauth
with_ldap
with_krb_srvnam
krb_srvtab
@@ -861,6 +862,7 @@ with_krb_srvnam
with_pam
with_bsd_auth
with_ldap
+with_oauth
with_bonjour
with_selinux
with_systemd
@@ -1570,6 +1572,7 @@ Optional Packages:
--with-pam build with PAM support
--with-bsd-auth build with BSD Authentication support
--with-ldap build with LDAP support
+ --with-oauth build with OAuth 2.0 support
--with-bonjour build with Bonjour support
--with-selinux build with SELinux support
--with-systemd build with systemd support
@@ -8377,6 +8380,42 @@ $as_echo "$with_ldap" >&6; }
+#
+# OAuth 2.0
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with OAuth support" >&5
+$as_echo_n "checking whether to build with OAuth support... " >&6; }
+
+
+
+# Check whether --with-oauth was given.
+if test "${with_oauth+set}" = set; then :
+ withval=$with_oauth;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_OAUTH 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-oauth option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_oauth=no
+
+fi
+
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_oauth" >&5
+$as_echo "$with_oauth" >&6; }
+
+
+
#
# Bonjour
#
@@ -13500,6 +13539,56 @@ fi
+if test "$with_oauth" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for i_init_session in -liddawc" >&5
+$as_echo_n "checking for i_init_session in -liddawc... " >&6; }
+if ${ac_cv_lib_iddawc_i_init_session+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-liddawc $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char i_init_session ();
+int
+main ()
+{
+return i_init_session ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_iddawc_i_init_session=yes
+else
+ ac_cv_lib_iddawc_i_init_session=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_iddawc_i_init_session" >&5
+$as_echo "$ac_cv_lib_iddawc_i_init_session" >&6; }
+if test "x$ac_cv_lib_iddawc_i_init_session" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBIDDAWC 1
+_ACEOF
+
+ LIBS="-liddawc $LIBS"
+
+else
+ as_fn_error $? "library 'iddawc' is required for OAuth support" "$LINENO" 5
+fi
+
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for security_compute_create_name in -lselinux" >&5
@@ -14513,6 +14602,17 @@ fi
done
+fi
+
+if test "$with_oauth" != no; then
+ ac_fn_c_check_header_mongrel "$LINENO" "iddawc.h" "ac_cv_header_iddawc_h" "$ac_includes_default"
+if test "x$ac_cv_header_iddawc_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <iddawc.h> is required for OAuth" "$LINENO" 5
+fi
+
+
fi
if test "$PORTNAME" = "win32" ; then
diff --git a/configure.ac b/configure.ac
index 19d1a80367..922608065f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -887,6 +887,17 @@ AC_MSG_RESULT([$with_ldap])
AC_SUBST(with_ldap)
+#
+# OAuth 2.0
+#
+AC_MSG_CHECKING([whether to build with OAuth support])
+PGAC_ARG_BOOL(with, oauth, no,
+ [build with OAuth 2.0 support],
+ [AC_DEFINE([USE_OAUTH], 1, [Define to 1 to build with OAuth 2.0 support. (--with-oauth)])])
+AC_MSG_RESULT([$with_oauth])
+AC_SUBST(with_oauth)
+
+
#
# Bonjour
#
@@ -1385,6 +1396,10 @@ fi
AC_SUBST(LDAP_LIBS_FE)
AC_SUBST(LDAP_LIBS_BE)
+if test "$with_oauth" = yes ; then
+ AC_CHECK_LIB(iddawc, i_init_session, [], [AC_MSG_ERROR([library 'iddawc' is required for OAuth support])])
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
AC_CHECK_LIB(selinux, security_compute_create_name, [],
@@ -1603,6 +1618,10 @@ elif test "$with_uuid" = ossp ; then
[AC_MSG_ERROR([header file <ossp/uuid.h> or <uuid.h> is required for OSSP UUID])])])
fi
+if test "$with_oauth" != no; then
+ AC_CHECK_HEADER(iddawc.h, [], [AC_MSG_ERROR([header file <iddawc.h> is required for OAuth])])
+fi
+
if test "$PORTNAME" = "win32" ; then
AC_CHECK_HEADERS(crtdefs.h)
fi
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index bbdc1c4bda..c9c61a9c99 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -193,6 +193,7 @@ with_ldap = @with_ldap@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
+with_oauth = @with_oauth@
with_system_tzdata = @with_system_tzdata@
with_uuid = @with_uuid@
with_zlib = @with_zlib@
diff --git a/src/include/common/oauth-common.h b/src/include/common/oauth-common.h
new file mode 100644
index 0000000000..3fa95ac7e8
--- /dev/null
+++ b/src/include/common/oauth-common.h
@@ -0,0 +1,19 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth-common.h
+ * Declarations for helper functions used for OAuth/OIDC authentication
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/common/oauth-common.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef OAUTH_COMMON_H
+#define OAUTH_COMMON_H
+
+/* Name of SASL mechanism per IANA */
+#define OAUTHBEARER_NAME "OAUTHBEARER"
+
+#endif /* OAUTH_COMMON_H */
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 635fbb2181..1b3332601e 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -319,6 +319,9 @@
/* Define to 1 if you have the `crypto' library (-lcrypto). */
#undef HAVE_LIBCRYPTO
+/* Define to 1 if you have the `iddawc' library (-liddawc). */
+#undef HAVE_LIBIDDAWC
+
/* Define to 1 if you have the `ldap' library (-lldap). */
#undef HAVE_LIBLDAP
@@ -922,6 +925,9 @@
/* Define to select named POSIX semaphores. */
#undef USE_NAMED_POSIX_SEMAPHORES
+/* Define to 1 to build with OAuth 2.0 support. (--with-oauth) */
+#undef USE_OAUTH
+
/* Define to 1 to build with OpenSSL support. (--with-ssl=openssl) */
#undef USE_OPENSSL
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 3c53393fa4..727305c578 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -62,6 +62,11 @@ OBJS += \
fe-secure-gssapi.o
endif
+ifeq ($(with_oauth),yes)
+OBJS += \
+ fe-auth-oauth.o
+endif
+
ifeq ($(PORTNAME), cygwin)
override shlib = cyg$(NAME)$(DLSUFFIX)
endif
@@ -83,7 +88,7 @@ endif
# that are built correctly for use in a shlib.
SHLIB_LINK_INTERNAL = -lpgcommon_shlib -lpgport_shlib
ifneq ($(PORTNAME), win32)
-SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
+SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -liddawc -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
else
SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi32 -lssl -lsocket -lnsl -lresolv -lintl -lm $(PTHREAD_LIBS), $(LIBS)) $(LDAP_LIBS_FE)
endif
diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
new file mode 100644
index 0000000000..383c9d4bdb
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth.c
@@ -0,0 +1,744 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth.c
+ * The front-end (client) implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include <iddawc.h>
+
+#include "postgres_fe.h"
+
+#include "common/base64.h"
+#include "common/hmac.h"
+#include "common/jsonapi.h"
+#include "common/oauth-common.h"
+#include "fe-auth.h"
+#include "mb/pg_wchar.h"
+
+/* The exported OAuth callback mechanism. */
+static void *oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism);
+static void oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen,
+ bool *done, bool *success);
+static bool oauth_channel_bound(void *opaq);
+static void oauth_free(void *opaq);
+
+const pg_fe_sasl_mech pg_oauth_mech = {
+ oauth_init,
+ oauth_exchange,
+ oauth_channel_bound,
+ oauth_free,
+};
+
+typedef enum
+{
+ FE_OAUTH_INIT,
+ FE_OAUTH_BEARER_SENT,
+ FE_OAUTH_SERVER_ERROR,
+} fe_oauth_state_enum;
+
+typedef struct
+{
+ fe_oauth_state_enum state;
+
+ PGconn *conn;
+} fe_oauth_state;
+
+static void *
+oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism)
+{
+ fe_oauth_state *state;
+
+ /*
+ * We only support one SASL mechanism here; anything else is programmer
+ * error.
+ */
+ Assert(sasl_mechanism != NULL);
+ Assert(!strcmp(sasl_mechanism, OAUTHBEARER_NAME));
+
+ state = malloc(sizeof(*state));
+ if (!state)
+ return NULL;
+
+ state->state = FE_OAUTH_INIT;
+ state->conn = conn;
+
+ return state;
+}
+
+static const char *
+iddawc_error_string(int errcode)
+{
+ switch (errcode)
+ {
+ case I_OK:
+ return "I_OK";
+
+ case I_ERROR:
+ return "I_ERROR";
+
+ case I_ERROR_PARAM:
+ return "I_ERROR_PARAM";
+
+ case I_ERROR_MEMORY:
+ return "I_ERROR_MEMORY";
+
+ case I_ERROR_UNAUTHORIZED:
+ return "I_ERROR_UNAUTHORIZED";
+
+ case I_ERROR_SERVER:
+ return "I_ERROR_SERVER";
+ }
+
+ return "<unknown>";
+}
+
+static void
+iddawc_error(PGconn *conn, int errcode, const char *msg)
+{
+ appendPQExpBufferStr(&conn->errorMessage, libpq_gettext(msg));
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext(" (iddawc error %s)\n"),
+ iddawc_error_string(errcode));
+}
+
+static void
+iddawc_request_error(PGconn *conn, struct _i_session *i, int err, const char *msg)
+{
+ const char *error_code;
+ const char *desc;
+
+ appendPQExpBuffer(&conn->errorMessage, "%s: ", libpq_gettext(msg));
+
+ error_code = i_get_str_parameter(i, I_OPT_ERROR);
+ if (!error_code)
+ {
+ /*
+ * The server didn't give us any useful information, so just print the
+ * error code.
+ */
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("(iddawc error %s)\n"),
+ iddawc_error_string(err));
+ return;
+ }
+
+ /* If the server gave a string description, print that too. */
+ desc = i_get_str_parameter(i, I_OPT_ERROR_DESCRIPTION);
+ if (desc)
+ appendPQExpBuffer(&conn->errorMessage, "%s ", desc);
+
+ appendPQExpBuffer(&conn->errorMessage, "(%s)\n", error_code);
+}
+
+static char *
+get_auth_token(PGconn *conn)
+{
+ PQExpBuffer token_buf = NULL;
+ struct _i_session session;
+ int err;
+ int auth_method;
+ bool user_prompted = false;
+ const char *verification_uri;
+ const char *user_code;
+ const char *access_token;
+ const char *token_type;
+ char *token = NULL;
+
+ if (!conn->oauth_discovery_uri)
+ return strdup(""); /* ask the server for one */
+
+ if (!conn->oauth_client_id)
+ {
+ /* We can't talk to a server without a client identifier. */
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("no oauth_client_id is set for the connection"));
+ return NULL;
+ }
+
+ i_init_session(&session);
+
+ token_buf = createPQExpBuffer();
+ if (!token_buf)
+ goto cleanup;
+
+ err = i_set_str_parameter(&session, I_OPT_OPENID_CONFIG_ENDPOINT, conn->oauth_discovery_uri);
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set OpenID config endpoint");
+ goto cleanup;
+ }
+
+ err = i_get_openid_config(&session);
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to fetch OpenID discovery document");
+ goto cleanup;
+ }
+
+ if (!i_get_str_parameter(&session, I_OPT_TOKEN_ENDPOINT))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer has no token endpoint"));
+ goto cleanup;
+ }
+
+ if (!i_get_str_parameter(&session, I_OPT_DEVICE_AUTHORIZATION_ENDPOINT))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer does not support device authorization"));
+ goto cleanup;
+ }
+
+ err = i_set_response_type(&session, I_RESPONSE_TYPE_DEVICE_CODE);
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set device code response type");
+ goto cleanup;
+ }
+
+ auth_method = I_TOKEN_AUTH_METHOD_NONE;
+ if (conn->oauth_client_secret && *conn->oauth_client_secret)
+ auth_method = I_TOKEN_AUTH_METHOD_SECRET_BASIC;
+
+ err = i_set_parameter_list(&session,
+ I_OPT_CLIENT_ID, conn->oauth_client_id,
+ I_OPT_CLIENT_SECRET, conn->oauth_client_secret,
+ I_OPT_TOKEN_METHOD, auth_method,
+ I_OPT_SCOPE, conn->oauth_scope,
+ I_OPT_NONE
+ );
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set client identifier");
+ goto cleanup;
+ }
+
+ err = i_run_device_auth_request(&session);
+ if (err)
+ {
+ iddawc_request_error(conn, &session, err,
+ "failed to obtain device authorization");
+ goto cleanup;
+ }
+
+ verification_uri = i_get_str_parameter(&session, I_OPT_DEVICE_AUTH_VERIFICATION_URI);
+ if (!verification_uri)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer did not provide a verification URI"));
+ goto cleanup;
+ }
+
+ user_code = i_get_str_parameter(&session, I_OPT_DEVICE_AUTH_USER_CODE);
+ if (!user_code)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer did not provide a user code"));
+ goto cleanup;
+ }
+
+ /*
+ * Poll the token endpoint until either the user logs in and authorizes the
+ * use of a token, or a hard failure occurs. We perform one ping _before_
+ * prompting the user, so that we don't make them do the work of logging in
+ * only to find that the token endpoint is completely unreachable.
+ */
+ err = i_run_token_request(&session);
+ while (err)
+ {
+ const char *error_code;
+ uint interval;
+
+ error_code = i_get_str_parameter(&session, I_OPT_ERROR);
+
+ /*
+ * authorization_pending and slow_down are the only acceptable errors;
+ * anything else and we bail.
+ */
+ if (!error_code || (strcmp(error_code, "authorization_pending")
+ && strcmp(error_code, "slow_down")))
+ {
+ iddawc_request_error(conn, &session, err,
+ "OAuth token retrieval failed");
+ goto cleanup;
+ }
+
+ if (!user_prompted)
+ {
+ /*
+ * Now that we know the token endpoint isn't broken, give the user
+ * the login instructions.
+ */
+ pqInternalNotice(&conn->noticeHooks,
+ "Visit %s and enter the code: %s",
+ verification_uri, user_code);
+
+ user_prompted = true;
+ }
+
+ /*
+ * We are required to wait between polls; the server tells us how long.
+ * TODO: if interval's not set, we need to default to five seconds
+ * TODO: sanity check the interval
+ */
+ interval = i_get_int_parameter(&session, I_OPT_DEVICE_AUTH_INTERVAL);
+
+ /*
+ * A slow_down error requires us to permanently increase our retry
+ * interval by five seconds. RFC 8628, Sec. 3.5.
+ */
+ if (!strcmp(error_code, "slow_down"))
+ {
+ interval += 5;
+ i_set_int_parameter(&session, I_OPT_DEVICE_AUTH_INTERVAL, interval);
+ }
+
+ sleep(interval);
+
+ /*
+ * XXX Reset the error code before every call, because iddawc won't do
+ * that for us. This matters if the server first sends a "pending" error
+ * code, then later hard-fails without sending an error code to
+ * overwrite the first one.
+ *
+ * That we have to do this at all seems like a bug in iddawc.
+ */
+ i_set_str_parameter(&session, I_OPT_ERROR, NULL);
+
+ err = i_run_token_request(&session);
+ }
+
+ access_token = i_get_str_parameter(&session, I_OPT_ACCESS_TOKEN);
+ token_type = i_get_str_parameter(&session, I_OPT_TOKEN_TYPE);
+
+ if (!access_token || !token_type || strcasecmp(token_type, "Bearer"))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer did not provide a bearer token"));
+ goto cleanup;
+ }
+
+ appendPQExpBufferStr(token_buf, "Bearer ");
+ appendPQExpBufferStr(token_buf, access_token);
+
+ if (PQExpBufferBroken(token_buf))
+ goto cleanup;
+
+ token = strdup(token_buf->data);
+
+cleanup:
+ if (token_buf)
+ destroyPQExpBuffer(token_buf);
+ i_clean_session(&session);
+
+ return token;
+}
+
+#define kvsep "\x01"
+
+static char *
+client_initial_response(PGconn *conn)
+{
+ static const char * const resp_format = "n,," kvsep "auth=%s" kvsep kvsep;
+
+ PQExpBuffer token_buf;
+ PQExpBuffer discovery_buf = NULL;
+ char *token = NULL;
+ char *response = NULL;
+
+ token_buf = createPQExpBuffer();
+ if (!token_buf)
+ goto cleanup;
+
+ /*
+ * If we don't yet have a discovery URI, but the user gave us an explicit
+ * issuer, use the .well-known discovery URI for that issuer.
+ */
+ if (!conn->oauth_discovery_uri && conn->oauth_issuer)
+ {
+ discovery_buf = createPQExpBuffer();
+ if (!discovery_buf)
+ goto cleanup;
+
+ appendPQExpBufferStr(discovery_buf, conn->oauth_issuer);
+ appendPQExpBufferStr(discovery_buf, "/.well-known/openid-configuration");
+
+ if (PQExpBufferBroken(discovery_buf))
+ goto cleanup;
+
+ conn->oauth_discovery_uri = strdup(discovery_buf->data);
+ }
+
+ token = get_auth_token(conn);
+ if (!token)
+ goto cleanup;
+
+ appendPQExpBuffer(token_buf, resp_format, token);
+ if (PQExpBufferBroken(token_buf))
+ goto cleanup;
+
+ response = strdup(token_buf->data);
+
+cleanup:
+ if (token)
+ free(token);
+ if (discovery_buf)
+ destroyPQExpBuffer(discovery_buf);
+ if (token_buf)
+ destroyPQExpBuffer(token_buf);
+
+ return response;
+}
+
+#define ERROR_STATUS_FIELD "status"
+#define ERROR_SCOPE_FIELD "scope"
+#define ERROR_OPENID_CONFIGURATION_FIELD "openid-configuration"
+
+struct json_ctx
+{
+ char *errmsg; /* any non-NULL value stops all processing */
+ PQExpBufferData errbuf; /* backing memory for errmsg */
+ int nested; /* nesting level (zero is the top) */
+
+ const char *target_field_name; /* points to a static allocation */
+ char **target_field; /* see below */
+
+ /* target_field, if set, points to one of the following: */
+ char *status;
+ char *scope;
+ char *discovery_uri;
+};
+
+#define oauth_json_has_error(ctx) \
+ (PQExpBufferDataBroken((ctx)->errbuf) || (ctx)->errmsg)
+
+#define oauth_json_set_error(ctx, ...) \
+ do { \
+ appendPQExpBuffer(&(ctx)->errbuf, __VA_ARGS__); \
+ (ctx)->errmsg = (ctx)->errbuf.data; \
+ } while (0)
+
+static void
+oauth_json_object_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (oauth_json_has_error(ctx))
+ return; /* short-circuit */
+
+ if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ ++ctx->nested;
+}
+
+static void
+oauth_json_object_end(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (oauth_json_has_error(ctx))
+ return; /* short-circuit */
+
+ --ctx->nested;
+}
+
+static void
+oauth_json_object_field_start(void *state, char *name, bool isnull)
+{
+ struct json_ctx *ctx = state;
+
+ if (oauth_json_has_error(ctx))
+ {
+ /* short-circuit */
+ free(name);
+ return;
+ }
+
+ if (ctx->nested == 1)
+ {
+ if (!strcmp(name, ERROR_STATUS_FIELD))
+ {
+ ctx->target_field_name = ERROR_STATUS_FIELD;
+ ctx->target_field = &ctx->status;
+ }
+ else if (!strcmp(name, ERROR_SCOPE_FIELD))
+ {
+ ctx->target_field_name = ERROR_SCOPE_FIELD;
+ ctx->target_field = &ctx->scope;
+ }
+ else if (!strcmp(name, ERROR_OPENID_CONFIGURATION_FIELD))
+ {
+ ctx->target_field_name = ERROR_OPENID_CONFIGURATION_FIELD;
+ ctx->target_field = &ctx->discovery_uri;
+ }
+ }
+
+ free(name);
+}
+
+static void
+oauth_json_array_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (oauth_json_has_error(ctx))
+ return; /* short-circuit */
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+}
+
+static void
+oauth_json_scalar(void *state, char *token, JsonTokenType type)
+{
+ struct json_ctx *ctx = state;
+
+ if (oauth_json_has_error(ctx))
+ {
+ /* short-circuit */
+ free(token);
+ return;
+ }
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ if (type == JSON_TOKEN_STRING)
+ {
+ *ctx->target_field = token;
+
+ ctx->target_field = NULL;
+ ctx->target_field_name = NULL;
+
+ return; /* don't free the token we're using */
+ }
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ free(token);
+}
+
+static bool
+handle_oauth_sasl_error(PGconn *conn, char *msg, int msglen)
+{
+ JsonLexContext lex = {0};
+ JsonSemAction sem = {0};
+ JsonParseErrorType err;
+ struct json_ctx ctx = {0};
+ char *errmsg = NULL;
+
+ /* Sanity check. */
+ if (strlen(msg) != msglen)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server's error message contained an embedded NULL"));
+ return false;
+ }
+
+ initJsonLexContextCstringLen(&lex, msg, msglen, PG_UTF8, true);
+
+ initPQExpBuffer(&ctx.errbuf);
+ sem.semstate = &ctx;
+
+ sem.object_start = oauth_json_object_start;
+ sem.object_end = oauth_json_object_end;
+ sem.object_field_start = oauth_json_object_field_start;
+ sem.array_start = oauth_json_array_start;
+ sem.scalar = oauth_json_scalar;
+
+ err = pg_parse_json(&lex, &sem);
+
+ if (err != JSON_SUCCESS)
+ {
+ errmsg = json_errdetail(err, &lex);
+ }
+ else if (PQExpBufferDataBroken(ctx.errbuf))
+ {
+ errmsg = libpq_gettext("out of memory");
+ }
+ else if (ctx.errmsg)
+ {
+ errmsg = ctx.errmsg;
+ }
+
+ if (errmsg)
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("failed to parse server's error response: %s"),
+ errmsg);
+
+ /* Don't need the error buffer or the JSON lexer anymore. */
+ termPQExpBuffer(&ctx.errbuf);
+ termJsonLexContext(&lex);
+
+ if (errmsg)
+ return false;
+
+ /* TODO: what if these override what the user already specified? */
+ if (ctx.discovery_uri)
+ {
+ if (conn->oauth_discovery_uri)
+ free(conn->oauth_discovery_uri);
+
+ conn->oauth_discovery_uri = ctx.discovery_uri;
+ }
+
+ if (ctx.scope)
+ {
+ if (conn->oauth_scope)
+ free(conn->oauth_scope);
+
+ conn->oauth_scope = ctx.scope;
+ }
+ /* TODO: missing error scope should clear any existing connection scope */
+
+ if (!ctx.status)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("server sent error response without a status"));
+ return false;
+ }
+
+ if (!strcmp(ctx.status, "invalid_token"))
+ {
+ /*
+ * invalid_token is the only error code we'll automatically retry for,
+ * but only if we have enough information to do so.
+ */
+ if (conn->oauth_discovery_uri)
+ conn->oauth_want_retry = true;
+ }
+ /* TODO: include status in hard failure message */
+
+ return true;
+}
+
+static void
+oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen,
+ bool *done, bool *success)
+{
+ fe_oauth_state *state = opaq;
+ PGconn *conn = state->conn;
+
+ *done = false;
+ *success = false;
+ *output = NULL;
+ *outputlen = 0;
+
+ switch (state->state)
+ {
+ case FE_OAUTH_INIT:
+ Assert(inputlen == -1);
+
+ *output = client_initial_response(conn);
+ if (!*output)
+ goto error;
+
+ *outputlen = strlen(*output);
+ state->state = FE_OAUTH_BEARER_SENT;
+
+ break;
+
+ case FE_OAUTH_BEARER_SENT:
+ if (final)
+ {
+ /* TODO: ensure there is no message content here. */
+ *done = true;
+ *success = true;
+
+ break;
+ }
+
+ /*
+ * Error message sent by the server.
+ */
+ if (!handle_oauth_sasl_error(conn, input, inputlen))
+ goto error;
+
+ /*
+ * Respond with the required dummy message (RFC 7628, sec. 3.2.3).
+ */
+ *output = strdup(kvsep);
+ *outputlen = strlen(*output); /* == 1 */
+
+ state->state = FE_OAUTH_SERVER_ERROR;
+ break;
+
+ case FE_OAUTH_SERVER_ERROR:
+ /*
+ * After an error, the server should send an error response to fail
+ * the SASL handshake, which is handled in higher layers.
+ *
+ * If we get here, the server either sent *another* challenge which
+ * isn't defined in the RFC, or completed the handshake successfully
+ * after telling us it was going to fail. Neither is acceptable.
+ */
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server sent additional OAuth data after error\n"));
+ goto error;
+
+ default:
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("invalid OAuth exchange state\n"));
+ goto error;
+ }
+
+ return;
+
+error:
+ *done = true;
+ *success = false;
+}
+
+static bool
+oauth_channel_bound(void *opaq)
+{
+ /* This mechanism does not support channel binding. */
+ return false;
+}
+
+static void
+oauth_free(void *opaq)
+{
+ fe_oauth_state *state = opaq;
+
+ free(state);
+}
diff --git a/src/interfaces/libpq/fe-auth-sasl.h b/src/interfaces/libpq/fe-auth-sasl.h
index da3c30b87b..b1bb382f70 100644
--- a/src/interfaces/libpq/fe-auth-sasl.h
+++ b/src/interfaces/libpq/fe-auth-sasl.h
@@ -65,6 +65,8 @@ typedef struct pg_fe_sasl_mech
*
* state: The opaque mechanism state returned by init()
*
+ * final: true if the server has sent a final exchange outcome
+ *
* input: The challenge data sent by the server, or NULL when
* generating a client-first initial response (that is, when
* the server expects the client to send a message to start
@@ -92,7 +94,8 @@ typedef struct pg_fe_sasl_mech
* Ignored if *done is false.
*--------
*/
- void (*exchange) (void *state, char *input, int inputlen,
+ void (*exchange) (void *state, bool final,
+ char *input, int inputlen,
char **output, int *outputlen,
bool *done, bool *success);
diff --git a/src/interfaces/libpq/fe-auth-scram.c b/src/interfaces/libpq/fe-auth-scram.c
index e616200704..681b76adbe 100644
--- a/src/interfaces/libpq/fe-auth-scram.c
+++ b/src/interfaces/libpq/fe-auth-scram.c
@@ -24,7 +24,8 @@
/* The exported SCRAM callback mechanism. */
static void *scram_init(PGconn *conn, const char *password,
const char *sasl_mechanism);
-static void scram_exchange(void *opaq, char *input, int inputlen,
+static void scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen,
bool *done, bool *success);
static bool scram_channel_bound(void *opaq);
@@ -206,7 +207,8 @@ scram_free(void *opaq)
* Exchange a SCRAM message with backend.
*/
static void
-scram_exchange(void *opaq, char *input, int inputlen,
+scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen,
bool *done, bool *success)
{
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index 6fceff561b..2567a34023 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -38,6 +38,7 @@
#endif
#include "common/md5.h"
+#include "common/oauth-common.h"
#include "common/scram-common.h"
#include "fe-auth.h"
#include "fe-auth-sasl.h"
@@ -422,7 +423,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
bool success;
const char *selected_mechanism;
PQExpBufferData mechanism_buf;
- char *password;
+ char *password = NULL;
initPQExpBuffer(&mechanism_buf);
@@ -444,8 +445,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
/*
* Parse the list of SASL authentication mechanisms in the
* AuthenticationSASL message, and select the best mechanism that we
- * support. SCRAM-SHA-256-PLUS and SCRAM-SHA-256 are the only ones
- * supported at the moment, listed by order of decreasing importance.
+ * support. Mechanisms are listed by order of decreasing importance.
*/
selected_mechanism = NULL;
for (;;)
@@ -485,6 +485,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
selected_mechanism = SCRAM_SHA_256_PLUS_NAME;
conn->sasl = &pg_scram_mech;
+ conn->password_needed = true;
}
#else
/*
@@ -522,7 +523,17 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
selected_mechanism = SCRAM_SHA_256_NAME;
conn->sasl = &pg_scram_mech;
+ conn->password_needed = true;
}
+#ifdef USE_OAUTH
+ else if (strcmp(mechanism_buf.data, OAUTHBEARER_NAME) == 0 &&
+ !selected_mechanism)
+ {
+ selected_mechanism = OAUTHBEARER_NAME;
+ conn->sasl = &pg_oauth_mech;
+ conn->password_needed = false;
+ }
+#endif
}
if (!selected_mechanism)
@@ -547,18 +558,19 @@ pg_SASL_init(PGconn *conn, int payloadlen)
/*
* First, select the password to use for the exchange, complaining if
- * there isn't one. Currently, all supported SASL mechanisms require a
- * password, so we can just go ahead here without further distinction.
+ * there isn't one and the SASL mechanism needs it.
*/
- conn->password_needed = true;
- password = conn->connhost[conn->whichhost].password;
- if (password == NULL)
- password = conn->pgpass;
- if (password == NULL || password[0] == '\0')
+ if (conn->password_needed)
{
- appendPQExpBufferStr(&conn->errorMessage,
- PQnoPasswordSupplied);
- goto error;
+ password = conn->connhost[conn->whichhost].password;
+ if (password == NULL)
+ password = conn->pgpass;
+ if (password == NULL || password[0] == '\0')
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ PQnoPasswordSupplied);
+ goto error;
+ }
}
Assert(conn->sasl);
@@ -576,7 +588,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
goto oom_error;
/* Get the mechanism-specific Initial Client Response, if any */
- conn->sasl->exchange(conn->sasl_state,
+ conn->sasl->exchange(conn->sasl_state, false,
NULL, -1,
&initialresponse, &initialresponselen,
&done, &success);
@@ -657,7 +669,7 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
/* For safety and convenience, ensure the buffer is NULL-terminated. */
challenge[payloadlen] = '\0';
- conn->sasl->exchange(conn->sasl_state,
+ conn->sasl->exchange(conn->sasl_state, final,
challenge, payloadlen,
&output, &outputlen,
&done, &success);
diff --git a/src/interfaces/libpq/fe-auth.h b/src/interfaces/libpq/fe-auth.h
index 049a8bb1a1..2a56774019 100644
--- a/src/interfaces/libpq/fe-auth.h
+++ b/src/interfaces/libpq/fe-auth.h
@@ -28,4 +28,7 @@ extern const pg_fe_sasl_mech pg_scram_mech;
extern char *pg_fe_scram_build_secret(const char *password,
const char **errstr);
+/* Mechanisms in fe-auth-oauth.c */
+extern const pg_fe_sasl_mech pg_oauth_mech;
+
#endif /* FE_AUTH_H */
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 1c5a2b43e9..5f78439586 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -344,6 +344,23 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"Target-Session-Attrs", "", 15, /* sizeof("prefer-standby") = 15 */
offsetof(struct pg_conn, target_session_attrs)},
+ /* OAuth v2 */
+ {"oauth_issuer", NULL, NULL, NULL,
+ "OAuth-Issuer", "", 40,
+ offsetof(struct pg_conn, oauth_issuer)},
+
+ {"oauth_client_id", NULL, NULL, NULL,
+ "OAuth-Client-ID", "", 40,
+ offsetof(struct pg_conn, oauth_client_id)},
+
+ {"oauth_client_secret", NULL, NULL, NULL,
+ "OAuth-Client-Secret", "", 40,
+ offsetof(struct pg_conn, oauth_client_secret)},
+
+ {"oauth_scope", NULL, NULL, NULL,
+ "OAuth-Scope", "", 15,
+ offsetof(struct pg_conn, oauth_scope)},
+
/* Terminating entry --- MUST BE LAST */
{NULL, NULL, NULL, NULL,
NULL, NULL, 0}
@@ -606,6 +623,7 @@ pqDropServerData(PGconn *conn)
conn->write_err_msg = NULL;
conn->be_pid = 0;
conn->be_key = 0;
+ /* conn->oauth_want_retry = false; TODO */
}
@@ -3381,6 +3399,16 @@ keep_going: /* We will come back to here until there is
/* Check to see if we should mention pgpassfile */
pgpassfileWarning(conn);
+#ifdef USE_OAUTH
+ if (conn->sasl == &pg_oauth_mech
+ && conn->oauth_want_retry)
+ {
+ /* TODO: only allow retry once */
+ need_new_connection = true;
+ goto keep_going;
+ }
+#endif
+
#ifdef ENABLE_GSS
/*
@@ -4161,6 +4189,16 @@ freePGconn(PGconn *conn)
free(conn->rowBuf);
if (conn->target_session_attrs)
free(conn->target_session_attrs);
+ if (conn->oauth_issuer)
+ free(conn->oauth_issuer);
+ if (conn->oauth_discovery_uri)
+ free(conn->oauth_discovery_uri);
+ if (conn->oauth_client_id)
+ free(conn->oauth_client_id);
+ if (conn->oauth_client_secret)
+ free(conn->oauth_client_secret);
+ if (conn->oauth_scope)
+ free(conn->oauth_scope);
termPQExpBuffer(&conn->errorMessage);
termPQExpBuffer(&conn->workBuffer);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index e0cee4b142..0dff13505a 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -394,6 +394,14 @@ struct pg_conn
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ /* OAuth v2 */
+ char *oauth_issuer; /* token issuer URL */
+ char *oauth_discovery_uri; /* URI of the issuer's discovery document */
+ char *oauth_client_id; /* client identifier */
+ char *oauth_client_secret; /* client secret */
+ char *oauth_scope; /* access token scope */
+ bool oauth_want_retry; /* should we retry on failure? */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
--
2.25.1
v3-0006-backend-add-OAUTHBEARER-SASL-mechanism.patch (text/x-patch, 35.5K) download
From 43ab0310ce8ee26a469167a2e4eae4c4bc295518 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 4 May 2021 16:21:11 -0700
Subject: [PATCH v3 6/9] backend: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) on the server side. This adds a new
auth method, oauth, to pg_hba.
Because OAuth implementations vary so wildly, and bearer token
validation is heavily dependent on the issuing party, authn/z is done by
communicating with an external program: the oauth_validator_command.
This command must do the following:
1. Receive the bearer token by reading its contents from a file
descriptor passed from the server. (The numeric value of this
descriptor may be inserted into the oauth_validator_command using the
%f specifier.)
This MUST be the first action the command performs. The server will
not begin reading stdout from the command until the token has been
read in full, so if the command tries to print anything and hits a
buffer limit, the backend will deadlock and time out.
2. Validate the bearer token. The correct way to do this depends on the
issuer, but it generally involves either cryptographic operations to
prove that the token was issued by a trusted party, or the
presentation of the bearer token to some other party so that _it_ can
perform validation.
The command MUST maintain confidentiality of the bearer token, since
in most cases it can be used just like a password. (There are ways to
cryptographically bind tokens to client certificates, but they are
way beyond the scope of this commit message.)
If the token cannot be validated, the command must exit with a
non-zero status. Further authentication/authorization is pointless if
the bearer token wasn't issued by someone you trust.
3. Authenticate the user, authorize the user, or both:
a. To authenticate the user, use the bearer token to retrieve some
trusted identifier string for the end user. The exact process for
this is, again, issuer-dependent. The command should print the
authenticated identity string to stdout, followed by a newline.
If the user cannot be authenticated, the validator should not
print anything to stdout. It should also exit with a non-zero
status, unless the token may be used to authorize the connection
through some other means (see below).
On a success, the command may then exit with a zero success code.
By default, the server will then check to make sure the identity
string matches the role that is being used (or matches a usermap
entry, if one is in use).
b. To optionally authorize the user, in combination with the HBA
option trust_validator_authz=1 (see below), the validator simply
returns a zero exit code if the client should be allowed to
connect with its presented role (which can be passed to the
command using the %r specifier), or a non-zero code otherwise.
The hard part is in determining whether the given token truly
authorizes the client to use the given role, which must
unfortunately be left as an exercise to the reader.
This obviously requires some care, as a poorly implemented token
validator may silently open the entire database to anyone with a
bearer token. But it may be a more portable approach, since OAuth
is designed as an authorization framework, not an authentication
framework. For example, the user's bearer token could carry an
"allow_superuser_access" claim, which would authorize pseudonymous
database access as any role. It's then up to the OAuth system
administrators to ensure that allow_superuser_access is doled out
only to the proper users.
c. It's possible that the user can be successfully authenticated but
isn't authorized to connect. In this case, the command may print
the authenticated ID and then fail with a non-zero exit code.
(This makes it easier to see what's going on in the Postgres
logs.)
4. Token validators may optionally log to stderr. This will be printed
verbatim into the Postgres server logs.
The oauth method supports the following HBA options (but note that two
of them are not optional, since we have no way of choosing sensible
defaults):
issuer: Required. The URL of the OAuth issuing party, which the client
must contact to receive a bearer token.
Some real-world examples as of time of writing:
- https://accounts.google.com
- https://login.microsoft.com/[tenant-id]/v2.0
scope: Required. The OAuth scope(s) required for the server to
authenticate and/or authorize the user. This is heavily
deployment-specific, but a simple example is "openid email".
map: Optional. Specify a standard PostgreSQL user map; this works
the same as with other auth methods such as peer. If a map is
not specified, the user ID returned by the token validator
must exactly match the role that's being requested (but see
trust_validator_authz, below).
trust_validator_authz:
Optional. When set to 1, this allows the token validator to
take full control of the authorization process. Standard user
mapping is skipped: if the validator command succeeds, the
client is allowed to connect under its desired role and no
further checks are done.
Unlike the client, servers support OAuth without needing to be built
against libiddawc (since the responsibility for "speaking" OAuth/OIDC
correctly is delegated entirely to the oauth_validator_command).
Several TODOs:
- port to platforms other than "modern Linux"
- overhaul the communication with oauth_validator_command, which is
currently a bad hack on OpenPipeStream()
- implement more sanity checks on the OAUTHBEARER message format and
tokens sent by the client
- implement more helpful handling of HBA misconfigurations
- properly interpolate JSON when generating error responses
- use logdetail during auth failures
- deal with role names that can't be safely passed to system() without
shell-escaping
- allow passing the configured issuer to the oauth_validator_command, to
deal with multi-issuer setups
- ...and more.
---
src/backend/libpq/Makefile | 1 +
src/backend/libpq/auth-oauth.c | 797 +++++++++++++++++++++++++++++++++
src/backend/libpq/auth-sasl.c | 10 +-
src/backend/libpq/auth-scram.c | 4 +-
src/backend/libpq/auth.c | 7 +
src/backend/libpq/hba.c | 29 +-
src/backend/utils/misc/guc.c | 12 +
src/include/libpq/hba.h | 8 +-
src/include/libpq/oauth.h | 24 +
src/include/libpq/sasl.h | 11 +
10 files changed, 889 insertions(+), 14 deletions(-)
create mode 100644 src/backend/libpq/auth-oauth.c
create mode 100644 src/include/libpq/oauth.h
diff --git a/src/backend/libpq/Makefile b/src/backend/libpq/Makefile
index 6d385fd6a4..98eb2a8242 100644
--- a/src/backend/libpq/Makefile
+++ b/src/backend/libpq/Makefile
@@ -15,6 +15,7 @@ include $(top_builddir)/src/Makefile.global
# be-fsstubs is here for historical reasons, probably belongs elsewhere
OBJS = \
+ auth-oauth.o \
auth-sasl.o \
auth-scram.o \
auth.o \
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
new file mode 100644
index 0000000000..c1232a31a0
--- /dev/null
+++ b/src/backend/libpq/auth-oauth.c
@@ -0,0 +1,797 @@
+/*-------------------------------------------------------------------------
+ *
+ * auth-oauth.c
+ * Server-side implementation of the SASL OAUTHBEARER mechanism.
+ *
+ * See the following RFC for more details:
+ * - RFC 7628: https://tools.ietf.org/html/rfc7628
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/backend/libpq/auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <unistd.h>
+#include <fcntl.h>
+
+#include "common/oauth-common.h"
+#include "lib/stringinfo.h"
+#include "libpq/auth.h"
+#include "libpq/hba.h"
+#include "libpq/oauth.h"
+#include "libpq/sasl.h"
+#include "storage/fd.h"
+
+/* GUC */
+char *oauth_validator_command;
+
+static void oauth_get_mechanisms(Port *port, StringInfo buf);
+static void *oauth_init(Port *port, const char *selected_mech, const char *shadow_pass);
+static int oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, const char **logdetail);
+
+/* Mechanism declaration */
+const pg_be_sasl_mech pg_be_oauth_mech = {
+ oauth_get_mechanisms,
+ oauth_init,
+ oauth_exchange,
+
+ PG_MAX_AUTH_TOKEN_LENGTH,
+};
+
+
+typedef enum
+{
+ OAUTH_STATE_INIT = 0,
+ OAUTH_STATE_ERROR,
+ OAUTH_STATE_FINISHED,
+} oauth_state;
+
+struct oauth_ctx
+{
+ oauth_state state;
+ Port *port;
+ const char *issuer;
+ const char *scope;
+};
+
+static char *sanitize_char(char c);
+static char *parse_kvpairs_for_auth(char **input);
+static void generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen);
+static bool validate(Port *port, const char *auth, const char **logdetail);
+static bool run_validator_command(Port *port, const char *token);
+static bool check_exit(FILE **fh, const char *command);
+static bool unset_cloexec(int fd);
+static bool username_ok_for_shell(const char *username);
+
+#define KVSEP 0x01
+#define AUTH_KEY "auth"
+#define BEARER_SCHEME "Bearer "
+
+static void
+oauth_get_mechanisms(Port *port, StringInfo buf)
+{
+ /* Only OAUTHBEARER is supported. */
+ appendStringInfoString(buf, OAUTHBEARER_NAME);
+ appendStringInfoChar(buf, '\0');
+}
+
+static void *
+oauth_init(Port *port, const char *selected_mech, const char *shadow_pass)
+{
+ struct oauth_ctx *ctx;
+
+ if (strcmp(selected_mech, OAUTHBEARER_NAME))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("client selected an invalid SASL authentication mechanism")));
+
+ ctx = palloc0(sizeof(*ctx));
+
+ ctx->state = OAUTH_STATE_INIT;
+ ctx->port = port;
+
+ Assert(port->hba);
+ ctx->issuer = port->hba->oauth_issuer;
+ ctx->scope = port->hba->oauth_scope;
+
+ return ctx;
+}
+
+static int
+oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, const char **logdetail)
+{
+ char *p;
+ char cbind_flag;
+ char *auth;
+
+ struct oauth_ctx *ctx = opaq;
+
+ *output = NULL;
+ *outputlen = -1;
+
+ /*
+ * If the client didn't include an "Initial Client Response" in the
+ * SASLInitialResponse message, send an empty challenge, to which the
+ * client will respond with the same data that usually comes in the
+ * Initial Client Response.
+ */
+ if (input == NULL)
+ {
+ Assert(ctx->state == OAUTH_STATE_INIT);
+
+ *output = pstrdup("");
+ *outputlen = 0;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ /*
+ * Check that the input length agrees with the string length of the input.
+ */
+ if (inputlen == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The message is empty.")));
+ if (inputlen != strlen(input))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message length does not match input length.")));
+
+ switch (ctx->state)
+ {
+ case OAUTH_STATE_INIT:
+ /* Handle this case below. */
+ break;
+
+ case OAUTH_STATE_ERROR:
+ /*
+ * Only one response is valid for the client during authentication
+ * failure: a single kvsep.
+ */
+ if (inputlen != 1 || *input != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Client did not send a kvsep response.")));
+
+ /* The (failed) handshake is now complete. */
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_FAILURE;
+
+ default:
+ elog(ERROR, "invalid OAUTHBEARER exchange state");
+ return PG_SASL_EXCHANGE_FAILURE;
+ }
+
+ /* Handle the client's initial message. */
+ p = pstrdup(input);
+
+ /*
+ * OAUTHBEARER does not currently define a channel binding (so there is no
+ * OAUTHBEARER-PLUS, and we do not accept a 'p' specifier). We accept a 'y'
+ * specifier purely for the remote chance that a future specification could
+ * define one; then future clients can still interoperate with this server
+ * implementation. 'n' is the expected case.
+ */
+ cbind_flag = *p;
+ switch (cbind_flag)
+ {
+ case 'p':
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The server does not support channel binding for OAuth, but the client message includes channel binding data.")));
+ break;
+
+ case 'y': /* fall through */
+ case 'n':
+ p++;
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Comma expected, but found character %s.",
+ sanitize_char(*p))));
+ p++;
+ break;
+
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected channel-binding flag %s.",
+ sanitize_char(cbind_flag))));
+ }
+
+ /*
+ * Forbid optional authzid (authorization identity). We don't support it.
+ */
+ if (*p == 'a')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("client uses authorization identity, but it is not supported")));
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected attribute %s in client-first-message.",
+ sanitize_char(*p))));
+ p++;
+
+ /* All remaining fields are separated by the RFC's kvsep (\x01). */
+ if (*p != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Key-value separator expected, but found character %s.",
+ sanitize_char(*p))));
+ p++;
+
+ auth = parse_kvpairs_for_auth(&p);
+ if (!auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message does not contain an auth value.")));
+
+ /* We should be at the end of our message. */
+ if (*p)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains additional data after the final terminator.")));
+
+ if (!validate(ctx->port, auth, logdetail))
+ {
+ generate_error_response(ctx, output, outputlen);
+
+ ctx->state = OAUTH_STATE_ERROR;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_SUCCESS;
+}
+
+/*
+ * Convert an arbitrary byte to printable form. For error messages.
+ *
+ * If it's a printable ASCII character, print it as a single character.
+ * otherwise, print it in hex.
+ *
+ * The returned pointer points to a static buffer.
+ */
+static char *
+sanitize_char(char c)
+{
+ static char buf[5];
+
+ if (c >= 0x21 && c <= 0x7E)
+ snprintf(buf, sizeof(buf), "'%c'", c);
+ else
+ snprintf(buf, sizeof(buf), "0x%02x", (unsigned char) c);
+ return buf;
+}
+
+/*
+ * Consumes all kvpairs in an OAUTHBEARER exchange message. If the "auth" key is
+ * found, its value is returned.
+ */
+static char *
+parse_kvpairs_for_auth(char **input)
+{
+ char *pos = *input;
+ char *auth = NULL;
+
+ /*
+ * The relevant ABNF, from Sec. 3.1:
+ *
+ * kvsep = %x01
+ * key = 1*(ALPHA)
+ * value = *(VCHAR / SP / HTAB / CR / LF )
+ * kvpair = key "=" value kvsep
+ * ;;gs2-header = See RFC 5801
+ * client-resp = (gs2-header kvsep *kvpair kvsep) / kvsep
+ *
+ * By the time we reach this code, the gs2-header and initial kvsep have
+ * already been validated. We start at the beginning of the first kvpair.
+ */
+
+ while (*pos)
+ {
+ char *end;
+ char *sep;
+ char *key;
+ char *value;
+
+ /*
+ * Find the end of this kvpair. Note that input is null-terminated by
+ * the SASL code, so the strchr() is bounded.
+ */
+ end = strchr(pos, KVSEP);
+ if (!end)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an unterminated key/value pair.")));
+ *end = '\0';
+
+ if (pos == end)
+ {
+ /* Empty kvpair, signifying the end of the list. */
+ *input = pos + 1;
+ return auth;
+ }
+
+ /*
+ * Find the end of the key name.
+ *
+ * TODO further validate the key/value grammar? empty keys, bad chars...
+ */
+ sep = strchr(pos, '=');
+ if (!sep)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains a key without a value.")));
+ *sep = '\0';
+
+ /* Both key and value are now safely terminated. */
+ key = pos;
+ value = sep + 1;
+
+ if (!strcmp(key, AUTH_KEY))
+ {
+ if (auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains multiple auth values.")));
+
+ auth = value;
+ }
+ else
+ {
+ /*
+ * The RFC also defines the host and port keys, but they are not
+ * required for OAUTHBEARER and we do not use them. Also, per
+ * Sec. 3.1, any key/value pairs we don't recognize must be ignored.
+ */
+ }
+
+ /* Move to the next pair. */
+ pos = end + 1;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message did not contain a final terminator.")));
+
+ return NULL; /* unreachable */
+}
+
+static void
+generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
+{
+ StringInfoData buf;
+
+ /*
+ * The admin needs to set an issuer and scope for OAuth to work. There's not
+ * really a way to hide this from the user, either, because we can't choose
+ * a "default" issuer, so be honest in the failure message.
+ *
+ * TODO: see if there's a better place to fail, earlier than this.
+ */
+ if (!ctx->issuer || !ctx->scope)
+ ereport(FATAL,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("OAuth is not properly configured for this user"),
+ errdetail_log("The issuer and scope parameters must be set in pg_hba.conf.")));
+
+
+ initStringInfo(&buf);
+
+ /*
+ * TODO: JSON escaping
+ */
+ appendStringInfo(&buf,
+ "{ "
+ "\"status\": \"invalid_token\", "
+ "\"openid-configuration\": \"%s/.well-known/openid-configuration\","
+ "\"scope\": \"%s\" "
+ "}",
+ ctx->issuer, ctx->scope);
+
+ *output = buf.data;
+ *outputlen = buf.len;
+}
+
+static bool
+validate(Port *port, const char *auth, const char **logdetail)
+{
+ static const char * const b64_set = "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-._~+/";
+
+ const char *token;
+ size_t span;
+ int ret;
+
+ /* TODO: handle logdetail when the test framework can check it */
+
+ /*
+ * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
+ * 2.1:
+ *
+ * b64token = 1*( ALPHA / DIGIT /
+ * "-" / "." / "_" / "~" / "+" / "/" ) *"="
+ * credentials = "Bearer" 1*SP b64token
+ *
+ * The "credentials" construction is what we receive in our auth value.
+ *
+ * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
+ * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
+ * compared case-insensitively. (This is not mentioned in RFC 6750, but it's
+ * pointed out in RFC 7628 Sec. 4.)
+ *
+ * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
+ */
+ if (strncasecmp(auth, BEARER_SCHEME, strlen(BEARER_SCHEME)))
+ return false;
+
+ /* Pull the bearer token out of the auth value. */
+ token = auth + strlen(BEARER_SCHEME);
+
+ /* Swallow any additional spaces. */
+ while (*token == ' ')
+ token++;
+
+ /*
+ * Before invoking the validator command, sanity-check the token format to
+ * avoid any injection attacks later in the chain. Invalid formats are
+ * technically a protocol violation, but don't reflect any information about
+ * the sensitive Bearer token back to the client; log at COMMERROR instead.
+ */
+
+ /* Tokens must not be empty. */
+ if (!*token)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Bearer token is empty.")));
+ return false;
+ }
+
+ /*
+ * Make sure the token contains only allowed characters. Tokens may end with
+ * any number of '=' characters.
+ */
+ span = strspn(token, b64_set);
+ while (token[span] == '=')
+ span++;
+
+ if (token[span] != '\0')
+ {
+ /*
+ * This error message could be more helpful by printing the problematic
+ * character(s), but that'd be a bit like printing a piece of someone's
+ * password into the logs.
+ */
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Bearer token is not in the correct format.")));
+ return false;
+ }
+
+ /* Have the validator check the token. */
+ if (!run_validator_command(port, token))
+ return false;
+
+ if (port->hba->oauth_skip_usermap)
+ {
+ /*
+ * If the validator is our authorization authority, we're done.
+ * Authentication may or may not have been performed depending on the
+ * validator implementation; all that matters is that the validator says
+ * the user can log in with the target role.
+ */
+ return true;
+ }
+
+ /* Make sure the validator authenticated the user. */
+ if (!port->authn_id)
+ {
+ /* TODO: use logdetail; reduce message duplication */
+ ereport(LOG,
+ (errmsg("OAuth bearer authentication failed for user \"%s\": validator provided no identity",
+ port->user_name)));
+ return false;
+ }
+
+ /* Finally, check the user map. */
+ ret = check_usermap(port->hba->usermap, port->user_name, port->authn_id,
+ false);
+ return (ret == STATUS_OK);
+}
+
+static bool
+run_validator_command(Port *port, const char *token)
+{
+ bool success = false;
+ int rc;
+ int pipefd[2];
+ int rfd = -1;
+ int wfd = -1;
+
+ StringInfoData command = { 0 };
+ char *p;
+ FILE *fh = NULL;
+
+ ssize_t written;
+ char *line = NULL;
+ size_t size = 0;
+ ssize_t len;
+
+ Assert(oauth_validator_command);
+
+ if (!oauth_validator_command[0])
+ {
+ ereport(COMMERROR,
+ (errmsg("oauth_validator_command is not set"),
+ errhint("To allow OAuth authenticated connections, set "
+ "oauth_validator_command in postgresql.conf.")));
+ return false;
+ }
+
+ /*
+ * Since popen() is unidirectional, open up a pipe for the other direction.
+ * Use CLOEXEC to ensure that our write end doesn't accidentally get copied
+ * into child processes, which would prevent us from closing it cleanly.
+ *
+ * XXX this is ugly. We should just read from the child process's stdout,
+ * but that's a lot more code.
+ * XXX by bypassing the popen API, we open the potential of process
+ * deadlock. Clearly document child process requirements (i.e. the child
+ * MUST read all data off of the pipe before writing anything).
+ * TODO: port to Windows using _pipe().
+ */
+ rc = pipe2(pipefd, O_CLOEXEC);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create child pipe: %m")));
+ return false;
+ }
+
+ rfd = pipefd[0];
+ wfd = pipefd[1];
+
+ /* Allow the read pipe be passed to the child. */
+ if (!unset_cloexec(rfd))
+ {
+ /* error message was already logged */
+ goto cleanup;
+ }
+
+ /*
+ * Construct the command, substituting any recognized %-specifiers:
+ *
+ * %f: the file descriptor of the input pipe
+ * %r: the role that the client wants to assume (port->user_name)
+ * %%: a literal '%'
+ */
+ initStringInfo(&command);
+
+ for (p = oauth_validator_command; *p; p++)
+ {
+ if (p[0] == '%')
+ {
+ switch (p[1])
+ {
+ case 'f':
+ appendStringInfo(&command, "%d", rfd);
+ p++;
+ break;
+ case 'r':
+ /*
+ * TODO: decide how this string should be escaped. The role
+ * is controlled by the client, so if we don't escape it,
+ * command injections are inevitable.
+ *
+ * This is probably an indication that the role name needs
+ * to be communicated to the validator process in some other
+ * way. For this proof of concept, just be incredibly strict
+ * about the characters that are allowed in user names.
+ */
+ if (!username_ok_for_shell(port->user_name))
+ goto cleanup;
+
+ appendStringInfoString(&command, port->user_name);
+ p++;
+ break;
+ case '%':
+ appendStringInfoChar(&command, '%');
+ p++;
+ break;
+ default:
+ appendStringInfoChar(&command, p[0]);
+ }
+ }
+ else
+ appendStringInfoChar(&command, p[0]);
+ }
+
+ /* Execute the command. */
+ fh = OpenPipeStream(command.data, "re");
+ /* TODO: handle failures */
+
+ /* We don't need the read end of the pipe anymore. */
+ close(rfd);
+ rfd = -1;
+
+ /* Give the command the token to validate. */
+ written = write(wfd, token, strlen(token));
+ if (written != strlen(token))
+ {
+ /* TODO must loop for short writes, EINTR et al */
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write token to child pipe: %m")));
+ goto cleanup;
+ }
+
+ close(wfd);
+ wfd = -1;
+
+ /*
+ * Read the command's response.
+ *
+ * TODO: getline() is probably too new to use, unfortunately.
+ * TODO: loop over all lines
+ */
+ if ((len = getline(&line, &size, fh)) >= 0)
+ {
+ /* TODO: fail if the authn_id doesn't end with a newline */
+ if (len > 0)
+ line[len - 1] = '\0';
+
+ set_authn_id(port, line);
+ }
+ else if (ferror(fh))
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from command \"%s\": %m",
+ command.data)));
+ goto cleanup;
+ }
+
+ /* Make sure the command exits cleanly. */
+ if (!check_exit(&fh, command.data))
+ {
+ /* error message already logged */
+ goto cleanup;
+ }
+
+ /* Done. */
+ success = true;
+
+cleanup:
+ if (line)
+ free(line);
+
+ /*
+ * In the successful case, the pipe fds are already closed. For the error
+ * case, always close out the pipe before waiting for the command, to
+ * prevent deadlock.
+ */
+ if (rfd >= 0)
+ close(rfd);
+ if (wfd >= 0)
+ close(wfd);
+
+ if (fh)
+ {
+ Assert(!success);
+ check_exit(&fh, command.data);
+ }
+
+ if (command.data)
+ pfree(command.data);
+
+ return success;
+}
+
+static bool
+check_exit(FILE **fh, const char *command)
+{
+ int rc;
+
+ rc = ClosePipeStream(*fh);
+ *fh = NULL;
+
+ if (rc == -1)
+ {
+ /* pclose() itself failed. */
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close pipe to command \"%s\": %m",
+ command)));
+ }
+ else if (rc != 0)
+ {
+ char *reason = wait_result_to_str(rc);
+
+ ereport(COMMERROR,
+ (errmsg("failed to execute command \"%s\": %s",
+ command, reason)));
+
+ pfree(reason);
+ }
+
+ return (rc == 0);
+}
+
+static bool
+unset_cloexec(int fd)
+{
+ int flags;
+ int rc;
+
+ flags = fcntl(fd, F_GETFD);
+ if (flags == -1)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not get fd flags for child pipe: %m")));
+ return false;
+ }
+
+ rc = fcntl(fd, F_SETFD, flags & ~FD_CLOEXEC);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not unset FD_CLOEXEC for child pipe: %m")));
+ return false;
+ }
+
+ return true;
+}
+
+/*
+ * XXX This should go away eventually and be replaced with either a proper
+ * escape or a different strategy for communication with the validator command.
+ */
+static bool
+username_ok_for_shell(const char *username)
+{
+ /* This set is borrowed from fe_utils' appendShellStringNoError(). */
+ static const char * const allowed = "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-_./:";
+ size_t span;
+
+ Assert(username && username[0]); /* should have already been checked */
+
+ span = strspn(username, allowed);
+ if (username[span] != '\0')
+ {
+ ereport(COMMERROR,
+ (errmsg("PostgreSQL user name contains unsafe characters and cannot be passed to the OAuth validator")));
+ return false;
+ }
+
+ return true;
+}
diff --git a/src/backend/libpq/auth-sasl.c b/src/backend/libpq/auth-sasl.c
index a1d7dbb6d5..0f461a6696 100644
--- a/src/backend/libpq/auth-sasl.c
+++ b/src/backend/libpq/auth-sasl.c
@@ -20,14 +20,6 @@
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
-/*
- * Maximum accepted size of SASL messages.
- *
- * The messages that the server or libpq generate are much smaller than this,
- * but have some headroom.
- */
-#define PG_MAX_SASL_MESSAGE_LENGTH 1024
-
/*
* Perform a SASL exchange with a libpq client, using a specific mechanism
* implementation.
@@ -103,7 +95,7 @@ CheckSASLAuth(const pg_be_sasl_mech *mech, Port *port, char *shadow_pass,
/* Get the actual SASL message */
initStringInfo(&buf);
- if (pq_getmessage(&buf, PG_MAX_SASL_MESSAGE_LENGTH))
+ if (pq_getmessage(&buf, mech->max_message_length))
{
/* EOF - pq_getmessage already logged error */
pfree(buf.data);
diff --git a/src/backend/libpq/auth-scram.c b/src/backend/libpq/auth-scram.c
index ee7f52218a..4049ace470 100644
--- a/src/backend/libpq/auth-scram.c
+++ b/src/backend/libpq/auth-scram.c
@@ -118,7 +118,9 @@ static int scram_exchange(void *opaq, const char *input, int inputlen,
const pg_be_sasl_mech pg_be_scram_mech = {
scram_get_mechanisms,
scram_init,
- scram_exchange
+ scram_exchange,
+
+ PG_MAX_SASL_MESSAGE_LENGTH
};
/*
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 3533b0bc50..5c30904e2b 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -30,6 +30,7 @@
#include "libpq/auth.h"
#include "libpq/crypt.h"
#include "libpq/libpq.h"
+#include "libpq/oauth.h"
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
#include "libpq/scram.h"
@@ -302,6 +303,9 @@ auth_failed(Port *port, int status, const char *logdetail)
case uaRADIUS:
errstr = gettext_noop("RADIUS authentication failed for user \"%s\"");
break;
+ case uaOAuth:
+ errstr = gettext_noop("OAuth bearer authentication failed for user \"%s\"");
+ break;
case uaCustom:
if (CustomAuthenticationError_hook)
errstr = CustomAuthenticationError_hook(port);
@@ -627,6 +631,9 @@ ClientAuthentication(Port *port)
case uaTrust:
status = STATUS_OK;
break;
+ case uaOAuth:
+ status = CheckSASLAuth(&pg_be_oauth_mech, port, NULL, NULL);
+ break;
case uaCustom:
if (CustomAuthenticationCheck_hook)
status = CustomAuthenticationCheck_hook(port);
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index ebae992964..f7f3059927 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -135,7 +135,8 @@ static const char *const UserAuthName[] =
"cert",
"radius",
"custom",
- "peer"
+ "peer",
+ "oauth",
};
@@ -1400,6 +1401,8 @@ parse_hba_line(TokenizedLine *tok_line, int elevel)
#endif
else if (strcmp(token->string, "radius") == 0)
parsedline->auth_method = uaRADIUS;
+ else if (strcmp(token->string, "oauth") == 0)
+ parsedline->auth_method = uaOAuth;
else if (strcmp(token->string, "custom") == 0)
parsedline->auth_method = uaCustom;
else
@@ -1728,8 +1731,9 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->auth_method != uaPeer &&
hbaline->auth_method != uaGSS &&
hbaline->auth_method != uaSSPI &&
+ hbaline->auth_method != uaOAuth &&
hbaline->auth_method != uaCert)
- INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, and cert"));
+ INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, oauth, and cert"));
hbaline->usermap = pstrdup(val);
}
else if (strcmp(name, "clientcert") == 0)
@@ -2113,6 +2117,27 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->radiusidentifiers = parsed_identifiers;
hbaline->radiusidentifiers_s = pstrdup(val);
}
+ else if (strcmp(name, "issuer") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("issuer", gettext_noop("oauth"));
+ hbaline->oauth_issuer = pstrdup(val);
+ }
+ else if (strcmp(name, "scope") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("scope", gettext_noop("oauth"));
+ hbaline->oauth_scope = pstrdup(val);
+ }
+ else if (strcmp(name, "trust_validator_authz") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("trust_validator_authz", gettext_noop("oauth"));
+ if (strcmp(val, "1") == 0)
+ hbaline->oauth_skip_usermap = true;
+ else
+ hbaline->oauth_skip_usermap = false;
+ }
else if (strcmp(name, "provider") == 0)
{
REQUIRE_AUTH_OPTION(uaCustom, "provider", "custom");
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 1e3650184b..791c7c83df 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -58,6 +58,7 @@
#include "libpq/auth.h"
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
+#include "libpq/oauth.h"
#include "miscadmin.h"
#include "optimizer/cost.h"
#include "optimizer/geqo.h"
@@ -4662,6 +4663,17 @@ static struct config_string ConfigureNamesString[] =
check_backtrace_functions, assign_backtrace_functions, NULL
},
+ {
+ {"oauth_validator_command", PGC_SIGHUP, CONN_AUTH_AUTH,
+ gettext_noop("Command to validate OAuth v2 bearer tokens."),
+ NULL,
+ GUC_SUPERUSER_ONLY
+ },
+ &oauth_validator_command,
+ "",
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, NULL, NULL, NULL, NULL
diff --git a/src/include/libpq/hba.h b/src/include/libpq/hba.h
index c5aef6994c..d46c2108eb 100644
--- a/src/include/libpq/hba.h
+++ b/src/include/libpq/hba.h
@@ -39,8 +39,9 @@ typedef enum UserAuth
uaCert,
uaRADIUS,
uaCustom,
- uaPeer
-#define USER_AUTH_LAST uaPeer /* Must be last value of this enum */
+ uaPeer,
+ uaOAuth
+#define USER_AUTH_LAST uaOAuth /* Must be last value of this enum */
} UserAuth;
/*
@@ -121,6 +122,9 @@ typedef struct HbaLine
char *radiusidentifiers_s;
List *radiusports;
char *radiusports_s;
+ char *oauth_issuer;
+ char *oauth_scope;
+ bool oauth_skip_usermap;
char *custom_provider;
} HbaLine;
diff --git a/src/include/libpq/oauth.h b/src/include/libpq/oauth.h
new file mode 100644
index 0000000000..870e426af1
--- /dev/null
+++ b/src/include/libpq/oauth.h
@@ -0,0 +1,24 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth.h
+ * Interface to libpq/auth-oauth.c
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/libpq/oauth.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_OAUTH_H
+#define PG_OAUTH_H
+
+#include "libpq/libpq-be.h"
+#include "libpq/sasl.h"
+
+extern char *oauth_validator_command;
+
+/* Implementation */
+extern const pg_be_sasl_mech pg_be_oauth_mech;
+
+#endif /* PG_OAUTH_H */
diff --git a/src/include/libpq/sasl.h b/src/include/libpq/sasl.h
index 71cc0dc251..3d481cc807 100644
--- a/src/include/libpq/sasl.h
+++ b/src/include/libpq/sasl.h
@@ -26,6 +26,14 @@
#define PG_SASL_EXCHANGE_SUCCESS 1
#define PG_SASL_EXCHANGE_FAILURE 2
+/*
+ * Maximum accepted size of SASL messages.
+ *
+ * The messages that the server or libpq generate are much smaller than this,
+ * but have some headroom.
+ */
+#define PG_MAX_SASL_MESSAGE_LENGTH 1024
+
/*
* Backend SASL mechanism callbacks.
*
@@ -127,6 +135,9 @@ typedef struct pg_be_sasl_mech
const char *input, int inputlen,
char **output, int *outputlen,
const char **logdetail);
+
+ /* The maximum size allowed for client SASLResponses. */
+ int max_message_length;
} pg_be_sasl_mech;
/* Common implementation for auth.c */
--
2.25.1
v3-0007-Add-a-very-simple-authn_id-extension.patch (text/x-patch, 2.8K) download
From 667fbd709f67232155cbaa3e09d1a4b4c02eeb22 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 18 May 2021 15:01:29 -0700
Subject: [PATCH v3 7/9] Add a very simple authn_id extension
...for retrieving the authn_id from the server in tests.
---
contrib/authn_id/Makefile | 19 +++++++++++++++++++
contrib/authn_id/authn_id--1.0.sql | 8 ++++++++
contrib/authn_id/authn_id.c | 28 ++++++++++++++++++++++++++++
contrib/authn_id/authn_id.control | 5 +++++
4 files changed, 60 insertions(+)
create mode 100644 contrib/authn_id/Makefile
create mode 100644 contrib/authn_id/authn_id--1.0.sql
create mode 100644 contrib/authn_id/authn_id.c
create mode 100644 contrib/authn_id/authn_id.control
diff --git a/contrib/authn_id/Makefile b/contrib/authn_id/Makefile
new file mode 100644
index 0000000000..46026358e0
--- /dev/null
+++ b/contrib/authn_id/Makefile
@@ -0,0 +1,19 @@
+# contrib/authn_id/Makefile
+
+MODULE_big = authn_id
+OBJS = authn_id.o
+
+EXTENSION = authn_id
+DATA = authn_id--1.0.sql
+PGFILEDESC = "authn_id - information about the authenticated user"
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/authn_id
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/authn_id/authn_id--1.0.sql b/contrib/authn_id/authn_id--1.0.sql
new file mode 100644
index 0000000000..af2a4d3991
--- /dev/null
+++ b/contrib/authn_id/authn_id--1.0.sql
@@ -0,0 +1,8 @@
+/* contrib/authn_id/authn_id--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION authn_id" to load this file. \quit
+
+CREATE FUNCTION authn_id() RETURNS text
+AS 'MODULE_PATHNAME', 'authn_id'
+LANGUAGE C IMMUTABLE;
diff --git a/contrib/authn_id/authn_id.c b/contrib/authn_id/authn_id.c
new file mode 100644
index 0000000000..0fecac36a8
--- /dev/null
+++ b/contrib/authn_id/authn_id.c
@@ -0,0 +1,28 @@
+/*
+ * Extension to expose the current user's authn_id.
+ *
+ * contrib/authn_id/authn_id.c
+ */
+
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "libpq/libpq-be.h"
+#include "miscadmin.h"
+#include "utils/builtins.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(authn_id);
+
+/*
+ * Returns the current user's authenticated identity.
+ */
+Datum
+authn_id(PG_FUNCTION_ARGS)
+{
+ if (!MyProcPort->authn_id)
+ PG_RETURN_NULL();
+
+ PG_RETURN_TEXT_P(cstring_to_text(MyProcPort->authn_id));
+}
diff --git a/contrib/authn_id/authn_id.control b/contrib/authn_id/authn_id.control
new file mode 100644
index 0000000000..e0f9e06bed
--- /dev/null
+++ b/contrib/authn_id/authn_id.control
@@ -0,0 +1,5 @@
+# authn_id extension
+comment = 'current user identity'
+default_version = '1.0'
+module_pathname = '$libdir/authn_id'
+relocatable = true
--
2.25.1
v3-0008-Add-pytest-suite-for-OAuth.patch (text/x-patch, 131.6K) download
From 1c24881d4b1e8777ce176d2c276fe8120bd6e648 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Fri, 4 Jun 2021 09:06:38 -0700
Subject: [PATCH v3 8/9] Add pytest suite for OAuth
Requires Python 3; on the first run of `make installcheck` the
dependencies will be installed into ./venv for you. See the README for
more details.
---
src/test/python/.gitignore | 2 +
src/test/python/Makefile | 38 +
src/test/python/README | 54 ++
src/test/python/client/__init__.py | 0
src/test/python/client/conftest.py | 126 +++
src/test/python/client/test_client.py | 180 ++++
src/test/python/client/test_oauth.py | 936 ++++++++++++++++++
src/test/python/pq3.py | 727 ++++++++++++++
src/test/python/pytest.ini | 4 +
src/test/python/requirements.txt | 7 +
src/test/python/server/__init__.py | 0
src/test/python/server/conftest.py | 45 +
src/test/python/server/test_oauth.py | 1012 ++++++++++++++++++++
src/test/python/server/test_server.py | 21 +
src/test/python/server/validate_bearer.py | 101 ++
src/test/python/server/validate_reflect.py | 34 +
src/test/python/test_internals.py | 138 +++
src/test/python/test_pq3.py | 558 +++++++++++
src/test/python/tls.py | 195 ++++
19 files changed, 4178 insertions(+)
create mode 100644 src/test/python/.gitignore
create mode 100644 src/test/python/Makefile
create mode 100644 src/test/python/README
create mode 100644 src/test/python/client/__init__.py
create mode 100644 src/test/python/client/conftest.py
create mode 100644 src/test/python/client/test_client.py
create mode 100644 src/test/python/client/test_oauth.py
create mode 100644 src/test/python/pq3.py
create mode 100644 src/test/python/pytest.ini
create mode 100644 src/test/python/requirements.txt
create mode 100644 src/test/python/server/__init__.py
create mode 100644 src/test/python/server/conftest.py
create mode 100644 src/test/python/server/test_oauth.py
create mode 100644 src/test/python/server/test_server.py
create mode 100755 src/test/python/server/validate_bearer.py
create mode 100755 src/test/python/server/validate_reflect.py
create mode 100644 src/test/python/test_internals.py
create mode 100644 src/test/python/test_pq3.py
create mode 100644 src/test/python/tls.py
diff --git a/src/test/python/.gitignore b/src/test/python/.gitignore
new file mode 100644
index 0000000000..0e8f027b2e
--- /dev/null
+++ b/src/test/python/.gitignore
@@ -0,0 +1,2 @@
+__pycache__/
+/venv/
diff --git a/src/test/python/Makefile b/src/test/python/Makefile
new file mode 100644
index 0000000000..b0695b6287
--- /dev/null
+++ b/src/test/python/Makefile
@@ -0,0 +1,38 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+# Only Python 3 is supported, but if it's named something different on your
+# system you can override it with the PYTHON3 variable.
+PYTHON3 := python3
+
+# All dependencies are placed into this directory. The default is .gitignored
+# for you, but you can override it if you'd like.
+VENV := ./venv
+
+override VBIN := $(VENV)/bin
+override PIP := $(VBIN)/pip
+override PYTEST := $(VBIN)/py.test
+override ISORT := $(VBIN)/isort
+override BLACK := $(VBIN)/black
+
+.PHONY: installcheck indent
+
+installcheck: $(PYTEST)
+ $(PYTEST) -v -rs
+
+indent: $(ISORT) $(BLACK)
+ $(ISORT) --profile black *.py client/*.py server/*.py
+ $(BLACK) *.py client/*.py server/*.py
+
+$(PYTEST) $(ISORT) $(BLACK) &: requirements.txt | $(PIP)
+ $(PIP) install --force-reinstall -r $<
+
+$(PIP):
+ $(PYTHON3) -m venv $(VENV)
+
+# A convenience recipe to rebuild psycopg2 against the local libpq.
+.PHONY: rebuild-psycopg2
+rebuild-psycopg2: | $(PIP)
+ $(PIP) install --force-reinstall --no-binary :all: $(shell grep psycopg2 requirements.txt)
diff --git a/src/test/python/README b/src/test/python/README
new file mode 100644
index 0000000000..0bda582c4b
--- /dev/null
+++ b/src/test/python/README
@@ -0,0 +1,54 @@
+A test suite for exercising both the libpq client and the server backend at the
+protocol level, based on pytest and Construct.
+
+The test suite currently assumes that the standard PG* environment variables
+point to the database under test and are sufficient to log in a superuser on
+that system. In other words, a bare `psql` needs to Just Work before the test
+suite can do its thing. For a newly built dev cluster, typically all that I need
+to do is a
+
+ export PGDATABASE=postgres
+
+but you can adjust as needed for your setup.
+
+## Requirements
+
+A supported version (3.6+) of Python.
+
+The first run of
+
+ make installcheck
+
+will install a local virtual environment and all needed dependencies. During
+development, if libpq changes incompatibly, you can issue
+
+ $ make rebuild-psycopg2
+
+to force a rebuild of the client library.
+
+## Hacking
+
+The code style is enforced by a _very_ opinionated autoformatter. Running the
+
+ make indent
+
+recipe will invoke it for you automatically. Don't fight the tool; part of the
+zen is in knowing that if the formatter makes your code ugly, there's probably a
+cleaner way to write your code.
+
+## Advanced Usage
+
+The Makefile is there for convenience, but you don't have to use it. Activate
+the virtualenv to be able to use pytest directly:
+
+ $ source venv/bin/activate
+ $ py.test -k oauth
+ ...
+ $ py.test ./server/test_server.py
+ ...
+ $ deactivate # puts the PATH et al back the way it was before
+
+To make quick smoke tests possible, slow tests have been marked explicitly. You
+can skip them by saying e.g.
+
+ $ py.test -m 'not slow'
diff --git a/src/test/python/client/__init__.py b/src/test/python/client/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/src/test/python/client/conftest.py b/src/test/python/client/conftest.py
new file mode 100644
index 0000000000..f38da7a138
--- /dev/null
+++ b/src/test/python/client/conftest.py
@@ -0,0 +1,126 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import socket
+import sys
+import threading
+
+import psycopg2
+import pytest
+
+import pq3
+
+BLOCKING_TIMEOUT = 2 # the number of seconds to wait for blocking calls
+
+
+@pytest.fixture
+def server_socket(unused_tcp_port_factory):
+ """
+ Returns a listening socket bound to an ephemeral port.
+ """
+ with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+ s.bind(("127.0.0.1", unused_tcp_port_factory()))
+ s.listen(1)
+ s.settimeout(BLOCKING_TIMEOUT)
+ yield s
+
+
+class ClientHandshake(threading.Thread):
+ """
+ A thread that connects to a local Postgres server using psycopg2. Once the
+ opening handshake completes, the connection will be immediately closed.
+ """
+
+ def __init__(self, *, port, **kwargs):
+ super().__init__()
+
+ kwargs["port"] = port
+ self._kwargs = kwargs
+
+ self.exception = None
+
+ def run(self):
+ try:
+ conn = psycopg2.connect(host="127.0.0.1", **self._kwargs)
+ conn.close()
+ except Exception as e:
+ self.exception = e
+
+ def check_completed(self, timeout=BLOCKING_TIMEOUT):
+ """
+ Joins the client thread. Raises an exception if the thread could not be
+ joined, or if it threw an exception itself. (The exception will be
+ cleared, so future calls to check_completed will succeed.)
+ """
+ self.join(timeout)
+
+ if self.is_alive():
+ raise TimeoutError("client thread did not handshake within the timeout")
+ elif self.exception:
+ e = self.exception
+ self.exception = None
+ raise e
+
+
+@pytest.fixture
+def accept(server_socket):
+ """
+ Returns a factory function that, when called, returns a pair (sock, client)
+ where sock is a server socket that has accepted a connection from client,
+ and client is an instance of ClientHandshake. Clients will complete their
+ handshakes and cleanly disconnect.
+
+ The default connstring options may be extended or overridden by passing
+ arbitrary keyword arguments. Keep in mind that you generally should not
+ override the host or port, since they point to the local test server.
+
+ For situations where a client needs to connect more than once to complete a
+ handshake, the accept function may be called more than once. (The client
+ returned for subsequent calls will always be the same client that was
+ returned for the first call.)
+
+ Tests must either complete the handshake so that the client thread can be
+ automatically joined during teardown, or else call client.check_completed()
+ and manually handle any expected errors.
+ """
+ _, port = server_socket.getsockname()
+
+ client = None
+ default_opts = dict(
+ port=port,
+ user=pq3.pguser(),
+ sslmode="disable",
+ )
+
+ def factory(**kwargs):
+ nonlocal client
+
+ if client is None:
+ opts = dict(default_opts)
+ opts.update(kwargs)
+
+ # The server_socket is already listening, so the client thread can
+ # be safely started; it'll block on the connection until we accept.
+ client = ClientHandshake(**opts)
+ client.start()
+
+ sock, _ = server_socket.accept()
+ return sock, client
+
+ yield factory
+ client.check_completed()
+
+
+@pytest.fixture
+def conn(accept):
+ """
+ Returns an accepted, wrapped pq3 connection to a psycopg2 client. The socket
+ will be closed when the test finishes, and the client will be checked for a
+ cleanly completed handshake.
+ """
+ sock, client = accept()
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ yield conn
diff --git a/src/test/python/client/test_client.py b/src/test/python/client/test_client.py
new file mode 100644
index 0000000000..c4c946fda4
--- /dev/null
+++ b/src/test/python/client/test_client.py
@@ -0,0 +1,180 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import sys
+
+import psycopg2
+import pytest
+from cryptography.hazmat.primitives import hashes, hmac
+
+import pq3
+
+
+def finish_handshake(conn):
+ """
+ Sends the AuthenticationOK message and the standard opening salvo of server
+ messages, then asserts that the client immediately sends a Terminate message
+ to close the connection cleanly.
+ """
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.OK)
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"client_encoding", value=b"UTF-8")
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"DateStyle", value=b"ISO, MDY")
+ pq3.send(conn, pq3.types.ReadyForQuery, status=b"I")
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.Terminate
+
+
+def test_handshake(conn):
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ finish_handshake(conn)
+
+
+def test_aborted_connection(accept):
+ """
+ Make sure the client correctly reports an early close during handshakes.
+ """
+ sock, client = accept()
+ sock.close()
+
+ expected = "server closed the connection unexpectedly"
+ with pytest.raises(psycopg2.OperationalError, match=expected):
+ client.check_completed()
+
+
+#
+# SCRAM-SHA-256 (see RFC 5802: https://tools.ietf.org/html/rfc5802)
+#
+
+
+@pytest.fixture
+def password():
+ """
+ Returns a password for use by both client and server.
+ """
+ # TODO: parameterize this with passwords that require SASLprep.
+ return "secret"
+
+
+@pytest.fixture
+def pwconn(accept, password):
+ """
+ Like the conn fixture, but uses a password in the connection.
+ """
+ sock, client = accept(password=password)
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ yield conn
+
+
+def sha256(data):
+ """The H(str) function from Section 2.2."""
+ digest = hashes.Hash(hashes.SHA256())
+ digest.update(data)
+ return digest.finalize()
+
+
+def hmac_256(key, data):
+ """The HMAC(key, str) function from Section 2.2."""
+ h = hmac.HMAC(key, hashes.SHA256())
+ h.update(data)
+ return h.finalize()
+
+
+def xor(a, b):
+ """The XOR operation from Section 2.2."""
+ res = bytearray(a)
+ for i, byte in enumerate(b):
+ res[i] ^= byte
+ return bytes(res)
+
+
+def h_i(data, salt, i):
+ """The Hi(str, salt, i) function from Section 2.2."""
+ assert i > 0
+
+ acc = hmac_256(data, salt + b"\x00\x00\x00\x01")
+ last = acc
+ i -= 1
+
+ while i:
+ u = hmac_256(data, last)
+ acc = xor(acc, u)
+
+ last = u
+ i -= 1
+
+ return acc
+
+
+def test_scram(pwconn, password):
+ startup = pq3.recv1(pwconn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ pwconn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"SCRAM-SHA-256", b""],
+ )
+
+ # Get the client-first-message.
+ pkt = pq3.recv1(pwconn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ initial = pq3.SASLInitialResponse.parse(pkt.payload)
+ assert initial.name == b"SCRAM-SHA-256"
+
+ c_bind, authzid, c_name, c_nonce = initial.data.split(b",")
+ assert c_bind == b"n" # no channel bindings on a plaintext connection
+ assert authzid == b"" # we don't support authzid currently
+ assert c_name == b"n=" # libpq doesn't honor the GS2 username
+ assert c_nonce.startswith(b"r=")
+
+ # Send the server-first-message.
+ salt = b"12345"
+ iterations = 2
+
+ s_nonce = c_nonce + b"somenonce"
+ s_salt = b"s=" + base64.b64encode(salt)
+ s_iterations = b"i=%d" % iterations
+
+ msg = b",".join([s_nonce, s_salt, s_iterations])
+ pq3.send(pwconn, pq3.types.AuthnRequest, type=pq3.authn.SASLContinue, body=msg)
+
+ # Get the client-final-message.
+ pkt = pq3.recv1(pwconn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ c_bind_final, c_nonce_final, c_proof = pkt.payload.split(b",")
+ assert c_bind_final == b"c=" + base64.b64encode(c_bind + b"," + authzid + b",")
+ assert c_nonce_final == s_nonce
+
+ # Calculate what the client proof should be.
+ salted_password = h_i(password.encode("ascii"), salt, iterations)
+ client_key = hmac_256(salted_password, b"Client Key")
+ stored_key = sha256(client_key)
+
+ auth_message = b",".join(
+ [c_name, c_nonce, s_nonce, s_salt, s_iterations, c_bind_final, c_nonce_final]
+ )
+ client_signature = hmac_256(stored_key, auth_message)
+ client_proof = xor(client_key, client_signature)
+
+ expected = b"p=" + base64.b64encode(client_proof)
+ assert c_proof == expected
+
+ # Send the correct server signature.
+ server_key = hmac_256(salted_password, b"Server Key")
+ server_signature = hmac_256(server_key, auth_message)
+
+ s_verify = b"v=" + base64.b64encode(server_signature)
+ pq3.send(pwconn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal, body=s_verify)
+
+ # Done!
+ finish_handshake(pwconn)
diff --git a/src/test/python/client/test_oauth.py b/src/test/python/client/test_oauth.py
new file mode 100644
index 0000000000..a754a9c0b6
--- /dev/null
+++ b/src/test/python/client/test_oauth.py
@@ -0,0 +1,936 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import http.server
+import json
+import secrets
+import sys
+import threading
+import time
+import urllib.parse
+
+import psycopg2
+import pytest
+
+import pq3
+
+from .conftest import BLOCKING_TIMEOUT
+
+
+def finish_handshake(conn):
+ """
+ Sends the AuthenticationOK message and the standard opening salvo of server
+ messages, then asserts that the client immediately sends a Terminate message
+ to close the connection cleanly.
+ """
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.OK)
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"client_encoding", value=b"UTF-8")
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"DateStyle", value=b"ISO, MDY")
+ pq3.send(conn, pq3.types.ReadyForQuery, status=b"I")
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.Terminate
+
+
+#
+# OAUTHBEARER (see RFC 7628: https://tools.ietf.org/html/rfc7628)
+#
+
+
+def start_oauth_handshake(conn):
+ """
+ Negotiates an OAUTHBEARER SASL challenge. Returns the client's initial
+ response data.
+ """
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn, pq3.types.AuthnRequest, type=pq3.authn.SASL, body=[b"OAUTHBEARER", b""]
+ )
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ initial = pq3.SASLInitialResponse.parse(pkt.payload)
+ assert initial.name == b"OAUTHBEARER"
+
+ return initial.data
+
+
+def get_auth_value(initial):
+ """
+ Finds the auth value (e.g. "Bearer somedata..." in the client's initial SASL
+ response.
+ """
+ kvpairs = initial.split(b"\x01")
+ assert kvpairs[0] == b"n,," # no channel binding or authzid
+ assert kvpairs[2] == b"" # ends with an empty kvpair
+ assert kvpairs[3] == b"" # ...and there's nothing after it
+ assert len(kvpairs) == 4
+
+ key, value = kvpairs[1].split(b"=", 2)
+ assert key == b"auth"
+
+ return value
+
+
+def xtest_oauth_success(conn): # TODO
+ initial = start_oauth_handshake(conn)
+
+ auth = get_auth_value(initial)
+ assert auth.startswith(b"Bearer ")
+
+ # Accept the token. TODO actually validate
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+class OpenIDProvider(threading.Thread):
+ """
+ A thread that runs a mock OpenID provider server.
+ """
+
+ def __init__(self, *, port):
+ super().__init__()
+
+ self.exception = None
+
+ addr = ("", port)
+ self.server = self._Server(addr, self._Handler)
+
+ # TODO: allow HTTPS only, somehow
+ oauth = self._OAuthState()
+ oauth.host = f"localhost:{port}"
+ oauth.issuer = f"http://localhost:{port}"
+
+ # The following endpoints are required to be advertised by providers,
+ # even though our chosen client implementation does not actually make
+ # use of them.
+ oauth.register_endpoint(
+ "authorization_endpoint", "POST", "/authorize", self._authorization_handler
+ )
+ oauth.register_endpoint("jwks_uri", "GET", "/keys", self._jwks_handler)
+
+ self.server.oauth = oauth
+
+ def run(self):
+ try:
+ self.server.serve_forever()
+ except Exception as e:
+ self.exception = e
+
+ def stop(self, timeout=BLOCKING_TIMEOUT):
+ """
+ Shuts down the server and joins its thread. Raises an exception if the
+ thread could not be joined, or if it threw an exception itself. Must
+ only be called once, after start().
+ """
+ self.server.shutdown()
+ self.join(timeout)
+
+ if self.is_alive():
+ raise TimeoutError("client thread did not handshake within the timeout")
+ elif self.exception:
+ e = self.exception
+ raise e
+
+ class _OAuthState(object):
+ def __init__(self):
+ self.endpoint_paths = {}
+ self._endpoints = {}
+
+ def register_endpoint(self, name, method, path, func):
+ if method not in self._endpoints:
+ self._endpoints[method] = {}
+
+ self._endpoints[method][path] = func
+ self.endpoint_paths[name] = path
+
+ def endpoint(self, method, path):
+ if method not in self._endpoints:
+ return None
+
+ return self._endpoints[method].get(path)
+
+ class _Server(http.server.HTTPServer):
+ def handle_error(self, request, addr):
+ self.shutdown_request(request)
+ raise
+
+ @staticmethod
+ def _jwks_handler(headers, params):
+ return 200, {"keys": []}
+
+ @staticmethod
+ def _authorization_handler(headers, params):
+ # We don't actually want this to be called during these tests -- we
+ # should be using the device authorization endpoint instead.
+ assert (
+ False
+ ), "authorization handler called instead of device authorization handler"
+
+ class _Handler(http.server.BaseHTTPRequestHandler):
+ timeout = BLOCKING_TIMEOUT
+
+ def _discovery_handler(self, headers, params):
+ oauth = self.server.oauth
+
+ doc = {
+ "issuer": oauth.issuer,
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ }
+
+ for name, path in oauth.endpoint_paths.items():
+ doc[name] = oauth.issuer + path
+
+ return 200, doc
+
+ def _handle(self, *, params=None, handler=None):
+ oauth = self.server.oauth
+ assert self.headers["Host"] == oauth.host
+
+ if handler is None:
+ handler = oauth.endpoint(self.command, self.path)
+ assert (
+ handler is not None
+ ), f"no registered endpoint for {self.command} {self.path}"
+
+ code, resp = handler(self.headers, params)
+
+ self.send_response(code)
+ self.send_header("Content-Type", "application/json")
+ self.end_headers()
+
+ resp = json.dumps(resp)
+ resp = resp.encode("utf-8")
+ self.wfile.write(resp)
+
+ self.close_connection = True
+
+ def do_GET(self):
+ if self.path == "/.well-known/openid-configuration":
+ self._handle(handler=self._discovery_handler)
+ return
+
+ self._handle()
+
+ def _request_body(self):
+ length = self.headers["Content-Length"]
+
+ # Handle only an explicit content-length.
+ assert length is not None
+ length = int(length)
+
+ return self.rfile.read(length).decode("utf-8")
+
+ def do_POST(self):
+ assert self.headers["Content-Type"] == "application/x-www-form-urlencoded"
+
+ body = self._request_body()
+ params = urllib.parse.parse_qs(body)
+
+ self._handle(params=params)
+
+
+@pytest.fixture
+def openid_provider(unused_tcp_port_factory):
+ """
+ A fixture that returns the OAuth state of a running OpenID provider server. The
+ server will be stopped when the fixture is torn down.
+ """
+ thread = OpenIDProvider(port=unused_tcp_port_factory())
+ thread.start()
+
+ try:
+ yield thread.server.oauth
+ finally:
+ thread.stop()
+
+
+@pytest.mark.parametrize("secret", [None, "", "hunter2"])
+@pytest.mark.parametrize("scope", [None, "", "openid email"])
+@pytest.mark.parametrize("retries", [0, 1])
+def test_oauth_with_explicit_issuer(
+ capfd, accept, openid_provider, retries, scope, secret
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ oauth_client_secret=secret,
+ oauth_scope=scope,
+ )
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+ verification_url = "https://example.com/device"
+
+ access_token = secrets.token_urlsafe()
+
+ def check_client_authn(headers, params):
+ if not secret:
+ assert params["client_id"] == [client_id]
+ return
+
+ # Require the client to use Basic authn; request-body credentials are
+ # NOT RECOMMENDED (RFC 6749, Sec. 2.3.1).
+ assert "Authorization" in headers
+
+ method, creds = headers["Authorization"].split()
+ assert method == "Basic"
+
+ expected = f"{client_id}:{secret}"
+ assert base64.b64decode(creds) == expected.encode("ascii")
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ check_client_authn(headers, params)
+
+ if scope:
+ assert params["scope"] == [scope]
+ else:
+ assert "scope" not in params
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": verification_url,
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ attempts = 0
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ check_client_authn(headers, params)
+
+ assert params["grant_type"] == ["urn:ietf:params:oauth:grant-type:device_code"]
+ assert params["device_code"] == [device_code]
+
+ now = time.monotonic()
+
+ with retry_lock:
+ nonlocal attempts
+
+ # If the test wants to force the client to retry, return an
+ # authorization_pending response and decrement the retry count.
+ if attempts < retries:
+ attempts += 1
+ return 400, {"error": "authorization_pending"}
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+ if retries:
+ # Finally, make sure that the client prompted the user with the expected
+ # authorization URL and user code.
+ expected = f"Visit {verification_url} and enter the code: {user_code}"
+ _, stderr = capfd.readouterr()
+ assert expected in stderr
+
+
+def test_oauth_requires_client_id(accept, openid_provider):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ # Do not set a client ID; this should cause a client error after the
+ # server asks for OAUTHBEARER and the client tries to contact the
+ # issuer.
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake.
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"OAUTHBEARER", b""],
+ )
+
+ # The client should disconnect at this point.
+ assert not conn.read()
+
+ expected_error = "no oauth_client_id is set"
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.slow
+@pytest.mark.parametrize("error_code", ["authorization_pending", "slow_down"])
+@pytest.mark.parametrize("retries", [1, 2])
+def test_oauth_retry_interval(accept, openid_provider, retries, error_code):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id="some-id",
+ )
+
+ expected_retry_interval = 1
+ access_token = secrets.token_urlsafe()
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ resp = {
+ "device_code": "my-device-code",
+ "user_code": "my-user-code",
+ "interval": expected_retry_interval,
+ "verification_uri": "https://example.com",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ attempts = 0
+ last_retry = None
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ now = time.monotonic()
+
+ with retry_lock:
+ nonlocal attempts, last_retry, expected_retry_interval
+
+ # Make sure the retry interval is being respected by the client.
+ if last_retry is not None:
+ interval = now - last_retry
+ assert interval >= expected_retry_interval
+
+ last_retry = now
+
+ # If the test wants to force the client to retry, return the desired
+ # error response and decrement the retry count.
+ if attempts < retries:
+ attempts += 1
+
+ # A slow_down code requires the client to additionally increase
+ # its interval by five seconds.
+ if error_code == "slow_down":
+ expected_retry_interval += 5
+
+ return 400, {"error": error_code}
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+@pytest.mark.parametrize(
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
+ {
+ "error": "invalid_client",
+ "error_description": "client authentication failed",
+ },
+ r"client authentication failed \(invalid_client\)",
+ id="authentication failure with description",
+ ),
+ pytest.param(
+ {"error": "invalid_request"},
+ r"\(invalid_request\)",
+ id="invalid request without description",
+ ),
+ pytest.param(
+ {},
+ r"failed to obtain device authorization",
+ id="broken error response",
+ ),
+ ],
+)
+def test_oauth_device_authorization_failures(
+ accept, openid_provider, failure_mode, error_pattern
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ )
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ return 400, failure_mode
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert False, "token endpoint was invoked unexpectedly"
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"OAUTHBEARER", b""],
+ )
+
+ # The client should not continue the connection due to the hardcoded
+ # provider failure; we disconnect here.
+
+ # Now make sure the client correctly failed.
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
+ {
+ "error": "expired_token",
+ "error_description": "the device code has expired",
+ },
+ r"the device code has expired \(expired_token\)",
+ id="expired token with description",
+ ),
+ pytest.param(
+ {"error": "access_denied"},
+ r"\(access_denied\)",
+ id="access denied without description",
+ ),
+ pytest.param(
+ {},
+ r"OAuth token retrieval failed",
+ id="broken error response",
+ ),
+ ],
+)
+@pytest.mark.parametrize("retries", [0, 1])
+def test_oauth_token_failures(
+ accept, openid_provider, retries, failure_mode, error_pattern
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ )
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ assert params["client_id"] == [client_id]
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": "https://example.com/device",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ with retry_lock:
+ nonlocal retries
+
+ # If the test wants to force the client to retry, return an
+ # authorization_pending response and decrement the retry count.
+ if retries > 0:
+ retries -= 1
+ return 400, {"error": "authorization_pending"}
+
+ return 400, failure_mode
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"OAUTHBEARER", b""],
+ )
+
+ # The client should not continue the connection due to the hardcoded
+ # provider failure; we disconnect here.
+
+ # Now make sure the client correctly failed.
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize("scope", [None, "openid email"])
+@pytest.mark.parametrize(
+ "base_response",
+ [
+ {"status": "invalid_token"},
+ {"extra_object": {"key": "value"}, "status": "invalid_token"},
+ {"extra_object": {"status": 1}, "status": "invalid_token"},
+ ],
+)
+def test_oauth_discovery(accept, openid_provider, base_response, scope):
+ sock, client = accept(oauth_client_id=secrets.token_hex())
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+ verification_url = "https://example.com/device"
+
+ access_token = secrets.token_urlsafe()
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ if scope:
+ assert params["scope"] == [scope]
+ else:
+ assert "scope" not in params
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": verification_url,
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert params["grant_type"] == ["urn:ietf:params:oauth:grant-type:device_code"]
+ assert params["device_code"] == [device_code]
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # For discovery, the client should send an empty auth header. See
+ # RFC 7628, Sec. 4.3.
+ auth = get_auth_value(initial)
+ assert auth == b""
+
+ # We will fail the first SASL exchange. First return a link to the
+ # discovery document, pointing to the test provider server.
+ resp = dict(base_response)
+
+ discovery_uri = f"{openid_provider.issuer}/.well-known/openid-configuration"
+ resp["openid-configuration"] = discovery_uri
+
+ if scope:
+ resp["scope"] = scope
+
+ resp = json.dumps(resp)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASLContinue,
+ body=resp.encode("ascii"),
+ )
+
+ # Per RFC, the client is required to send a dummy ^A response.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+ assert pkt.payload == b"\x01"
+
+ # Now fail the SASL exchange.
+ pq3.send(
+ conn,
+ pq3.types.ErrorResponse,
+ fields=[
+ b"SFATAL",
+ b"C28000",
+ b"Mdoesn't matter",
+ b"",
+ ],
+ )
+
+ # The client will connect to us a second time, using the parameters we sent
+ # it.
+ sock, _ = accept()
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+@pytest.mark.parametrize(
+ "response,expected_error",
+ [
+ pytest.param(
+ "abcde",
+ 'Token "abcde" is invalid',
+ id="bad JSON: invalid syntax",
+ ),
+ pytest.param(
+ '"abcde"',
+ "top-level element must be an object",
+ id="bad JSON: top-level element is a string",
+ ),
+ pytest.param(
+ "[]",
+ "top-level element must be an object",
+ id="bad JSON: top-level element is an array",
+ ),
+ pytest.param(
+ "{}",
+ "server sent error response without a status",
+ id="bad JSON: no status member",
+ ),
+ pytest.param(
+ '{ "status": null }',
+ 'field "status" must be a string',
+ id="bad JSON: null status member",
+ ),
+ pytest.param(
+ '{ "status": 0 }',
+ 'field "status" must be a string',
+ id="bad JSON: int status member",
+ ),
+ pytest.param(
+ '{ "status": [ "bad" ] }',
+ 'field "status" must be a string',
+ id="bad JSON: array status member",
+ ),
+ pytest.param(
+ '{ "status": { "bad": "bad" } }',
+ 'field "status" must be a string',
+ id="bad JSON: object status member",
+ ),
+ pytest.param(
+ '{ "nested": { "status": "bad" } }',
+ "server sent error response without a status",
+ id="bad JSON: nested status",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token" ',
+ "The input string ended unexpectedly",
+ id="bad JSON: unterminated object",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token" } { }',
+ 'Expected end of input, but found "{"',
+ id="bad JSON: trailing data",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "openid-configuration": 1 }',
+ 'field "openid-configuration" must be a string',
+ id="bad JSON: int openid-configuration member",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "openid-configuration": 1 }',
+ 'field "openid-configuration" must be a string',
+ id="bad JSON: int openid-configuration member",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "scope": 1 }',
+ 'field "scope" must be a string',
+ id="bad JSON: int scope member",
+ ),
+ ],
+)
+def test_oauth_discovery_server_error(accept, response, expected_error):
+ sock, client = accept(oauth_client_id=secrets.token_hex())
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # Fail the SASL exchange with an invalid JSON response.
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASLContinue,
+ body=response.encode("utf-8"),
+ )
+
+ # The client should disconnect, so the socket is closed here. (If
+ # the client doesn't disconnect, it will report a different error
+ # below and the test will fail.)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "sasl_err,resp_type,resp_payload,expected_error",
+ [
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.ErrorResponse,
+ dict(
+ fields=[b"SFATAL", b"C28000", b"Mexpected error message", b""],
+ ),
+ "expected error message",
+ id="standard server error: invalid_request",
+ ),
+ pytest.param(
+ {"status": "invalid_token"},
+ pq3.types.ErrorResponse,
+ dict(
+ fields=[b"SFATAL", b"C28000", b"Mexpected error message", b""],
+ ),
+ "expected error message",
+ id="standard server error: invalid_token without discovery URI",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASLContinue, body=b""),
+ "server sent additional OAuth data",
+ id="broken server: additional challenge after error",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASLFinal),
+ "server sent additional OAuth data",
+ id="broken server: SASL success after error",
+ ),
+ ],
+)
+def test_oauth_server_error(accept, sasl_err, resp_type, resp_payload, expected_error):
+ sock, client = accept()
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ start_oauth_handshake(conn)
+
+ # Ignore the client data. Return an error "challenge".
+ resp = json.dumps(sasl_err)
+ resp = resp.encode("utf-8")
+
+ pq3.send(
+ conn, pq3.types.AuthnRequest, type=pq3.authn.SASLContinue, body=resp
+ )
+
+ # Per RFC, the client is required to send a dummy ^A response.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+ assert pkt.payload == b"\x01"
+
+ # Now fail the SASL exchange (in either a valid way, or an invalid
+ # one, depending on the test).
+ pq3.send(conn, resp_type, **resp_payload)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
diff --git a/src/test/python/pq3.py b/src/test/python/pq3.py
new file mode 100644
index 0000000000..3a22dad0b6
--- /dev/null
+++ b/src/test/python/pq3.py
@@ -0,0 +1,727 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import getpass
+import io
+import os
+import ssl
+import sys
+import textwrap
+
+from construct import *
+
+import tls
+
+
+def protocol(major, minor):
+ """
+ Returns the protocol version, in integer format, corresponding to the given
+ major and minor version numbers.
+ """
+ return (major << 16) | minor
+
+
+# Startup
+
+StringList = GreedyRange(NullTerminated(GreedyBytes))
+
+
+class KeyValueAdapter(Adapter):
+ """
+ Turns a key-value store into a null-terminated list of null-terminated
+ strings, as presented on the wire in the startup packet.
+ """
+
+ def _encode(self, obj, context, path):
+ if isinstance(obj, list):
+ return obj
+
+ l = []
+
+ for k, v in obj.items():
+ if isinstance(k, str):
+ k = k.encode("utf-8")
+ l.append(k)
+
+ if isinstance(v, str):
+ v = v.encode("utf-8")
+ l.append(v)
+
+ l.append(b"")
+ return l
+
+ def _decode(self, obj, context, path):
+ # TODO: turn a list back into a dict
+ return obj
+
+
+KeyValues = KeyValueAdapter(StringList)
+
+_startup_payload = Switch(
+ this.proto,
+ {
+ protocol(3, 0): KeyValues,
+ },
+ default=GreedyBytes,
+)
+
+
+def _default_protocol(this):
+ try:
+ if isinstance(this.payload, (list, dict)):
+ return protocol(3, 0)
+ except AttributeError:
+ pass # no payload passed during build
+
+ return 0
+
+
+def _startup_payload_len(this):
+ """
+ The payload field has a fixed size based on the length of the packet. But
+ if the caller hasn't supplied an explicit length at build time, we have to
+ build the payload to figure out how long it is, which requires us to know
+ the length first... This function exists solely to break the cycle.
+ """
+ assert this._building, "_startup_payload_len() cannot be called during parsing"
+
+ try:
+ payload = this.payload
+ except AttributeError:
+ return 0 # no payload
+
+ if isinstance(payload, bytes):
+ # already serialized; just use the given length
+ return len(payload)
+
+ try:
+ proto = this.proto
+ except AttributeError:
+ proto = _default_protocol(this)
+
+ data = _startup_payload.build(payload, proto=proto)
+ return len(data)
+
+
+Startup = Struct(
+ "len" / Default(Int32sb, lambda this: _startup_payload_len(this) + 8),
+ "proto" / Default(Hex(Int32sb), _default_protocol),
+ "payload" / FixedSized(this.len - 8, Default(_startup_payload, b"")),
+)
+
+# Pq3
+
+# Adapted from construct.core.EnumIntegerString
+class EnumNamedByte:
+ def __init__(self, val, name):
+ self._val = val
+ self._name = name
+
+ def __int__(self):
+ return ord(self._val)
+
+ def __str__(self):
+ return "(enum) %s %r" % (self._name, self._val)
+
+ def __repr__(self):
+ return "EnumNamedByte(%r)" % self._val
+
+ def __eq__(self, other):
+ if isinstance(other, EnumNamedByte):
+ other = other._val
+ if not isinstance(other, bytes):
+ return NotImplemented
+
+ return self._val == other
+
+ def __hash__(self):
+ return hash(self._val)
+
+
+# Adapted from construct.core.Enum
+class ByteEnum(Adapter):
+ def __init__(self, **mapping):
+ super(ByteEnum, self).__init__(Byte)
+ self.namemapping = {k: EnumNamedByte(v, k) for k, v in mapping.items()}
+ self.decmapping = {v: EnumNamedByte(v, k) for k, v in mapping.items()}
+
+ def __getattr__(self, name):
+ if name in self.namemapping:
+ return self.decmapping[self.namemapping[name]]
+ raise AttributeError
+
+ def _decode(self, obj, context, path):
+ b = bytes([obj])
+ try:
+ return self.decmapping[b]
+ except KeyError:
+ return EnumNamedByte(b, "(unknown)")
+
+ def _encode(self, obj, context, path):
+ if isinstance(obj, int):
+ return obj
+ elif isinstance(obj, bytes):
+ return ord(obj)
+ return int(obj)
+
+
+types = ByteEnum(
+ ErrorResponse=b"E",
+ ReadyForQuery=b"Z",
+ Query=b"Q",
+ EmptyQueryResponse=b"I",
+ AuthnRequest=b"R",
+ PasswordMessage=b"p",
+ BackendKeyData=b"K",
+ CommandComplete=b"C",
+ ParameterStatus=b"S",
+ DataRow=b"D",
+ Terminate=b"X",
+)
+
+
+authn = Enum(
+ Int32ub,
+ OK=0,
+ SASL=10,
+ SASLContinue=11,
+ SASLFinal=12,
+)
+
+
+_authn_body = Switch(
+ this.type,
+ {
+ authn.OK: Terminated,
+ authn.SASL: StringList,
+ },
+ default=GreedyBytes,
+)
+
+
+def _data_len(this):
+ assert this._building, "_data_len() cannot be called during parsing"
+
+ if not hasattr(this, "data") or this.data is None:
+ return -1
+
+ return len(this.data)
+
+
+# The protocol reuses the PasswordMessage for several authentication response
+# types, and there's no good way to figure out which is which without keeping
+# state for the entire stream. So this is a separate Construct that can be
+# explicitly parsed/built by code that knows it's needed.
+SASLInitialResponse = Struct(
+ "name" / NullTerminated(GreedyBytes),
+ "len" / Default(Int32sb, lambda this: _data_len(this)),
+ "data"
+ / IfThenElse(
+ # Allow tests to explicitly pass an incorrect length during testing, by
+ # not enforcing a FixedSized during build. (The len calculation above
+ # defaults to the correct size.)
+ this._building,
+ Optional(GreedyBytes),
+ If(this.len != -1, Default(FixedSized(this.len, GreedyBytes), b"")),
+ ),
+ Terminated, # make sure the entire response is consumed
+)
+
+
+_column = FocusedSeq(
+ "data",
+ "len" / Default(Int32sb, lambda this: _data_len(this)),
+ "data" / If(this.len != -1, FixedSized(this.len, GreedyBytes)),
+)
+
+
+_payload_map = {
+ types.ErrorResponse: Struct("fields" / StringList),
+ types.ReadyForQuery: Struct("status" / Bytes(1)),
+ types.Query: Struct("query" / NullTerminated(GreedyBytes)),
+ types.EmptyQueryResponse: Terminated,
+ types.AuthnRequest: Struct("type" / authn, "body" / Default(_authn_body, b"")),
+ types.BackendKeyData: Struct("pid" / Int32ub, "key" / Hex(Int32ub)),
+ types.CommandComplete: Struct("tag" / NullTerminated(GreedyBytes)),
+ types.ParameterStatus: Struct(
+ "name" / NullTerminated(GreedyBytes), "value" / NullTerminated(GreedyBytes)
+ ),
+ types.DataRow: Struct("columns" / Default(PrefixedArray(Int16sb, _column), b"")),
+ types.Terminate: Terminated,
+}
+
+
+_payload = FocusedSeq(
+ "_payload",
+ "_payload"
+ / Switch(
+ this._.type,
+ _payload_map,
+ default=GreedyBytes,
+ ),
+ Terminated, # make sure every payload consumes the entire packet
+)
+
+
+def _payload_len(this):
+ """
+ See _startup_payload_len() for an explanation.
+ """
+ assert this._building, "_payload_len() cannot be called during parsing"
+
+ try:
+ payload = this.payload
+ except AttributeError:
+ return 0 # no payload
+
+ if isinstance(payload, bytes):
+ # already serialized; just use the given length
+ return len(payload)
+
+ data = _payload.build(payload, type=this.type)
+ return len(data)
+
+
+Pq3 = Struct(
+ "type" / types,
+ "len" / Default(Int32ub, lambda this: _payload_len(this) + 4),
+ "payload" / FixedSized(this.len - 4, Default(_payload, b"")),
+)
+
+
+# Environment
+
+
+def pghost():
+ return os.environ.get("PGHOST", default="localhost")
+
+
+def pgport():
+ return int(os.environ.get("PGPORT", default=5432))
+
+
+def pguser():
+ try:
+ return os.environ["PGUSER"]
+ except KeyError:
+ return getpass.getuser()
+
+
+def pgdatabase():
+ return os.environ.get("PGDATABASE", default="postgres")
+
+
+# Connections
+
+
+def _hexdump_translation_map():
+ """
+ For hexdumps. Translates any unprintable or non-ASCII bytes into '.'.
+ """
+ input = bytearray()
+
+ for i in range(128):
+ c = chr(i)
+
+ if not c.isprintable():
+ input += bytes([i])
+
+ input += bytes(range(128, 256))
+
+ return bytes.maketrans(input, b"." * len(input))
+
+
+class _DebugStream(object):
+ """
+ Wraps a file-like object and adds hexdumps of the read and write data. Call
+ end_packet() on a _DebugStream to write the accumulated hexdumps to the
+ output stream, along with the packet that was sent.
+ """
+
+ _translation_map = _hexdump_translation_map()
+
+ def __init__(self, stream, out=sys.stdout):
+ """
+ Creates a new _DebugStream wrapping the given stream (which must have
+ been created by wrap()). All attributes not provided by the _DebugStream
+ are delegated to the wrapped stream. out is the text stream to which
+ hexdumps are written.
+ """
+ self.raw = stream
+ self._out = out
+ self._rbuf = io.BytesIO()
+ self._wbuf = io.BytesIO()
+
+ def __getattr__(self, name):
+ return getattr(self.raw, name)
+
+ def __setattr__(self, name, value):
+ if name in ("raw", "_out", "_rbuf", "_wbuf"):
+ return object.__setattr__(self, name, value)
+
+ setattr(self.raw, name, value)
+
+ def read(self, *args, **kwargs):
+ buf = self.raw.read(*args, **kwargs)
+
+ self._rbuf.write(buf)
+ return buf
+
+ def write(self, b):
+ self._wbuf.write(b)
+ return self.raw.write(b)
+
+ def recv(self, *args):
+ buf = self.raw.recv(*args)
+
+ self._rbuf.write(buf)
+ return buf
+
+ def _flush(self, buf, prefix):
+ width = 16
+ hexwidth = width * 3 - 1
+
+ count = 0
+ buf.seek(0)
+
+ while True:
+ line = buf.read(16)
+
+ if not line:
+ if count:
+ self._out.write("\n") # separate the output block with a newline
+ return
+
+ self._out.write("%s %04X:\t" % (prefix, count))
+ self._out.write("%*s\t" % (-hexwidth, line.hex(" ")))
+ self._out.write(line.translate(self._translation_map).decode("ascii"))
+ self._out.write("\n")
+
+ count += 16
+
+ def print_debug(self, obj, *, prefix=""):
+ contents = ""
+ if obj is not None:
+ contents = str(obj)
+
+ for line in contents.splitlines():
+ self._out.write("%s%s\n" % (prefix, line))
+
+ self._out.write("\n")
+
+ def flush_debug(self, *, prefix=""):
+ self._flush(self._rbuf, prefix + "<")
+ self._rbuf = io.BytesIO()
+
+ self._flush(self._wbuf, prefix + ">")
+ self._wbuf = io.BytesIO()
+
+ def end_packet(self, pkt, *, read=False, prefix="", indent=" "):
+ """
+ Marks the end of a logical "packet" of data. A string representation of
+ pkt will be printed, and the debug buffers will be flushed with an
+ indent. All lines can be optionally prefixed.
+
+ If read is True, the packet representation is written after the debug
+ buffers; otherwise the default of False (meaning write) causes the
+ packet representation to be dumped first. This is meant to capture the
+ logical flow of layer translation.
+ """
+ write = not read
+
+ if write:
+ self.print_debug(pkt, prefix=prefix + "> ")
+
+ self.flush_debug(prefix=prefix + indent)
+
+ if read:
+ self.print_debug(pkt, prefix=prefix + "< ")
+
+
+@contextlib.contextmanager
+def wrap(socket, *, debug_stream=None):
+ """
+ Transforms a raw socket into a connection that can be used for Construct
+ building and parsing. The return value is a context manager and can be used
+ in a with statement.
+ """
+ # It is critical that buffering be disabled here, so that we can still
+ # manipulate the raw socket without desyncing the stream.
+ with socket.makefile("rwb", buffering=0) as sfile:
+ # Expose the original socket's recv() on the SocketIO object we return.
+ def recv(self, *args):
+ return socket.recv(*args)
+
+ sfile.recv = recv.__get__(sfile)
+
+ conn = sfile
+ if debug_stream:
+ conn = _DebugStream(conn, debug_stream)
+
+ try:
+ yield conn
+ finally:
+ if debug_stream:
+ conn.flush_debug(prefix="? ")
+
+
+def _send(stream, cls, obj):
+ debugging = hasattr(stream, "flush_debug")
+ out = io.BytesIO()
+
+ # Ideally we would build directly to the passed stream, but because we need
+ # to reparse the generated output for the debugging case, build to an
+ # intermediate BytesIO and send it instead.
+ cls.build_stream(obj, out)
+ buf = out.getvalue()
+
+ stream.write(buf)
+ if debugging:
+ pkt = cls.parse(buf)
+ stream.end_packet(pkt)
+
+ stream.flush()
+
+
+def send(stream, packet_type, payload_data=None, **payloadkw):
+ """
+ Sends a packet on the given pq3 connection. type is the pq3.types member
+ that should be assigned to the packet. If payload_data is given, it will be
+ used as the packet payload; otherwise the key/value pairs in payloadkw will
+ be the payload contents.
+ """
+ data = payloadkw
+
+ if payload_data is not None:
+ if payloadkw:
+ raise ValueError(
+ "payload_data and payload keywords may not be used simultaneously"
+ )
+
+ data = payload_data
+
+ _send(stream, Pq3, dict(type=packet_type, payload=data))
+
+
+def send_startup(stream, proto=None, **kwargs):
+ """
+ Sends a startup packet on the given pq3 connection. In most cases you should
+ use the handshake functions instead, which will do this for you.
+
+ By default, a protocol version 3 packet will be sent. This can be overridden
+ with the proto parameter.
+ """
+ pkt = {}
+
+ if proto is not None:
+ pkt["proto"] = proto
+ if kwargs:
+ pkt["payload"] = kwargs
+
+ _send(stream, Startup, pkt)
+
+
+def recv1(stream, *, cls=Pq3):
+ """
+ Receives a single pq3 packet from the given stream and returns it.
+ """
+ resp = cls.parse_stream(stream)
+
+ debugging = hasattr(stream, "flush_debug")
+ if debugging:
+ stream.end_packet(resp, read=True)
+
+ return resp
+
+
+def handshake(stream, **kwargs):
+ """
+ Performs a libpq v3 startup handshake. kwargs should contain the key/value
+ parameters to send to the server in the startup packet.
+ """
+ # Send our startup parameters.
+ send_startup(stream, **kwargs)
+
+ # Receive and dump packets until the server indicates it's ready for our
+ # first query.
+ while True:
+ resp = recv1(stream)
+ if resp is None:
+ raise RuntimeError("server closed connection during handshake")
+
+ if resp.type == types.ReadyForQuery:
+ return
+ elif resp.type == types.ErrorResponse:
+ raise RuntimeError(
+ f"received error response from peer: {resp.payload.fields!r}"
+ )
+
+
+# TLS
+
+
+class _TLSStream(object):
+ """
+ A file-like object that performs TLS encryption/decryption on a wrapped
+ stream. Differs from ssl.SSLSocket in that we have full visibility and
+ control over the TLS layer.
+ """
+
+ def __init__(self, stream, context):
+ self._stream = stream
+ self._debugging = hasattr(stream, "flush_debug")
+
+ self._in = ssl.MemoryBIO()
+ self._out = ssl.MemoryBIO()
+ self._ssl = context.wrap_bio(self._in, self._out)
+
+ def handshake(self):
+ try:
+ self._pump(lambda: self._ssl.do_handshake())
+ finally:
+ self._flush_debug(prefix="? ")
+
+ def read(self, *args):
+ return self._pump(lambda: self._ssl.read(*args))
+
+ def write(self, *args):
+ return self._pump(lambda: self._ssl.write(*args))
+
+ def _decode(self, buf):
+ """
+ Attempts to decode a buffer of TLS data into a packet representation
+ that can be printed.
+
+ TODO: handle buffers (and record fragments) that don't align with packet
+ boundaries.
+ """
+ end = len(buf)
+ bio = io.BytesIO(buf)
+
+ ret = io.StringIO()
+
+ while bio.tell() < end:
+ record = tls.Plaintext.parse_stream(bio)
+
+ if ret.tell() > 0:
+ ret.write("\n")
+ ret.write("[Record] ")
+ ret.write(str(record))
+ ret.write("\n")
+
+ if record.type == tls.ContentType.handshake:
+ record_cls = tls.Handshake
+ else:
+ continue
+
+ innerlen = len(record.fragment)
+ inner = io.BytesIO(record.fragment)
+
+ while inner.tell() < innerlen:
+ msg = record_cls.parse_stream(inner)
+
+ indented = "[Message] " + str(msg)
+ indented = textwrap.indent(indented, " ")
+
+ ret.write("\n")
+ ret.write(indented)
+ ret.write("\n")
+
+ return ret.getvalue()
+
+ def flush(self):
+ if not self._out.pending:
+ self._stream.flush()
+ return
+
+ buf = self._out.read()
+ self._stream.write(buf)
+
+ if self._debugging:
+ pkt = self._decode(buf)
+ self._stream.end_packet(pkt, prefix=" ")
+
+ self._stream.flush()
+
+ def _pump(self, operation):
+ while True:
+ try:
+ return operation()
+ except (ssl.SSLWantReadError, ssl.SSLWantWriteError) as e:
+ want = e
+ self._read_write(want)
+
+ def _recv(self, maxsize):
+ buf = self._stream.recv(4096)
+ if not buf:
+ self._in.write_eof()
+ return
+
+ self._in.write(buf)
+
+ if not self._debugging:
+ return
+
+ pkt = self._decode(buf)
+ self._stream.end_packet(pkt, read=True, prefix=" ")
+
+ def _read_write(self, want):
+ # XXX This needs work. So many corner cases yet to handle. For one,
+ # doing blocking writes in flush may lead to distributed deadlock if the
+ # peer is already blocking on its writes.
+
+ if isinstance(want, ssl.SSLWantWriteError):
+ assert self._out.pending, "SSL backend wants write without data"
+
+ self.flush()
+
+ if isinstance(want, ssl.SSLWantReadError):
+ self._recv(4096)
+
+ def _flush_debug(self, prefix):
+ if not self._debugging:
+ return
+
+ self._stream.flush_debug(prefix=prefix)
+
+
+@contextlib.contextmanager
+def tls_handshake(stream, context):
+ """
+ Performs a TLS handshake over the given stream (which must have been created
+ via a call to wrap()), and returns a new stream which transparently tunnels
+ data over the TLS connection.
+
+ If the passed stream has debugging enabled, the returned stream will also
+ have debugging, using the same output IO.
+ """
+ debugging = hasattr(stream, "flush_debug")
+
+ # Send our startup parameters.
+ send_startup(stream, proto=protocol(1234, 5679))
+
+ # Look at the SSL response.
+ resp = stream.read(1)
+ if debugging:
+ stream.flush_debug(prefix=" ")
+
+ if resp == b"N":
+ raise RuntimeError("server does not support SSLRequest")
+ if resp != b"S":
+ raise RuntimeError(f"unexpected response of type {resp!r} during TLS startup")
+
+ tls = _TLSStream(stream, context)
+ tls.handshake()
+
+ if debugging:
+ tls = _DebugStream(tls, stream._out)
+
+ try:
+ yield tls
+ # TODO: teardown/unwrap the connection?
+ finally:
+ if debugging:
+ tls.flush_debug(prefix="? ")
diff --git a/src/test/python/pytest.ini b/src/test/python/pytest.ini
new file mode 100644
index 0000000000..ab7a6e7fb9
--- /dev/null
+++ b/src/test/python/pytest.ini
@@ -0,0 +1,4 @@
+[pytest]
+
+markers =
+ slow: mark test as slow
diff --git a/src/test/python/requirements.txt b/src/test/python/requirements.txt
new file mode 100644
index 0000000000..32f105ea84
--- /dev/null
+++ b/src/test/python/requirements.txt
@@ -0,0 +1,7 @@
+black
+cryptography~=3.4.6
+construct~=2.10.61
+isort~=5.6
+psycopg2~=2.8.6
+pytest~=6.1
+pytest-asyncio~=0.14.0
diff --git a/src/test/python/server/__init__.py b/src/test/python/server/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/src/test/python/server/conftest.py b/src/test/python/server/conftest.py
new file mode 100644
index 0000000000..ba7342a453
--- /dev/null
+++ b/src/test/python/server/conftest.py
@@ -0,0 +1,45 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import socket
+import sys
+
+import pytest
+
+import pq3
+
+
+@pytest.fixture
+def connect():
+ """
+ A factory fixture that, when called, returns a socket connected to a
+ Postgres server, wrapped in a pq3 connection. The calling test will be
+ skipped automatically if a server is not running at PGHOST:PGPORT, so it's
+ best to connect as soon as possible after the test case begins, to avoid
+ doing unnecessary work.
+ """
+ # Set up an ExitStack to handle safe cleanup of all of the moving pieces.
+ with contextlib.ExitStack() as stack:
+
+ def conn_factory():
+ addr = (pq3.pghost(), pq3.pgport())
+
+ try:
+ sock = socket.create_connection(addr, timeout=2)
+ except ConnectionError as e:
+ pytest.skip(f"unable to connect to {addr}: {e}")
+
+ # Have ExitStack close our socket.
+ stack.enter_context(sock)
+
+ # Wrap the connection in a pq3 layer and have ExitStack clean it up
+ # too.
+ wrap_ctx = pq3.wrap(sock, debug_stream=sys.stdout)
+ conn = stack.enter_context(wrap_ctx)
+
+ return conn
+
+ yield conn_factory
diff --git a/src/test/python/server/test_oauth.py b/src/test/python/server/test_oauth.py
new file mode 100644
index 0000000000..cb5ca7fa23
--- /dev/null
+++ b/src/test/python/server/test_oauth.py
@@ -0,0 +1,1012 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import contextlib
+import json
+import os
+import pathlib
+import secrets
+import shlex
+import shutil
+import socket
+import struct
+from multiprocessing import shared_memory
+
+import psycopg2
+import pytest
+from psycopg2 import sql
+
+import pq3
+
+MAX_SASL_MESSAGE_LENGTH = 65535
+
+INVALID_AUTHORIZATION_ERRCODE = b"28000"
+PROTOCOL_VIOLATION_ERRCODE = b"08P01"
+FEATURE_NOT_SUPPORTED_ERRCODE = b"0A000"
+
+SHARED_MEM_NAME = "oauth-pytest"
+MAX_TOKEN_SIZE = 4096
+MAX_UINT16 = 2 ** 16 - 1
+
+
+def skip_if_no_postgres():
+ """
+ Used by the oauth_ctx fixture to skip this test module if no Postgres server
+ is running.
+
+ This logic is nearly duplicated with the conn fixture. Ideally oauth_ctx
+ would depend on that, but a module-scope fixture can't depend on a
+ test-scope fixture, and we haven't reached the rule of three yet.
+ """
+ addr = (pq3.pghost(), pq3.pgport())
+
+ try:
+ with socket.create_connection(addr, timeout=2):
+ pass
+ except ConnectionError as e:
+ pytest.skip(f"unable to connect to {addr}: {e}")
+
+
+@contextlib.contextmanager
+def prepend_file(path, lines):
+ """
+ A context manager that prepends a file on disk with the desired lines of
+ text. When the context manager is exited, the file will be restored to its
+ original contents.
+ """
+ # First make a backup of the original file.
+ bak = path + ".bak"
+ shutil.copy2(path, bak)
+
+ try:
+ # Write the new lines, followed by the original file content.
+ with open(path, "w") as new, open(bak, "r") as orig:
+ new.writelines(lines)
+ shutil.copyfileobj(orig, new)
+
+ # Return control to the calling code.
+ yield
+
+ finally:
+ # Put the backup back into place.
+ os.replace(bak, path)
+
+
+@pytest.fixture(scope="module")
+def oauth_ctx():
+ """
+ Creates a database and user that use the oauth auth method. The context
+ object contains the dbname and user attributes as strings to be used during
+ connection, as well as the issuer and scope that have been set in the HBA
+ configuration.
+
+ This fixture assumes that the standard PG* environment variables point to a
+ server running on a local machine, and that the PGUSER has rights to create
+ databases and roles.
+ """
+ skip_if_no_postgres() # don't bother running these tests without a server
+
+ id = secrets.token_hex(4)
+
+ class Context:
+ dbname = "oauth_test_" + id
+
+ user = "oauth_user_" + id
+ map_user = "oauth_map_user_" + id
+ authz_user = "oauth_authz_user_" + id
+
+ issuer = "https://example.com/" + id
+ scope = "openid " + id
+
+ ctx = Context()
+ hba_lines = (
+ f'host {ctx.dbname} {ctx.map_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" map=oauth\n',
+ f'host {ctx.dbname} {ctx.authz_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" trust_validator_authz=1\n',
+ f'host {ctx.dbname} all samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}"\n',
+ )
+ ident_lines = (r"oauth /^(.*)@example\.com$ \1",)
+
+ conn = psycopg2.connect("")
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+
+ # Create our roles and database.
+ user = sql.Identifier(ctx.user)
+ map_user = sql.Identifier(ctx.map_user)
+ authz_user = sql.Identifier(ctx.authz_user)
+ dbname = sql.Identifier(ctx.dbname)
+
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(map_user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(authz_user))
+ c.execute(sql.SQL("CREATE DATABASE {};").format(dbname))
+
+ # Make this test script the server's oauth_validator.
+ path = pathlib.Path(__file__).parent / "validate_bearer.py"
+ path = str(path.absolute())
+
+ cmd = f"{shlex.quote(path)} {SHARED_MEM_NAME} <&%f"
+ c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (cmd,))
+
+ # Replace pg_hba and pg_ident.
+ c.execute("SHOW hba_file;")
+ hba = c.fetchone()[0]
+
+ c.execute("SHOW ident_file;")
+ ident = c.fetchone()[0]
+
+ with prepend_file(hba, hba_lines), prepend_file(ident, ident_lines):
+ c.execute("SELECT pg_reload_conf();")
+
+ # Use the new database and user.
+ yield ctx
+
+ # Put things back the way they were.
+ c.execute("SELECT pg_reload_conf();")
+
+ c.execute("ALTER SYSTEM RESET oauth_validator_command;")
+ c.execute(sql.SQL("DROP DATABASE {};").format(dbname))
+ c.execute(sql.SQL("DROP ROLE {};").format(authz_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(map_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(user))
+
+
+@pytest.fixture()
+def conn(oauth_ctx, connect):
+ """
+ A convenience wrapper for connect(). The main purpose of this fixture is to
+ make sure oauth_ctx runs its setup code before the connection is made.
+ """
+ return connect()
+
+
+@pytest.fixture(scope="module", autouse=True)
+def authn_id_extension(oauth_ctx):
+ """
+ Performs a `CREATE EXTENSION authn_id` in the test database. This fixture is
+ autoused, so tests don't need to rely on it.
+ """
+ conn = psycopg2.connect(database=oauth_ctx.dbname)
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+ c.execute("CREATE EXTENSION authn_id;")
+
+
+@pytest.fixture(scope="session")
+def shared_mem():
+ """
+ Yields a shared memory segment that can be used for communication between
+ the bearer_token fixture and ./validate_bearer.py.
+ """
+ size = MAX_TOKEN_SIZE + 2 # two byte length prefix
+ mem = shared_memory.SharedMemory(SHARED_MEM_NAME, create=True, size=size)
+
+ try:
+ with contextlib.closing(mem):
+ yield mem
+ finally:
+ mem.unlink()
+
+
+@pytest.fixture()
+def bearer_token(shared_mem):
+ """
+ Returns a factory function that, when called, will store a Bearer token in
+ shared_mem. If token is None (the default), a new token will be generated
+ using secrets.token_urlsafe() and returned; otherwise the passed token will
+ be used as-is.
+
+ When token is None, the generated token size in bytes may be specified as an
+ argument; if unset, a small 16-byte token will be generated. The token size
+ may not exceed MAX_TOKEN_SIZE in any case.
+
+ The return value is the token, converted to a bytes object.
+
+ As a special case for testing failure modes, accept_any may be set to True.
+ This signals to the validator command that any bearer token should be
+ accepted. The returned token in this case may be used or discarded as needed
+ by the test.
+ """
+
+ def set_token(token=None, *, size=16, accept_any=False):
+ if token is not None:
+ size = len(token)
+
+ if size > MAX_TOKEN_SIZE:
+ raise ValueError(f"token size {size} exceeds maximum size {MAX_TOKEN_SIZE}")
+
+ if token is None:
+ if size % 4:
+ raise ValueError(f"requested token size {size} is not a multiple of 4")
+
+ token = secrets.token_urlsafe(size // 4 * 3)
+ assert len(token) == size
+
+ try:
+ token = token.encode("ascii")
+ except AttributeError:
+ pass # already encoded
+
+ if accept_any:
+ # Two-byte magic value.
+ shared_mem.buf[:2] = struct.pack("H", MAX_UINT16)
+ else:
+ # Two-byte length prefix, then the token data.
+ shared_mem.buf[:2] = struct.pack("H", len(token))
+ shared_mem.buf[2 : size + 2] = token
+
+ return token
+
+ return set_token
+
+
+def begin_oauth_handshake(conn, oauth_ctx, *, user=None):
+ if user is None:
+ user = oauth_ctx.authz_user
+
+ pq3.send_startup(conn, user=user, database=oauth_ctx.dbname)
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.AuthnRequest
+
+ # The server should advertise exactly one mechanism.
+ assert resp.payload.type == pq3.authn.SASL
+ assert resp.payload.body == [b"OAUTHBEARER", b""]
+
+
+def send_initial_response(conn, *, auth=None, bearer=None):
+ """
+ Sends the OAUTHBEARER initial response on the connection, using the given
+ bearer token. Alternatively to a bearer token, the initial response's auth
+ field may be explicitly specified to test corner cases.
+ """
+ if bearer is not None and auth is not None:
+ raise ValueError("exactly one of the auth and bearer kwargs must be set")
+
+ if bearer is not None:
+ auth = b"Bearer " + bearer
+
+ if auth is None:
+ raise ValueError("exactly one of the auth and bearer kwargs must be set")
+
+ initial = pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"n,,\x01auth=" + auth + b"\x01\x01",
+ )
+ )
+ pq3.send(conn, pq3.types.PasswordMessage, initial)
+
+
+def expect_handshake_success(conn):
+ """
+ Validates that the server responds with an AuthnOK message, and then drains
+ the connection until a ReadyForQuery message is received.
+ """
+ resp = pq3.recv1(conn)
+
+ assert resp.type == pq3.types.AuthnRequest
+ assert resp.payload.type == pq3.authn.OK
+ assert not resp.payload.body
+
+ receive_until(conn, pq3.types.ReadyForQuery)
+
+
+def expect_handshake_failure(conn, oauth_ctx):
+ """
+ Performs the OAUTHBEARER SASL failure "handshake" and validates the server's
+ side of the conversation, including the final ErrorResponse.
+ """
+
+ # We expect a discovery "challenge" back from the server before the authn
+ # failure message.
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.AuthnRequest
+
+ req = resp.payload
+ assert req.type == pq3.authn.SASLContinue
+
+ body = json.loads(req.body)
+ assert body["status"] == "invalid_token"
+ assert body["scope"] == oauth_ctx.scope
+
+ expected_config = oauth_ctx.issuer + "/.well-known/openid-configuration"
+ assert body["openid-configuration"] == expected_config
+
+ # Send the dummy response to complete the failed handshake.
+ pq3.send(conn, pq3.types.PasswordMessage, b"\x01")
+ resp = pq3.recv1(conn)
+
+ err = ExpectedError(INVALID_AUTHORIZATION_ERRCODE, "bearer authentication failed")
+ err.match(resp)
+
+
+def receive_until(conn, type):
+ """
+ receive_until pulls packets off the pq3 connection until a packet with the
+ desired type is found, or an error response is received.
+ """
+ while True:
+ pkt = pq3.recv1(conn)
+
+ if pkt.type == type:
+ return pkt
+ elif pkt.type == pq3.types.ErrorResponse:
+ raise RuntimeError(
+ f"received error response from peer: {pkt.payload.fields!r}"
+ )
+
+
+@pytest.mark.parametrize("token_len", [16, 1024, 4096])
+@pytest.mark.parametrize(
+ "auth_prefix",
+ [
+ b"Bearer ",
+ b"bearer ",
+ b"Bearer ",
+ ],
+)
+def test_oauth(conn, oauth_ctx, bearer_token, auth_prefix, token_len):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Generate our bearer token with the desired length.
+ token = bearer_token(size=token_len)
+ auth = auth_prefix + token
+
+ send_initial_response(conn, auth=auth)
+ expect_handshake_success(conn)
+
+ # Make sure that the server has not set an authenticated ID.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT authn_id();")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ assert row.columns == [None]
+
+
+@pytest.mark.parametrize(
+ "token_value",
+ [
+ "abcdzA==",
+ "123456M=",
+ "x-._~+/x",
+ ],
+)
+def test_oauth_bearer_corner_cases(conn, oauth_ctx, bearer_token, token_value):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ send_initial_response(conn, bearer=bearer_token(token_value))
+
+ expect_handshake_success(conn)
+
+
+@pytest.mark.parametrize(
+ "user,authn_id,should_succeed",
+ [
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: ctx.user,
+ True,
+ id="validator authn: succeeds when authn_id == username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: None,
+ False,
+ id="validator authn: fails when authn_id is not set",
+ ),
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: ctx.authz_user,
+ False,
+ id="validator authn: fails when authn_id != username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: ctx.map_user + "@example.com",
+ True,
+ id="validator with map: succeeds when authn_id matches map",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: None,
+ False,
+ id="validator with map: fails when authn_id is not set",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: ctx.map_user + "@example.net",
+ False,
+ id="validator with map: fails when authn_id doesn't match map",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: None,
+ True,
+ id="validator authz: succeeds with no authn_id",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "",
+ True,
+ id="validator authz: succeeds with empty authn_id",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "postgres",
+ True,
+ id="validator authz: succeeds with basic username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "me@example.com",
+ True,
+ id="validator authz: succeeds with email address",
+ ),
+ ],
+)
+def test_oauth_authn_id(conn, oauth_ctx, bearer_token, user, authn_id, should_succeed):
+ token = None
+
+ authn_id = authn_id(oauth_ctx)
+ if authn_id is not None:
+ authn_id = authn_id.encode("ascii")
+
+ # As a hack to get the validator to reflect arbitrary output from this
+ # test, encode the desired output as a base64 token. The validator will
+ # key on the leading "output=" to differentiate this from the random
+ # tokens generated by secrets.token_urlsafe().
+ output = b"output=" + authn_id + b"\n"
+ token = base64.urlsafe_b64encode(output)
+
+ token = bearer_token(token)
+ username = user(oauth_ctx)
+
+ begin_oauth_handshake(conn, oauth_ctx, user=username)
+ send_initial_response(conn, bearer=token)
+
+ if not should_succeed:
+ expect_handshake_failure(conn, oauth_ctx)
+ return
+
+ expect_handshake_success(conn)
+
+ # Check the reported authn_id.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT authn_id();")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ assert row.columns == [authn_id]
+
+
+class ExpectedError(object):
+ def __init__(self, code, msg=None, detail=None):
+ self.code = code
+ self.msg = msg
+ self.detail = detail
+
+ # Protect against the footgun of an accidental empty string, which will
+ # "match" anything. If you don't want to match message or detail, just
+ # don't pass them.
+ if self.msg == "":
+ raise ValueError("msg must be non-empty or None")
+ if self.detail == "":
+ raise ValueError("detail must be non-empty or None")
+
+ def _getfield(self, resp, type):
+ """
+ Searches an ErrorResponse for a single field of the given type (e.g.
+ "M", "C", "D") and returns its value. Asserts if it doesn't find exactly
+ one field.
+ """
+ prefix = type.encode("ascii")
+ fields = [f for f in resp.payload.fields if f.startswith(prefix)]
+
+ assert len(fields) == 1
+ return fields[0][1:] # strip off the type byte
+
+ def match(self, resp):
+ """
+ Checks that the given response matches the expected code, message, and
+ detail (if given). The error code must match exactly. The expected
+ message and detail must be contained within the actual strings.
+ """
+ assert resp.type == pq3.types.ErrorResponse
+
+ code = self._getfield(resp, "C")
+ assert code == self.code
+
+ if self.msg:
+ msg = self._getfield(resp, "M")
+ expected = self.msg.encode("utf-8")
+ assert expected in msg
+
+ if self.detail:
+ detail = self._getfield(resp, "D")
+ expected = self.detail.encode("utf-8")
+ assert expected in detail
+
+
+def test_oauth_rejected_bearer(conn, oauth_ctx, bearer_token):
+ # Generate a new bearer token, which we will proceed not to use.
+ _ = bearer_token()
+
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send a bearer token that doesn't match what the validator expects. It
+ # should fail the connection.
+ send_initial_response(conn, bearer=b"xxxxxx")
+
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.parametrize(
+ "bad_bearer",
+ [
+ b"Bearer ",
+ b"Bearer a===b",
+ b"Bearer hello!",
+ b"Bearer me@example.com",
+ b'OAuth realm="Example"',
+ b"",
+ ],
+)
+def test_oauth_invalid_bearer(conn, oauth_ctx, bearer_token, bad_bearer):
+ # Tell the validator to accept any token. This ensures that the invalid
+ # bearer tokens are rejected before the validation step.
+ _ = bearer_token(accept_any=True)
+
+ begin_oauth_handshake(conn, oauth_ctx)
+ send_initial_response(conn, auth=bad_bearer)
+
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.slow
+@pytest.mark.parametrize(
+ "resp_type,resp,err",
+ [
+ pytest.param(
+ None,
+ None,
+ None,
+ marks=pytest.mark.slow,
+ id="no response (expect timeout)",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"hello",
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not send a kvsep response",
+ ),
+ id="bad dummy response",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"\x01\x01",
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not send a kvsep response",
+ ),
+ id="multiple kvseps",
+ ),
+ pytest.param(
+ pq3.types.Query,
+ dict(query=b""),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "expected SASL response"),
+ id="bad response message type",
+ ),
+ ],
+)
+def test_oauth_bad_response_to_error_challenge(conn, oauth_ctx, resp_type, resp, err):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send an empty auth initial response, which will force an authn failure.
+ send_initial_response(conn, auth=b"")
+
+ # We expect a discovery "challenge" back from the server before the authn
+ # failure message.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.AuthnRequest
+
+ req = pkt.payload
+ assert req.type == pq3.authn.SASLContinue
+
+ body = json.loads(req.body)
+ assert body["status"] == "invalid_token"
+
+ if resp_type is None:
+ # Do not send the dummy response. We should time out and not get a
+ # response from the server.
+ with pytest.raises(socket.timeout):
+ conn.read(1)
+
+ # Done with the test.
+ return
+
+ # Send the bad response.
+ pq3.send(conn, resp_type, resp)
+
+ # Make sure the server fails the connection correctly.
+ pkt = pq3.recv1(conn)
+ err.match(pkt)
+
+
+@pytest.mark.parametrize(
+ "type,payload,err",
+ [
+ pytest.param(
+ pq3.types.ErrorResponse,
+ dict(fields=[b""]),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "expected SASL response"),
+ id="error response in initial message",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"x" * (MAX_SASL_MESSAGE_LENGTH + 1),
+ ExpectedError(
+ INVALID_AUTHORIZATION_ERRCODE, "bearer authentication failed"
+ ),
+ id="overlong initial response data",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"SCRAM-SHA-256")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE, "invalid SASL authentication mechanism"
+ ),
+ id="bad SASL mechanism selection",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", len=2, data=b"x")),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "insufficient data"),
+ id="SASL data underflow",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", len=0, data=b"x")),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "invalid message format"),
+ id="SASL data overflow",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "message is empty",
+ ),
+ id="empty",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"n,,\x01auth=\x01\x01\0")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "length does not match input length",
+ ),
+ id="contains null byte",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected channel-binding flag", # XXX this is a bit strange
+ ),
+ id="initial error response",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"p=tls-server-end-point,,\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "server does not support channel binding",
+ ),
+ id="uses channel binding",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"x,,\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected channel-binding flag",
+ ),
+ id="invalid channel binding specifier",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Comma expected",
+ ),
+ id="bad GS2 header: missing channel binding terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,a")),
+ ExpectedError(
+ FEATURE_NOT_SUPPORTED_ERRCODE,
+ "client uses authorization identity",
+ ),
+ id="bad GS2 header: authzid in use",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,b,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected attribute",
+ ),
+ id="bad GS2 header: extra attribute",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected attribute 0x00", # XXX this is a bit strange
+ ),
+ id="bad GS2 header: missing authzid terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Key-value separator expected",
+ ),
+ id="missing initial kvsep",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Key-value separator expected",
+ ),
+ id="missing initial kvsep",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "does not contain an auth value",
+ ),
+ id="missing auth value: empty key-value list",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01host=example.com\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "does not contain an auth value",
+ ),
+ id="missing auth value: other keys present",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01host=example.com")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "unterminated key/value pair",
+ ),
+ id="missing value terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not contain a final terminator",
+ ),
+ id="missing list terminator: empty list",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01auth=Bearer 0\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not contain a final terminator",
+ ),
+ id="missing list terminator: with auth value",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01auth=Bearer 0\x01\x01blah")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "additional data after the final terminator",
+ ),
+ id="additional key after terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01key\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "key without a value",
+ ),
+ id="key without value",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"y,,\x01auth=Bearer 0\x01auth=Bearer 1\x01\x01",
+ )
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "contains multiple auth values",
+ ),
+ id="multiple auth values",
+ ),
+ ],
+)
+def test_oauth_bad_initial_response(conn, oauth_ctx, type, payload, err):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # The server expects a SASL response; give it something else instead.
+ if not isinstance(payload, dict):
+ payload = dict(payload_data=payload)
+ pq3.send(conn, type, **payload)
+
+ resp = pq3.recv1(conn)
+ err.match(resp)
+
+
+def test_oauth_empty_initial_response(conn, oauth_ctx, bearer_token):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send an initial response without data.
+ initial = pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER"))
+ pq3.send(conn, pq3.types.PasswordMessage, initial)
+
+ # The server should respond with an empty challenge so we can send the data
+ # it wants.
+ pkt = pq3.recv1(conn)
+
+ assert pkt.type == pq3.types.AuthnRequest
+ assert pkt.payload.type == pq3.authn.SASLContinue
+ assert not pkt.payload.body
+
+ # Now send the initial data.
+ data = b"n,,\x01auth=Bearer " + bearer_token() + b"\x01\x01"
+ pq3.send(conn, pq3.types.PasswordMessage, data)
+
+ # Server should now complete the handshake.
+ expect_handshake_success(conn)
+
+
+@pytest.fixture()
+def set_validator():
+ """
+ A per-test fixture that allows a test to override the setting of
+ oauth_validator_command for the cluster. The setting will be reverted during
+ teardown.
+
+ Passing None will perform an ALTER SYSTEM RESET.
+ """
+ conn = psycopg2.connect("")
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+
+ # Save the previous value.
+ c.execute("SHOW oauth_validator_command;")
+ prev_cmd = c.fetchone()[0]
+
+ def setter(cmd):
+ c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (cmd,))
+ c.execute("SELECT pg_reload_conf();")
+
+ yield setter
+
+ # Restore the previous value.
+ c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (prev_cmd,))
+ c.execute("SELECT pg_reload_conf();")
+
+
+def test_oauth_no_validator(oauth_ctx, set_validator, connect, bearer_token):
+ # Clear out our validator command, then establish a new connection.
+ set_validator("")
+ conn = connect()
+
+ begin_oauth_handshake(conn, oauth_ctx)
+ send_initial_response(conn, bearer=bearer_token())
+
+ # The server should fail the connection.
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+def test_oauth_validator_role(oauth_ctx, set_validator, connect):
+ # Switch the validator implementation. This validator will reflect the
+ # PGUSER as the authenticated identity.
+ path = pathlib.Path(__file__).parent / "validate_reflect.py"
+ path = str(path.absolute())
+
+ set_validator(f"{shlex.quote(path)} '%r' <&%f")
+ conn = connect()
+
+ # Log in. Note that the reflection validator ignores the bearer token.
+ begin_oauth_handshake(conn, oauth_ctx, user=oauth_ctx.user)
+ send_initial_response(conn, bearer=b"dontcare")
+ expect_handshake_success(conn)
+
+ # Check the user identity.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT authn_id();")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ expected = oauth_ctx.user.encode("utf-8")
+ assert row.columns == [expected]
+
+
+def test_oauth_role_with_shell_unsafe_characters(oauth_ctx, set_validator, connect):
+ """
+ XXX This test pins undesirable behavior. We should be able to handle any
+ valid Postgres role name.
+ """
+ # Switch the validator implementation. This validator will reflect the
+ # PGUSER as the authenticated identity.
+ path = pathlib.Path(__file__).parent / "validate_reflect.py"
+ path = str(path.absolute())
+
+ set_validator(f"{shlex.quote(path)} '%r' <&%f")
+ conn = connect()
+
+ unsafe_username = "hello'there"
+ begin_oauth_handshake(conn, oauth_ctx, user=unsafe_username)
+
+ # The server should reject the handshake.
+ send_initial_response(conn, bearer=b"dontcare")
+ expect_handshake_failure(conn, oauth_ctx)
diff --git a/src/test/python/server/test_server.py b/src/test/python/server/test_server.py
new file mode 100644
index 0000000000..02126dba79
--- /dev/null
+++ b/src/test/python/server/test_server.py
@@ -0,0 +1,21 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import pq3
+
+
+def test_handshake(connect):
+ """Basic sanity check."""
+ conn = connect()
+
+ pq3.handshake(conn, user=pq3.pguser(), database=pq3.pgdatabase())
+
+ pq3.send(conn, pq3.types.Query, query=b"")
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.EmptyQueryResponse
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.ReadyForQuery
diff --git a/src/test/python/server/validate_bearer.py b/src/test/python/server/validate_bearer.py
new file mode 100755
index 0000000000..2cc73ff154
--- /dev/null
+++ b/src/test/python/server/validate_bearer.py
@@ -0,0 +1,101 @@
+#! /usr/bin/env python3
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+# DO NOT USE THIS OAUTH VALIDATOR IN PRODUCTION. It doesn't actually validate
+# anything, and it logs the bearer token data, which is sensitive.
+#
+# This executable is used as an oauth_validator_command in concert with
+# test_oauth.py. Memory is shared and communicated from that test module's
+# bearer_token() fixture.
+#
+# This script must run under the Postgres server environment; keep the
+# dependency list fairly standard.
+
+import base64
+import binascii
+import contextlib
+import struct
+import sys
+from multiprocessing import shared_memory
+
+MAX_UINT16 = 2 ** 16 - 1
+
+
+def remove_shm_from_resource_tracker():
+ """
+ Monkey-patch multiprocessing.resource_tracker so SharedMemory won't be
+ tracked. Pulled from this thread, where there are more details:
+
+ https://bugs.python.org/issue38119
+
+ TL;DR: all clients of shared memory segments automatically destroy them on
+ process exit, which makes shared memory segments much less useful. This
+ monkeypatch removes that behavior so that we can defer to the test to manage
+ the segment lifetime.
+
+ Ideally a future Python patch will pull in this fix and then the entire
+ function can go away.
+ """
+ from multiprocessing import resource_tracker
+
+ def fix_register(name, rtype):
+ if rtype == "shared_memory":
+ return
+ return resource_tracker._resource_tracker.register(self, name, rtype)
+
+ resource_tracker.register = fix_register
+
+ def fix_unregister(name, rtype):
+ if rtype == "shared_memory":
+ return
+ return resource_tracker._resource_tracker.unregister(self, name, rtype)
+
+ resource_tracker.unregister = fix_unregister
+
+ if "shared_memory" in resource_tracker._CLEANUP_FUNCS:
+ del resource_tracker._CLEANUP_FUNCS["shared_memory"]
+
+
+def main(args):
+ remove_shm_from_resource_tracker() # XXX remove some day
+
+ # Get the expected token from the currently running test.
+ shared_mem_name = args[0]
+
+ mem = shared_memory.SharedMemory(shared_mem_name)
+ with contextlib.closing(mem):
+ # First two bytes are the token length.
+ size = struct.unpack("H", mem.buf[:2])[0]
+
+ if size == MAX_UINT16:
+ # Special case: the test wants us to accept any token.
+ sys.stderr.write("accepting token without validation\n")
+ return
+
+ # The remainder of the buffer contains the expected token.
+ assert size <= (mem.size - 2)
+ expected_token = mem.buf[2 : size + 2].tobytes()
+
+ mem.buf[:] = b"\0" * mem.size # scribble over the token
+
+ token = sys.stdin.buffer.read()
+ if token != expected_token:
+ sys.exit(f"failed to match Bearer token ({token!r} != {expected_token!r})")
+
+ # See if the test wants us to print anything. If so, it will have encoded
+ # the desired output in the token with an "output=" prefix.
+ try:
+ # altchars="-_" corresponds to the urlsafe alphabet.
+ data = base64.b64decode(token, altchars="-_", validate=True)
+
+ if data.startswith(b"output="):
+ sys.stdout.buffer.write(data[7:])
+
+ except binascii.Error:
+ pass
+
+
+if __name__ == "__main__":
+ main(sys.argv[1:])
diff --git a/src/test/python/server/validate_reflect.py b/src/test/python/server/validate_reflect.py
new file mode 100755
index 0000000000..24c3a7e715
--- /dev/null
+++ b/src/test/python/server/validate_reflect.py
@@ -0,0 +1,34 @@
+#! /usr/bin/env python3
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+# DO NOT USE THIS OAUTH VALIDATOR IN PRODUCTION. It ignores the bearer token
+# entirely and automatically logs the user in.
+#
+# This executable is used as an oauth_validator_command in concert with
+# test_oauth.py. It expects the user's desired role name as an argument; the
+# actual token will be discarded and the user will be logged in with the role
+# name as the authenticated identity.
+#
+# This script must run under the Postgres server environment; keep the
+# dependency list fairly standard.
+
+import sys
+
+
+def main(args):
+ # We have to read the entire token as our first action to unblock the
+ # server, but we won't actually use it.
+ _ = sys.stdin.buffer.read()
+
+ if len(args) != 1:
+ sys.exit("usage: ./validate_reflect.py ROLE")
+
+ # Log the user in as the provided role.
+ role = args[0]
+ print(role)
+
+
+if __name__ == "__main__":
+ main(sys.argv[1:])
diff --git a/src/test/python/test_internals.py b/src/test/python/test_internals.py
new file mode 100644
index 0000000000..dee4855fc0
--- /dev/null
+++ b/src/test/python/test_internals.py
@@ -0,0 +1,138 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import io
+
+from pq3 import _DebugStream
+
+
+def test_DebugStream_read():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+
+ stream = _DebugStream(under, out)
+
+ res = stream.read(5)
+ assert res == b"abcde"
+
+ res = stream.read(16)
+ assert res == b"fghijklmnopqrstu"
+
+ stream.flush_debug()
+
+ res = stream.read()
+ assert res == b"vwxyz"
+
+ stream.flush_debug()
+
+ expected = (
+ "< 0000:\t61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 70\tabcdefghijklmnop\n"
+ "< 0010:\t71 72 73 74 75 \tqrstu\n"
+ "\n"
+ "< 0000:\t76 77 78 79 7a \tvwxyz\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_write():
+ under = io.BytesIO()
+ out = io.StringIO()
+
+ stream = _DebugStream(under, out)
+
+ stream.write(b"\x00\x01\x02")
+ stream.flush()
+
+ assert under.getvalue() == b"\x00\x01\x02"
+
+ stream.write(b"\xc0\xc1\xc2")
+ stream.flush()
+
+ assert under.getvalue() == b"\x00\x01\x02\xc0\xc1\xc2"
+
+ stream.flush_debug()
+
+ expected = "> 0000:\t00 01 02 c0 c1 c2 \t......\n\n"
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_read_write():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+ stream = _DebugStream(under, out)
+
+ res = stream.read(5)
+ assert res == b"abcde"
+
+ stream.write(b"xxxxx")
+ stream.flush()
+
+ assert under.getvalue() == b"abcdexxxxxklmnopqrstuvwxyz"
+
+ res = stream.read(5)
+ assert res == b"klmno"
+
+ stream.write(b"xxxxx")
+ stream.flush()
+
+ assert under.getvalue() == b"abcdexxxxxklmnoxxxxxuvwxyz"
+
+ stream.flush_debug()
+
+ expected = (
+ "< 0000:\t61 62 63 64 65 6b 6c 6d 6e 6f \tabcdeklmno\n"
+ "\n"
+ "> 0000:\t78 78 78 78 78 78 78 78 78 78 \txxxxxxxxxx\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_end_packet():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+ stream = _DebugStream(under, out)
+
+ stream.read(5)
+ stream.end_packet("read description", read=True, indent=" ")
+
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("write description", indent=" ")
+
+ stream.read(5)
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("read/write combo for read", read=True, indent=" ")
+
+ stream.read(5)
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("read/write combo for write", indent=" ")
+
+ expected = (
+ " < 0000:\t61 62 63 64 65 \tabcde\n"
+ "\n"
+ "< read description\n"
+ "\n"
+ "> write description\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ " < 0000:\t6b 6c 6d 6e 6f \tklmno\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ "< read/write combo for read\n"
+ "\n"
+ "> read/write combo for write\n"
+ "\n"
+ " < 0000:\t75 76 77 78 79 \tuvwxy\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
diff --git a/src/test/python/test_pq3.py b/src/test/python/test_pq3.py
new file mode 100644
index 0000000000..e0c0e0568d
--- /dev/null
+++ b/src/test/python/test_pq3.py
@@ -0,0 +1,558 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import getpass
+import io
+import struct
+import sys
+
+import pytest
+from construct import Container, PaddingError, StreamError, TerminatedError
+
+import pq3
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"\x00\x00\x00\x10\x00\x04\x00\x00abcdefgh",
+ Container(len=16, proto=0x40000, payload=b"abcdefgh"),
+ b"",
+ id="8-byte payload",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x08\x00\x04\x00\x00",
+ Container(len=8, proto=0x40000, payload=b""),
+ b"",
+ id="no payload",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x09\x00\x04\x00\x00abcde",
+ Container(len=9, proto=0x40000, payload=b"a"),
+ b"bcde",
+ id="1-byte payload and extra padding",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x0B\x00\x03\x00\x00hi\x00",
+ Container(len=11, proto=pq3.protocol(3, 0), payload=[b"hi"]),
+ b"",
+ id="implied parameter list when using proto version 3.0",
+ ),
+ ],
+)
+def test_Startup_parse(raw, expected, extra):
+ with io.BytesIO(raw) as stream:
+ actual = pq3.Startup.parse_stream(stream)
+
+ assert actual == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "packet,expected_bytes",
+ [
+ pytest.param(
+ dict(),
+ b"\x00\x00\x00\x08\x00\x00\x00\x00",
+ id="nothing set",
+ ),
+ pytest.param(
+ dict(len=10, proto=0x12345678),
+ b"\x00\x00\x00\x0A\x12\x34\x56\x78\x00\x00",
+ id="len and proto set explicitly",
+ ),
+ pytest.param(
+ dict(proto=0x12345678),
+ b"\x00\x00\x00\x08\x12\x34\x56\x78",
+ id="implied len with no payload",
+ ),
+ pytest.param(
+ dict(proto=0x12345678, payload=b"abcd"),
+ b"\x00\x00\x00\x0C\x12\x34\x56\x78abcd",
+ id="implied len with payload",
+ ),
+ pytest.param(
+ dict(payload=[b""]),
+ b"\x00\x00\x00\x09\x00\x03\x00\x00\x00",
+ id="implied proto version 3 when sending parameters",
+ ),
+ pytest.param(
+ dict(payload=[b"hi", b""]),
+ b"\x00\x00\x00\x0C\x00\x03\x00\x00hi\x00\x00",
+ id="implied proto version 3 and len when sending more than one parameter",
+ ),
+ pytest.param(
+ dict(payload=dict(user="jsmith", database="postgres")),
+ b"\x00\x00\x00\x27\x00\x03\x00\x00user\x00jsmith\x00database\x00postgres\x00\x00",
+ id="auto-serialization of dict parameters",
+ ),
+ ],
+)
+def test_Startup_build(packet, expected_bytes):
+ actual = pq3.Startup.build(packet)
+ assert actual == expected_bytes
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"*\x00\x00\x00\x08abcd",
+ dict(type=b"*", len=8, payload=b"abcd"),
+ b"",
+ id="4-byte payload",
+ ),
+ pytest.param(
+ b"*\x00\x00\x00\x04",
+ dict(type=b"*", len=4, payload=b""),
+ b"",
+ id="no payload",
+ ),
+ pytest.param(
+ b"*\x00\x00\x00\x05xabcd",
+ dict(type=b"*", len=5, payload=b"x"),
+ b"abcd",
+ id="1-byte payload with extra padding",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x08\x00\x00\x00\x00",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=8,
+ payload=dict(type=pq3.authn.OK, body=None),
+ ),
+ b"",
+ id="AuthenticationOk",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x12\x00\x00\x00\x0AEXTERNAL\x00\x00",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=18,
+ payload=dict(type=pq3.authn.SASL, body=[b"EXTERNAL", b""]),
+ ),
+ b"",
+ id="AuthenticationSASL",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0B12345",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=13,
+ payload=dict(type=pq3.authn.SASLContinue, body=b"12345"),
+ ),
+ b"",
+ id="AuthenticationSASLContinue",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0C12345",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=13,
+ payload=dict(type=pq3.authn.SASLFinal, body=b"12345"),
+ ),
+ b"",
+ id="AuthenticationSASLFinal",
+ ),
+ pytest.param(
+ b"p\x00\x00\x00\x0Bhunter2",
+ dict(
+ type=pq3.types.PasswordMessage,
+ len=11,
+ payload=b"hunter2",
+ ),
+ b"",
+ id="PasswordMessage",
+ ),
+ pytest.param(
+ b"K\x00\x00\x00\x0C\x00\x00\x00\x00\x12\x34\x56\x78",
+ dict(
+ type=pq3.types.BackendKeyData,
+ len=12,
+ payload=dict(pid=0, key=0x12345678),
+ ),
+ b"",
+ id="BackendKeyData",
+ ),
+ pytest.param(
+ b"C\x00\x00\x00\x08SET\x00",
+ dict(
+ type=pq3.types.CommandComplete,
+ len=8,
+ payload=dict(tag=b"SET"),
+ ),
+ b"",
+ id="CommandComplete",
+ ),
+ pytest.param(
+ b"E\x00\x00\x00\x11Mbad!\x00Mdog!\x00\x00",
+ dict(type=b"E", len=17, payload=dict(fields=[b"Mbad!", b"Mdog!", b""])),
+ b"",
+ id="ErrorResponse",
+ ),
+ pytest.param(
+ b"S\x00\x00\x00\x08a\x00b\x00",
+ dict(
+ type=pq3.types.ParameterStatus,
+ len=8,
+ payload=dict(name=b"a", value=b"b"),
+ ),
+ b"",
+ id="ParameterStatus",
+ ),
+ pytest.param(
+ b"Z\x00\x00\x00\x05x",
+ dict(type=b"Z", len=5, payload=dict(status=b"x")),
+ b"",
+ id="ReadyForQuery",
+ ),
+ pytest.param(
+ b"Q\x00\x00\x00\x06!\x00",
+ dict(type=pq3.types.Query, len=6, payload=dict(query=b"!")),
+ b"",
+ id="Query",
+ ),
+ pytest.param(
+ b"D\x00\x00\x00\x0B\x00\x01\x00\x00\x00\x01!",
+ dict(type=pq3.types.DataRow, len=11, payload=dict(columns=[b"!"])),
+ b"",
+ id="DataRow",
+ ),
+ pytest.param(
+ b"D\x00\x00\x00\x06\x00\x00extra",
+ dict(type=pq3.types.DataRow, len=6, payload=dict(columns=[])),
+ b"extra",
+ id="DataRow with extra data",
+ ),
+ pytest.param(
+ b"I\x00\x00\x00\x04",
+ dict(type=pq3.types.EmptyQueryResponse, len=4, payload=None),
+ b"",
+ id="EmptyQueryResponse",
+ ),
+ pytest.param(
+ b"I\x00\x00\x00\x04\xFF",
+ dict(type=b"I", len=4, payload=None),
+ b"\xFF",
+ id="EmptyQueryResponse with extra bytes",
+ ),
+ pytest.param(
+ b"X\x00\x00\x00\x04",
+ dict(type=pq3.types.Terminate, len=4, payload=None),
+ b"",
+ id="Terminate",
+ ),
+ ],
+)
+def test_Pq3_parse(raw, expected, extra):
+ with io.BytesIO(raw) as stream:
+ actual = pq3.Pq3.parse_stream(stream)
+
+ assert actual == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(type=b"*", len=5),
+ b"*\x00\x00\x00\x05\x00",
+ id="type and len set explicitly",
+ ),
+ pytest.param(
+ dict(type=b"*"),
+ b"*\x00\x00\x00\x04",
+ id="implied len with no payload",
+ ),
+ pytest.param(
+ dict(type=b"*", payload=b"1234"),
+ b"*\x00\x00\x00\x081234",
+ id="implied len with payload",
+ ),
+ pytest.param(
+ dict(type=pq3.types.AuthnRequest, payload=dict(type=pq3.authn.OK)),
+ b"R\x00\x00\x00\x08\x00\x00\x00\x00",
+ id="implied len/type for AuthenticationOK",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(
+ type=pq3.authn.SASL,
+ body=[b"SCRAM-SHA-256-PLUS", b"SCRAM-SHA-256", b""],
+ ),
+ ),
+ b"R\x00\x00\x00\x2A\x00\x00\x00\x0ASCRAM-SHA-256-PLUS\x00SCRAM-SHA-256\x00\x00",
+ id="implied len/type for AuthenticationSASL",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(type=pq3.authn.SASLContinue, body=b"12345"),
+ ),
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0B12345",
+ id="implied len/type for AuthenticationSASLContinue",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(type=pq3.authn.SASLFinal, body=b"12345"),
+ ),
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0C12345",
+ id="implied len/type for AuthenticationSASLFinal",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.PasswordMessage,
+ payload=b"hunter2",
+ ),
+ b"p\x00\x00\x00\x0Bhunter2",
+ id="implied len/type for PasswordMessage",
+ ),
+ pytest.param(
+ dict(type=pq3.types.BackendKeyData, payload=dict(pid=1, key=7)),
+ b"K\x00\x00\x00\x0C\x00\x00\x00\x01\x00\x00\x00\x07",
+ id="implied len/type for BackendKeyData",
+ ),
+ pytest.param(
+ dict(type=pq3.types.CommandComplete, payload=dict(tag=b"SET")),
+ b"C\x00\x00\x00\x08SET\x00",
+ id="implied len/type for CommandComplete",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ErrorResponse, payload=dict(fields=[b"error", b""])),
+ b"E\x00\x00\x00\x0Berror\x00\x00",
+ id="implied len/type for ErrorResponse",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ParameterStatus, payload=dict(name=b"a", value=b"b")),
+ b"S\x00\x00\x00\x08a\x00b\x00",
+ id="implied len/type for ParameterStatus",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ReadyForQuery, payload=dict(status=b"I")),
+ b"Z\x00\x00\x00\x05I",
+ id="implied len/type for ReadyForQuery",
+ ),
+ pytest.param(
+ dict(type=pq3.types.Query, payload=dict(query=b"SELECT 1;")),
+ b"Q\x00\x00\x00\x0eSELECT 1;\x00",
+ id="implied len/type for Query",
+ ),
+ pytest.param(
+ dict(type=pq3.types.DataRow, payload=dict(columns=[b"abcd"])),
+ b"D\x00\x00\x00\x0E\x00\x01\x00\x00\x00\x04abcd",
+ id="implied len/type for DataRow",
+ ),
+ pytest.param(
+ dict(type=pq3.types.EmptyQueryResponse),
+ b"I\x00\x00\x00\x04",
+ id="implied len for EmptyQueryResponse",
+ ),
+ pytest.param(
+ dict(type=pq3.types.Terminate),
+ b"X\x00\x00\x00\x04",
+ id="implied len for Terminate",
+ ),
+ ],
+)
+def test_Pq3_build(fields, expected):
+ actual = pq3.Pq3.build(fields)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"\x00\x00",
+ dict(columns=[]),
+ b"",
+ id="no columns",
+ ),
+ pytest.param(
+ b"\x00\x01\x00\x00\x00\x04abcd",
+ dict(columns=[b"abcd"]),
+ b"",
+ id="one column",
+ ),
+ pytest.param(
+ b"\x00\x02\x00\x00\x00\x04abcd\x00\x00\x00\x01x",
+ dict(columns=[b"abcd", b"x"]),
+ b"",
+ id="multiple columns",
+ ),
+ pytest.param(
+ b"\x00\x02\x00\x00\x00\x00\x00\x00\x00\x01x",
+ dict(columns=[b"", b"x"]),
+ b"",
+ id="empty column value",
+ ),
+ pytest.param(
+ b"\x00\x02\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF",
+ dict(columns=[None, None]),
+ b"",
+ id="null columns",
+ ),
+ ],
+)
+def test_DataRow_parse(raw, expected, extra):
+ pkt = b"D" + struct.pack("!i", len(raw) + 4) + raw
+ with io.BytesIO(pkt) as stream:
+ actual = pq3.Pq3.parse_stream(stream)
+
+ assert actual.type == pq3.types.DataRow
+ assert actual.payload == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(),
+ b"\x00\x00",
+ id="no columns",
+ ),
+ pytest.param(
+ dict(columns=[None, None]),
+ b"\x00\x02\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF",
+ id="null columns",
+ ),
+ ],
+)
+def test_DataRow_build(fields, expected):
+ actual = pq3.Pq3.build(dict(type=pq3.types.DataRow, payload=fields))
+
+ expected = b"D" + struct.pack("!i", len(expected) + 4) + expected
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "raw,expected,exception",
+ [
+ pytest.param(
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ dict(name=b"EXTERNAL", len=-1, data=None),
+ None,
+ id="no initial response",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\x02me",
+ dict(name=b"EXTERNAL", len=2, data=b"me"),
+ None,
+ id="initial response",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\x02meextra",
+ None,
+ TerminatedError,
+ id="extra data",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\xFFme",
+ None,
+ StreamError,
+ id="underflow",
+ ),
+ ],
+)
+def test_SASLInitialResponse_parse(raw, expected, exception):
+ ctx = contextlib.nullcontext()
+ if exception:
+ ctx = pytest.raises(exception)
+
+ with ctx:
+ actual = pq3.SASLInitialResponse.parse(raw)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(name=b"EXTERNAL"),
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ id="no initial response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=None),
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ id="no initial response (explicit None)",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=b""),
+ b"EXTERNAL\x00\x00\x00\x00\x00",
+ id="empty response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=b"me@example.com"),
+ b"EXTERNAL\x00\x00\x00\x00\x0Eme@example.com",
+ id="initial response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", len=2, data=b"me@example.com"),
+ b"EXTERNAL\x00\x00\x00\x00\x02me@example.com",
+ id="data overflow",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", len=14, data=b"me"),
+ b"EXTERNAL\x00\x00\x00\x00\x0Eme",
+ id="data underflow",
+ ),
+ ],
+)
+def test_SASLInitialResponse_build(fields, expected):
+ actual = pq3.SASLInitialResponse.build(fields)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "version,expected_bytes",
+ [
+ pytest.param((3, 0), b"\x00\x03\x00\x00", id="version 3"),
+ pytest.param((1234, 5679), b"\x04\xd2\x16\x2f", id="SSLRequest"),
+ ],
+)
+def test_protocol(version, expected_bytes):
+ # Make sure the integer returned by protocol is correctly serialized on the
+ # wire.
+ assert struct.pack("!i", pq3.protocol(*version)) == expected_bytes
+
+
+@pytest.mark.parametrize(
+ "envvar,func,expected",
+ [
+ ("PGHOST", pq3.pghost, "localhost"),
+ ("PGPORT", pq3.pgport, 5432),
+ ("PGUSER", pq3.pguser, getpass.getuser()),
+ ("PGDATABASE", pq3.pgdatabase, "postgres"),
+ ],
+)
+def test_env_defaults(monkeypatch, envvar, func, expected):
+ monkeypatch.delenv(envvar, raising=False)
+
+ actual = func()
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "envvars,func,expected",
+ [
+ (dict(PGHOST="otherhost"), pq3.pghost, "otherhost"),
+ (dict(PGPORT="6789"), pq3.pgport, 6789),
+ (dict(PGUSER="postgres"), pq3.pguser, "postgres"),
+ (dict(PGDATABASE="template1"), pq3.pgdatabase, "template1"),
+ ],
+)
+def test_env(monkeypatch, envvars, func, expected):
+ for k, v in envvars.items():
+ monkeypatch.setenv(k, v)
+
+ actual = func()
+ assert actual == expected
diff --git a/src/test/python/tls.py b/src/test/python/tls.py
new file mode 100644
index 0000000000..075c02c1ca
--- /dev/null
+++ b/src/test/python/tls.py
@@ -0,0 +1,195 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+from construct import *
+
+#
+# TLS 1.3
+#
+# Most of the types below are transcribed from RFC 8446:
+#
+# https://tools.ietf.org/html/rfc8446
+#
+
+
+def _Vector(size_field, element):
+ return Prefixed(size_field, GreedyRange(element))
+
+
+# Alerts
+
+AlertLevel = Enum(
+ Byte,
+ warning=1,
+ fatal=2,
+)
+
+AlertDescription = Enum(
+ Byte,
+ close_notify=0,
+ unexpected_message=10,
+ bad_record_mac=20,
+ decryption_failed_RESERVED=21,
+ record_overflow=22,
+ decompression_failure=30,
+ handshake_failure=40,
+ no_certificate_RESERVED=41,
+ bad_certificate=42,
+ unsupported_certificate=43,
+ certificate_revoked=44,
+ certificate_expired=45,
+ certificate_unknown=46,
+ illegal_parameter=47,
+ unknown_ca=48,
+ access_denied=49,
+ decode_error=50,
+ decrypt_error=51,
+ export_restriction_RESERVED=60,
+ protocol_version=70,
+ insufficient_security=71,
+ internal_error=80,
+ user_canceled=90,
+ no_renegotiation=100,
+ unsupported_extension=110,
+)
+
+Alert = Struct(
+ "level" / AlertLevel,
+ "description" / AlertDescription,
+)
+
+
+# Extensions
+
+ExtensionType = Enum(
+ Int16ub,
+ server_name=0,
+ max_fragment_length=1,
+ status_request=5,
+ supported_groups=10,
+ signature_algorithms=13,
+ use_srtp=14,
+ heartbeat=15,
+ application_layer_protocol_negotiation=16,
+ signed_certificate_timestamp=18,
+ client_certificate_type=19,
+ server_certificate_type=20,
+ padding=21,
+ pre_shared_key=41,
+ early_data=42,
+ supported_versions=43,
+ cookie=44,
+ psk_key_exchange_modes=45,
+ certificate_authorities=47,
+ oid_filters=48,
+ post_handshake_auth=49,
+ signature_algorithms_cert=50,
+ key_share=51,
+)
+
+Extension = Struct(
+ "extension_type" / ExtensionType,
+ "extension_data" / Prefixed(Int16ub, GreedyBytes),
+)
+
+
+# ClientHello
+
+
+class _CipherSuiteAdapter(Adapter):
+ class _hextuple(tuple):
+ def __repr__(self):
+ return f"(0x{self[0]:02X}, 0x{self[1]:02X})"
+
+ def _encode(self, obj, context, path):
+ return bytes(obj)
+
+ def _decode(self, obj, context, path):
+ assert len(obj) == 2
+ return self._hextuple(obj)
+
+
+ProtocolVersion = Hex(Int16ub)
+
+Random = Hex(Bytes(32))
+
+CipherSuite = _CipherSuiteAdapter(Byte[2])
+
+ClientHello = Struct(
+ "legacy_version" / ProtocolVersion,
+ "random" / Random,
+ "legacy_session_id" / Prefixed(Byte, Hex(GreedyBytes)),
+ "cipher_suites" / _Vector(Int16ub, CipherSuite),
+ "legacy_compression_methods" / Prefixed(Byte, GreedyBytes),
+ "extensions" / _Vector(Int16ub, Extension),
+)
+
+# ServerHello
+
+ServerHello = Struct(
+ "legacy_version" / ProtocolVersion,
+ "random" / Random,
+ "legacy_session_id_echo" / Prefixed(Byte, Hex(GreedyBytes)),
+ "cipher_suite" / CipherSuite,
+ "legacy_compression_method" / Hex(Byte),
+ "extensions" / _Vector(Int16ub, Extension),
+)
+
+# Handshake
+
+HandshakeType = Enum(
+ Byte,
+ client_hello=1,
+ server_hello=2,
+ new_session_ticket=4,
+ end_of_early_data=5,
+ encrypted_extensions=8,
+ certificate=11,
+ certificate_request=13,
+ certificate_verify=15,
+ finished=20,
+ key_update=24,
+ message_hash=254,
+)
+
+Handshake = Struct(
+ "msg_type" / HandshakeType,
+ "length" / Int24ub,
+ "payload"
+ / Switch(
+ this.msg_type,
+ {
+ HandshakeType.client_hello: ClientHello,
+ HandshakeType.server_hello: ServerHello,
+ # HandshakeType.end_of_early_data: EndOfEarlyData,
+ # HandshakeType.encrypted_extensions: EncryptedExtensions,
+ # HandshakeType.certificate_request: CertificateRequest,
+ # HandshakeType.certificate: Certificate,
+ # HandshakeType.certificate_verify: CertificateVerify,
+ # HandshakeType.finished: Finished,
+ # HandshakeType.new_session_ticket: NewSessionTicket,
+ # HandshakeType.key_update: KeyUpdate,
+ },
+ default=FixedSized(this.length, GreedyBytes),
+ ),
+)
+
+# Records
+
+ContentType = Enum(
+ Byte,
+ invalid=0,
+ change_cipher_spec=20,
+ alert=21,
+ handshake=22,
+ application_data=23,
+)
+
+Plaintext = Struct(
+ "type" / ContentType,
+ "legacy_record_version" / ProtocolVersion,
+ "length" / Int16ub,
+ "fragment" / FixedSized(this.length, GreedyBytes),
+)
--
2.25.1
v3-0009-contrib-oauth-switch-to-pluggable-auth-API.patch (text/x-patch, 13.7K) download
From d275b329aaed6c6ef2403e4c313725a1ae88fa40 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Fri, 4 Mar 2022 08:48:47 -0800
Subject: [PATCH v3 9/9] contrib/oauth: switch to pluggable auth API
Move the core server implementation to contrib/oauth as a pluggable
provider, using the RegisterAuthProvider() API. oauth_validator_command
has been moved from core to a custom GUC. Tests have been updated to
handle the new implementations.
Some server modifications remain:
- Adding new HBA options for custom providers
- Registering support for usermaps
- Allowing custom SASL mechanisms to declare their own maximum message
length
This patch is optional; you can apply/revert it to compare the two
approaches.
---
contrib/oauth/Makefile | 16 +++++++
.../auth-oauth.c => contrib/oauth/oauth.c | 47 +++++++++++++++----
src/backend/libpq/Makefile | 1 -
src/backend/libpq/auth.c | 7 ---
src/backend/libpq/hba.c | 21 +++++----
src/backend/utils/misc/guc.c | 12 -----
src/include/libpq/hba.h | 3 +-
src/include/libpq/oauth.h | 24 ----------
src/test/python/server/test_oauth.py | 20 ++++----
9 files changed, 78 insertions(+), 73 deletions(-)
create mode 100644 contrib/oauth/Makefile
rename src/backend/libpq/auth-oauth.c => contrib/oauth/oauth.c (95%)
delete mode 100644 src/include/libpq/oauth.h
diff --git a/contrib/oauth/Makefile b/contrib/oauth/Makefile
new file mode 100644
index 0000000000..880bc1fef3
--- /dev/null
+++ b/contrib/oauth/Makefile
@@ -0,0 +1,16 @@
+# contrib/oauth/Makefile
+
+MODULE_big = oauth
+OBJS = oauth.o
+PGFILEDESC = "oauth - auth provider supporting OAuth 2.0/OIDC"
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/oauth
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/backend/libpq/auth-oauth.c b/contrib/oauth/oauth.c
similarity index 95%
rename from src/backend/libpq/auth-oauth.c
rename to contrib/oauth/oauth.c
index c1232a31a0..3a6dab19d9 100644
--- a/src/backend/libpq/auth-oauth.c
+++ b/contrib/oauth/oauth.c
@@ -1,33 +1,39 @@
-/*-------------------------------------------------------------------------
+/* -------------------------------------------------------------------------
*
- * auth-oauth.c
+ * oauth.c
* Server-side implementation of the SASL OAUTHBEARER mechanism.
*
* See the following RFC for more details:
* - RFC 7628: https://tools.ietf.org/html/rfc7628
*
- * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
- * src/backend/libpq/auth-oauth.c
+ * contrib/oauth/oauth.c
*
- *-------------------------------------------------------------------------
+ * -------------------------------------------------------------------------
*/
+
#include "postgres.h"
#include <unistd.h>
#include <fcntl.h>
#include "common/oauth-common.h"
+#include "fmgr.h"
#include "lib/stringinfo.h"
#include "libpq/auth.h"
#include "libpq/hba.h"
-#include "libpq/oauth.h"
#include "libpq/sasl.h"
#include "storage/fd.h"
+#include "utils/guc.h"
+
+PG_MODULE_MAGIC;
+
+void _PG_init(void);
/* GUC */
-char *oauth_validator_command;
+static char *oauth_validator_command;
static void oauth_get_mechanisms(Port *port, StringInfo buf);
static void *oauth_init(Port *port, const char *selected_mech, const char *shadow_pass);
@@ -35,7 +41,7 @@ static int oauth_exchange(void *opaq, const char *input, int inputlen,
char **output, int *outputlen, const char **logdetail);
/* Mechanism declaration */
-const pg_be_sasl_mech pg_be_oauth_mech = {
+static const pg_be_sasl_mech oauth_mech = {
oauth_get_mechanisms,
oauth_init,
oauth_exchange,
@@ -795,3 +801,28 @@ username_ok_for_shell(const char *username)
return true;
}
+
+static int CheckOAuth(Port *port)
+{
+ return CheckSASLAuth(&oauth_mech, port, NULL, NULL);
+}
+
+static const char *OAuthError(Port *port)
+{
+ return psprintf("OAuth bearer authentication failed for user \"%s\"",
+ port->user_name);
+}
+
+void
+_PG_init(void)
+{
+ RegisterAuthProvider("oauth", CheckOAuth, OAuthError);
+
+ DefineCustomStringVariable("oauth.validator_command",
+ gettext_noop("Command to validate OAuth v2 bearer tokens."),
+ NULL,
+ &oauth_validator_command,
+ "",
+ PGC_SIGHUP, GUC_SUPERUSER_ONLY,
+ NULL, NULL, NULL);
+}
diff --git a/src/backend/libpq/Makefile b/src/backend/libpq/Makefile
index 98eb2a8242..6d385fd6a4 100644
--- a/src/backend/libpq/Makefile
+++ b/src/backend/libpq/Makefile
@@ -15,7 +15,6 @@ include $(top_builddir)/src/Makefile.global
# be-fsstubs is here for historical reasons, probably belongs elsewhere
OBJS = \
- auth-oauth.o \
auth-sasl.o \
auth-scram.o \
auth.o \
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 5c30904e2b..3533b0bc50 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -30,7 +30,6 @@
#include "libpq/auth.h"
#include "libpq/crypt.h"
#include "libpq/libpq.h"
-#include "libpq/oauth.h"
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
#include "libpq/scram.h"
@@ -303,9 +302,6 @@ auth_failed(Port *port, int status, const char *logdetail)
case uaRADIUS:
errstr = gettext_noop("RADIUS authentication failed for user \"%s\"");
break;
- case uaOAuth:
- errstr = gettext_noop("OAuth bearer authentication failed for user \"%s\"");
- break;
case uaCustom:
if (CustomAuthenticationError_hook)
errstr = CustomAuthenticationError_hook(port);
@@ -631,9 +627,6 @@ ClientAuthentication(Port *port)
case uaTrust:
status = STATUS_OK;
break;
- case uaOAuth:
- status = CheckSASLAuth(&pg_be_oauth_mech, port, NULL, NULL);
- break;
case uaCustom:
if (CustomAuthenticationCheck_hook)
status = CustomAuthenticationCheck_hook(port);
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index f7f3059927..fb51c53cc0 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -136,7 +136,6 @@ static const char *const UserAuthName[] =
"radius",
"custom",
"peer",
- "oauth",
};
@@ -1401,8 +1400,6 @@ parse_hba_line(TokenizedLine *tok_line, int elevel)
#endif
else if (strcmp(token->string, "radius") == 0)
parsedline->auth_method = uaRADIUS;
- else if (strcmp(token->string, "oauth") == 0)
- parsedline->auth_method = uaOAuth;
else if (strcmp(token->string, "custom") == 0)
parsedline->auth_method = uaCustom;
else
@@ -1731,9 +1728,9 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->auth_method != uaPeer &&
hbaline->auth_method != uaGSS &&
hbaline->auth_method != uaSSPI &&
- hbaline->auth_method != uaOAuth &&
- hbaline->auth_method != uaCert)
- INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, oauth, and cert"));
+ hbaline->auth_method != uaCert &&
+ hbaline->auth_method != uaCustom)
+ INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, cert, and custom"));
hbaline->usermap = pstrdup(val);
}
else if (strcmp(name, "clientcert") == 0)
@@ -2119,19 +2116,25 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
}
else if (strcmp(name, "issuer") == 0)
{
- if (hbaline->auth_method != uaOAuth)
+ if (hbaline->auth_method != uaCustom
+ && (custom_provider_name != NULL
+ && strcmp(custom_provider_name, "oauth")))
INVALID_AUTH_OPTION("issuer", gettext_noop("oauth"));
hbaline->oauth_issuer = pstrdup(val);
}
else if (strcmp(name, "scope") == 0)
{
- if (hbaline->auth_method != uaOAuth)
+ if (hbaline->auth_method != uaCustom
+ && (custom_provider_name != NULL
+ && strcmp(custom_provider_name, "oauth")))
INVALID_AUTH_OPTION("scope", gettext_noop("oauth"));
hbaline->oauth_scope = pstrdup(val);
}
else if (strcmp(name, "trust_validator_authz") == 0)
{
- if (hbaline->auth_method != uaOAuth)
+ if (hbaline->auth_method != uaCustom
+ && (custom_provider_name != NULL
+ && strcmp(custom_provider_name, "oauth")))
INVALID_AUTH_OPTION("trust_validator_authz", gettext_noop("oauth"));
if (strcmp(val, "1") == 0)
hbaline->oauth_skip_usermap = true;
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 791c7c83df..1e3650184b 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -58,7 +58,6 @@
#include "libpq/auth.h"
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
-#include "libpq/oauth.h"
#include "miscadmin.h"
#include "optimizer/cost.h"
#include "optimizer/geqo.h"
@@ -4663,17 +4662,6 @@ static struct config_string ConfigureNamesString[] =
check_backtrace_functions, assign_backtrace_functions, NULL
},
- {
- {"oauth_validator_command", PGC_SIGHUP, CONN_AUTH_AUTH,
- gettext_noop("Command to validate OAuth v2 bearer tokens."),
- NULL,
- GUC_SUPERUSER_ONLY
- },
- &oauth_validator_command,
- "",
- NULL, NULL, NULL
- },
-
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, NULL, NULL, NULL, NULL
diff --git a/src/include/libpq/hba.h b/src/include/libpq/hba.h
index d46c2108eb..0c6a7dd823 100644
--- a/src/include/libpq/hba.h
+++ b/src/include/libpq/hba.h
@@ -40,8 +40,7 @@ typedef enum UserAuth
uaRADIUS,
uaCustom,
uaPeer,
- uaOAuth
-#define USER_AUTH_LAST uaOAuth /* Must be last value of this enum */
+#define USER_AUTH_LAST uaPeer /* Must be last value of this enum */
} UserAuth;
/*
diff --git a/src/include/libpq/oauth.h b/src/include/libpq/oauth.h
deleted file mode 100644
index 870e426af1..0000000000
--- a/src/include/libpq/oauth.h
+++ /dev/null
@@ -1,24 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * oauth.h
- * Interface to libpq/auth-oauth.c
- *
- * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
- * Portions Copyright (c) 1994, Regents of the University of California
- *
- * src/include/libpq/oauth.h
- *
- *-------------------------------------------------------------------------
- */
-#ifndef PG_OAUTH_H
-#define PG_OAUTH_H
-
-#include "libpq/libpq-be.h"
-#include "libpq/sasl.h"
-
-extern char *oauth_validator_command;
-
-/* Implementation */
-extern const pg_be_sasl_mech pg_be_oauth_mech;
-
-#endif /* PG_OAUTH_H */
diff --git a/src/test/python/server/test_oauth.py b/src/test/python/server/test_oauth.py
index cb5ca7fa23..07fc25edc2 100644
--- a/src/test/python/server/test_oauth.py
+++ b/src/test/python/server/test_oauth.py
@@ -103,9 +103,9 @@ def oauth_ctx():
ctx = Context()
hba_lines = (
- f'host {ctx.dbname} {ctx.map_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" map=oauth\n',
- f'host {ctx.dbname} {ctx.authz_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" trust_validator_authz=1\n',
- f'host {ctx.dbname} all samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}"\n',
+ f'host {ctx.dbname} {ctx.map_user} samehost custom provider=oauth issuer="{ctx.issuer}" scope="{ctx.scope}" map=oauth\n',
+ f'host {ctx.dbname} {ctx.authz_user} samehost custom provider=oauth issuer="{ctx.issuer}" scope="{ctx.scope}" trust_validator_authz=1\n',
+ f'host {ctx.dbname} all samehost custom provider=oauth issuer="{ctx.issuer}" scope="{ctx.scope}"\n',
)
ident_lines = (r"oauth /^(.*)@example\.com$ \1",)
@@ -126,12 +126,12 @@ def oauth_ctx():
c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(authz_user))
c.execute(sql.SQL("CREATE DATABASE {};").format(dbname))
- # Make this test script the server's oauth_validator.
+ # Make this test script the server's oauth validator.
path = pathlib.Path(__file__).parent / "validate_bearer.py"
path = str(path.absolute())
cmd = f"{shlex.quote(path)} {SHARED_MEM_NAME} <&%f"
- c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (cmd,))
+ c.execute("ALTER SYSTEM SET oauth.validator_command TO %s;", (cmd,))
# Replace pg_hba and pg_ident.
c.execute("SHOW hba_file;")
@@ -149,7 +149,7 @@ def oauth_ctx():
# Put things back the way they were.
c.execute("SELECT pg_reload_conf();")
- c.execute("ALTER SYSTEM RESET oauth_validator_command;")
+ c.execute("ALTER SYSTEM RESET oauth.validator_command;")
c.execute(sql.SQL("DROP DATABASE {};").format(dbname))
c.execute(sql.SQL("DROP ROLE {};").format(authz_user))
c.execute(sql.SQL("DROP ROLE {};").format(map_user))
@@ -930,7 +930,7 @@ def test_oauth_empty_initial_response(conn, oauth_ctx, bearer_token):
def set_validator():
"""
A per-test fixture that allows a test to override the setting of
- oauth_validator_command for the cluster. The setting will be reverted during
+ oauth.validator_command for the cluster. The setting will be reverted during
teardown.
Passing None will perform an ALTER SYSTEM RESET.
@@ -942,17 +942,17 @@ def set_validator():
c = conn.cursor()
# Save the previous value.
- c.execute("SHOW oauth_validator_command;")
+ c.execute("SHOW oauth.validator_command;")
prev_cmd = c.fetchone()[0]
def setter(cmd):
- c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (cmd,))
+ c.execute("ALTER SYSTEM SET oauth.validator_command TO %s;", (cmd,))
c.execute("SELECT pg_reload_conf();")
yield setter
# Restore the previous value.
- c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (prev_cmd,))
+ c.execute("ALTER SYSTEM SET oauth.validator_command TO %s;", (prev_cmd,))
c.execute("SELECT pg_reload_conf();")
--
2.25.1
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-03-22 21:48 samay sharma <smilingsamay@gmail.com>
parent: Jacob Champion <pchampion@vmware.com>
1 sibling, 1 reply; 424+ messages in thread
From: samay sharma @ 2022-03-22 21:48 UTC (permalink / raw)
To: Jacob Champion <pchampion@vmware.com>; +Cc: pgsql-hackers; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>
Hi Jacob,
Thank you for porting this on top of the pluggable auth methods API. I've
addressed the feedback around other backend changes in my latest patch, but
the client side changes still remain. I had a few questions to understand
them better.
(a) What specifically do the client side changes in the patch implement?
(b) Are the changes you made on the client side specific to OAUTH or are
they about making SASL more generic? As an additional question, if someone
wanted to implement something similar on top of your patch, would they
still have to make client side changes?
Regards,
Samay
On Fri, Mar 4, 2022 at 11:13 AM Jacob Champion <pchampion@vmware.com> wrote:
> Hi all,
>
> v3 rebases this patchset over the top of Samay's pluggable auth
> provider API [1], included here as patches 0001-3. The final patch in
> the set ports the server implementation from a core feature to a
> contrib module; to switch between the two approaches, simply leave out
> that final patch.
>
> There are still some backend changes that must be made to get this
> working, as pointed out in 0009, and obviously libpq support still
> requires code changes.
>
> --Jacob
>
> [1]
> https://www.postgresql.org/message-id/flat/CAJxrbyxTRn5P8J-p%2BwHLwFahK5y56PhK28VOb55jqMO05Y-DJw%40m...
>
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-03-22 22:44 Jacob Champion <pchampion@vmware.com>
parent: samay sharma <smilingsamay@gmail.com>
0 siblings, 0 replies; 424+ messages in thread
From: Jacob Champion @ 2022-03-22 22:44 UTC (permalink / raw)
To: smilingsamay@gmail.com <smilingsamay@gmail.com>; +Cc: hlinnaka@iki.fi <hlinnaka@iki.fi>; pgsql-hackers; michael@paquier.xyz <michael@paquier.xyz>
On Tue, 2022-03-22 at 14:48 -0700, samay sharma wrote:
> Thank you for porting this on top of the pluggable auth methods API.
> I've addressed the feedback around other backend changes in my latest
> patch, but the client side changes still remain. I had a few
> questions to understand them better.
>
> (a) What specifically do the client side changes in the patch implement?
Hi Samay,
The client-side changes are an implementation of the OAuth 2.0 Device
Authorization Grant [1] in libpq. The majority of the OAuth logic is
handled by the third-party iddawc library.
The server tells the client what OIDC provider to contact, and then
libpq prompts you to log into that provider on your
smartphone/browser/etc. using a one-time code. After you give libpq
permission to act on your behalf, the Bearer token gets sent to libpq
via a direct connection, and libpq forwards it to the server so that
the server can determine whether you're allowed in.
> (b) Are the changes you made on the client side specific to OAUTH or
> are they about making SASL more generic?
The original patchset included changes to make SASL more generic. Many
of those changes have since been merged, and the remaining code is
mostly OAuth-specific, but there are still improvements to be made.
(And there's some JSON crud to sift through in the first couple of
patches. I'm still mad that the OAUTHBEARER spec requires clients to
parse JSON in the first place.)
> As an additional question,
> if someone wanted to implement something similar on top of your
> patch, would they still have to make client side changes?
Any new SASL mechanisms require changes to libpq at this point. You
need to implement a new pg_sasl_mech, modify pg_SASL_init() to select
the mechanism correctly, and add whatever connection string options you
need, along with the associated state in pg_conn. Patch 0004 has all
the client-side magic for OAUTHBEARER.
--Jacob
[1] https://datatracker.ietf.org/doc/html/rfc8628
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-03-26 00:00 Jacob Champion <pchampion@vmware.com>
parent: Jacob Champion <pchampion@vmware.com>
1 sibling, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2022-03-26 00:00 UTC (permalink / raw)
To: pgsql-hackers; +Cc: hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>
On Fri, 2022-03-04 at 19:13 +0000, Jacob Champion wrote:
> v3 rebases this patchset over the top of Samay's pluggable auth
> provider API [1], included here as patches 0001-3.
v4 rebases over the latest version of the pluggable auth patchset
(included as 0001-4). Note that there's a recent conflict as
of d4781d887; use an older commit as the base (or wait for the other
thread to be updated).
--Jacob
Attachments: download all (10 files, .tar.gz)
v4-0001-Add-support-for-custom-authentication-methods.patch (text/x-patch, 11.6K) download
From 575431b4e035c266b55a25414f802fbf8ba16b97 Mon Sep 17 00:00:00 2001
From: Samay Sharma <smilingsamay@gmail.com>
Date: Tue, 15 Feb 2022 22:23:29 -0800
Subject: [PATCH v4 01/10] Add support for custom authentication methods
Currently, PostgreSQL supports only a set of pre-defined authentication
methods. This patch adds support for 2 hooks which allow users to add
their custom authentication methods by defining a check function and an
error function. Users can then use these methods by using a new "custom"
keyword in pg_hba.conf and specifying the authentication provider they
want to use.
---
src/backend/libpq/auth.c | 108 ++++++++++++++++++++++++++++++++-------
src/backend/libpq/hba.c | 44 ++++++++++++++++
src/include/libpq/auth.h | 37 ++++++++++++++
src/include/libpq/hba.h | 2 +
4 files changed, 172 insertions(+), 19 deletions(-)
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index efc53f3135..375ee33892 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -47,8 +47,6 @@
*----------------------------------------------------------------
*/
static void auth_failed(Port *port, int status, const char *logdetail);
-static char *recv_password_packet(Port *port);
-static void set_authn_id(Port *port, const char *id);
/*----------------------------------------------------------------
@@ -206,22 +204,11 @@ static int pg_SSPI_make_upn(char *accountname,
static int CheckRADIUSAuth(Port *port);
static int PerformRadiusTransaction(const char *server, const char *secret, const char *portstr, const char *identifier, const char *user_name, const char *passwd);
-
-/*
- * Maximum accepted size of GSS and SSPI authentication tokens.
- * We also use this as a limit on ordinary password packet lengths.
- *
- * Kerberos tickets are usually quite small, but the TGTs issued by Windows
- * domain controllers include an authorization field known as the Privilege
- * Attribute Certificate (PAC), which contains the user's Windows permissions
- * (group memberships etc.). The PAC is copied into all tickets obtained on
- * the basis of this TGT (even those issued by Unix realms which the Windows
- * realm trusts), and can be several kB in size. The maximum token size
- * accepted by Windows systems is determined by the MaxAuthToken Windows
- * registry setting. Microsoft recommends that it is not set higher than
- * 65535 bytes, so that seems like a reasonable limit for us as well.
+/*----------------------------------------------------------------
+ * Custom Authentication
+ *----------------------------------------------------------------
*/
-#define PG_MAX_AUTH_TOKEN_LENGTH 65535
+static List *custom_auth_providers = NIL;
/*----------------------------------------------------------------
* Global authentication functions
@@ -311,6 +298,15 @@ auth_failed(Port *port, int status, const char *logdetail)
case uaRADIUS:
errstr = gettext_noop("RADIUS authentication failed for user \"%s\"");
break;
+ case uaCustom:
+ {
+ CustomAuthProvider *provider = get_provider_by_name(port->hba->custom_provider);
+ if (provider->auth_error_hook)
+ errstr = provider->auth_error_hook(port);
+ else
+ errstr = gettext_noop("Custom authentication failed for user \"%s\"");
+ break;
+ }
default:
errstr = gettext_noop("authentication failed for user \"%s\": invalid authentication method");
break;
@@ -345,7 +341,7 @@ auth_failed(Port *port, int status, const char *logdetail)
* lifetime of the Port, so it is safe to pass a string that is managed by an
* external library.
*/
-static void
+void
set_authn_id(Port *port, const char *id)
{
Assert(id);
@@ -630,6 +626,13 @@ ClientAuthentication(Port *port)
case uaTrust:
status = STATUS_OK;
break;
+ case uaCustom:
+ {
+ CustomAuthProvider *provider = get_provider_by_name(port->hba->custom_provider);
+ if (provider->auth_check_hook)
+ status = provider->auth_check_hook(port);
+ break;
+ }
}
if ((status == STATUS_OK && port->hba->clientcert == clientCertFull)
@@ -689,7 +692,7 @@ sendAuthRequest(Port *port, AuthRequest areq, const char *extradata, int extrale
*
* Returns NULL if couldn't get password, else palloc'd string.
*/
-static char *
+char *
recv_password_packet(Port *port)
{
StringInfoData buf;
@@ -3343,3 +3346,70 @@ PerformRadiusTransaction(const char *server, const char *secret, const char *por
}
} /* while (true) */
}
+
+/*----------------------------------------------------------------
+ * Custom authentication
+ *----------------------------------------------------------------
+ */
+
+/*
+ * RegisterAuthProvider registers a custom authentication provider to be
+ * used for authentication. It validates the inputs and adds the provider
+ * name and it's hooks to a list of loaded providers. The right provider's
+ * hooks can then be called based on the provider name specified in
+ * pg_hba.conf.
+ *
+ * This function should be called in _PG_init() by any extension looking to
+ * add a custom authentication method.
+ */
+void RegisterAuthProvider(const char *provider_name,
+ CustomAuthenticationCheck_hook_type AuthenticationCheckFunction,
+ CustomAuthenticationError_hook_type AuthenticationErrorFunction)
+{
+ CustomAuthProvider *provider = NULL;
+ MemoryContext old_context;
+
+ if (provider_name == NULL)
+ {
+ ereport(ERROR,
+ (errmsg("cannot register authentication provider without name")));
+ }
+
+ if (AuthenticationCheckFunction == NULL)
+ {
+ ereport(ERROR,
+ (errmsg("cannot register authentication provider without a check function")));
+ }
+
+ /*
+ * Allocate in top memory context as we need to read this whenever
+ * we parse pg_hba.conf
+ */
+ old_context = MemoryContextSwitchTo(TopMemoryContext);
+ provider = palloc(sizeof(CustomAuthProvider));
+ provider->name = MemoryContextStrdup(TopMemoryContext,provider_name);
+ provider->auth_check_hook = AuthenticationCheckFunction;
+ provider->auth_error_hook = AuthenticationErrorFunction;
+ custom_auth_providers = lappend(custom_auth_providers, provider);
+ MemoryContextSwitchTo(old_context);
+}
+
+/*
+ * Returns the authentication provider (which includes it's
+ * callback functions) based on name specified.
+ */
+CustomAuthProvider *get_provider_by_name(const char *name)
+{
+ ListCell *lc;
+
+ foreach(lc, custom_auth_providers)
+ {
+ CustomAuthProvider *provider = (CustomAuthProvider *) lfirst(lc);
+ if (strcmp(provider->name,name) == 0)
+ {
+ return provider;
+ }
+ }
+
+ return NULL;
+}
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index 90953c38f3..9f15252789 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -31,6 +31,7 @@
#include "common/ip.h"
#include "common/string.h"
#include "funcapi.h"
+#include "libpq/auth.h"
#include "libpq/ifaddr.h"
#include "libpq/libpq.h"
#include "miscadmin.h"
@@ -134,6 +135,7 @@ static const char *const UserAuthName[] =
"ldap",
"cert",
"radius",
+ "custom",
"peer"
};
@@ -1399,6 +1401,8 @@ parse_hba_line(TokenizedLine *tok_line, int elevel)
#endif
else if (strcmp(token->string, "radius") == 0)
parsedline->auth_method = uaRADIUS;
+ else if (strcmp(token->string, "custom") == 0)
+ parsedline->auth_method = uaCustom;
else
{
ereport(elevel,
@@ -1691,6 +1695,14 @@ parse_hba_line(TokenizedLine *tok_line, int elevel)
parsedline->clientcert = clientCertFull;
}
+ /*
+ * Ensure that the provider name is specified for custom authentication method.
+ */
+ if (parsedline->auth_method == uaCustom)
+ {
+ MANDATORY_AUTH_ARG(parsedline->custom_provider, "provider", "custom");
+ }
+
return parsedline;
}
@@ -2102,6 +2114,31 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->radiusidentifiers = parsed_identifiers;
hbaline->radiusidentifiers_s = pstrdup(val);
}
+ else if (strcmp(name, "provider") == 0)
+ {
+ REQUIRE_AUTH_OPTION(uaCustom, "provider", "custom");
+
+ /*
+ * Verify that the provider mentioned is loaded via shared_preload_libraries.
+ */
+
+ if (get_provider_by_name(val) == NULL)
+ {
+ ereport(elevel,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ errmsg("cannot use authentication provider %s",val),
+ errhint("Load authentication provider via shared_preload_libraries."),
+ errcontext("line %d of configuration file \"%s\"",
+ line_num, HbaFileName)));
+ *err_msg = psprintf("cannot use authentication provider %s", val);
+
+ return false;
+ }
+ else
+ {
+ hbaline->custom_provider = pstrdup(val);
+ }
+ }
else
{
ereport(elevel,
@@ -2442,6 +2479,13 @@ gethba_options(HbaLine *hba)
CStringGetTextDatum(psprintf("radiusports=%s", hba->radiusports_s));
}
+ if (hba->auth_method == uaCustom)
+ {
+ if (hba->custom_provider)
+ options[noptions++] =
+ CStringGetTextDatum(psprintf("provider=%s",hba->custom_provider));
+ }
+
/* If you add more options, consider increasing MAX_HBA_OPTIONS. */
Assert(noptions <= MAX_HBA_OPTIONS);
diff --git a/src/include/libpq/auth.h b/src/include/libpq/auth.h
index 6d7ee1acb9..7aff98d919 100644
--- a/src/include/libpq/auth.h
+++ b/src/include/libpq/auth.h
@@ -23,9 +23,46 @@ extern char *pg_krb_realm;
extern void ClientAuthentication(Port *port);
extern void sendAuthRequest(Port *port, AuthRequest areq, const char *extradata,
int extralen);
+extern void set_authn_id(Port *port, const char *id);
+extern char *recv_password_packet(Port *port);
/* Hook for plugins to get control in ClientAuthentication() */
+typedef int (*CustomAuthenticationCheck_hook_type) (Port *);
typedef void (*ClientAuthentication_hook_type) (Port *, int);
extern PGDLLIMPORT ClientAuthentication_hook_type ClientAuthentication_hook;
+/* Hook for plugins to report error messages in auth_failed() */
+typedef const char * (*CustomAuthenticationError_hook_type) (Port *);
+
+extern void RegisterAuthProvider
+ (const char *provider_name,
+ CustomAuthenticationCheck_hook_type CustomAuthenticationCheck_hook,
+ CustomAuthenticationError_hook_type CustomAuthenticationError_hook);
+
+/* Declarations for custom authentication providers */
+typedef struct CustomAuthProvider
+{
+ const char *name;
+ CustomAuthenticationCheck_hook_type auth_check_hook;
+ CustomAuthenticationError_hook_type auth_error_hook;
+} CustomAuthProvider;
+
+extern CustomAuthProvider *get_provider_by_name(const char *name);
+
+/*
+ * Maximum accepted size of GSS and SSPI authentication tokens.
+ * We also use this as a limit on ordinary password packet lengths.
+ *
+ * Kerberos tickets are usually quite small, but the TGTs issued by Windows
+ * domain controllers include an authorization field known as the Privilege
+ * Attribute Certificate (PAC), which contains the user's Windows permissions
+ * (group memberships etc.). The PAC is copied into all tickets obtained on
+ * the basis of this TGT (even those issued by Unix realms which the Windows
+ * realm trusts), and can be several kB in size. The maximum token size
+ * accepted by Windows systems is determined by the MaxAuthToken Windows
+ * registry setting. Microsoft recommends that it is not set higher than
+ * 65535 bytes, so that seems like a reasonable limit for us as well.
+ */
+#define PG_MAX_AUTH_TOKEN_LENGTH 65535
+
#endif /* AUTH_H */
diff --git a/src/include/libpq/hba.h b/src/include/libpq/hba.h
index 8d9f3821b1..48490c44ed 100644
--- a/src/include/libpq/hba.h
+++ b/src/include/libpq/hba.h
@@ -38,6 +38,7 @@ typedef enum UserAuth
uaLDAP,
uaCert,
uaRADIUS,
+ uaCustom,
uaPeer
#define USER_AUTH_LAST uaPeer /* Must be last value of this enum */
} UserAuth;
@@ -120,6 +121,7 @@ typedef struct HbaLine
char *radiusidentifiers_s;
List *radiusports;
char *radiusports_s;
+ char *custom_provider;
} HbaLine;
typedef struct IdentLine
--
2.25.1
v4-0002-Add-sample-extension-to-test-custom-auth-provider.patch (text/x-patch, 4.4K) download
From cb4131d8424861826e443708bc0f4e6baa76c871 Mon Sep 17 00:00:00 2001
From: Samay Sharma <smilingsamay@gmail.com>
Date: Tue, 15 Feb 2022 22:28:40 -0800
Subject: [PATCH v4 02/10] Add sample extension to test custom auth provider
hooks
This change adds a new extension to src/test/modules to
test the custom authentication provider hooks. In this
extension, we use an array to define which users to
authenticate and what passwords to use. We then get
encrypted passwords from the client and match them with
the encrypted version of the password in the array.
---
src/include/libpq/scram.h | 2 +-
src/test/modules/test_auth_provider/Makefile | 16 ++++
.../test_auth_provider/test_auth_provider.c | 86 +++++++++++++++++++
3 files changed, 103 insertions(+), 1 deletion(-)
create mode 100644 src/test/modules/test_auth_provider/Makefile
create mode 100644 src/test/modules/test_auth_provider/test_auth_provider.c
diff --git a/src/include/libpq/scram.h b/src/include/libpq/scram.h
index e60992a0d2..c51e848c24 100644
--- a/src/include/libpq/scram.h
+++ b/src/include/libpq/scram.h
@@ -18,7 +18,7 @@
#include "libpq/sasl.h"
/* SASL implementation callbacks */
-extern const pg_be_sasl_mech pg_be_scram_mech;
+extern PGDLLIMPORT const pg_be_sasl_mech pg_be_scram_mech;
/* Routines to handle and check SCRAM-SHA-256 secret */
extern char *pg_be_scram_build_secret(const char *password);
diff --git a/src/test/modules/test_auth_provider/Makefile b/src/test/modules/test_auth_provider/Makefile
new file mode 100644
index 0000000000..17971a5c7a
--- /dev/null
+++ b/src/test/modules/test_auth_provider/Makefile
@@ -0,0 +1,16 @@
+# src/test/modules/test_auth_provider/Makefile
+
+MODULE_big = test_auth_provider
+OBJS = test_auth_provider.o
+PGFILEDESC = "test_auth_provider - provider to test auth hooks"
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/test_auth_provider
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/test_auth_provider/test_auth_provider.c b/src/test/modules/test_auth_provider/test_auth_provider.c
new file mode 100644
index 0000000000..7c4b1f3500
--- /dev/null
+++ b/src/test/modules/test_auth_provider/test_auth_provider.c
@@ -0,0 +1,86 @@
+/* -------------------------------------------------------------------------
+ *
+ * test_auth_provider.c
+ * example authentication provider plugin
+ *
+ * Copyright (c) 2022, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * contrib/test_auth_provider/test_auth_provider.c
+ *
+ * -------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "libpq/auth.h"
+#include "libpq/libpq.h"
+#include "libpq/scram.h"
+
+PG_MODULE_MAGIC;
+
+void _PG_init(void);
+
+static char *get_encrypted_password_for_user(char *user_name);
+
+/*
+ * List of usernames / passwords to approve. Here we are not
+ * getting passwords from Postgres but from this list. In a more real-life
+ * extension, you can fetch valid credentials and authentication tokens /
+ * passwords from an external authentication provider.
+ */
+char credentials[3][3][50] = {
+ {"bob","alice","carol"},
+ {"bob123","alice123","carol123"}
+};
+
+static int TestAuthenticationCheck(Port *port)
+{
+ int result = STATUS_ERROR;
+ char *real_pass;
+ const char *logdetail = NULL;
+
+ real_pass = get_encrypted_password_for_user(port->user_name);
+ if (real_pass)
+ {
+ result = CheckSASLAuth(&pg_be_scram_mech, port, real_pass, &logdetail);
+ pfree(real_pass);
+ }
+
+ if (result == STATUS_OK)
+ set_authn_id(port, port->user_name);
+
+ return result;
+}
+
+/*
+ * Get SCRAM encrypted version of the password for user.
+ */
+static char *
+get_encrypted_password_for_user(char *user_name)
+{
+ char *password = NULL;
+ int i;
+ for (i=0; i<3; i++)
+ {
+ if (strcmp(user_name, credentials[0][i]) == 0)
+ {
+ password = pstrdup(pg_be_scram_build_secret(credentials[1][i]));
+ }
+ }
+
+ return password;
+}
+
+static const char *TestAuthenticationError(Port *port)
+{
+ char *error_message = (char *)palloc (100);
+ sprintf(error_message, "Test authentication failed for user %s", port->user_name);
+ return error_message;
+}
+
+void
+_PG_init(void)
+{
+ RegisterAuthProvider("test", TestAuthenticationCheck, TestAuthenticationError);
+}
--
2.25.1
v4-0003-Add-tests-for-test_auth_provider-extension.patch (text/x-patch, 6.1K) download
From 24702486bfaca691d6ca9388544fb23e6d765055 Mon Sep 17 00:00:00 2001
From: Samay Sharma <smilingsamay@gmail.com>
Date: Wed, 16 Feb 2022 12:28:36 -0800
Subject: [PATCH v4 03/10] Add tests for test_auth_provider extension
Add tap tests for test_auth_provider extension allow make check in
src/test/modules to run them.
---
src/test/modules/Makefile | 1 +
src/test/modules/test_auth_provider/Makefile | 2 +
.../test_auth_provider/t/001_custom_auth.pl | 125 ++++++++++++++++++
3 files changed, 128 insertions(+)
create mode 100644 src/test/modules/test_auth_provider/t/001_custom_auth.pl
diff --git a/src/test/modules/Makefile b/src/test/modules/Makefile
index 9090226daa..d0d461ef9e 100644
--- a/src/test/modules/Makefile
+++ b/src/test/modules/Makefile
@@ -14,6 +14,7 @@ SUBDIRS = \
plsample \
snapshot_too_old \
spgist_name_ops \
+ test_auth_provider \
test_bloomfilter \
test_ddl_deparse \
test_extensions \
diff --git a/src/test/modules/test_auth_provider/Makefile b/src/test/modules/test_auth_provider/Makefile
index 17971a5c7a..7d601cf7d5 100644
--- a/src/test/modules/test_auth_provider/Makefile
+++ b/src/test/modules/test_auth_provider/Makefile
@@ -4,6 +4,8 @@ MODULE_big = test_auth_provider
OBJS = test_auth_provider.o
PGFILEDESC = "test_auth_provider - provider to test auth hooks"
+TAP_TESTS = 1
+
ifdef USE_PGXS
PG_CONFIG = pg_config
PGXS := $(shell $(PG_CONFIG) --pgxs)
diff --git a/src/test/modules/test_auth_provider/t/001_custom_auth.pl b/src/test/modules/test_auth_provider/t/001_custom_auth.pl
new file mode 100644
index 0000000000..3b7472dc7f
--- /dev/null
+++ b/src/test/modules/test_auth_provider/t/001_custom_auth.pl
@@ -0,0 +1,125 @@
+
+# Copyright (c) 2021-2022, PostgreSQL Global Development Group
+
+# Set of tests for testing custom authentication hooks.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+# Delete pg_hba.conf from the given node, add a new entry to it
+# and then execute a reload to refresh it.
+sub reset_pg_hba
+{
+ my $node = shift;
+ my $hba_method = shift;
+
+ unlink($node->data_dir . '/pg_hba.conf');
+ # just for testing purposes, use a continuation line
+ $node->append_conf('pg_hba.conf', "local all all\\\n $hba_method");
+ $node->reload;
+ return;
+}
+
+# Test if you get expected results in pg_hba_file_rules error column after
+# changing pg_hba.conf and reloading it.
+sub test_hba_reload
+{
+ my ($node, $method, $expected_res) = @_;
+ my $status_string = 'failed';
+ $status_string = 'success' if ($expected_res eq 0);
+ my $testname = "pg_hba.conf reload $status_string for method $method";
+
+ reset_pg_hba($node, $method);
+
+ my ($cmdret, $stdout, $stderr) = $node->psql("postgres",
+ "select count(*) from pg_hba_file_rules where error is not null",extra_params => ['-U','bob']);
+
+ is($stdout, $expected_res, $testname);
+}
+
+# Test access for a single role, useful to wrap all tests into one. Extra
+# named parameters are passed to connect_ok/fails as-is.
+sub test_role
+{
+ local $Test::Builder::Level = $Test::Builder::Level + 1;
+
+ my ($node, $role, $method, $expected_res, %params) = @_;
+ my $status_string = 'failed';
+ $status_string = 'success' if ($expected_res eq 0);
+
+ my $connstr = "user=$role";
+ my $testname =
+ "authentication $status_string for method $method, role $role";
+
+ if ($expected_res eq 0)
+ {
+ $node->connect_ok($connstr, $testname, %params);
+ }
+ else
+ {
+ # No checks of the error message, only the status code.
+ $node->connect_fails($connstr, $testname, %params);
+ }
+}
+
+# Initialize server node
+my $node = PostgreSQL::Test::Cluster->new('server');
+$node->init;
+$node->append_conf('postgresql.conf', "log_connections = on\n");
+$node->append_conf('postgresql.conf', "shared_preload_libraries = 'test_auth_provider.so'\n");
+$node->start;
+
+$node->safe_psql('postgres', "CREATE ROLE bob SUPERUSER LOGIN;");
+$node->safe_psql('postgres', "CREATE ROLE alice LOGIN;");
+$node->safe_psql('postgres', "CREATE ROLE test LOGIN;");
+
+# Add custom auth method to pg_hba.conf
+reset_pg_hba($node, 'custom provider=test');
+
+# Test that users are able to login with correct passwords.
+$ENV{"PGPASSWORD"} = 'bob123';
+test_role($node, 'bob', 'custom', 0, log_like => [qr/connection authorized: user=bob/]);
+$ENV{"PGPASSWORD"} = 'alice123';
+test_role($node, 'alice', 'custom', 0, log_like => [qr/connection authorized: user=alice/]);
+
+# Test that bad passwords are rejected.
+$ENV{"PGPASSWORD"} = 'badpassword';
+test_role($node, 'bob', 'custom', 2, log_unlike => [qr/connection authorized:/]);
+test_role($node, 'alice', 'custom', 2, log_unlike => [qr/connection authorized:/]);
+
+# Test that users not in authentication list are rejected.
+test_role($node, 'test', 'custom', 2, log_unlike => [qr/connection authorized:/]);
+
+$ENV{"PGPASSWORD"} = 'bob123';
+
+# Tests for invalid auth options
+
+# Test that an incorrect provider name is not accepted.
+test_hba_reload($node, 'custom provider=wrong', 1);
+
+# Test that specifying provider option with different auth method is not allowed.
+test_hba_reload($node, 'trust provider=test', 1);
+
+# Test that provider name is a mandatory option for custom auth.
+test_hba_reload($node, 'custom', 1);
+
+# Test that correct provider name allows reload to succeed.
+test_hba_reload($node, 'custom provider=test', 0);
+
+# Custom auth modules require mentioning extension in shared_preload_libraries.
+
+# Remove extension from shared_preload_libraries and try to restart.
+$node->adjust_conf('postgresql.conf', 'shared_preload_libraries', "''");
+command_fails(['pg_ctl', '-w', '-D', $node->data_dir, '-l', $node->logfile, 'restart'],'restart with empty shared_preload_libraries failed');
+
+# Fix shared_preload_libraries and confirm that you can now restart.
+$node->adjust_conf('postgresql.conf', 'shared_preload_libraries', "'test_auth_provider.so'");
+command_ok(['pg_ctl', '-w', '-D', $node->data_dir, '-l', $node->logfile,'start'],'restart with correct shared_preload_libraries succeeded');
+
+# Test that we can connect again
+test_role($node, 'bob', 'custom', 0, log_like => [qr/connection authorized: user=bob/]);
+
+done_testing();
--
2.25.1
v4-0004-Add-support-for-map-and-custom-auth-options.patch (text/x-patch, 11.7K) download
From c30970a354b23f26eaf3e1db7c7d7759f2f828b3 Mon Sep 17 00:00:00 2001
From: Samay Sharma <smilingsamay@gmail.com>
Date: Mon, 14 Mar 2022 14:54:08 -0700
Subject: [PATCH v4 04/10] Add support for "map" and custom auth options
This commit allows extensions to now specify, validate and use
custom options for their custom auth methods. This is done by
exposing a validation function hook which can be defined by
extensions. The valid options are then stored as key / value
pairs which can be used while checking authentication. We also
allow custom auth providers to use the "map" option to use
usermaps.
The test module was updated to use custom options and new tests
were added.
---
src/backend/libpq/auth.c | 4 +-
src/backend/libpq/hba.c | 76 +++++++++++++++----
src/include/libpq/auth.h | 17 +++--
src/include/libpq/hba.h | 8 ++
.../test_auth_provider/t/001_custom_auth.pl | 22 ++++++
.../test_auth_provider/test_auth_provider.c | 50 +++++++++++-
6 files changed, 157 insertions(+), 20 deletions(-)
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 375ee33892..4a8a63922a 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -3364,7 +3364,8 @@ PerformRadiusTransaction(const char *server, const char *secret, const char *por
*/
void RegisterAuthProvider(const char *provider_name,
CustomAuthenticationCheck_hook_type AuthenticationCheckFunction,
- CustomAuthenticationError_hook_type AuthenticationErrorFunction)
+ CustomAuthenticationError_hook_type AuthenticationErrorFunction,
+ CustomAuthenticationValidateOptions_hook_type AuthenticationOptionsFunction)
{
CustomAuthProvider *provider = NULL;
MemoryContext old_context;
@@ -3390,6 +3391,7 @@ void RegisterAuthProvider(const char *provider_name,
provider->name = MemoryContextStrdup(TopMemoryContext,provider_name);
provider->auth_check_hook = AuthenticationCheckFunction;
provider->auth_error_hook = AuthenticationErrorFunction;
+ provider->auth_options_hook = AuthenticationOptionsFunction;
custom_auth_providers = lappend(custom_auth_providers, provider);
MemoryContextSwitchTo(old_context);
}
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index 9f15252789..42cb1ce51d 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -1729,8 +1729,9 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->auth_method != uaPeer &&
hbaline->auth_method != uaGSS &&
hbaline->auth_method != uaSSPI &&
- hbaline->auth_method != uaCert)
- INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, and cert"));
+ hbaline->auth_method != uaCert &&
+ hbaline->auth_method != uaCustom)
+ INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, cert and custom"));
hbaline->usermap = pstrdup(val);
}
else if (strcmp(name, "clientcert") == 0)
@@ -2121,7 +2122,6 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
/*
* Verify that the provider mentioned is loaded via shared_preload_libraries.
*/
-
if (get_provider_by_name(val) == NULL)
{
ereport(elevel,
@@ -2129,7 +2129,7 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
errmsg("cannot use authentication provider %s",val),
errhint("Load authentication provider via shared_preload_libraries."),
errcontext("line %d of configuration file \"%s\"",
- line_num, HbaFileName)));
+ line_num, HbaFileName)));
*err_msg = psprintf("cannot use authentication provider %s", val);
return false;
@@ -2141,15 +2141,55 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
}
else
{
- ereport(elevel,
- (errcode(ERRCODE_CONFIG_FILE_ERROR),
- errmsg("unrecognized authentication option name: \"%s\"",
- name),
- errcontext("line %d of configuration file \"%s\"",
- line_num, HbaFileName)));
- *err_msg = psprintf("unrecognized authentication option name: \"%s\"",
- name);
- return false;
+ /*
+ * Allow custom providers to validate their options if they have an
+ * option validation function defined.
+ */
+ if (hbaline->auth_method == uaCustom && (hbaline->custom_provider != NULL))
+ {
+ bool valid_option = false;
+ CustomAuthProvider *provider = get_provider_by_name(hbaline->custom_provider);
+ if (provider->auth_options_hook)
+ {
+ valid_option = provider->auth_options_hook(name, val, hbaline, err_msg);
+ if (valid_option)
+ {
+ CustomOption *option = palloc(sizeof(CustomOption));
+ option->name = pstrdup(name);
+ option->value = pstrdup(val);
+ hbaline->custom_auth_options = lappend(hbaline->custom_auth_options,
+ option);
+ }
+ }
+ else
+ {
+ *err_msg = psprintf("unrecognized authentication option name: \"%s\"",
+ name);
+ }
+
+ /* Report the error returned by the provider as it is */
+ if (!valid_option)
+ {
+ ereport(elevel,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ errmsg("%s", *err_msg),
+ errcontext("line %d of configuration file \"%s\"",
+ line_num, HbaFileName)));
+ return false;
+ }
+ }
+ else
+ {
+ ereport(elevel,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ errmsg("unrecognized authentication option name: \"%s\"",
+ name),
+ errcontext("line %d of configuration file \"%s\"",
+ line_num, HbaFileName)));
+ *err_msg = psprintf("unrecognized authentication option name: \"%s\"",
+ name);
+ return false;
+ }
}
return true;
}
@@ -2484,6 +2524,16 @@ gethba_options(HbaLine *hba)
if (hba->custom_provider)
options[noptions++] =
CStringGetTextDatum(psprintf("provider=%s",hba->custom_provider));
+ if (hba->custom_auth_options)
+ {
+ ListCell *lc;
+ foreach(lc, hba->custom_auth_options)
+ {
+ CustomOption *option = (CustomOption *)lfirst(lc);
+ options[noptions++] =
+ CStringGetTextDatum(psprintf("%s=%s",option->name, option->value));
+ }
+ }
}
/* If you add more options, consider increasing MAX_HBA_OPTIONS. */
diff --git a/src/include/libpq/auth.h b/src/include/libpq/auth.h
index 7aff98d919..cbdc63b4df 100644
--- a/src/include/libpq/auth.h
+++ b/src/include/libpq/auth.h
@@ -31,22 +31,29 @@ typedef int (*CustomAuthenticationCheck_hook_type) (Port *);
typedef void (*ClientAuthentication_hook_type) (Port *, int);
extern PGDLLIMPORT ClientAuthentication_hook_type ClientAuthentication_hook;
+/* Declarations for custom authentication providers */
+
/* Hook for plugins to report error messages in auth_failed() */
typedef const char * (*CustomAuthenticationError_hook_type) (Port *);
-extern void RegisterAuthProvider
- (const char *provider_name,
- CustomAuthenticationCheck_hook_type CustomAuthenticationCheck_hook,
- CustomAuthenticationError_hook_type CustomAuthenticationError_hook);
+/* Hook for plugins to validate custom authentication options */
+typedef bool (*CustomAuthenticationValidateOptions_hook_type)
+ (char *, char *, HbaLine *, char **);
-/* Declarations for custom authentication providers */
typedef struct CustomAuthProvider
{
const char *name;
CustomAuthenticationCheck_hook_type auth_check_hook;
CustomAuthenticationError_hook_type auth_error_hook;
+ CustomAuthenticationValidateOptions_hook_type auth_options_hook;
} CustomAuthProvider;
+extern void RegisterAuthProvider
+ (const char *provider_name,
+ CustomAuthenticationCheck_hook_type CustomAuthenticationCheck_hook,
+ CustomAuthenticationError_hook_type CustomAuthenticationError_hook,
+ CustomAuthenticationValidateOptions_hook_type CustomAuthenticationOptions_hook);
+
extern CustomAuthProvider *get_provider_by_name(const char *name);
/*
diff --git a/src/include/libpq/hba.h b/src/include/libpq/hba.h
index 48490c44ed..31a00c4b71 100644
--- a/src/include/libpq/hba.h
+++ b/src/include/libpq/hba.h
@@ -78,6 +78,13 @@ typedef enum ClientCertName
clientCertDN
} ClientCertName;
+/* Struct for custom options defined by custom auth plugins */
+typedef struct CustomOption
+{
+ char *name;
+ char *value;
+}CustomOption;
+
typedef struct HbaLine
{
int linenumber;
@@ -122,6 +129,7 @@ typedef struct HbaLine
List *radiusports;
char *radiusports_s;
char *custom_provider;
+ List *custom_auth_options;
} HbaLine;
typedef struct IdentLine
diff --git a/src/test/modules/test_auth_provider/t/001_custom_auth.pl b/src/test/modules/test_auth_provider/t/001_custom_auth.pl
index 3b7472dc7f..e964c2f723 100644
--- a/src/test/modules/test_auth_provider/t/001_custom_auth.pl
+++ b/src/test/modules/test_auth_provider/t/001_custom_auth.pl
@@ -109,6 +109,28 @@ test_hba_reload($node, 'custom', 1);
# Test that correct provider name allows reload to succeed.
test_hba_reload($node, 'custom provider=test', 0);
+# Tests for custom auth options
+
+# Test that a custom option doesn't work without a provider.
+test_hba_reload($node, 'custom allow=bob', 1);
+
+# Test that options other than allowed ones are not accepted.
+test_hba_reload($node, 'custom provider=test wrong=true', 1);
+
+# Test that only valid values are accepted for allowed options.
+test_hba_reload($node, 'custom provider=test allow=wrong', 1);
+
+# Test that setting allow option for a user doesn't look at the password.
+test_hba_reload($node, 'custom provider=test allow=bob', 0);
+$ENV{"PGPASSWORD"} = 'bad123';
+test_role($node, 'bob', 'custom', 0, log_like => [qr/connection authorized: user=bob/]);
+
+# Password is still checked for other users.
+test_role($node, 'alice', 'custom', 2, log_unlike => [qr/connection authorized:/]);
+
+# Reset the password for future tests.
+$ENV{"PGPASSWORD"} = 'bob123';
+
# Custom auth modules require mentioning extension in shared_preload_libraries.
# Remove extension from shared_preload_libraries and try to restart.
diff --git a/src/test/modules/test_auth_provider/test_auth_provider.c b/src/test/modules/test_auth_provider/test_auth_provider.c
index 7c4b1f3500..5ac425f5b6 100644
--- a/src/test/modules/test_auth_provider/test_auth_provider.c
+++ b/src/test/modules/test_auth_provider/test_auth_provider.c
@@ -39,7 +39,27 @@ static int TestAuthenticationCheck(Port *port)
int result = STATUS_ERROR;
char *real_pass;
const char *logdetail = NULL;
+ ListCell *lc;
+ /*
+ * If user's name is in the the "allow" list, do not request password
+ * for them and allow them to authenticate.
+ */
+ foreach(lc,port->hba->custom_auth_options)
+ {
+ CustomOption *option = (CustomOption *) lfirst(lc);
+ if (strcmp(option->name, "allow") == 0 &&
+ strcmp(option->value, port->user_name) == 0)
+ {
+ set_authn_id(port, port->user_name);
+ return STATUS_OK;
+ }
+ }
+
+ /*
+ * Encrypt the password and validate that it's the same as the one
+ * returned by the client.
+ */
real_pass = get_encrypted_password_for_user(port->user_name);
if (real_pass)
{
@@ -79,8 +99,36 @@ static const char *TestAuthenticationError(Port *port)
return error_message;
}
+/*
+ * Returns if the options passed are supported by the extension
+ * and are valid. Currently only "allow" is supported.
+ */
+static bool TestAuthenticationOptions(char *name, char *val, HbaLine *hbaline, char **err_msg)
+{
+ /* Validate that an actual user is in the "allow" list. */
+ if (strcmp(name,"allow") == 0)
+ {
+ for (int i=0;i<3;i++)
+ {
+ if (strcmp(val,credentials[0][i]) == 0)
+ {
+ return true;
+ }
+ }
+
+ *err_msg = psprintf("\"%s\" is not valid value for option \"%s\"", val, name);
+ return false;
+ }
+ else
+ {
+ *err_msg = psprintf("option \"%s\" not recognized by \"%s\" provider", val, hbaline->custom_provider);
+ return false;
+ }
+}
+
void
_PG_init(void)
{
- RegisterAuthProvider("test", TestAuthenticationCheck, TestAuthenticationError);
+ RegisterAuthProvider("test", TestAuthenticationCheck,
+ TestAuthenticationError,TestAuthenticationOptions);
}
--
2.25.1
v4-0005-common-jsonapi-support-FRONTEND-clients.patch (text/x-patch, 20.4K) download
From 0ca324b52c94760e799974e5661fe29c3912d2d8 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Mon, 3 May 2021 15:38:26 -0700
Subject: [PATCH v4 05/10] common/jsonapi: support FRONTEND clients
Based on a patch by Michael Paquier.
For frontend code, use PQExpBuffer instead of StringInfo. This requires
us to track allocation failures so that we can return JSON_OUT_OF_MEMORY
as needed. json_errdetail() now allocates its error message inside
memory owned by the JsonLexContext, so clients don't need to worry about
freeing it.
For convenience, the backend now has destroyJsonLexContext() to mirror
other create/destroy APIs. The frontend has init/term versions of the
API to handle stack-allocated JsonLexContexts.
We can now partially revert b44669b2ca, now that json_errdetail() works
correctly.
---
src/backend/utils/adt/jsonfuncs.c | 4 +-
src/bin/pg_verifybackup/parse_manifest.c | 13 +-
src/bin/pg_verifybackup/t/005_bad_manifest.pl | 2 +-
src/common/Makefile | 2 +-
src/common/jsonapi.c | 290 +++++++++++++-----
src/include/common/jsonapi.h | 47 ++-
6 files changed, 270 insertions(+), 88 deletions(-)
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index 29664aa6e4..7d32a99d8c 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -723,9 +723,7 @@ json_object_keys(PG_FUNCTION_ARGS)
pg_parse_json_or_ereport(lex, sem);
/* keys are now in state->result */
- pfree(lex->strval->data);
- pfree(lex->strval);
- pfree(lex);
+ destroyJsonLexContext(lex);
pfree(sem);
MemoryContextSwitchTo(oldcontext);
diff --git a/src/bin/pg_verifybackup/parse_manifest.c b/src/bin/pg_verifybackup/parse_manifest.c
index 6364b01282..4b38fd3963 100644
--- a/src/bin/pg_verifybackup/parse_manifest.c
+++ b/src/bin/pg_verifybackup/parse_manifest.c
@@ -119,7 +119,7 @@ void
json_parse_manifest(JsonManifestParseContext *context, char *buffer,
size_t size)
{
- JsonLexContext *lex;
+ JsonLexContext lex = {0};
JsonParseErrorType json_error;
JsonSemAction sem;
JsonManifestParseState parse;
@@ -129,8 +129,8 @@ json_parse_manifest(JsonManifestParseContext *context, char *buffer,
parse.state = JM_EXPECT_TOPLEVEL_START;
parse.saw_version_field = false;
- /* Create a JSON lexing context. */
- lex = makeJsonLexContextCstringLen(buffer, size, PG_UTF8, true);
+ /* Initialize a JSON lexing context. */
+ initJsonLexContextCstringLen(&lex, buffer, size, PG_UTF8, true);
/* Set up semantic actions. */
sem.semstate = &parse;
@@ -145,14 +145,17 @@ json_parse_manifest(JsonManifestParseContext *context, char *buffer,
sem.scalar = json_manifest_scalar;
/* Run the actual JSON parser. */
- json_error = pg_parse_json(lex, &sem);
+ json_error = pg_parse_json(&lex, &sem);
if (json_error != JSON_SUCCESS)
- json_manifest_parse_failure(context, "parsing failed");
+ json_manifest_parse_failure(context, json_errdetail(json_error, &lex));
if (parse.state != JM_EXPECT_EOF)
json_manifest_parse_failure(context, "manifest ended unexpectedly");
/* Verify the manifest checksum. */
verify_manifest_checksum(&parse, buffer, size);
+
+ /* Clean up. */
+ termJsonLexContext(&lex);
}
/*
diff --git a/src/bin/pg_verifybackup/t/005_bad_manifest.pl b/src/bin/pg_verifybackup/t/005_bad_manifest.pl
index 118beb53d7..f2692972fe 100644
--- a/src/bin/pg_verifybackup/t/005_bad_manifest.pl
+++ b/src/bin/pg_verifybackup/t/005_bad_manifest.pl
@@ -16,7 +16,7 @@ my $tempdir = PostgreSQL::Test::Utils::tempdir;
test_bad_manifest(
'input string ended unexpectedly',
- qr/could not parse backup manifest: parsing failed/,
+ qr/could not parse backup manifest: The input string ended unexpectedly/,
<<EOM);
{
EOM
diff --git a/src/common/Makefile b/src/common/Makefile
index f627349835..694da03658 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -40,7 +40,7 @@ override CPPFLAGS += -DVAL_LDFLAGS_EX="\"$(LDFLAGS_EX)\""
override CPPFLAGS += -DVAL_LDFLAGS_SL="\"$(LDFLAGS_SL)\""
override CPPFLAGS += -DVAL_LIBS="\"$(LIBS)\""
-override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common $(CPPFLAGS)
+override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common -I$(libpq_srcdir) $(CPPFLAGS)
LIBS += $(PTHREAD_LIBS)
# If you add objects here, see also src/tools/msvc/Mkvcbuild.pm
diff --git a/src/common/jsonapi.c b/src/common/jsonapi.c
index 6666077a93..7fc5eaf460 100644
--- a/src/common/jsonapi.c
+++ b/src/common/jsonapi.c
@@ -20,10 +20,39 @@
#include "common/jsonapi.h"
#include "mb/pg_wchar.h"
-#ifndef FRONTEND
+#ifdef FRONTEND
+#include "pqexpbuffer.h"
+#else
+#include "lib/stringinfo.h"
#include "miscadmin.h"
#endif
+/*
+ * In backend, we will use palloc/pfree along with StringInfo. In frontend, use
+ * malloc and PQExpBuffer, and return JSON_OUT_OF_MEMORY on out-of-memory.
+ */
+#ifdef FRONTEND
+
+#define STRDUP(s) strdup(s)
+#define ALLOC(size) malloc(size)
+
+#define appendStrVal appendPQExpBuffer
+#define appendStrValChar appendPQExpBufferChar
+#define createStrVal createPQExpBuffer
+#define resetStrVal resetPQExpBuffer
+
+#else /* !FRONTEND */
+
+#define STRDUP(s) pstrdup(s)
+#define ALLOC(size) palloc(size)
+
+#define appendStrVal appendStringInfo
+#define appendStrValChar appendStringInfoChar
+#define createStrVal makeStringInfo
+#define resetStrVal resetStringInfo
+
+#endif
+
/*
* The context of the parser is maintained by the recursive descent
* mechanism, but is passed explicitly to the error reporting routine
@@ -132,10 +161,12 @@ IsValidJsonNumber(const char *str, int len)
return (!numeric_error) && (total_len == dummy_lex.input_length);
}
+#ifndef FRONTEND
+
/*
* makeJsonLexContextCstringLen
*
- * lex constructor, with or without StringInfo object for de-escaped lexemes.
+ * lex constructor, with or without a string object for de-escaped lexemes.
*
* Without is better as it makes the processing faster, so only make one
* if really required.
@@ -145,13 +176,66 @@ makeJsonLexContextCstringLen(char *json, int len, int encoding, bool need_escape
{
JsonLexContext *lex = palloc0(sizeof(JsonLexContext));
+ initJsonLexContextCstringLen(lex, json, len, encoding, need_escapes);
+
+ return lex;
+}
+
+void
+destroyJsonLexContext(JsonLexContext *lex)
+{
+ termJsonLexContext(lex);
+ pfree(lex);
+}
+
+#endif /* !FRONTEND */
+
+void
+initJsonLexContextCstringLen(JsonLexContext *lex, char *json, int len, int encoding, bool need_escapes)
+{
lex->input = lex->token_terminator = lex->line_start = json;
lex->line_number = 1;
lex->input_length = len;
lex->input_encoding = encoding;
- if (need_escapes)
- lex->strval = makeStringInfo();
- return lex;
+ lex->parse_strval = need_escapes;
+ if (lex->parse_strval)
+ {
+ /*
+ * This call can fail in FRONTEND code. We defer error handling to time
+ * of use (json_lex_string()) since there's no way to signal failure
+ * here, and we might not need to parse any strings anyway.
+ */
+ lex->strval = createStrVal();
+ }
+ lex->errormsg = NULL;
+}
+
+void
+termJsonLexContext(JsonLexContext *lex)
+{
+ static const JsonLexContext empty = {0};
+
+ if (lex->strval)
+ {
+#ifdef FRONTEND
+ destroyPQExpBuffer(lex->strval);
+#else
+ pfree(lex->strval->data);
+ pfree(lex->strval);
+#endif
+ }
+
+ if (lex->errormsg)
+ {
+#ifdef FRONTEND
+ destroyPQExpBuffer(lex->errormsg);
+#else
+ pfree(lex->errormsg->data);
+ pfree(lex->errormsg);
+#endif
+ }
+
+ *lex = empty;
}
/*
@@ -217,7 +301,7 @@ json_count_array_elements(JsonLexContext *lex, int *elements)
* etc, so doing this with a copy makes that safe.
*/
memcpy(©lex, lex, sizeof(JsonLexContext));
- copylex.strval = NULL; /* not interested in values here */
+ copylex.parse_strval = false; /* not interested in values here */
copylex.lex_level++;
count = 0;
@@ -279,14 +363,21 @@ parse_scalar(JsonLexContext *lex, JsonSemAction *sem)
/* extract the de-escaped string value, or the raw lexeme */
if (lex_peek(lex) == JSON_TOKEN_STRING)
{
- if (lex->strval != NULL)
- val = pstrdup(lex->strval->data);
+ if (lex->parse_strval)
+ {
+ val = STRDUP(lex->strval->data);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
}
else
{
int len = (lex->token_terminator - lex->token_start);
- val = palloc(len + 1);
+ val = ALLOC(len + 1);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+
memcpy(val, lex->token_start, len);
val[len] = '\0';
}
@@ -320,8 +411,12 @@ parse_object_field(JsonLexContext *lex, JsonSemAction *sem)
if (lex_peek(lex) != JSON_TOKEN_STRING)
return report_parse_error(JSON_PARSE_STRING, lex);
- if ((ostart != NULL || oend != NULL) && lex->strval != NULL)
- fname = pstrdup(lex->strval->data);
+ if ((ostart != NULL || oend != NULL) && lex->parse_strval)
+ {
+ fname = STRDUP(lex->strval->data);
+ if (fname == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
result = json_lex(lex);
if (result != JSON_SUCCESS)
return result;
@@ -368,6 +463,10 @@ parse_object(JsonLexContext *lex, JsonSemAction *sem)
JsonParseErrorType result;
#ifndef FRONTEND
+ /*
+ * TODO: clients need some way to put a bound on stack growth. Parse level
+ * limits maybe?
+ */
check_stack_depth();
#endif
@@ -676,8 +775,15 @@ json_lex_string(JsonLexContext *lex)
int len;
int hi_surrogate = -1;
- if (lex->strval != NULL)
- resetStringInfo(lex->strval);
+ if (lex->parse_strval)
+ {
+#ifdef FRONTEND
+ /* make sure initialization succeeded */
+ if (lex->strval == NULL)
+ return JSON_OUT_OF_MEMORY;
+#endif
+ resetStrVal(lex->strval);
+ }
Assert(lex->input_length > 0);
s = lex->token_start;
@@ -737,7 +843,7 @@ json_lex_string(JsonLexContext *lex)
return JSON_UNICODE_ESCAPE_FORMAT;
}
}
- if (lex->strval != NULL)
+ if (lex->parse_strval)
{
/*
* Combine surrogate pairs.
@@ -797,19 +903,19 @@ json_lex_string(JsonLexContext *lex)
unicode_to_utf8(ch, (unsigned char *) utf8str);
utf8len = pg_utf_mblen((unsigned char *) utf8str);
- appendBinaryStringInfo(lex->strval, utf8str, utf8len);
+ appendBinaryPQExpBuffer(lex->strval, utf8str, utf8len);
}
else if (ch <= 0x007f)
{
/* The ASCII range is the same in all encodings */
- appendStringInfoChar(lex->strval, (char) ch);
+ appendPQExpBufferChar(lex->strval, (char) ch);
}
else
return JSON_UNICODE_HIGH_ESCAPE;
#endif /* FRONTEND */
}
}
- else if (lex->strval != NULL)
+ else if (lex->parse_strval)
{
if (hi_surrogate != -1)
return JSON_UNICODE_LOW_SURROGATE;
@@ -819,22 +925,22 @@ json_lex_string(JsonLexContext *lex)
case '"':
case '\\':
case '/':
- appendStringInfoChar(lex->strval, *s);
+ appendStrValChar(lex->strval, *s);
break;
case 'b':
- appendStringInfoChar(lex->strval, '\b');
+ appendStrValChar(lex->strval, '\b');
break;
case 'f':
- appendStringInfoChar(lex->strval, '\f');
+ appendStrValChar(lex->strval, '\f');
break;
case 'n':
- appendStringInfoChar(lex->strval, '\n');
+ appendStrValChar(lex->strval, '\n');
break;
case 'r':
- appendStringInfoChar(lex->strval, '\r');
+ appendStrValChar(lex->strval, '\r');
break;
case 't':
- appendStringInfoChar(lex->strval, '\t');
+ appendStrValChar(lex->strval, '\t');
break;
default:
/* Not a valid string escape, so signal error. */
@@ -858,12 +964,12 @@ json_lex_string(JsonLexContext *lex)
}
}
- else if (lex->strval != NULL)
+ else if (lex->parse_strval)
{
if (hi_surrogate != -1)
return JSON_UNICODE_LOW_SURROGATE;
- appendStringInfoChar(lex->strval, *s);
+ appendStrValChar(lex->strval, *s);
}
}
@@ -871,6 +977,11 @@ json_lex_string(JsonLexContext *lex)
if (hi_surrogate != -1)
return JSON_UNICODE_LOW_SURROGATE;
+#ifdef FRONTEND
+ if (lex->parse_strval && PQExpBufferBroken(lex->strval))
+ return JSON_OUT_OF_MEMORY;
+#endif
+
/* Hooray, we found the end of the string! */
lex->prev_token_terminator = lex->token_terminator;
lex->token_terminator = s + 1;
@@ -1043,72 +1154,93 @@ report_parse_error(JsonParseContext ctx, JsonLexContext *lex)
return JSON_SUCCESS; /* silence stupider compilers */
}
-
-#ifndef FRONTEND
-/*
- * Extract the current token from a lexing context, for error reporting.
- */
-static char *
-extract_token(JsonLexContext *lex)
-{
- int toklen = lex->token_terminator - lex->token_start;
- char *token = palloc(toklen + 1);
-
- memcpy(token, lex->token_start, toklen);
- token[toklen] = '\0';
- return token;
-}
-
/*
* Construct a detail message for a JSON error.
*
- * Note that the error message generated by this routine may not be
- * palloc'd, making it unsafe for frontend code as there is no way to
- * know if this can be safery pfree'd or not.
+ * The returned allocation is either static or owned by the JsonLexContext and
+ * should not be freed.
*/
char *
json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
{
+ int toklen = lex->token_terminator - lex->token_start;
+
+ if (error == JSON_OUT_OF_MEMORY)
+ {
+ /* Short circuit. Allocating anything for this case is unhelpful. */
+ return _("out of memory");
+ }
+
+ if (lex->errormsg)
+ resetStrVal(lex->errormsg);
+ else
+ lex->errormsg = createStrVal();
+
switch (error)
{
case JSON_SUCCESS:
/* fall through to the error code after switch */
break;
case JSON_ESCAPING_INVALID:
- return psprintf(_("Escape sequence \"\\%s\" is invalid."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Escape sequence \"\\%.*s\" is invalid."),
+ toklen, lex->token_start);
+ break;
case JSON_ESCAPING_REQUIRED:
- return psprintf(_("Character with value 0x%02x must be escaped."),
- (unsigned char) *(lex->token_terminator));
+ appendStrVal(lex->errormsg,
+ _("Character with value 0x%02x must be escaped."),
+ (unsigned char) *(lex->token_terminator));
+ break;
case JSON_EXPECTED_END:
- return psprintf(_("Expected end of input, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected end of input, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_ARRAY_FIRST:
- return psprintf(_("Expected array element or \"]\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected array element or \"]\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_ARRAY_NEXT:
- return psprintf(_("Expected \",\" or \"]\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \",\" or \"]\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_COLON:
- return psprintf(_("Expected \":\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \":\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_JSON:
- return psprintf(_("Expected JSON value, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected JSON value, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_MORE:
return _("The input string ended unexpectedly.");
case JSON_EXPECTED_OBJECT_FIRST:
- return psprintf(_("Expected string or \"}\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected string or \"}\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_OBJECT_NEXT:
- return psprintf(_("Expected \",\" or \"}\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \",\" or \"}\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_STRING:
- return psprintf(_("Expected string, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected string, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_INVALID_TOKEN:
- return psprintf(_("Token \"%s\" is invalid."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Token \"%.*s\" is invalid."),
+ toklen, lex->token_start);
+ break;
+ case JSON_OUT_OF_MEMORY:
+ /* should have been handled above; use the error path */
+ break;
case JSON_UNICODE_CODE_POINT_ZERO:
return _("\\u0000 cannot be converted to text.");
case JSON_UNICODE_ESCAPE_FORMAT:
@@ -1122,12 +1254,22 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
return _("Unicode low surrogate must follow a high surrogate.");
}
- /*
- * We don't use a default: case, so that the compiler will warn about
- * unhandled enum values. But this needs to be here anyway to cover the
- * possibility of an incorrect input.
- */
- elog(ERROR, "unexpected json parse error type: %d", (int) error);
- return NULL;
-}
+ /* Note that lex->errormsg can be NULL in FRONTEND code. */
+ if (lex->errormsg && !lex->errormsg->data[0])
+ {
+ /*
+ * We don't use a default: case, so that the compiler will warn about
+ * unhandled enum values. But this needs to be here anyway to cover the
+ * possibility of an incorrect input.
+ */
+ appendStrVal(lex->errormsg,
+ "unexpected json parse error type: %d", (int) error);
+ }
+
+#ifdef FRONTEND
+ if (PQExpBufferBroken(lex->errormsg))
+ return _("out of memory while constructing error description");
#endif
+
+ return lex->errormsg->data;
+}
diff --git a/src/include/common/jsonapi.h b/src/include/common/jsonapi.h
index 52cb4a9339..d7cafc84fe 100644
--- a/src/include/common/jsonapi.h
+++ b/src/include/common/jsonapi.h
@@ -14,8 +14,6 @@
#ifndef JSONAPI_H
#define JSONAPI_H
-#include "lib/stringinfo.h"
-
typedef enum
{
JSON_TOKEN_INVALID,
@@ -48,6 +46,7 @@ typedef enum
JSON_EXPECTED_OBJECT_NEXT,
JSON_EXPECTED_STRING,
JSON_INVALID_TOKEN,
+ JSON_OUT_OF_MEMORY,
JSON_UNICODE_CODE_POINT_ZERO,
JSON_UNICODE_ESCAPE_FORMAT,
JSON_UNICODE_HIGH_ESCAPE,
@@ -55,6 +54,17 @@ typedef enum
JSON_UNICODE_LOW_SURROGATE
} JsonParseErrorType;
+/*
+ * Don't depend on the internal type header for strval; if callers need access
+ * then they can include the appropriate header themselves.
+ */
+#ifdef FRONTEND
+#define StrValType PQExpBufferData
+#else
+#define StrValType StringInfoData
+#endif
+
+typedef struct StrValType StrValType;
/*
* All the fields in this structure should be treated as read-only.
@@ -81,7 +91,9 @@ typedef struct JsonLexContext
int lex_level;
int line_number; /* line number, starting from 1 */
char *line_start; /* where that line starts within input */
- StringInfo strval;
+ bool parse_strval;
+ StrValType *strval; /* only used if parse_strval == true */
+ StrValType *errormsg;
} JsonLexContext;
typedef void (*json_struct_action) (void *state);
@@ -141,9 +153,10 @@ extern JsonSemAction nullSemAction;
*/
extern JsonParseErrorType json_count_array_elements(JsonLexContext *lex,
int *elements);
+#ifndef FRONTEND
/*
- * constructor for JsonLexContext, with or without strval element.
+ * allocating constructor for JsonLexContext, with or without strval element.
* If supplied, the strval element will contain a de-escaped version of
* the lexeme. However, doing this imposes a performance penalty, so
* it should be avoided if the de-escaped lexeme is not required.
@@ -153,6 +166,32 @@ extern JsonLexContext *makeJsonLexContextCstringLen(char *json,
int encoding,
bool need_escapes);
+/*
+ * Counterpart to makeJsonLexContextCstringLen(): clears and deallocates lex.
+ * The context pointer should not be used after this call.
+ */
+extern void destroyJsonLexContext(JsonLexContext *lex);
+
+#endif /* !FRONTEND */
+
+/*
+ * stack constructor for JsonLexContext, with or without strval element.
+ * If supplied, the strval element will contain a de-escaped version of
+ * the lexeme. However, doing this imposes a performance penalty, so
+ * it should be avoided if the de-escaped lexeme is not required.
+ */
+extern void initJsonLexContextCstringLen(JsonLexContext *lex,
+ char *json,
+ int len,
+ int encoding,
+ bool need_escapes);
+
+/*
+ * Counterpart to initJsonLexContextCstringLen(): clears the contents of lex,
+ * but does not deallocate lex itself.
+ */
+extern void termJsonLexContext(JsonLexContext *lex);
+
/* lex one token */
extern JsonParseErrorType json_lex(JsonLexContext *lex);
--
2.25.1
v4-0006-libpq-add-OAUTHBEARER-SASL-mechanism.patch (text/x-patch, 35.6K) download
From f4752b6daea9b519ff3482094a1f445a4731cc15 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 13 Apr 2021 10:27:27 -0700
Subject: [PATCH v4 06/10] libpq: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) and OAuth 2.0 Device Authorization
Grants (RFC 8628) on the client side. When speaking to a OAuth-enabled
server, it looks a bit like this:
$ psql 'host=example.org oauth_client_id=f02c6361-0635-...'
Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG
The OAuth issuer must support device authorization. No other OAuth flows
are currently implemented.
The client implementation requires libiddawc and its development
headers. Configure --with-oauth (and --with-includes/--with-libraries to
point at the iddawc installation, if it's in a custom location).
Several TODOs:
- don't retry forever if the server won't accept our token
- perform several sanity checks on the OAuth issuer's responses
- handle cases where the client has been set up with an issuer and
scope, but the Postgres server wants to use something different
- improve error debuggability during the OAuth handshake
- ...and more.
---
configure | 100 ++++
configure.ac | 19 +
src/Makefile.global.in | 1 +
src/include/common/oauth-common.h | 19 +
src/include/pg_config.h.in | 6 +
src/interfaces/libpq/Makefile | 7 +-
src/interfaces/libpq/fe-auth-oauth.c | 744 +++++++++++++++++++++++++++
src/interfaces/libpq/fe-auth-sasl.h | 5 +-
src/interfaces/libpq/fe-auth-scram.c | 6 +-
src/interfaces/libpq/fe-auth.c | 42 +-
src/interfaces/libpq/fe-auth.h | 3 +
src/interfaces/libpq/fe-connect.c | 38 ++
src/interfaces/libpq/libpq-int.h | 8 +
13 files changed, 979 insertions(+), 19 deletions(-)
create mode 100644 src/include/common/oauth-common.h
create mode 100644 src/interfaces/libpq/fe-auth-oauth.c
diff --git a/configure b/configure
index e066cbe2c8..42a3304681 100755
--- a/configure
+++ b/configure
@@ -718,6 +718,7 @@ with_uuid
with_readline
with_systemd
with_selinux
+with_oauth
with_ldap
with_krb_srvnam
krb_srvtab
@@ -861,6 +862,7 @@ with_krb_srvnam
with_pam
with_bsd_auth
with_ldap
+with_oauth
with_bonjour
with_selinux
with_systemd
@@ -1570,6 +1572,7 @@ Optional Packages:
--with-pam build with PAM support
--with-bsd-auth build with BSD Authentication support
--with-ldap build with LDAP support
+ --with-oauth build with OAuth 2.0 support
--with-bonjour build with Bonjour support
--with-selinux build with SELinux support
--with-systemd build with systemd support
@@ -8377,6 +8380,42 @@ $as_echo "$with_ldap" >&6; }
+#
+# OAuth 2.0
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with OAuth support" >&5
+$as_echo_n "checking whether to build with OAuth support... " >&6; }
+
+
+
+# Check whether --with-oauth was given.
+if test "${with_oauth+set}" = set; then :
+ withval=$with_oauth;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_OAUTH 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-oauth option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_oauth=no
+
+fi
+
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_oauth" >&5
+$as_echo "$with_oauth" >&6; }
+
+
+
#
# Bonjour
#
@@ -13503,6 +13542,56 @@ fi
+if test "$with_oauth" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for i_init_session in -liddawc" >&5
+$as_echo_n "checking for i_init_session in -liddawc... " >&6; }
+if ${ac_cv_lib_iddawc_i_init_session+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-liddawc $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char i_init_session ();
+int
+main ()
+{
+return i_init_session ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_iddawc_i_init_session=yes
+else
+ ac_cv_lib_iddawc_i_init_session=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_iddawc_i_init_session" >&5
+$as_echo "$ac_cv_lib_iddawc_i_init_session" >&6; }
+if test "x$ac_cv_lib_iddawc_i_init_session" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBIDDAWC 1
+_ACEOF
+
+ LIBS="-liddawc $LIBS"
+
+else
+ as_fn_error $? "library 'iddawc' is required for OAuth support" "$LINENO" 5
+fi
+
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for security_compute_create_name in -lselinux" >&5
@@ -14516,6 +14605,17 @@ fi
done
+fi
+
+if test "$with_oauth" != no; then
+ ac_fn_c_check_header_mongrel "$LINENO" "iddawc.h" "ac_cv_header_iddawc_h" "$ac_includes_default"
+if test "x$ac_cv_header_iddawc_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <iddawc.h> is required for OAuth" "$LINENO" 5
+fi
+
+
fi
if test "$PORTNAME" = "win32" ; then
diff --git a/configure.ac b/configure.ac
index 078381e568..4050f91dbd 100644
--- a/configure.ac
+++ b/configure.ac
@@ -887,6 +887,17 @@ AC_MSG_RESULT([$with_ldap])
AC_SUBST(with_ldap)
+#
+# OAuth 2.0
+#
+AC_MSG_CHECKING([whether to build with OAuth support])
+PGAC_ARG_BOOL(with, oauth, no,
+ [build with OAuth 2.0 support],
+ [AC_DEFINE([USE_OAUTH], 1, [Define to 1 to build with OAuth 2.0 support. (--with-oauth)])])
+AC_MSG_RESULT([$with_oauth])
+AC_SUBST(with_oauth)
+
+
#
# Bonjour
#
@@ -1388,6 +1399,10 @@ fi
AC_SUBST(LDAP_LIBS_FE)
AC_SUBST(LDAP_LIBS_BE)
+if test "$with_oauth" = yes ; then
+ AC_CHECK_LIB(iddawc, i_init_session, [], [AC_MSG_ERROR([library 'iddawc' is required for OAuth support])])
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
AC_CHECK_LIB(selinux, security_compute_create_name, [],
@@ -1606,6 +1621,10 @@ elif test "$with_uuid" = ossp ; then
[AC_MSG_ERROR([header file <ossp/uuid.h> or <uuid.h> is required for OSSP UUID])])])
fi
+if test "$with_oauth" != no; then
+ AC_CHECK_HEADER(iddawc.h, [], [AC_MSG_ERROR([header file <iddawc.h> is required for OAuth])])
+fi
+
if test "$PORTNAME" = "win32" ; then
AC_CHECK_HEADERS(crtdefs.h)
fi
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index bbdc1c4bda..c9c61a9c99 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -193,6 +193,7 @@ with_ldap = @with_ldap@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
+with_oauth = @with_oauth@
with_system_tzdata = @with_system_tzdata@
with_uuid = @with_uuid@
with_zlib = @with_zlib@
diff --git a/src/include/common/oauth-common.h b/src/include/common/oauth-common.h
new file mode 100644
index 0000000000..3fa95ac7e8
--- /dev/null
+++ b/src/include/common/oauth-common.h
@@ -0,0 +1,19 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth-common.h
+ * Declarations for helper functions used for OAuth/OIDC authentication
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/common/oauth-common.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef OAUTH_COMMON_H
+#define OAUTH_COMMON_H
+
+/* Name of SASL mechanism per IANA */
+#define OAUTHBEARER_NAME "OAUTHBEARER"
+
+#endif /* OAUTH_COMMON_H */
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 635fbb2181..1b3332601e 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -319,6 +319,9 @@
/* Define to 1 if you have the `crypto' library (-lcrypto). */
#undef HAVE_LIBCRYPTO
+/* Define to 1 if you have the `iddawc' library (-liddawc). */
+#undef HAVE_LIBIDDAWC
+
/* Define to 1 if you have the `ldap' library (-lldap). */
#undef HAVE_LIBLDAP
@@ -922,6 +925,9 @@
/* Define to select named POSIX semaphores. */
#undef USE_NAMED_POSIX_SEMAPHORES
+/* Define to 1 to build with OAuth 2.0 support. (--with-oauth) */
+#undef USE_OAUTH
+
/* Define to 1 to build with OpenSSL support. (--with-ssl=openssl) */
#undef USE_OPENSSL
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 3c53393fa4..727305c578 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -62,6 +62,11 @@ OBJS += \
fe-secure-gssapi.o
endif
+ifeq ($(with_oauth),yes)
+OBJS += \
+ fe-auth-oauth.o
+endif
+
ifeq ($(PORTNAME), cygwin)
override shlib = cyg$(NAME)$(DLSUFFIX)
endif
@@ -83,7 +88,7 @@ endif
# that are built correctly for use in a shlib.
SHLIB_LINK_INTERNAL = -lpgcommon_shlib -lpgport_shlib
ifneq ($(PORTNAME), win32)
-SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
+SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -liddawc -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
else
SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi32 -lssl -lsocket -lnsl -lresolv -lintl -lm $(PTHREAD_LIBS), $(LIBS)) $(LDAP_LIBS_FE)
endif
diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
new file mode 100644
index 0000000000..383c9d4bdb
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth.c
@@ -0,0 +1,744 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth.c
+ * The front-end (client) implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include <iddawc.h>
+
+#include "postgres_fe.h"
+
+#include "common/base64.h"
+#include "common/hmac.h"
+#include "common/jsonapi.h"
+#include "common/oauth-common.h"
+#include "fe-auth.h"
+#include "mb/pg_wchar.h"
+
+/* The exported OAuth callback mechanism. */
+static void *oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism);
+static void oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen,
+ bool *done, bool *success);
+static bool oauth_channel_bound(void *opaq);
+static void oauth_free(void *opaq);
+
+const pg_fe_sasl_mech pg_oauth_mech = {
+ oauth_init,
+ oauth_exchange,
+ oauth_channel_bound,
+ oauth_free,
+};
+
+typedef enum
+{
+ FE_OAUTH_INIT,
+ FE_OAUTH_BEARER_SENT,
+ FE_OAUTH_SERVER_ERROR,
+} fe_oauth_state_enum;
+
+typedef struct
+{
+ fe_oauth_state_enum state;
+
+ PGconn *conn;
+} fe_oauth_state;
+
+static void *
+oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism)
+{
+ fe_oauth_state *state;
+
+ /*
+ * We only support one SASL mechanism here; anything else is programmer
+ * error.
+ */
+ Assert(sasl_mechanism != NULL);
+ Assert(!strcmp(sasl_mechanism, OAUTHBEARER_NAME));
+
+ state = malloc(sizeof(*state));
+ if (!state)
+ return NULL;
+
+ state->state = FE_OAUTH_INIT;
+ state->conn = conn;
+
+ return state;
+}
+
+static const char *
+iddawc_error_string(int errcode)
+{
+ switch (errcode)
+ {
+ case I_OK:
+ return "I_OK";
+
+ case I_ERROR:
+ return "I_ERROR";
+
+ case I_ERROR_PARAM:
+ return "I_ERROR_PARAM";
+
+ case I_ERROR_MEMORY:
+ return "I_ERROR_MEMORY";
+
+ case I_ERROR_UNAUTHORIZED:
+ return "I_ERROR_UNAUTHORIZED";
+
+ case I_ERROR_SERVER:
+ return "I_ERROR_SERVER";
+ }
+
+ return "<unknown>";
+}
+
+static void
+iddawc_error(PGconn *conn, int errcode, const char *msg)
+{
+ appendPQExpBufferStr(&conn->errorMessage, libpq_gettext(msg));
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext(" (iddawc error %s)\n"),
+ iddawc_error_string(errcode));
+}
+
+static void
+iddawc_request_error(PGconn *conn, struct _i_session *i, int err, const char *msg)
+{
+ const char *error_code;
+ const char *desc;
+
+ appendPQExpBuffer(&conn->errorMessage, "%s: ", libpq_gettext(msg));
+
+ error_code = i_get_str_parameter(i, I_OPT_ERROR);
+ if (!error_code)
+ {
+ /*
+ * The server didn't give us any useful information, so just print the
+ * error code.
+ */
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("(iddawc error %s)\n"),
+ iddawc_error_string(err));
+ return;
+ }
+
+ /* If the server gave a string description, print that too. */
+ desc = i_get_str_parameter(i, I_OPT_ERROR_DESCRIPTION);
+ if (desc)
+ appendPQExpBuffer(&conn->errorMessage, "%s ", desc);
+
+ appendPQExpBuffer(&conn->errorMessage, "(%s)\n", error_code);
+}
+
+static char *
+get_auth_token(PGconn *conn)
+{
+ PQExpBuffer token_buf = NULL;
+ struct _i_session session;
+ int err;
+ int auth_method;
+ bool user_prompted = false;
+ const char *verification_uri;
+ const char *user_code;
+ const char *access_token;
+ const char *token_type;
+ char *token = NULL;
+
+ if (!conn->oauth_discovery_uri)
+ return strdup(""); /* ask the server for one */
+
+ if (!conn->oauth_client_id)
+ {
+ /* We can't talk to a server without a client identifier. */
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("no oauth_client_id is set for the connection"));
+ return NULL;
+ }
+
+ i_init_session(&session);
+
+ token_buf = createPQExpBuffer();
+ if (!token_buf)
+ goto cleanup;
+
+ err = i_set_str_parameter(&session, I_OPT_OPENID_CONFIG_ENDPOINT, conn->oauth_discovery_uri);
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set OpenID config endpoint");
+ goto cleanup;
+ }
+
+ err = i_get_openid_config(&session);
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to fetch OpenID discovery document");
+ goto cleanup;
+ }
+
+ if (!i_get_str_parameter(&session, I_OPT_TOKEN_ENDPOINT))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer has no token endpoint"));
+ goto cleanup;
+ }
+
+ if (!i_get_str_parameter(&session, I_OPT_DEVICE_AUTHORIZATION_ENDPOINT))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer does not support device authorization"));
+ goto cleanup;
+ }
+
+ err = i_set_response_type(&session, I_RESPONSE_TYPE_DEVICE_CODE);
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set device code response type");
+ goto cleanup;
+ }
+
+ auth_method = I_TOKEN_AUTH_METHOD_NONE;
+ if (conn->oauth_client_secret && *conn->oauth_client_secret)
+ auth_method = I_TOKEN_AUTH_METHOD_SECRET_BASIC;
+
+ err = i_set_parameter_list(&session,
+ I_OPT_CLIENT_ID, conn->oauth_client_id,
+ I_OPT_CLIENT_SECRET, conn->oauth_client_secret,
+ I_OPT_TOKEN_METHOD, auth_method,
+ I_OPT_SCOPE, conn->oauth_scope,
+ I_OPT_NONE
+ );
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set client identifier");
+ goto cleanup;
+ }
+
+ err = i_run_device_auth_request(&session);
+ if (err)
+ {
+ iddawc_request_error(conn, &session, err,
+ "failed to obtain device authorization");
+ goto cleanup;
+ }
+
+ verification_uri = i_get_str_parameter(&session, I_OPT_DEVICE_AUTH_VERIFICATION_URI);
+ if (!verification_uri)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer did not provide a verification URI"));
+ goto cleanup;
+ }
+
+ user_code = i_get_str_parameter(&session, I_OPT_DEVICE_AUTH_USER_CODE);
+ if (!user_code)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer did not provide a user code"));
+ goto cleanup;
+ }
+
+ /*
+ * Poll the token endpoint until either the user logs in and authorizes the
+ * use of a token, or a hard failure occurs. We perform one ping _before_
+ * prompting the user, so that we don't make them do the work of logging in
+ * only to find that the token endpoint is completely unreachable.
+ */
+ err = i_run_token_request(&session);
+ while (err)
+ {
+ const char *error_code;
+ uint interval;
+
+ error_code = i_get_str_parameter(&session, I_OPT_ERROR);
+
+ /*
+ * authorization_pending and slow_down are the only acceptable errors;
+ * anything else and we bail.
+ */
+ if (!error_code || (strcmp(error_code, "authorization_pending")
+ && strcmp(error_code, "slow_down")))
+ {
+ iddawc_request_error(conn, &session, err,
+ "OAuth token retrieval failed");
+ goto cleanup;
+ }
+
+ if (!user_prompted)
+ {
+ /*
+ * Now that we know the token endpoint isn't broken, give the user
+ * the login instructions.
+ */
+ pqInternalNotice(&conn->noticeHooks,
+ "Visit %s and enter the code: %s",
+ verification_uri, user_code);
+
+ user_prompted = true;
+ }
+
+ /*
+ * We are required to wait between polls; the server tells us how long.
+ * TODO: if interval's not set, we need to default to five seconds
+ * TODO: sanity check the interval
+ */
+ interval = i_get_int_parameter(&session, I_OPT_DEVICE_AUTH_INTERVAL);
+
+ /*
+ * A slow_down error requires us to permanently increase our retry
+ * interval by five seconds. RFC 8628, Sec. 3.5.
+ */
+ if (!strcmp(error_code, "slow_down"))
+ {
+ interval += 5;
+ i_set_int_parameter(&session, I_OPT_DEVICE_AUTH_INTERVAL, interval);
+ }
+
+ sleep(interval);
+
+ /*
+ * XXX Reset the error code before every call, because iddawc won't do
+ * that for us. This matters if the server first sends a "pending" error
+ * code, then later hard-fails without sending an error code to
+ * overwrite the first one.
+ *
+ * That we have to do this at all seems like a bug in iddawc.
+ */
+ i_set_str_parameter(&session, I_OPT_ERROR, NULL);
+
+ err = i_run_token_request(&session);
+ }
+
+ access_token = i_get_str_parameter(&session, I_OPT_ACCESS_TOKEN);
+ token_type = i_get_str_parameter(&session, I_OPT_TOKEN_TYPE);
+
+ if (!access_token || !token_type || strcasecmp(token_type, "Bearer"))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer did not provide a bearer token"));
+ goto cleanup;
+ }
+
+ appendPQExpBufferStr(token_buf, "Bearer ");
+ appendPQExpBufferStr(token_buf, access_token);
+
+ if (PQExpBufferBroken(token_buf))
+ goto cleanup;
+
+ token = strdup(token_buf->data);
+
+cleanup:
+ if (token_buf)
+ destroyPQExpBuffer(token_buf);
+ i_clean_session(&session);
+
+ return token;
+}
+
+#define kvsep "\x01"
+
+static char *
+client_initial_response(PGconn *conn)
+{
+ static const char * const resp_format = "n,," kvsep "auth=%s" kvsep kvsep;
+
+ PQExpBuffer token_buf;
+ PQExpBuffer discovery_buf = NULL;
+ char *token = NULL;
+ char *response = NULL;
+
+ token_buf = createPQExpBuffer();
+ if (!token_buf)
+ goto cleanup;
+
+ /*
+ * If we don't yet have a discovery URI, but the user gave us an explicit
+ * issuer, use the .well-known discovery URI for that issuer.
+ */
+ if (!conn->oauth_discovery_uri && conn->oauth_issuer)
+ {
+ discovery_buf = createPQExpBuffer();
+ if (!discovery_buf)
+ goto cleanup;
+
+ appendPQExpBufferStr(discovery_buf, conn->oauth_issuer);
+ appendPQExpBufferStr(discovery_buf, "/.well-known/openid-configuration");
+
+ if (PQExpBufferBroken(discovery_buf))
+ goto cleanup;
+
+ conn->oauth_discovery_uri = strdup(discovery_buf->data);
+ }
+
+ token = get_auth_token(conn);
+ if (!token)
+ goto cleanup;
+
+ appendPQExpBuffer(token_buf, resp_format, token);
+ if (PQExpBufferBroken(token_buf))
+ goto cleanup;
+
+ response = strdup(token_buf->data);
+
+cleanup:
+ if (token)
+ free(token);
+ if (discovery_buf)
+ destroyPQExpBuffer(discovery_buf);
+ if (token_buf)
+ destroyPQExpBuffer(token_buf);
+
+ return response;
+}
+
+#define ERROR_STATUS_FIELD "status"
+#define ERROR_SCOPE_FIELD "scope"
+#define ERROR_OPENID_CONFIGURATION_FIELD "openid-configuration"
+
+struct json_ctx
+{
+ char *errmsg; /* any non-NULL value stops all processing */
+ PQExpBufferData errbuf; /* backing memory for errmsg */
+ int nested; /* nesting level (zero is the top) */
+
+ const char *target_field_name; /* points to a static allocation */
+ char **target_field; /* see below */
+
+ /* target_field, if set, points to one of the following: */
+ char *status;
+ char *scope;
+ char *discovery_uri;
+};
+
+#define oauth_json_has_error(ctx) \
+ (PQExpBufferDataBroken((ctx)->errbuf) || (ctx)->errmsg)
+
+#define oauth_json_set_error(ctx, ...) \
+ do { \
+ appendPQExpBuffer(&(ctx)->errbuf, __VA_ARGS__); \
+ (ctx)->errmsg = (ctx)->errbuf.data; \
+ } while (0)
+
+static void
+oauth_json_object_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (oauth_json_has_error(ctx))
+ return; /* short-circuit */
+
+ if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ ++ctx->nested;
+}
+
+static void
+oauth_json_object_end(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (oauth_json_has_error(ctx))
+ return; /* short-circuit */
+
+ --ctx->nested;
+}
+
+static void
+oauth_json_object_field_start(void *state, char *name, bool isnull)
+{
+ struct json_ctx *ctx = state;
+
+ if (oauth_json_has_error(ctx))
+ {
+ /* short-circuit */
+ free(name);
+ return;
+ }
+
+ if (ctx->nested == 1)
+ {
+ if (!strcmp(name, ERROR_STATUS_FIELD))
+ {
+ ctx->target_field_name = ERROR_STATUS_FIELD;
+ ctx->target_field = &ctx->status;
+ }
+ else if (!strcmp(name, ERROR_SCOPE_FIELD))
+ {
+ ctx->target_field_name = ERROR_SCOPE_FIELD;
+ ctx->target_field = &ctx->scope;
+ }
+ else if (!strcmp(name, ERROR_OPENID_CONFIGURATION_FIELD))
+ {
+ ctx->target_field_name = ERROR_OPENID_CONFIGURATION_FIELD;
+ ctx->target_field = &ctx->discovery_uri;
+ }
+ }
+
+ free(name);
+}
+
+static void
+oauth_json_array_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (oauth_json_has_error(ctx))
+ return; /* short-circuit */
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+}
+
+static void
+oauth_json_scalar(void *state, char *token, JsonTokenType type)
+{
+ struct json_ctx *ctx = state;
+
+ if (oauth_json_has_error(ctx))
+ {
+ /* short-circuit */
+ free(token);
+ return;
+ }
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ if (type == JSON_TOKEN_STRING)
+ {
+ *ctx->target_field = token;
+
+ ctx->target_field = NULL;
+ ctx->target_field_name = NULL;
+
+ return; /* don't free the token we're using */
+ }
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ free(token);
+}
+
+static bool
+handle_oauth_sasl_error(PGconn *conn, char *msg, int msglen)
+{
+ JsonLexContext lex = {0};
+ JsonSemAction sem = {0};
+ JsonParseErrorType err;
+ struct json_ctx ctx = {0};
+ char *errmsg = NULL;
+
+ /* Sanity check. */
+ if (strlen(msg) != msglen)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server's error message contained an embedded NULL"));
+ return false;
+ }
+
+ initJsonLexContextCstringLen(&lex, msg, msglen, PG_UTF8, true);
+
+ initPQExpBuffer(&ctx.errbuf);
+ sem.semstate = &ctx;
+
+ sem.object_start = oauth_json_object_start;
+ sem.object_end = oauth_json_object_end;
+ sem.object_field_start = oauth_json_object_field_start;
+ sem.array_start = oauth_json_array_start;
+ sem.scalar = oauth_json_scalar;
+
+ err = pg_parse_json(&lex, &sem);
+
+ if (err != JSON_SUCCESS)
+ {
+ errmsg = json_errdetail(err, &lex);
+ }
+ else if (PQExpBufferDataBroken(ctx.errbuf))
+ {
+ errmsg = libpq_gettext("out of memory");
+ }
+ else if (ctx.errmsg)
+ {
+ errmsg = ctx.errmsg;
+ }
+
+ if (errmsg)
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("failed to parse server's error response: %s"),
+ errmsg);
+
+ /* Don't need the error buffer or the JSON lexer anymore. */
+ termPQExpBuffer(&ctx.errbuf);
+ termJsonLexContext(&lex);
+
+ if (errmsg)
+ return false;
+
+ /* TODO: what if these override what the user already specified? */
+ if (ctx.discovery_uri)
+ {
+ if (conn->oauth_discovery_uri)
+ free(conn->oauth_discovery_uri);
+
+ conn->oauth_discovery_uri = ctx.discovery_uri;
+ }
+
+ if (ctx.scope)
+ {
+ if (conn->oauth_scope)
+ free(conn->oauth_scope);
+
+ conn->oauth_scope = ctx.scope;
+ }
+ /* TODO: missing error scope should clear any existing connection scope */
+
+ if (!ctx.status)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("server sent error response without a status"));
+ return false;
+ }
+
+ if (!strcmp(ctx.status, "invalid_token"))
+ {
+ /*
+ * invalid_token is the only error code we'll automatically retry for,
+ * but only if we have enough information to do so.
+ */
+ if (conn->oauth_discovery_uri)
+ conn->oauth_want_retry = true;
+ }
+ /* TODO: include status in hard failure message */
+
+ return true;
+}
+
+static void
+oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen,
+ bool *done, bool *success)
+{
+ fe_oauth_state *state = opaq;
+ PGconn *conn = state->conn;
+
+ *done = false;
+ *success = false;
+ *output = NULL;
+ *outputlen = 0;
+
+ switch (state->state)
+ {
+ case FE_OAUTH_INIT:
+ Assert(inputlen == -1);
+
+ *output = client_initial_response(conn);
+ if (!*output)
+ goto error;
+
+ *outputlen = strlen(*output);
+ state->state = FE_OAUTH_BEARER_SENT;
+
+ break;
+
+ case FE_OAUTH_BEARER_SENT:
+ if (final)
+ {
+ /* TODO: ensure there is no message content here. */
+ *done = true;
+ *success = true;
+
+ break;
+ }
+
+ /*
+ * Error message sent by the server.
+ */
+ if (!handle_oauth_sasl_error(conn, input, inputlen))
+ goto error;
+
+ /*
+ * Respond with the required dummy message (RFC 7628, sec. 3.2.3).
+ */
+ *output = strdup(kvsep);
+ *outputlen = strlen(*output); /* == 1 */
+
+ state->state = FE_OAUTH_SERVER_ERROR;
+ break;
+
+ case FE_OAUTH_SERVER_ERROR:
+ /*
+ * After an error, the server should send an error response to fail
+ * the SASL handshake, which is handled in higher layers.
+ *
+ * If we get here, the server either sent *another* challenge which
+ * isn't defined in the RFC, or completed the handshake successfully
+ * after telling us it was going to fail. Neither is acceptable.
+ */
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server sent additional OAuth data after error\n"));
+ goto error;
+
+ default:
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("invalid OAuth exchange state\n"));
+ goto error;
+ }
+
+ return;
+
+error:
+ *done = true;
+ *success = false;
+}
+
+static bool
+oauth_channel_bound(void *opaq)
+{
+ /* This mechanism does not support channel binding. */
+ return false;
+}
+
+static void
+oauth_free(void *opaq)
+{
+ fe_oauth_state *state = opaq;
+
+ free(state);
+}
diff --git a/src/interfaces/libpq/fe-auth-sasl.h b/src/interfaces/libpq/fe-auth-sasl.h
index da3c30b87b..b1bb382f70 100644
--- a/src/interfaces/libpq/fe-auth-sasl.h
+++ b/src/interfaces/libpq/fe-auth-sasl.h
@@ -65,6 +65,8 @@ typedef struct pg_fe_sasl_mech
*
* state: The opaque mechanism state returned by init()
*
+ * final: true if the server has sent a final exchange outcome
+ *
* input: The challenge data sent by the server, or NULL when
* generating a client-first initial response (that is, when
* the server expects the client to send a message to start
@@ -92,7 +94,8 @@ typedef struct pg_fe_sasl_mech
* Ignored if *done is false.
*--------
*/
- void (*exchange) (void *state, char *input, int inputlen,
+ void (*exchange) (void *state, bool final,
+ char *input, int inputlen,
char **output, int *outputlen,
bool *done, bool *success);
diff --git a/src/interfaces/libpq/fe-auth-scram.c b/src/interfaces/libpq/fe-auth-scram.c
index e616200704..681b76adbe 100644
--- a/src/interfaces/libpq/fe-auth-scram.c
+++ b/src/interfaces/libpq/fe-auth-scram.c
@@ -24,7 +24,8 @@
/* The exported SCRAM callback mechanism. */
static void *scram_init(PGconn *conn, const char *password,
const char *sasl_mechanism);
-static void scram_exchange(void *opaq, char *input, int inputlen,
+static void scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen,
bool *done, bool *success);
static bool scram_channel_bound(void *opaq);
@@ -206,7 +207,8 @@ scram_free(void *opaq)
* Exchange a SCRAM message with backend.
*/
static void
-scram_exchange(void *opaq, char *input, int inputlen,
+scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen,
bool *done, bool *success)
{
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index 6fceff561b..2567a34023 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -38,6 +38,7 @@
#endif
#include "common/md5.h"
+#include "common/oauth-common.h"
#include "common/scram-common.h"
#include "fe-auth.h"
#include "fe-auth-sasl.h"
@@ -422,7 +423,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
bool success;
const char *selected_mechanism;
PQExpBufferData mechanism_buf;
- char *password;
+ char *password = NULL;
initPQExpBuffer(&mechanism_buf);
@@ -444,8 +445,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
/*
* Parse the list of SASL authentication mechanisms in the
* AuthenticationSASL message, and select the best mechanism that we
- * support. SCRAM-SHA-256-PLUS and SCRAM-SHA-256 are the only ones
- * supported at the moment, listed by order of decreasing importance.
+ * support. Mechanisms are listed by order of decreasing importance.
*/
selected_mechanism = NULL;
for (;;)
@@ -485,6 +485,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
selected_mechanism = SCRAM_SHA_256_PLUS_NAME;
conn->sasl = &pg_scram_mech;
+ conn->password_needed = true;
}
#else
/*
@@ -522,7 +523,17 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
selected_mechanism = SCRAM_SHA_256_NAME;
conn->sasl = &pg_scram_mech;
+ conn->password_needed = true;
}
+#ifdef USE_OAUTH
+ else if (strcmp(mechanism_buf.data, OAUTHBEARER_NAME) == 0 &&
+ !selected_mechanism)
+ {
+ selected_mechanism = OAUTHBEARER_NAME;
+ conn->sasl = &pg_oauth_mech;
+ conn->password_needed = false;
+ }
+#endif
}
if (!selected_mechanism)
@@ -547,18 +558,19 @@ pg_SASL_init(PGconn *conn, int payloadlen)
/*
* First, select the password to use for the exchange, complaining if
- * there isn't one. Currently, all supported SASL mechanisms require a
- * password, so we can just go ahead here without further distinction.
+ * there isn't one and the SASL mechanism needs it.
*/
- conn->password_needed = true;
- password = conn->connhost[conn->whichhost].password;
- if (password == NULL)
- password = conn->pgpass;
- if (password == NULL || password[0] == '\0')
+ if (conn->password_needed)
{
- appendPQExpBufferStr(&conn->errorMessage,
- PQnoPasswordSupplied);
- goto error;
+ password = conn->connhost[conn->whichhost].password;
+ if (password == NULL)
+ password = conn->pgpass;
+ if (password == NULL || password[0] == '\0')
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ PQnoPasswordSupplied);
+ goto error;
+ }
}
Assert(conn->sasl);
@@ -576,7 +588,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
goto oom_error;
/* Get the mechanism-specific Initial Client Response, if any */
- conn->sasl->exchange(conn->sasl_state,
+ conn->sasl->exchange(conn->sasl_state, false,
NULL, -1,
&initialresponse, &initialresponselen,
&done, &success);
@@ -657,7 +669,7 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
/* For safety and convenience, ensure the buffer is NULL-terminated. */
challenge[payloadlen] = '\0';
- conn->sasl->exchange(conn->sasl_state,
+ conn->sasl->exchange(conn->sasl_state, final,
challenge, payloadlen,
&output, &outputlen,
&done, &success);
diff --git a/src/interfaces/libpq/fe-auth.h b/src/interfaces/libpq/fe-auth.h
index 049a8bb1a1..2a56774019 100644
--- a/src/interfaces/libpq/fe-auth.h
+++ b/src/interfaces/libpq/fe-auth.h
@@ -28,4 +28,7 @@ extern const pg_fe_sasl_mech pg_scram_mech;
extern char *pg_fe_scram_build_secret(const char *password,
const char **errstr);
+/* Mechanisms in fe-auth-oauth.c */
+extern const pg_fe_sasl_mech pg_oauth_mech;
+
#endif /* FE_AUTH_H */
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index cf554d389f..fdd30d71de 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -344,6 +344,23 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"Target-Session-Attrs", "", 15, /* sizeof("prefer-standby") = 15 */
offsetof(struct pg_conn, target_session_attrs)},
+ /* OAuth v2 */
+ {"oauth_issuer", NULL, NULL, NULL,
+ "OAuth-Issuer", "", 40,
+ offsetof(struct pg_conn, oauth_issuer)},
+
+ {"oauth_client_id", NULL, NULL, NULL,
+ "OAuth-Client-ID", "", 40,
+ offsetof(struct pg_conn, oauth_client_id)},
+
+ {"oauth_client_secret", NULL, NULL, NULL,
+ "OAuth-Client-Secret", "", 40,
+ offsetof(struct pg_conn, oauth_client_secret)},
+
+ {"oauth_scope", NULL, NULL, NULL,
+ "OAuth-Scope", "", 15,
+ offsetof(struct pg_conn, oauth_scope)},
+
/* Terminating entry --- MUST BE LAST */
{NULL, NULL, NULL, NULL,
NULL, NULL, 0}
@@ -606,6 +623,7 @@ pqDropServerData(PGconn *conn)
conn->write_err_msg = NULL;
conn->be_pid = 0;
conn->be_key = 0;
+ /* conn->oauth_want_retry = false; TODO */
}
@@ -3386,6 +3404,16 @@ keep_going: /* We will come back to here until there is
/* Check to see if we should mention pgpassfile */
pgpassfileWarning(conn);
+#ifdef USE_OAUTH
+ if (conn->sasl == &pg_oauth_mech
+ && conn->oauth_want_retry)
+ {
+ /* TODO: only allow retry once */
+ need_new_connection = true;
+ goto keep_going;
+ }
+#endif
+
#ifdef ENABLE_GSS
/*
@@ -4166,6 +4194,16 @@ freePGconn(PGconn *conn)
free(conn->rowBuf);
if (conn->target_session_attrs)
free(conn->target_session_attrs);
+ if (conn->oauth_issuer)
+ free(conn->oauth_issuer);
+ if (conn->oauth_discovery_uri)
+ free(conn->oauth_discovery_uri);
+ if (conn->oauth_client_id)
+ free(conn->oauth_client_id);
+ if (conn->oauth_client_secret)
+ free(conn->oauth_client_secret);
+ if (conn->oauth_scope)
+ free(conn->oauth_scope);
termPQExpBuffer(&conn->errorMessage);
termPQExpBuffer(&conn->workBuffer);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index e0cee4b142..0dff13505a 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -394,6 +394,14 @@ struct pg_conn
char *ssl_max_protocol_version; /* maximum TLS protocol version */
char *target_session_attrs; /* desired session properties */
+ /* OAuth v2 */
+ char *oauth_issuer; /* token issuer URL */
+ char *oauth_discovery_uri; /* URI of the issuer's discovery document */
+ char *oauth_client_id; /* client identifier */
+ char *oauth_client_secret; /* client secret */
+ char *oauth_scope; /* access token scope */
+ bool oauth_want_retry; /* should we retry on failure? */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
--
2.25.1
v4-0007-backend-add-OAUTHBEARER-SASL-mechanism.patch (text/x-patch, 35.6K) download
From b3ceda62e9cc6cbbc24c63c05c5ce072ae771c1b Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 4 May 2021 16:21:11 -0700
Subject: [PATCH v4 07/10] backend: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) on the server side. This adds a new
auth method, oauth, to pg_hba.
Because OAuth implementations vary so wildly, and bearer token
validation is heavily dependent on the issuing party, authn/z is done by
communicating with an external program: the oauth_validator_command.
This command must do the following:
1. Receive the bearer token by reading its contents from a file
descriptor passed from the server. (The numeric value of this
descriptor may be inserted into the oauth_validator_command using the
%f specifier.)
This MUST be the first action the command performs. The server will
not begin reading stdout from the command until the token has been
read in full, so if the command tries to print anything and hits a
buffer limit, the backend will deadlock and time out.
2. Validate the bearer token. The correct way to do this depends on the
issuer, but it generally involves either cryptographic operations to
prove that the token was issued by a trusted party, or the
presentation of the bearer token to some other party so that _it_ can
perform validation.
The command MUST maintain confidentiality of the bearer token, since
in most cases it can be used just like a password. (There are ways to
cryptographically bind tokens to client certificates, but they are
way beyond the scope of this commit message.)
If the token cannot be validated, the command must exit with a
non-zero status. Further authentication/authorization is pointless if
the bearer token wasn't issued by someone you trust.
3. Authenticate the user, authorize the user, or both:
a. To authenticate the user, use the bearer token to retrieve some
trusted identifier string for the end user. The exact process for
this is, again, issuer-dependent. The command should print the
authenticated identity string to stdout, followed by a newline.
If the user cannot be authenticated, the validator should not
print anything to stdout. It should also exit with a non-zero
status, unless the token may be used to authorize the connection
through some other means (see below).
On a success, the command may then exit with a zero success code.
By default, the server will then check to make sure the identity
string matches the role that is being used (or matches a usermap
entry, if one is in use).
b. To optionally authorize the user, in combination with the HBA
option trust_validator_authz=1 (see below), the validator simply
returns a zero exit code if the client should be allowed to
connect with its presented role (which can be passed to the
command using the %r specifier), or a non-zero code otherwise.
The hard part is in determining whether the given token truly
authorizes the client to use the given role, which must
unfortunately be left as an exercise to the reader.
This obviously requires some care, as a poorly implemented token
validator may silently open the entire database to anyone with a
bearer token. But it may be a more portable approach, since OAuth
is designed as an authorization framework, not an authentication
framework. For example, the user's bearer token could carry an
"allow_superuser_access" claim, which would authorize pseudonymous
database access as any role. It's then up to the OAuth system
administrators to ensure that allow_superuser_access is doled out
only to the proper users.
c. It's possible that the user can be successfully authenticated but
isn't authorized to connect. In this case, the command may print
the authenticated ID and then fail with a non-zero exit code.
(This makes it easier to see what's going on in the Postgres
logs.)
4. Token validators may optionally log to stderr. This will be printed
verbatim into the Postgres server logs.
The oauth method supports the following HBA options (but note that two
of them are not optional, since we have no way of choosing sensible
defaults):
issuer: Required. The URL of the OAuth issuing party, which the client
must contact to receive a bearer token.
Some real-world examples as of time of writing:
- https://accounts.google.com
- https://login.microsoft.com/[tenant-id]/v2.0
scope: Required. The OAuth scope(s) required for the server to
authenticate and/or authorize the user. This is heavily
deployment-specific, but a simple example is "openid email".
map: Optional. Specify a standard PostgreSQL user map; this works
the same as with other auth methods such as peer. If a map is
not specified, the user ID returned by the token validator
must exactly match the role that's being requested (but see
trust_validator_authz, below).
trust_validator_authz:
Optional. When set to 1, this allows the token validator to
take full control of the authorization process. Standard user
mapping is skipped: if the validator command succeeds, the
client is allowed to connect under its desired role and no
further checks are done.
Unlike the client, servers support OAuth without needing to be built
against libiddawc (since the responsibility for "speaking" OAuth/OIDC
correctly is delegated entirely to the oauth_validator_command).
Several TODOs:
- port to platforms other than "modern Linux"
- overhaul the communication with oauth_validator_command, which is
currently a bad hack on OpenPipeStream()
- implement more sanity checks on the OAUTHBEARER message format and
tokens sent by the client
- implement more helpful handling of HBA misconfigurations
- properly interpolate JSON when generating error responses
- use logdetail during auth failures
- deal with role names that can't be safely passed to system() without
shell-escaping
- allow passing the configured issuer to the oauth_validator_command, to
deal with multi-issuer setups
- ...and more.
---
src/backend/libpq/Makefile | 1 +
src/backend/libpq/auth-oauth.c | 797 +++++++++++++++++++++++++++++++++
src/backend/libpq/auth-sasl.c | 10 +-
src/backend/libpq/auth-scram.c | 4 +-
src/backend/libpq/auth.c | 7 +
src/backend/libpq/hba.c | 29 +-
src/backend/utils/misc/guc.c | 12 +
src/include/libpq/hba.h | 8 +-
src/include/libpq/oauth.h | 24 +
src/include/libpq/sasl.h | 11 +
10 files changed, 889 insertions(+), 14 deletions(-)
create mode 100644 src/backend/libpq/auth-oauth.c
create mode 100644 src/include/libpq/oauth.h
diff --git a/src/backend/libpq/Makefile b/src/backend/libpq/Makefile
index 6d385fd6a4..98eb2a8242 100644
--- a/src/backend/libpq/Makefile
+++ b/src/backend/libpq/Makefile
@@ -15,6 +15,7 @@ include $(top_builddir)/src/Makefile.global
# be-fsstubs is here for historical reasons, probably belongs elsewhere
OBJS = \
+ auth-oauth.o \
auth-sasl.o \
auth-scram.o \
auth.o \
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
new file mode 100644
index 0000000000..c1232a31a0
--- /dev/null
+++ b/src/backend/libpq/auth-oauth.c
@@ -0,0 +1,797 @@
+/*-------------------------------------------------------------------------
+ *
+ * auth-oauth.c
+ * Server-side implementation of the SASL OAUTHBEARER mechanism.
+ *
+ * See the following RFC for more details:
+ * - RFC 7628: https://tools.ietf.org/html/rfc7628
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/backend/libpq/auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <unistd.h>
+#include <fcntl.h>
+
+#include "common/oauth-common.h"
+#include "lib/stringinfo.h"
+#include "libpq/auth.h"
+#include "libpq/hba.h"
+#include "libpq/oauth.h"
+#include "libpq/sasl.h"
+#include "storage/fd.h"
+
+/* GUC */
+char *oauth_validator_command;
+
+static void oauth_get_mechanisms(Port *port, StringInfo buf);
+static void *oauth_init(Port *port, const char *selected_mech, const char *shadow_pass);
+static int oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, const char **logdetail);
+
+/* Mechanism declaration */
+const pg_be_sasl_mech pg_be_oauth_mech = {
+ oauth_get_mechanisms,
+ oauth_init,
+ oauth_exchange,
+
+ PG_MAX_AUTH_TOKEN_LENGTH,
+};
+
+
+typedef enum
+{
+ OAUTH_STATE_INIT = 0,
+ OAUTH_STATE_ERROR,
+ OAUTH_STATE_FINISHED,
+} oauth_state;
+
+struct oauth_ctx
+{
+ oauth_state state;
+ Port *port;
+ const char *issuer;
+ const char *scope;
+};
+
+static char *sanitize_char(char c);
+static char *parse_kvpairs_for_auth(char **input);
+static void generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen);
+static bool validate(Port *port, const char *auth, const char **logdetail);
+static bool run_validator_command(Port *port, const char *token);
+static bool check_exit(FILE **fh, const char *command);
+static bool unset_cloexec(int fd);
+static bool username_ok_for_shell(const char *username);
+
+#define KVSEP 0x01
+#define AUTH_KEY "auth"
+#define BEARER_SCHEME "Bearer "
+
+static void
+oauth_get_mechanisms(Port *port, StringInfo buf)
+{
+ /* Only OAUTHBEARER is supported. */
+ appendStringInfoString(buf, OAUTHBEARER_NAME);
+ appendStringInfoChar(buf, '\0');
+}
+
+static void *
+oauth_init(Port *port, const char *selected_mech, const char *shadow_pass)
+{
+ struct oauth_ctx *ctx;
+
+ if (strcmp(selected_mech, OAUTHBEARER_NAME))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("client selected an invalid SASL authentication mechanism")));
+
+ ctx = palloc0(sizeof(*ctx));
+
+ ctx->state = OAUTH_STATE_INIT;
+ ctx->port = port;
+
+ Assert(port->hba);
+ ctx->issuer = port->hba->oauth_issuer;
+ ctx->scope = port->hba->oauth_scope;
+
+ return ctx;
+}
+
+static int
+oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, const char **logdetail)
+{
+ char *p;
+ char cbind_flag;
+ char *auth;
+
+ struct oauth_ctx *ctx = opaq;
+
+ *output = NULL;
+ *outputlen = -1;
+
+ /*
+ * If the client didn't include an "Initial Client Response" in the
+ * SASLInitialResponse message, send an empty challenge, to which the
+ * client will respond with the same data that usually comes in the
+ * Initial Client Response.
+ */
+ if (input == NULL)
+ {
+ Assert(ctx->state == OAUTH_STATE_INIT);
+
+ *output = pstrdup("");
+ *outputlen = 0;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ /*
+ * Check that the input length agrees with the string length of the input.
+ */
+ if (inputlen == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The message is empty.")));
+ if (inputlen != strlen(input))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message length does not match input length.")));
+
+ switch (ctx->state)
+ {
+ case OAUTH_STATE_INIT:
+ /* Handle this case below. */
+ break;
+
+ case OAUTH_STATE_ERROR:
+ /*
+ * Only one response is valid for the client during authentication
+ * failure: a single kvsep.
+ */
+ if (inputlen != 1 || *input != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Client did not send a kvsep response.")));
+
+ /* The (failed) handshake is now complete. */
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_FAILURE;
+
+ default:
+ elog(ERROR, "invalid OAUTHBEARER exchange state");
+ return PG_SASL_EXCHANGE_FAILURE;
+ }
+
+ /* Handle the client's initial message. */
+ p = pstrdup(input);
+
+ /*
+ * OAUTHBEARER does not currently define a channel binding (so there is no
+ * OAUTHBEARER-PLUS, and we do not accept a 'p' specifier). We accept a 'y'
+ * specifier purely for the remote chance that a future specification could
+ * define one; then future clients can still interoperate with this server
+ * implementation. 'n' is the expected case.
+ */
+ cbind_flag = *p;
+ switch (cbind_flag)
+ {
+ case 'p':
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The server does not support channel binding for OAuth, but the client message includes channel binding data.")));
+ break;
+
+ case 'y': /* fall through */
+ case 'n':
+ p++;
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Comma expected, but found character %s.",
+ sanitize_char(*p))));
+ p++;
+ break;
+
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected channel-binding flag %s.",
+ sanitize_char(cbind_flag))));
+ }
+
+ /*
+ * Forbid optional authzid (authorization identity). We don't support it.
+ */
+ if (*p == 'a')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("client uses authorization identity, but it is not supported")));
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected attribute %s in client-first-message.",
+ sanitize_char(*p))));
+ p++;
+
+ /* All remaining fields are separated by the RFC's kvsep (\x01). */
+ if (*p != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Key-value separator expected, but found character %s.",
+ sanitize_char(*p))));
+ p++;
+
+ auth = parse_kvpairs_for_auth(&p);
+ if (!auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message does not contain an auth value.")));
+
+ /* We should be at the end of our message. */
+ if (*p)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains additional data after the final terminator.")));
+
+ if (!validate(ctx->port, auth, logdetail))
+ {
+ generate_error_response(ctx, output, outputlen);
+
+ ctx->state = OAUTH_STATE_ERROR;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_SUCCESS;
+}
+
+/*
+ * Convert an arbitrary byte to printable form. For error messages.
+ *
+ * If it's a printable ASCII character, print it as a single character.
+ * otherwise, print it in hex.
+ *
+ * The returned pointer points to a static buffer.
+ */
+static char *
+sanitize_char(char c)
+{
+ static char buf[5];
+
+ if (c >= 0x21 && c <= 0x7E)
+ snprintf(buf, sizeof(buf), "'%c'", c);
+ else
+ snprintf(buf, sizeof(buf), "0x%02x", (unsigned char) c);
+ return buf;
+}
+
+/*
+ * Consumes all kvpairs in an OAUTHBEARER exchange message. If the "auth" key is
+ * found, its value is returned.
+ */
+static char *
+parse_kvpairs_for_auth(char **input)
+{
+ char *pos = *input;
+ char *auth = NULL;
+
+ /*
+ * The relevant ABNF, from Sec. 3.1:
+ *
+ * kvsep = %x01
+ * key = 1*(ALPHA)
+ * value = *(VCHAR / SP / HTAB / CR / LF )
+ * kvpair = key "=" value kvsep
+ * ;;gs2-header = See RFC 5801
+ * client-resp = (gs2-header kvsep *kvpair kvsep) / kvsep
+ *
+ * By the time we reach this code, the gs2-header and initial kvsep have
+ * already been validated. We start at the beginning of the first kvpair.
+ */
+
+ while (*pos)
+ {
+ char *end;
+ char *sep;
+ char *key;
+ char *value;
+
+ /*
+ * Find the end of this kvpair. Note that input is null-terminated by
+ * the SASL code, so the strchr() is bounded.
+ */
+ end = strchr(pos, KVSEP);
+ if (!end)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an unterminated key/value pair.")));
+ *end = '\0';
+
+ if (pos == end)
+ {
+ /* Empty kvpair, signifying the end of the list. */
+ *input = pos + 1;
+ return auth;
+ }
+
+ /*
+ * Find the end of the key name.
+ *
+ * TODO further validate the key/value grammar? empty keys, bad chars...
+ */
+ sep = strchr(pos, '=');
+ if (!sep)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains a key without a value.")));
+ *sep = '\0';
+
+ /* Both key and value are now safely terminated. */
+ key = pos;
+ value = sep + 1;
+
+ if (!strcmp(key, AUTH_KEY))
+ {
+ if (auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains multiple auth values.")));
+
+ auth = value;
+ }
+ else
+ {
+ /*
+ * The RFC also defines the host and port keys, but they are not
+ * required for OAUTHBEARER and we do not use them. Also, per
+ * Sec. 3.1, any key/value pairs we don't recognize must be ignored.
+ */
+ }
+
+ /* Move to the next pair. */
+ pos = end + 1;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message did not contain a final terminator.")));
+
+ return NULL; /* unreachable */
+}
+
+static void
+generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
+{
+ StringInfoData buf;
+
+ /*
+ * The admin needs to set an issuer and scope for OAuth to work. There's not
+ * really a way to hide this from the user, either, because we can't choose
+ * a "default" issuer, so be honest in the failure message.
+ *
+ * TODO: see if there's a better place to fail, earlier than this.
+ */
+ if (!ctx->issuer || !ctx->scope)
+ ereport(FATAL,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("OAuth is not properly configured for this user"),
+ errdetail_log("The issuer and scope parameters must be set in pg_hba.conf.")));
+
+
+ initStringInfo(&buf);
+
+ /*
+ * TODO: JSON escaping
+ */
+ appendStringInfo(&buf,
+ "{ "
+ "\"status\": \"invalid_token\", "
+ "\"openid-configuration\": \"%s/.well-known/openid-configuration\","
+ "\"scope\": \"%s\" "
+ "}",
+ ctx->issuer, ctx->scope);
+
+ *output = buf.data;
+ *outputlen = buf.len;
+}
+
+static bool
+validate(Port *port, const char *auth, const char **logdetail)
+{
+ static const char * const b64_set = "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-._~+/";
+
+ const char *token;
+ size_t span;
+ int ret;
+
+ /* TODO: handle logdetail when the test framework can check it */
+
+ /*
+ * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
+ * 2.1:
+ *
+ * b64token = 1*( ALPHA / DIGIT /
+ * "-" / "." / "_" / "~" / "+" / "/" ) *"="
+ * credentials = "Bearer" 1*SP b64token
+ *
+ * The "credentials" construction is what we receive in our auth value.
+ *
+ * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
+ * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
+ * compared case-insensitively. (This is not mentioned in RFC 6750, but it's
+ * pointed out in RFC 7628 Sec. 4.)
+ *
+ * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
+ */
+ if (strncasecmp(auth, BEARER_SCHEME, strlen(BEARER_SCHEME)))
+ return false;
+
+ /* Pull the bearer token out of the auth value. */
+ token = auth + strlen(BEARER_SCHEME);
+
+ /* Swallow any additional spaces. */
+ while (*token == ' ')
+ token++;
+
+ /*
+ * Before invoking the validator command, sanity-check the token format to
+ * avoid any injection attacks later in the chain. Invalid formats are
+ * technically a protocol violation, but don't reflect any information about
+ * the sensitive Bearer token back to the client; log at COMMERROR instead.
+ */
+
+ /* Tokens must not be empty. */
+ if (!*token)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Bearer token is empty.")));
+ return false;
+ }
+
+ /*
+ * Make sure the token contains only allowed characters. Tokens may end with
+ * any number of '=' characters.
+ */
+ span = strspn(token, b64_set);
+ while (token[span] == '=')
+ span++;
+
+ if (token[span] != '\0')
+ {
+ /*
+ * This error message could be more helpful by printing the problematic
+ * character(s), but that'd be a bit like printing a piece of someone's
+ * password into the logs.
+ */
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Bearer token is not in the correct format.")));
+ return false;
+ }
+
+ /* Have the validator check the token. */
+ if (!run_validator_command(port, token))
+ return false;
+
+ if (port->hba->oauth_skip_usermap)
+ {
+ /*
+ * If the validator is our authorization authority, we're done.
+ * Authentication may or may not have been performed depending on the
+ * validator implementation; all that matters is that the validator says
+ * the user can log in with the target role.
+ */
+ return true;
+ }
+
+ /* Make sure the validator authenticated the user. */
+ if (!port->authn_id)
+ {
+ /* TODO: use logdetail; reduce message duplication */
+ ereport(LOG,
+ (errmsg("OAuth bearer authentication failed for user \"%s\": validator provided no identity",
+ port->user_name)));
+ return false;
+ }
+
+ /* Finally, check the user map. */
+ ret = check_usermap(port->hba->usermap, port->user_name, port->authn_id,
+ false);
+ return (ret == STATUS_OK);
+}
+
+static bool
+run_validator_command(Port *port, const char *token)
+{
+ bool success = false;
+ int rc;
+ int pipefd[2];
+ int rfd = -1;
+ int wfd = -1;
+
+ StringInfoData command = { 0 };
+ char *p;
+ FILE *fh = NULL;
+
+ ssize_t written;
+ char *line = NULL;
+ size_t size = 0;
+ ssize_t len;
+
+ Assert(oauth_validator_command);
+
+ if (!oauth_validator_command[0])
+ {
+ ereport(COMMERROR,
+ (errmsg("oauth_validator_command is not set"),
+ errhint("To allow OAuth authenticated connections, set "
+ "oauth_validator_command in postgresql.conf.")));
+ return false;
+ }
+
+ /*
+ * Since popen() is unidirectional, open up a pipe for the other direction.
+ * Use CLOEXEC to ensure that our write end doesn't accidentally get copied
+ * into child processes, which would prevent us from closing it cleanly.
+ *
+ * XXX this is ugly. We should just read from the child process's stdout,
+ * but that's a lot more code.
+ * XXX by bypassing the popen API, we open the potential of process
+ * deadlock. Clearly document child process requirements (i.e. the child
+ * MUST read all data off of the pipe before writing anything).
+ * TODO: port to Windows using _pipe().
+ */
+ rc = pipe2(pipefd, O_CLOEXEC);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create child pipe: %m")));
+ return false;
+ }
+
+ rfd = pipefd[0];
+ wfd = pipefd[1];
+
+ /* Allow the read pipe be passed to the child. */
+ if (!unset_cloexec(rfd))
+ {
+ /* error message was already logged */
+ goto cleanup;
+ }
+
+ /*
+ * Construct the command, substituting any recognized %-specifiers:
+ *
+ * %f: the file descriptor of the input pipe
+ * %r: the role that the client wants to assume (port->user_name)
+ * %%: a literal '%'
+ */
+ initStringInfo(&command);
+
+ for (p = oauth_validator_command; *p; p++)
+ {
+ if (p[0] == '%')
+ {
+ switch (p[1])
+ {
+ case 'f':
+ appendStringInfo(&command, "%d", rfd);
+ p++;
+ break;
+ case 'r':
+ /*
+ * TODO: decide how this string should be escaped. The role
+ * is controlled by the client, so if we don't escape it,
+ * command injections are inevitable.
+ *
+ * This is probably an indication that the role name needs
+ * to be communicated to the validator process in some other
+ * way. For this proof of concept, just be incredibly strict
+ * about the characters that are allowed in user names.
+ */
+ if (!username_ok_for_shell(port->user_name))
+ goto cleanup;
+
+ appendStringInfoString(&command, port->user_name);
+ p++;
+ break;
+ case '%':
+ appendStringInfoChar(&command, '%');
+ p++;
+ break;
+ default:
+ appendStringInfoChar(&command, p[0]);
+ }
+ }
+ else
+ appendStringInfoChar(&command, p[0]);
+ }
+
+ /* Execute the command. */
+ fh = OpenPipeStream(command.data, "re");
+ /* TODO: handle failures */
+
+ /* We don't need the read end of the pipe anymore. */
+ close(rfd);
+ rfd = -1;
+
+ /* Give the command the token to validate. */
+ written = write(wfd, token, strlen(token));
+ if (written != strlen(token))
+ {
+ /* TODO must loop for short writes, EINTR et al */
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write token to child pipe: %m")));
+ goto cleanup;
+ }
+
+ close(wfd);
+ wfd = -1;
+
+ /*
+ * Read the command's response.
+ *
+ * TODO: getline() is probably too new to use, unfortunately.
+ * TODO: loop over all lines
+ */
+ if ((len = getline(&line, &size, fh)) >= 0)
+ {
+ /* TODO: fail if the authn_id doesn't end with a newline */
+ if (len > 0)
+ line[len - 1] = '\0';
+
+ set_authn_id(port, line);
+ }
+ else if (ferror(fh))
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from command \"%s\": %m",
+ command.data)));
+ goto cleanup;
+ }
+
+ /* Make sure the command exits cleanly. */
+ if (!check_exit(&fh, command.data))
+ {
+ /* error message already logged */
+ goto cleanup;
+ }
+
+ /* Done. */
+ success = true;
+
+cleanup:
+ if (line)
+ free(line);
+
+ /*
+ * In the successful case, the pipe fds are already closed. For the error
+ * case, always close out the pipe before waiting for the command, to
+ * prevent deadlock.
+ */
+ if (rfd >= 0)
+ close(rfd);
+ if (wfd >= 0)
+ close(wfd);
+
+ if (fh)
+ {
+ Assert(!success);
+ check_exit(&fh, command.data);
+ }
+
+ if (command.data)
+ pfree(command.data);
+
+ return success;
+}
+
+static bool
+check_exit(FILE **fh, const char *command)
+{
+ int rc;
+
+ rc = ClosePipeStream(*fh);
+ *fh = NULL;
+
+ if (rc == -1)
+ {
+ /* pclose() itself failed. */
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close pipe to command \"%s\": %m",
+ command)));
+ }
+ else if (rc != 0)
+ {
+ char *reason = wait_result_to_str(rc);
+
+ ereport(COMMERROR,
+ (errmsg("failed to execute command \"%s\": %s",
+ command, reason)));
+
+ pfree(reason);
+ }
+
+ return (rc == 0);
+}
+
+static bool
+unset_cloexec(int fd)
+{
+ int flags;
+ int rc;
+
+ flags = fcntl(fd, F_GETFD);
+ if (flags == -1)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not get fd flags for child pipe: %m")));
+ return false;
+ }
+
+ rc = fcntl(fd, F_SETFD, flags & ~FD_CLOEXEC);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not unset FD_CLOEXEC for child pipe: %m")));
+ return false;
+ }
+
+ return true;
+}
+
+/*
+ * XXX This should go away eventually and be replaced with either a proper
+ * escape or a different strategy for communication with the validator command.
+ */
+static bool
+username_ok_for_shell(const char *username)
+{
+ /* This set is borrowed from fe_utils' appendShellStringNoError(). */
+ static const char * const allowed = "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-_./:";
+ size_t span;
+
+ Assert(username && username[0]); /* should have already been checked */
+
+ span = strspn(username, allowed);
+ if (username[span] != '\0')
+ {
+ ereport(COMMERROR,
+ (errmsg("PostgreSQL user name contains unsafe characters and cannot be passed to the OAuth validator")));
+ return false;
+ }
+
+ return true;
+}
diff --git a/src/backend/libpq/auth-sasl.c b/src/backend/libpq/auth-sasl.c
index a1d7dbb6d5..0f461a6696 100644
--- a/src/backend/libpq/auth-sasl.c
+++ b/src/backend/libpq/auth-sasl.c
@@ -20,14 +20,6 @@
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
-/*
- * Maximum accepted size of SASL messages.
- *
- * The messages that the server or libpq generate are much smaller than this,
- * but have some headroom.
- */
-#define PG_MAX_SASL_MESSAGE_LENGTH 1024
-
/*
* Perform a SASL exchange with a libpq client, using a specific mechanism
* implementation.
@@ -103,7 +95,7 @@ CheckSASLAuth(const pg_be_sasl_mech *mech, Port *port, char *shadow_pass,
/* Get the actual SASL message */
initStringInfo(&buf);
- if (pq_getmessage(&buf, PG_MAX_SASL_MESSAGE_LENGTH))
+ if (pq_getmessage(&buf, mech->max_message_length))
{
/* EOF - pq_getmessage already logged error */
pfree(buf.data);
diff --git a/src/backend/libpq/auth-scram.c b/src/backend/libpq/auth-scram.c
index ee7f52218a..4049ace470 100644
--- a/src/backend/libpq/auth-scram.c
+++ b/src/backend/libpq/auth-scram.c
@@ -118,7 +118,9 @@ static int scram_exchange(void *opaq, const char *input, int inputlen,
const pg_be_sasl_mech pg_be_scram_mech = {
scram_get_mechanisms,
scram_init,
- scram_exchange
+ scram_exchange,
+
+ PG_MAX_SASL_MESSAGE_LENGTH
};
/*
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 4a8a63922a..17042d84ad 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -30,6 +30,7 @@
#include "libpq/auth.h"
#include "libpq/crypt.h"
#include "libpq/libpq.h"
+#include "libpq/oauth.h"
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
#include "libpq/scram.h"
@@ -298,6 +299,9 @@ auth_failed(Port *port, int status, const char *logdetail)
case uaRADIUS:
errstr = gettext_noop("RADIUS authentication failed for user \"%s\"");
break;
+ case uaOAuth:
+ errstr = gettext_noop("OAuth bearer authentication failed for user \"%s\"");
+ break;
case uaCustom:
{
CustomAuthProvider *provider = get_provider_by_name(port->hba->custom_provider);
@@ -626,6 +630,9 @@ ClientAuthentication(Port *port)
case uaTrust:
status = STATUS_OK;
break;
+ case uaOAuth:
+ status = CheckSASLAuth(&pg_be_oauth_mech, port, NULL, NULL);
+ break;
case uaCustom:
{
CustomAuthProvider *provider = get_provider_by_name(port->hba->custom_provider);
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index 42cb1ce51d..cd3b1cc140 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -136,7 +136,8 @@ static const char *const UserAuthName[] =
"cert",
"radius",
"custom",
- "peer"
+ "peer",
+ "oauth",
};
@@ -1401,6 +1402,8 @@ parse_hba_line(TokenizedLine *tok_line, int elevel)
#endif
else if (strcmp(token->string, "radius") == 0)
parsedline->auth_method = uaRADIUS;
+ else if (strcmp(token->string, "oauth") == 0)
+ parsedline->auth_method = uaOAuth;
else if (strcmp(token->string, "custom") == 0)
parsedline->auth_method = uaCustom;
else
@@ -1730,8 +1733,9 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->auth_method != uaGSS &&
hbaline->auth_method != uaSSPI &&
hbaline->auth_method != uaCert &&
+ hbaline->auth_method != uaOAuth &&
hbaline->auth_method != uaCustom)
- INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, cert and custom"));
+ INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, cert, oauth, and custom"));
hbaline->usermap = pstrdup(val);
}
else if (strcmp(name, "clientcert") == 0)
@@ -2115,6 +2119,27 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->radiusidentifiers = parsed_identifiers;
hbaline->radiusidentifiers_s = pstrdup(val);
}
+ else if (strcmp(name, "issuer") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("issuer", gettext_noop("oauth"));
+ hbaline->oauth_issuer = pstrdup(val);
+ }
+ else if (strcmp(name, "scope") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("scope", gettext_noop("oauth"));
+ hbaline->oauth_scope = pstrdup(val);
+ }
+ else if (strcmp(name, "trust_validator_authz") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("trust_validator_authz", gettext_noop("oauth"));
+ if (strcmp(val, "1") == 0)
+ hbaline->oauth_skip_usermap = true;
+ else
+ hbaline->oauth_skip_usermap = false;
+ }
else if (strcmp(name, "provider") == 0)
{
REQUIRE_AUTH_OPTION(uaCustom, "provider", "custom");
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index f70f7f5c01..9a5b2aa496 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -59,6 +59,7 @@
#include "libpq/auth.h"
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
+#include "libpq/oauth.h"
#include "miscadmin.h"
#include "optimizer/cost.h"
#include "optimizer/geqo.h"
@@ -4666,6 +4667,17 @@ static struct config_string ConfigureNamesString[] =
check_backtrace_functions, assign_backtrace_functions, NULL
},
+ {
+ {"oauth_validator_command", PGC_SIGHUP, CONN_AUTH_AUTH,
+ gettext_noop("Command to validate OAuth v2 bearer tokens."),
+ NULL,
+ GUC_SUPERUSER_ONLY
+ },
+ &oauth_validator_command,
+ "",
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, NULL, NULL, NULL, NULL
diff --git a/src/include/libpq/hba.h b/src/include/libpq/hba.h
index 31a00c4b71..e405103a2e 100644
--- a/src/include/libpq/hba.h
+++ b/src/include/libpq/hba.h
@@ -39,8 +39,9 @@ typedef enum UserAuth
uaCert,
uaRADIUS,
uaCustom,
- uaPeer
-#define USER_AUTH_LAST uaPeer /* Must be last value of this enum */
+ uaPeer,
+ uaOAuth
+#define USER_AUTH_LAST uaOAuth /* Must be last value of this enum */
} UserAuth;
/*
@@ -128,6 +129,9 @@ typedef struct HbaLine
char *radiusidentifiers_s;
List *radiusports;
char *radiusports_s;
+ char *oauth_issuer;
+ char *oauth_scope;
+ bool oauth_skip_usermap;
char *custom_provider;
List *custom_auth_options;
} HbaLine;
diff --git a/src/include/libpq/oauth.h b/src/include/libpq/oauth.h
new file mode 100644
index 0000000000..870e426af1
--- /dev/null
+++ b/src/include/libpq/oauth.h
@@ -0,0 +1,24 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth.h
+ * Interface to libpq/auth-oauth.c
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/libpq/oauth.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_OAUTH_H
+#define PG_OAUTH_H
+
+#include "libpq/libpq-be.h"
+#include "libpq/sasl.h"
+
+extern char *oauth_validator_command;
+
+/* Implementation */
+extern const pg_be_sasl_mech pg_be_oauth_mech;
+
+#endif /* PG_OAUTH_H */
diff --git a/src/include/libpq/sasl.h b/src/include/libpq/sasl.h
index 39ccf8f0e3..f7d905591a 100644
--- a/src/include/libpq/sasl.h
+++ b/src/include/libpq/sasl.h
@@ -26,6 +26,14 @@
#define PG_SASL_EXCHANGE_SUCCESS 1
#define PG_SASL_EXCHANGE_FAILURE 2
+/*
+ * Maximum accepted size of SASL messages.
+ *
+ * The messages that the server or libpq generate are much smaller than this,
+ * but have some headroom.
+ */
+#define PG_MAX_SASL_MESSAGE_LENGTH 1024
+
/*
* Backend SASL mechanism callbacks.
*
@@ -127,6 +135,9 @@ typedef struct pg_be_sasl_mech
const char *input, int inputlen,
char **output, int *outputlen,
const char **logdetail);
+
+ /* The maximum size allowed for client SASLResponses. */
+ int max_message_length;
} pg_be_sasl_mech;
/* Common implementation for auth.c */
--
2.25.1
v4-0008-Add-a-very-simple-authn_id-extension.patch (text/x-patch, 2.8K) download
From 9dd8e024fde29239829e822b8f2b82028044cd8b Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 18 May 2021 15:01:29 -0700
Subject: [PATCH v4 08/10] Add a very simple authn_id extension
...for retrieving the authn_id from the server in tests.
---
contrib/authn_id/Makefile | 19 +++++++++++++++++++
contrib/authn_id/authn_id--1.0.sql | 8 ++++++++
contrib/authn_id/authn_id.c | 28 ++++++++++++++++++++++++++++
contrib/authn_id/authn_id.control | 5 +++++
4 files changed, 60 insertions(+)
create mode 100644 contrib/authn_id/Makefile
create mode 100644 contrib/authn_id/authn_id--1.0.sql
create mode 100644 contrib/authn_id/authn_id.c
create mode 100644 contrib/authn_id/authn_id.control
diff --git a/contrib/authn_id/Makefile b/contrib/authn_id/Makefile
new file mode 100644
index 0000000000..46026358e0
--- /dev/null
+++ b/contrib/authn_id/Makefile
@@ -0,0 +1,19 @@
+# contrib/authn_id/Makefile
+
+MODULE_big = authn_id
+OBJS = authn_id.o
+
+EXTENSION = authn_id
+DATA = authn_id--1.0.sql
+PGFILEDESC = "authn_id - information about the authenticated user"
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/authn_id
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/authn_id/authn_id--1.0.sql b/contrib/authn_id/authn_id--1.0.sql
new file mode 100644
index 0000000000..af2a4d3991
--- /dev/null
+++ b/contrib/authn_id/authn_id--1.0.sql
@@ -0,0 +1,8 @@
+/* contrib/authn_id/authn_id--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION authn_id" to load this file. \quit
+
+CREATE FUNCTION authn_id() RETURNS text
+AS 'MODULE_PATHNAME', 'authn_id'
+LANGUAGE C IMMUTABLE;
diff --git a/contrib/authn_id/authn_id.c b/contrib/authn_id/authn_id.c
new file mode 100644
index 0000000000..0fecac36a8
--- /dev/null
+++ b/contrib/authn_id/authn_id.c
@@ -0,0 +1,28 @@
+/*
+ * Extension to expose the current user's authn_id.
+ *
+ * contrib/authn_id/authn_id.c
+ */
+
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "libpq/libpq-be.h"
+#include "miscadmin.h"
+#include "utils/builtins.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(authn_id);
+
+/*
+ * Returns the current user's authenticated identity.
+ */
+Datum
+authn_id(PG_FUNCTION_ARGS)
+{
+ if (!MyProcPort->authn_id)
+ PG_RETURN_NULL();
+
+ PG_RETURN_TEXT_P(cstring_to_text(MyProcPort->authn_id));
+}
diff --git a/contrib/authn_id/authn_id.control b/contrib/authn_id/authn_id.control
new file mode 100644
index 0000000000..e0f9e06bed
--- /dev/null
+++ b/contrib/authn_id/authn_id.control
@@ -0,0 +1,5 @@
+# authn_id extension
+comment = 'current user identity'
+default_version = '1.0'
+module_pathname = '$libdir/authn_id'
+relocatable = true
--
2.25.1
v4-0009-Add-pytest-suite-for-OAuth.patch (text/x-patch, 131.6K) download
From 6d8fd9e5b352fd0847c9454ced2b763a6b11e73f Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Fri, 4 Jun 2021 09:06:38 -0700
Subject: [PATCH v4 09/10] Add pytest suite for OAuth
Requires Python 3; on the first run of `make installcheck` the
dependencies will be installed into ./venv for you. See the README for
more details.
---
src/test/python/.gitignore | 2 +
src/test/python/Makefile | 38 +
src/test/python/README | 54 ++
src/test/python/client/__init__.py | 0
src/test/python/client/conftest.py | 126 +++
src/test/python/client/test_client.py | 180 ++++
src/test/python/client/test_oauth.py | 936 ++++++++++++++++++
src/test/python/pq3.py | 727 ++++++++++++++
src/test/python/pytest.ini | 4 +
src/test/python/requirements.txt | 7 +
src/test/python/server/__init__.py | 0
src/test/python/server/conftest.py | 45 +
src/test/python/server/test_oauth.py | 1012 ++++++++++++++++++++
src/test/python/server/test_server.py | 21 +
src/test/python/server/validate_bearer.py | 101 ++
src/test/python/server/validate_reflect.py | 34 +
src/test/python/test_internals.py | 138 +++
src/test/python/test_pq3.py | 558 +++++++++++
src/test/python/tls.py | 195 ++++
19 files changed, 4178 insertions(+)
create mode 100644 src/test/python/.gitignore
create mode 100644 src/test/python/Makefile
create mode 100644 src/test/python/README
create mode 100644 src/test/python/client/__init__.py
create mode 100644 src/test/python/client/conftest.py
create mode 100644 src/test/python/client/test_client.py
create mode 100644 src/test/python/client/test_oauth.py
create mode 100644 src/test/python/pq3.py
create mode 100644 src/test/python/pytest.ini
create mode 100644 src/test/python/requirements.txt
create mode 100644 src/test/python/server/__init__.py
create mode 100644 src/test/python/server/conftest.py
create mode 100644 src/test/python/server/test_oauth.py
create mode 100644 src/test/python/server/test_server.py
create mode 100755 src/test/python/server/validate_bearer.py
create mode 100755 src/test/python/server/validate_reflect.py
create mode 100644 src/test/python/test_internals.py
create mode 100644 src/test/python/test_pq3.py
create mode 100644 src/test/python/tls.py
diff --git a/src/test/python/.gitignore b/src/test/python/.gitignore
new file mode 100644
index 0000000000..0e8f027b2e
--- /dev/null
+++ b/src/test/python/.gitignore
@@ -0,0 +1,2 @@
+__pycache__/
+/venv/
diff --git a/src/test/python/Makefile b/src/test/python/Makefile
new file mode 100644
index 0000000000..b0695b6287
--- /dev/null
+++ b/src/test/python/Makefile
@@ -0,0 +1,38 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+# Only Python 3 is supported, but if it's named something different on your
+# system you can override it with the PYTHON3 variable.
+PYTHON3 := python3
+
+# All dependencies are placed into this directory. The default is .gitignored
+# for you, but you can override it if you'd like.
+VENV := ./venv
+
+override VBIN := $(VENV)/bin
+override PIP := $(VBIN)/pip
+override PYTEST := $(VBIN)/py.test
+override ISORT := $(VBIN)/isort
+override BLACK := $(VBIN)/black
+
+.PHONY: installcheck indent
+
+installcheck: $(PYTEST)
+ $(PYTEST) -v -rs
+
+indent: $(ISORT) $(BLACK)
+ $(ISORT) --profile black *.py client/*.py server/*.py
+ $(BLACK) *.py client/*.py server/*.py
+
+$(PYTEST) $(ISORT) $(BLACK) &: requirements.txt | $(PIP)
+ $(PIP) install --force-reinstall -r $<
+
+$(PIP):
+ $(PYTHON3) -m venv $(VENV)
+
+# A convenience recipe to rebuild psycopg2 against the local libpq.
+.PHONY: rebuild-psycopg2
+rebuild-psycopg2: | $(PIP)
+ $(PIP) install --force-reinstall --no-binary :all: $(shell grep psycopg2 requirements.txt)
diff --git a/src/test/python/README b/src/test/python/README
new file mode 100644
index 0000000000..0bda582c4b
--- /dev/null
+++ b/src/test/python/README
@@ -0,0 +1,54 @@
+A test suite for exercising both the libpq client and the server backend at the
+protocol level, based on pytest and Construct.
+
+The test suite currently assumes that the standard PG* environment variables
+point to the database under test and are sufficient to log in a superuser on
+that system. In other words, a bare `psql` needs to Just Work before the test
+suite can do its thing. For a newly built dev cluster, typically all that I need
+to do is a
+
+ export PGDATABASE=postgres
+
+but you can adjust as needed for your setup.
+
+## Requirements
+
+A supported version (3.6+) of Python.
+
+The first run of
+
+ make installcheck
+
+will install a local virtual environment and all needed dependencies. During
+development, if libpq changes incompatibly, you can issue
+
+ $ make rebuild-psycopg2
+
+to force a rebuild of the client library.
+
+## Hacking
+
+The code style is enforced by a _very_ opinionated autoformatter. Running the
+
+ make indent
+
+recipe will invoke it for you automatically. Don't fight the tool; part of the
+zen is in knowing that if the formatter makes your code ugly, there's probably a
+cleaner way to write your code.
+
+## Advanced Usage
+
+The Makefile is there for convenience, but you don't have to use it. Activate
+the virtualenv to be able to use pytest directly:
+
+ $ source venv/bin/activate
+ $ py.test -k oauth
+ ...
+ $ py.test ./server/test_server.py
+ ...
+ $ deactivate # puts the PATH et al back the way it was before
+
+To make quick smoke tests possible, slow tests have been marked explicitly. You
+can skip them by saying e.g.
+
+ $ py.test -m 'not slow'
diff --git a/src/test/python/client/__init__.py b/src/test/python/client/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/src/test/python/client/conftest.py b/src/test/python/client/conftest.py
new file mode 100644
index 0000000000..f38da7a138
--- /dev/null
+++ b/src/test/python/client/conftest.py
@@ -0,0 +1,126 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import socket
+import sys
+import threading
+
+import psycopg2
+import pytest
+
+import pq3
+
+BLOCKING_TIMEOUT = 2 # the number of seconds to wait for blocking calls
+
+
+@pytest.fixture
+def server_socket(unused_tcp_port_factory):
+ """
+ Returns a listening socket bound to an ephemeral port.
+ """
+ with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+ s.bind(("127.0.0.1", unused_tcp_port_factory()))
+ s.listen(1)
+ s.settimeout(BLOCKING_TIMEOUT)
+ yield s
+
+
+class ClientHandshake(threading.Thread):
+ """
+ A thread that connects to a local Postgres server using psycopg2. Once the
+ opening handshake completes, the connection will be immediately closed.
+ """
+
+ def __init__(self, *, port, **kwargs):
+ super().__init__()
+
+ kwargs["port"] = port
+ self._kwargs = kwargs
+
+ self.exception = None
+
+ def run(self):
+ try:
+ conn = psycopg2.connect(host="127.0.0.1", **self._kwargs)
+ conn.close()
+ except Exception as e:
+ self.exception = e
+
+ def check_completed(self, timeout=BLOCKING_TIMEOUT):
+ """
+ Joins the client thread. Raises an exception if the thread could not be
+ joined, or if it threw an exception itself. (The exception will be
+ cleared, so future calls to check_completed will succeed.)
+ """
+ self.join(timeout)
+
+ if self.is_alive():
+ raise TimeoutError("client thread did not handshake within the timeout")
+ elif self.exception:
+ e = self.exception
+ self.exception = None
+ raise e
+
+
+@pytest.fixture
+def accept(server_socket):
+ """
+ Returns a factory function that, when called, returns a pair (sock, client)
+ where sock is a server socket that has accepted a connection from client,
+ and client is an instance of ClientHandshake. Clients will complete their
+ handshakes and cleanly disconnect.
+
+ The default connstring options may be extended or overridden by passing
+ arbitrary keyword arguments. Keep in mind that you generally should not
+ override the host or port, since they point to the local test server.
+
+ For situations where a client needs to connect more than once to complete a
+ handshake, the accept function may be called more than once. (The client
+ returned for subsequent calls will always be the same client that was
+ returned for the first call.)
+
+ Tests must either complete the handshake so that the client thread can be
+ automatically joined during teardown, or else call client.check_completed()
+ and manually handle any expected errors.
+ """
+ _, port = server_socket.getsockname()
+
+ client = None
+ default_opts = dict(
+ port=port,
+ user=pq3.pguser(),
+ sslmode="disable",
+ )
+
+ def factory(**kwargs):
+ nonlocal client
+
+ if client is None:
+ opts = dict(default_opts)
+ opts.update(kwargs)
+
+ # The server_socket is already listening, so the client thread can
+ # be safely started; it'll block on the connection until we accept.
+ client = ClientHandshake(**opts)
+ client.start()
+
+ sock, _ = server_socket.accept()
+ return sock, client
+
+ yield factory
+ client.check_completed()
+
+
+@pytest.fixture
+def conn(accept):
+ """
+ Returns an accepted, wrapped pq3 connection to a psycopg2 client. The socket
+ will be closed when the test finishes, and the client will be checked for a
+ cleanly completed handshake.
+ """
+ sock, client = accept()
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ yield conn
diff --git a/src/test/python/client/test_client.py b/src/test/python/client/test_client.py
new file mode 100644
index 0000000000..c4c946fda4
--- /dev/null
+++ b/src/test/python/client/test_client.py
@@ -0,0 +1,180 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import sys
+
+import psycopg2
+import pytest
+from cryptography.hazmat.primitives import hashes, hmac
+
+import pq3
+
+
+def finish_handshake(conn):
+ """
+ Sends the AuthenticationOK message and the standard opening salvo of server
+ messages, then asserts that the client immediately sends a Terminate message
+ to close the connection cleanly.
+ """
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.OK)
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"client_encoding", value=b"UTF-8")
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"DateStyle", value=b"ISO, MDY")
+ pq3.send(conn, pq3.types.ReadyForQuery, status=b"I")
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.Terminate
+
+
+def test_handshake(conn):
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ finish_handshake(conn)
+
+
+def test_aborted_connection(accept):
+ """
+ Make sure the client correctly reports an early close during handshakes.
+ """
+ sock, client = accept()
+ sock.close()
+
+ expected = "server closed the connection unexpectedly"
+ with pytest.raises(psycopg2.OperationalError, match=expected):
+ client.check_completed()
+
+
+#
+# SCRAM-SHA-256 (see RFC 5802: https://tools.ietf.org/html/rfc5802)
+#
+
+
+@pytest.fixture
+def password():
+ """
+ Returns a password for use by both client and server.
+ """
+ # TODO: parameterize this with passwords that require SASLprep.
+ return "secret"
+
+
+@pytest.fixture
+def pwconn(accept, password):
+ """
+ Like the conn fixture, but uses a password in the connection.
+ """
+ sock, client = accept(password=password)
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ yield conn
+
+
+def sha256(data):
+ """The H(str) function from Section 2.2."""
+ digest = hashes.Hash(hashes.SHA256())
+ digest.update(data)
+ return digest.finalize()
+
+
+def hmac_256(key, data):
+ """The HMAC(key, str) function from Section 2.2."""
+ h = hmac.HMAC(key, hashes.SHA256())
+ h.update(data)
+ return h.finalize()
+
+
+def xor(a, b):
+ """The XOR operation from Section 2.2."""
+ res = bytearray(a)
+ for i, byte in enumerate(b):
+ res[i] ^= byte
+ return bytes(res)
+
+
+def h_i(data, salt, i):
+ """The Hi(str, salt, i) function from Section 2.2."""
+ assert i > 0
+
+ acc = hmac_256(data, salt + b"\x00\x00\x00\x01")
+ last = acc
+ i -= 1
+
+ while i:
+ u = hmac_256(data, last)
+ acc = xor(acc, u)
+
+ last = u
+ i -= 1
+
+ return acc
+
+
+def test_scram(pwconn, password):
+ startup = pq3.recv1(pwconn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ pwconn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"SCRAM-SHA-256", b""],
+ )
+
+ # Get the client-first-message.
+ pkt = pq3.recv1(pwconn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ initial = pq3.SASLInitialResponse.parse(pkt.payload)
+ assert initial.name == b"SCRAM-SHA-256"
+
+ c_bind, authzid, c_name, c_nonce = initial.data.split(b",")
+ assert c_bind == b"n" # no channel bindings on a plaintext connection
+ assert authzid == b"" # we don't support authzid currently
+ assert c_name == b"n=" # libpq doesn't honor the GS2 username
+ assert c_nonce.startswith(b"r=")
+
+ # Send the server-first-message.
+ salt = b"12345"
+ iterations = 2
+
+ s_nonce = c_nonce + b"somenonce"
+ s_salt = b"s=" + base64.b64encode(salt)
+ s_iterations = b"i=%d" % iterations
+
+ msg = b",".join([s_nonce, s_salt, s_iterations])
+ pq3.send(pwconn, pq3.types.AuthnRequest, type=pq3.authn.SASLContinue, body=msg)
+
+ # Get the client-final-message.
+ pkt = pq3.recv1(pwconn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ c_bind_final, c_nonce_final, c_proof = pkt.payload.split(b",")
+ assert c_bind_final == b"c=" + base64.b64encode(c_bind + b"," + authzid + b",")
+ assert c_nonce_final == s_nonce
+
+ # Calculate what the client proof should be.
+ salted_password = h_i(password.encode("ascii"), salt, iterations)
+ client_key = hmac_256(salted_password, b"Client Key")
+ stored_key = sha256(client_key)
+
+ auth_message = b",".join(
+ [c_name, c_nonce, s_nonce, s_salt, s_iterations, c_bind_final, c_nonce_final]
+ )
+ client_signature = hmac_256(stored_key, auth_message)
+ client_proof = xor(client_key, client_signature)
+
+ expected = b"p=" + base64.b64encode(client_proof)
+ assert c_proof == expected
+
+ # Send the correct server signature.
+ server_key = hmac_256(salted_password, b"Server Key")
+ server_signature = hmac_256(server_key, auth_message)
+
+ s_verify = b"v=" + base64.b64encode(server_signature)
+ pq3.send(pwconn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal, body=s_verify)
+
+ # Done!
+ finish_handshake(pwconn)
diff --git a/src/test/python/client/test_oauth.py b/src/test/python/client/test_oauth.py
new file mode 100644
index 0000000000..a754a9c0b6
--- /dev/null
+++ b/src/test/python/client/test_oauth.py
@@ -0,0 +1,936 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import http.server
+import json
+import secrets
+import sys
+import threading
+import time
+import urllib.parse
+
+import psycopg2
+import pytest
+
+import pq3
+
+from .conftest import BLOCKING_TIMEOUT
+
+
+def finish_handshake(conn):
+ """
+ Sends the AuthenticationOK message and the standard opening salvo of server
+ messages, then asserts that the client immediately sends a Terminate message
+ to close the connection cleanly.
+ """
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.OK)
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"client_encoding", value=b"UTF-8")
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"DateStyle", value=b"ISO, MDY")
+ pq3.send(conn, pq3.types.ReadyForQuery, status=b"I")
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.Terminate
+
+
+#
+# OAUTHBEARER (see RFC 7628: https://tools.ietf.org/html/rfc7628)
+#
+
+
+def start_oauth_handshake(conn):
+ """
+ Negotiates an OAUTHBEARER SASL challenge. Returns the client's initial
+ response data.
+ """
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn, pq3.types.AuthnRequest, type=pq3.authn.SASL, body=[b"OAUTHBEARER", b""]
+ )
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ initial = pq3.SASLInitialResponse.parse(pkt.payload)
+ assert initial.name == b"OAUTHBEARER"
+
+ return initial.data
+
+
+def get_auth_value(initial):
+ """
+ Finds the auth value (e.g. "Bearer somedata..." in the client's initial SASL
+ response.
+ """
+ kvpairs = initial.split(b"\x01")
+ assert kvpairs[0] == b"n,," # no channel binding or authzid
+ assert kvpairs[2] == b"" # ends with an empty kvpair
+ assert kvpairs[3] == b"" # ...and there's nothing after it
+ assert len(kvpairs) == 4
+
+ key, value = kvpairs[1].split(b"=", 2)
+ assert key == b"auth"
+
+ return value
+
+
+def xtest_oauth_success(conn): # TODO
+ initial = start_oauth_handshake(conn)
+
+ auth = get_auth_value(initial)
+ assert auth.startswith(b"Bearer ")
+
+ # Accept the token. TODO actually validate
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+class OpenIDProvider(threading.Thread):
+ """
+ A thread that runs a mock OpenID provider server.
+ """
+
+ def __init__(self, *, port):
+ super().__init__()
+
+ self.exception = None
+
+ addr = ("", port)
+ self.server = self._Server(addr, self._Handler)
+
+ # TODO: allow HTTPS only, somehow
+ oauth = self._OAuthState()
+ oauth.host = f"localhost:{port}"
+ oauth.issuer = f"http://localhost:{port}"
+
+ # The following endpoints are required to be advertised by providers,
+ # even though our chosen client implementation does not actually make
+ # use of them.
+ oauth.register_endpoint(
+ "authorization_endpoint", "POST", "/authorize", self._authorization_handler
+ )
+ oauth.register_endpoint("jwks_uri", "GET", "/keys", self._jwks_handler)
+
+ self.server.oauth = oauth
+
+ def run(self):
+ try:
+ self.server.serve_forever()
+ except Exception as e:
+ self.exception = e
+
+ def stop(self, timeout=BLOCKING_TIMEOUT):
+ """
+ Shuts down the server and joins its thread. Raises an exception if the
+ thread could not be joined, or if it threw an exception itself. Must
+ only be called once, after start().
+ """
+ self.server.shutdown()
+ self.join(timeout)
+
+ if self.is_alive():
+ raise TimeoutError("client thread did not handshake within the timeout")
+ elif self.exception:
+ e = self.exception
+ raise e
+
+ class _OAuthState(object):
+ def __init__(self):
+ self.endpoint_paths = {}
+ self._endpoints = {}
+
+ def register_endpoint(self, name, method, path, func):
+ if method not in self._endpoints:
+ self._endpoints[method] = {}
+
+ self._endpoints[method][path] = func
+ self.endpoint_paths[name] = path
+
+ def endpoint(self, method, path):
+ if method not in self._endpoints:
+ return None
+
+ return self._endpoints[method].get(path)
+
+ class _Server(http.server.HTTPServer):
+ def handle_error(self, request, addr):
+ self.shutdown_request(request)
+ raise
+
+ @staticmethod
+ def _jwks_handler(headers, params):
+ return 200, {"keys": []}
+
+ @staticmethod
+ def _authorization_handler(headers, params):
+ # We don't actually want this to be called during these tests -- we
+ # should be using the device authorization endpoint instead.
+ assert (
+ False
+ ), "authorization handler called instead of device authorization handler"
+
+ class _Handler(http.server.BaseHTTPRequestHandler):
+ timeout = BLOCKING_TIMEOUT
+
+ def _discovery_handler(self, headers, params):
+ oauth = self.server.oauth
+
+ doc = {
+ "issuer": oauth.issuer,
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ }
+
+ for name, path in oauth.endpoint_paths.items():
+ doc[name] = oauth.issuer + path
+
+ return 200, doc
+
+ def _handle(self, *, params=None, handler=None):
+ oauth = self.server.oauth
+ assert self.headers["Host"] == oauth.host
+
+ if handler is None:
+ handler = oauth.endpoint(self.command, self.path)
+ assert (
+ handler is not None
+ ), f"no registered endpoint for {self.command} {self.path}"
+
+ code, resp = handler(self.headers, params)
+
+ self.send_response(code)
+ self.send_header("Content-Type", "application/json")
+ self.end_headers()
+
+ resp = json.dumps(resp)
+ resp = resp.encode("utf-8")
+ self.wfile.write(resp)
+
+ self.close_connection = True
+
+ def do_GET(self):
+ if self.path == "/.well-known/openid-configuration":
+ self._handle(handler=self._discovery_handler)
+ return
+
+ self._handle()
+
+ def _request_body(self):
+ length = self.headers["Content-Length"]
+
+ # Handle only an explicit content-length.
+ assert length is not None
+ length = int(length)
+
+ return self.rfile.read(length).decode("utf-8")
+
+ def do_POST(self):
+ assert self.headers["Content-Type"] == "application/x-www-form-urlencoded"
+
+ body = self._request_body()
+ params = urllib.parse.parse_qs(body)
+
+ self._handle(params=params)
+
+
+@pytest.fixture
+def openid_provider(unused_tcp_port_factory):
+ """
+ A fixture that returns the OAuth state of a running OpenID provider server. The
+ server will be stopped when the fixture is torn down.
+ """
+ thread = OpenIDProvider(port=unused_tcp_port_factory())
+ thread.start()
+
+ try:
+ yield thread.server.oauth
+ finally:
+ thread.stop()
+
+
+@pytest.mark.parametrize("secret", [None, "", "hunter2"])
+@pytest.mark.parametrize("scope", [None, "", "openid email"])
+@pytest.mark.parametrize("retries", [0, 1])
+def test_oauth_with_explicit_issuer(
+ capfd, accept, openid_provider, retries, scope, secret
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ oauth_client_secret=secret,
+ oauth_scope=scope,
+ )
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+ verification_url = "https://example.com/device"
+
+ access_token = secrets.token_urlsafe()
+
+ def check_client_authn(headers, params):
+ if not secret:
+ assert params["client_id"] == [client_id]
+ return
+
+ # Require the client to use Basic authn; request-body credentials are
+ # NOT RECOMMENDED (RFC 6749, Sec. 2.3.1).
+ assert "Authorization" in headers
+
+ method, creds = headers["Authorization"].split()
+ assert method == "Basic"
+
+ expected = f"{client_id}:{secret}"
+ assert base64.b64decode(creds) == expected.encode("ascii")
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ check_client_authn(headers, params)
+
+ if scope:
+ assert params["scope"] == [scope]
+ else:
+ assert "scope" not in params
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": verification_url,
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ attempts = 0
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ check_client_authn(headers, params)
+
+ assert params["grant_type"] == ["urn:ietf:params:oauth:grant-type:device_code"]
+ assert params["device_code"] == [device_code]
+
+ now = time.monotonic()
+
+ with retry_lock:
+ nonlocal attempts
+
+ # If the test wants to force the client to retry, return an
+ # authorization_pending response and decrement the retry count.
+ if attempts < retries:
+ attempts += 1
+ return 400, {"error": "authorization_pending"}
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+ if retries:
+ # Finally, make sure that the client prompted the user with the expected
+ # authorization URL and user code.
+ expected = f"Visit {verification_url} and enter the code: {user_code}"
+ _, stderr = capfd.readouterr()
+ assert expected in stderr
+
+
+def test_oauth_requires_client_id(accept, openid_provider):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ # Do not set a client ID; this should cause a client error after the
+ # server asks for OAUTHBEARER and the client tries to contact the
+ # issuer.
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake.
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"OAUTHBEARER", b""],
+ )
+
+ # The client should disconnect at this point.
+ assert not conn.read()
+
+ expected_error = "no oauth_client_id is set"
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.slow
+@pytest.mark.parametrize("error_code", ["authorization_pending", "slow_down"])
+@pytest.mark.parametrize("retries", [1, 2])
+def test_oauth_retry_interval(accept, openid_provider, retries, error_code):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id="some-id",
+ )
+
+ expected_retry_interval = 1
+ access_token = secrets.token_urlsafe()
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ resp = {
+ "device_code": "my-device-code",
+ "user_code": "my-user-code",
+ "interval": expected_retry_interval,
+ "verification_uri": "https://example.com",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ attempts = 0
+ last_retry = None
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ now = time.monotonic()
+
+ with retry_lock:
+ nonlocal attempts, last_retry, expected_retry_interval
+
+ # Make sure the retry interval is being respected by the client.
+ if last_retry is not None:
+ interval = now - last_retry
+ assert interval >= expected_retry_interval
+
+ last_retry = now
+
+ # If the test wants to force the client to retry, return the desired
+ # error response and decrement the retry count.
+ if attempts < retries:
+ attempts += 1
+
+ # A slow_down code requires the client to additionally increase
+ # its interval by five seconds.
+ if error_code == "slow_down":
+ expected_retry_interval += 5
+
+ return 400, {"error": error_code}
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+@pytest.mark.parametrize(
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
+ {
+ "error": "invalid_client",
+ "error_description": "client authentication failed",
+ },
+ r"client authentication failed \(invalid_client\)",
+ id="authentication failure with description",
+ ),
+ pytest.param(
+ {"error": "invalid_request"},
+ r"\(invalid_request\)",
+ id="invalid request without description",
+ ),
+ pytest.param(
+ {},
+ r"failed to obtain device authorization",
+ id="broken error response",
+ ),
+ ],
+)
+def test_oauth_device_authorization_failures(
+ accept, openid_provider, failure_mode, error_pattern
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ )
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ return 400, failure_mode
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert False, "token endpoint was invoked unexpectedly"
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"OAUTHBEARER", b""],
+ )
+
+ # The client should not continue the connection due to the hardcoded
+ # provider failure; we disconnect here.
+
+ # Now make sure the client correctly failed.
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
+ {
+ "error": "expired_token",
+ "error_description": "the device code has expired",
+ },
+ r"the device code has expired \(expired_token\)",
+ id="expired token with description",
+ ),
+ pytest.param(
+ {"error": "access_denied"},
+ r"\(access_denied\)",
+ id="access denied without description",
+ ),
+ pytest.param(
+ {},
+ r"OAuth token retrieval failed",
+ id="broken error response",
+ ),
+ ],
+)
+@pytest.mark.parametrize("retries", [0, 1])
+def test_oauth_token_failures(
+ accept, openid_provider, retries, failure_mode, error_pattern
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ )
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ assert params["client_id"] == [client_id]
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": "https://example.com/device",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ with retry_lock:
+ nonlocal retries
+
+ # If the test wants to force the client to retry, return an
+ # authorization_pending response and decrement the retry count.
+ if retries > 0:
+ retries -= 1
+ return 400, {"error": "authorization_pending"}
+
+ return 400, failure_mode
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"OAUTHBEARER", b""],
+ )
+
+ # The client should not continue the connection due to the hardcoded
+ # provider failure; we disconnect here.
+
+ # Now make sure the client correctly failed.
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize("scope", [None, "openid email"])
+@pytest.mark.parametrize(
+ "base_response",
+ [
+ {"status": "invalid_token"},
+ {"extra_object": {"key": "value"}, "status": "invalid_token"},
+ {"extra_object": {"status": 1}, "status": "invalid_token"},
+ ],
+)
+def test_oauth_discovery(accept, openid_provider, base_response, scope):
+ sock, client = accept(oauth_client_id=secrets.token_hex())
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+ verification_url = "https://example.com/device"
+
+ access_token = secrets.token_urlsafe()
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ if scope:
+ assert params["scope"] == [scope]
+ else:
+ assert "scope" not in params
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": verification_url,
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert params["grant_type"] == ["urn:ietf:params:oauth:grant-type:device_code"]
+ assert params["device_code"] == [device_code]
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # For discovery, the client should send an empty auth header. See
+ # RFC 7628, Sec. 4.3.
+ auth = get_auth_value(initial)
+ assert auth == b""
+
+ # We will fail the first SASL exchange. First return a link to the
+ # discovery document, pointing to the test provider server.
+ resp = dict(base_response)
+
+ discovery_uri = f"{openid_provider.issuer}/.well-known/openid-configuration"
+ resp["openid-configuration"] = discovery_uri
+
+ if scope:
+ resp["scope"] = scope
+
+ resp = json.dumps(resp)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASLContinue,
+ body=resp.encode("ascii"),
+ )
+
+ # Per RFC, the client is required to send a dummy ^A response.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+ assert pkt.payload == b"\x01"
+
+ # Now fail the SASL exchange.
+ pq3.send(
+ conn,
+ pq3.types.ErrorResponse,
+ fields=[
+ b"SFATAL",
+ b"C28000",
+ b"Mdoesn't matter",
+ b"",
+ ],
+ )
+
+ # The client will connect to us a second time, using the parameters we sent
+ # it.
+ sock, _ = accept()
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+@pytest.mark.parametrize(
+ "response,expected_error",
+ [
+ pytest.param(
+ "abcde",
+ 'Token "abcde" is invalid',
+ id="bad JSON: invalid syntax",
+ ),
+ pytest.param(
+ '"abcde"',
+ "top-level element must be an object",
+ id="bad JSON: top-level element is a string",
+ ),
+ pytest.param(
+ "[]",
+ "top-level element must be an object",
+ id="bad JSON: top-level element is an array",
+ ),
+ pytest.param(
+ "{}",
+ "server sent error response without a status",
+ id="bad JSON: no status member",
+ ),
+ pytest.param(
+ '{ "status": null }',
+ 'field "status" must be a string',
+ id="bad JSON: null status member",
+ ),
+ pytest.param(
+ '{ "status": 0 }',
+ 'field "status" must be a string',
+ id="bad JSON: int status member",
+ ),
+ pytest.param(
+ '{ "status": [ "bad" ] }',
+ 'field "status" must be a string',
+ id="bad JSON: array status member",
+ ),
+ pytest.param(
+ '{ "status": { "bad": "bad" } }',
+ 'field "status" must be a string',
+ id="bad JSON: object status member",
+ ),
+ pytest.param(
+ '{ "nested": { "status": "bad" } }',
+ "server sent error response without a status",
+ id="bad JSON: nested status",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token" ',
+ "The input string ended unexpectedly",
+ id="bad JSON: unterminated object",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token" } { }',
+ 'Expected end of input, but found "{"',
+ id="bad JSON: trailing data",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "openid-configuration": 1 }',
+ 'field "openid-configuration" must be a string',
+ id="bad JSON: int openid-configuration member",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "openid-configuration": 1 }',
+ 'field "openid-configuration" must be a string',
+ id="bad JSON: int openid-configuration member",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "scope": 1 }',
+ 'field "scope" must be a string',
+ id="bad JSON: int scope member",
+ ),
+ ],
+)
+def test_oauth_discovery_server_error(accept, response, expected_error):
+ sock, client = accept(oauth_client_id=secrets.token_hex())
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # Fail the SASL exchange with an invalid JSON response.
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASLContinue,
+ body=response.encode("utf-8"),
+ )
+
+ # The client should disconnect, so the socket is closed here. (If
+ # the client doesn't disconnect, it will report a different error
+ # below and the test will fail.)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "sasl_err,resp_type,resp_payload,expected_error",
+ [
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.ErrorResponse,
+ dict(
+ fields=[b"SFATAL", b"C28000", b"Mexpected error message", b""],
+ ),
+ "expected error message",
+ id="standard server error: invalid_request",
+ ),
+ pytest.param(
+ {"status": "invalid_token"},
+ pq3.types.ErrorResponse,
+ dict(
+ fields=[b"SFATAL", b"C28000", b"Mexpected error message", b""],
+ ),
+ "expected error message",
+ id="standard server error: invalid_token without discovery URI",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASLContinue, body=b""),
+ "server sent additional OAuth data",
+ id="broken server: additional challenge after error",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASLFinal),
+ "server sent additional OAuth data",
+ id="broken server: SASL success after error",
+ ),
+ ],
+)
+def test_oauth_server_error(accept, sasl_err, resp_type, resp_payload, expected_error):
+ sock, client = accept()
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ start_oauth_handshake(conn)
+
+ # Ignore the client data. Return an error "challenge".
+ resp = json.dumps(sasl_err)
+ resp = resp.encode("utf-8")
+
+ pq3.send(
+ conn, pq3.types.AuthnRequest, type=pq3.authn.SASLContinue, body=resp
+ )
+
+ # Per RFC, the client is required to send a dummy ^A response.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+ assert pkt.payload == b"\x01"
+
+ # Now fail the SASL exchange (in either a valid way, or an invalid
+ # one, depending on the test).
+ pq3.send(conn, resp_type, **resp_payload)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
diff --git a/src/test/python/pq3.py b/src/test/python/pq3.py
new file mode 100644
index 0000000000..3a22dad0b6
--- /dev/null
+++ b/src/test/python/pq3.py
@@ -0,0 +1,727 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import getpass
+import io
+import os
+import ssl
+import sys
+import textwrap
+
+from construct import *
+
+import tls
+
+
+def protocol(major, minor):
+ """
+ Returns the protocol version, in integer format, corresponding to the given
+ major and minor version numbers.
+ """
+ return (major << 16) | minor
+
+
+# Startup
+
+StringList = GreedyRange(NullTerminated(GreedyBytes))
+
+
+class KeyValueAdapter(Adapter):
+ """
+ Turns a key-value store into a null-terminated list of null-terminated
+ strings, as presented on the wire in the startup packet.
+ """
+
+ def _encode(self, obj, context, path):
+ if isinstance(obj, list):
+ return obj
+
+ l = []
+
+ for k, v in obj.items():
+ if isinstance(k, str):
+ k = k.encode("utf-8")
+ l.append(k)
+
+ if isinstance(v, str):
+ v = v.encode("utf-8")
+ l.append(v)
+
+ l.append(b"")
+ return l
+
+ def _decode(self, obj, context, path):
+ # TODO: turn a list back into a dict
+ return obj
+
+
+KeyValues = KeyValueAdapter(StringList)
+
+_startup_payload = Switch(
+ this.proto,
+ {
+ protocol(3, 0): KeyValues,
+ },
+ default=GreedyBytes,
+)
+
+
+def _default_protocol(this):
+ try:
+ if isinstance(this.payload, (list, dict)):
+ return protocol(3, 0)
+ except AttributeError:
+ pass # no payload passed during build
+
+ return 0
+
+
+def _startup_payload_len(this):
+ """
+ The payload field has a fixed size based on the length of the packet. But
+ if the caller hasn't supplied an explicit length at build time, we have to
+ build the payload to figure out how long it is, which requires us to know
+ the length first... This function exists solely to break the cycle.
+ """
+ assert this._building, "_startup_payload_len() cannot be called during parsing"
+
+ try:
+ payload = this.payload
+ except AttributeError:
+ return 0 # no payload
+
+ if isinstance(payload, bytes):
+ # already serialized; just use the given length
+ return len(payload)
+
+ try:
+ proto = this.proto
+ except AttributeError:
+ proto = _default_protocol(this)
+
+ data = _startup_payload.build(payload, proto=proto)
+ return len(data)
+
+
+Startup = Struct(
+ "len" / Default(Int32sb, lambda this: _startup_payload_len(this) + 8),
+ "proto" / Default(Hex(Int32sb), _default_protocol),
+ "payload" / FixedSized(this.len - 8, Default(_startup_payload, b"")),
+)
+
+# Pq3
+
+# Adapted from construct.core.EnumIntegerString
+class EnumNamedByte:
+ def __init__(self, val, name):
+ self._val = val
+ self._name = name
+
+ def __int__(self):
+ return ord(self._val)
+
+ def __str__(self):
+ return "(enum) %s %r" % (self._name, self._val)
+
+ def __repr__(self):
+ return "EnumNamedByte(%r)" % self._val
+
+ def __eq__(self, other):
+ if isinstance(other, EnumNamedByte):
+ other = other._val
+ if not isinstance(other, bytes):
+ return NotImplemented
+
+ return self._val == other
+
+ def __hash__(self):
+ return hash(self._val)
+
+
+# Adapted from construct.core.Enum
+class ByteEnum(Adapter):
+ def __init__(self, **mapping):
+ super(ByteEnum, self).__init__(Byte)
+ self.namemapping = {k: EnumNamedByte(v, k) for k, v in mapping.items()}
+ self.decmapping = {v: EnumNamedByte(v, k) for k, v in mapping.items()}
+
+ def __getattr__(self, name):
+ if name in self.namemapping:
+ return self.decmapping[self.namemapping[name]]
+ raise AttributeError
+
+ def _decode(self, obj, context, path):
+ b = bytes([obj])
+ try:
+ return self.decmapping[b]
+ except KeyError:
+ return EnumNamedByte(b, "(unknown)")
+
+ def _encode(self, obj, context, path):
+ if isinstance(obj, int):
+ return obj
+ elif isinstance(obj, bytes):
+ return ord(obj)
+ return int(obj)
+
+
+types = ByteEnum(
+ ErrorResponse=b"E",
+ ReadyForQuery=b"Z",
+ Query=b"Q",
+ EmptyQueryResponse=b"I",
+ AuthnRequest=b"R",
+ PasswordMessage=b"p",
+ BackendKeyData=b"K",
+ CommandComplete=b"C",
+ ParameterStatus=b"S",
+ DataRow=b"D",
+ Terminate=b"X",
+)
+
+
+authn = Enum(
+ Int32ub,
+ OK=0,
+ SASL=10,
+ SASLContinue=11,
+ SASLFinal=12,
+)
+
+
+_authn_body = Switch(
+ this.type,
+ {
+ authn.OK: Terminated,
+ authn.SASL: StringList,
+ },
+ default=GreedyBytes,
+)
+
+
+def _data_len(this):
+ assert this._building, "_data_len() cannot be called during parsing"
+
+ if not hasattr(this, "data") or this.data is None:
+ return -1
+
+ return len(this.data)
+
+
+# The protocol reuses the PasswordMessage for several authentication response
+# types, and there's no good way to figure out which is which without keeping
+# state for the entire stream. So this is a separate Construct that can be
+# explicitly parsed/built by code that knows it's needed.
+SASLInitialResponse = Struct(
+ "name" / NullTerminated(GreedyBytes),
+ "len" / Default(Int32sb, lambda this: _data_len(this)),
+ "data"
+ / IfThenElse(
+ # Allow tests to explicitly pass an incorrect length during testing, by
+ # not enforcing a FixedSized during build. (The len calculation above
+ # defaults to the correct size.)
+ this._building,
+ Optional(GreedyBytes),
+ If(this.len != -1, Default(FixedSized(this.len, GreedyBytes), b"")),
+ ),
+ Terminated, # make sure the entire response is consumed
+)
+
+
+_column = FocusedSeq(
+ "data",
+ "len" / Default(Int32sb, lambda this: _data_len(this)),
+ "data" / If(this.len != -1, FixedSized(this.len, GreedyBytes)),
+)
+
+
+_payload_map = {
+ types.ErrorResponse: Struct("fields" / StringList),
+ types.ReadyForQuery: Struct("status" / Bytes(1)),
+ types.Query: Struct("query" / NullTerminated(GreedyBytes)),
+ types.EmptyQueryResponse: Terminated,
+ types.AuthnRequest: Struct("type" / authn, "body" / Default(_authn_body, b"")),
+ types.BackendKeyData: Struct("pid" / Int32ub, "key" / Hex(Int32ub)),
+ types.CommandComplete: Struct("tag" / NullTerminated(GreedyBytes)),
+ types.ParameterStatus: Struct(
+ "name" / NullTerminated(GreedyBytes), "value" / NullTerminated(GreedyBytes)
+ ),
+ types.DataRow: Struct("columns" / Default(PrefixedArray(Int16sb, _column), b"")),
+ types.Terminate: Terminated,
+}
+
+
+_payload = FocusedSeq(
+ "_payload",
+ "_payload"
+ / Switch(
+ this._.type,
+ _payload_map,
+ default=GreedyBytes,
+ ),
+ Terminated, # make sure every payload consumes the entire packet
+)
+
+
+def _payload_len(this):
+ """
+ See _startup_payload_len() for an explanation.
+ """
+ assert this._building, "_payload_len() cannot be called during parsing"
+
+ try:
+ payload = this.payload
+ except AttributeError:
+ return 0 # no payload
+
+ if isinstance(payload, bytes):
+ # already serialized; just use the given length
+ return len(payload)
+
+ data = _payload.build(payload, type=this.type)
+ return len(data)
+
+
+Pq3 = Struct(
+ "type" / types,
+ "len" / Default(Int32ub, lambda this: _payload_len(this) + 4),
+ "payload" / FixedSized(this.len - 4, Default(_payload, b"")),
+)
+
+
+# Environment
+
+
+def pghost():
+ return os.environ.get("PGHOST", default="localhost")
+
+
+def pgport():
+ return int(os.environ.get("PGPORT", default=5432))
+
+
+def pguser():
+ try:
+ return os.environ["PGUSER"]
+ except KeyError:
+ return getpass.getuser()
+
+
+def pgdatabase():
+ return os.environ.get("PGDATABASE", default="postgres")
+
+
+# Connections
+
+
+def _hexdump_translation_map():
+ """
+ For hexdumps. Translates any unprintable or non-ASCII bytes into '.'.
+ """
+ input = bytearray()
+
+ for i in range(128):
+ c = chr(i)
+
+ if not c.isprintable():
+ input += bytes([i])
+
+ input += bytes(range(128, 256))
+
+ return bytes.maketrans(input, b"." * len(input))
+
+
+class _DebugStream(object):
+ """
+ Wraps a file-like object and adds hexdumps of the read and write data. Call
+ end_packet() on a _DebugStream to write the accumulated hexdumps to the
+ output stream, along with the packet that was sent.
+ """
+
+ _translation_map = _hexdump_translation_map()
+
+ def __init__(self, stream, out=sys.stdout):
+ """
+ Creates a new _DebugStream wrapping the given stream (which must have
+ been created by wrap()). All attributes not provided by the _DebugStream
+ are delegated to the wrapped stream. out is the text stream to which
+ hexdumps are written.
+ """
+ self.raw = stream
+ self._out = out
+ self._rbuf = io.BytesIO()
+ self._wbuf = io.BytesIO()
+
+ def __getattr__(self, name):
+ return getattr(self.raw, name)
+
+ def __setattr__(self, name, value):
+ if name in ("raw", "_out", "_rbuf", "_wbuf"):
+ return object.__setattr__(self, name, value)
+
+ setattr(self.raw, name, value)
+
+ def read(self, *args, **kwargs):
+ buf = self.raw.read(*args, **kwargs)
+
+ self._rbuf.write(buf)
+ return buf
+
+ def write(self, b):
+ self._wbuf.write(b)
+ return self.raw.write(b)
+
+ def recv(self, *args):
+ buf = self.raw.recv(*args)
+
+ self._rbuf.write(buf)
+ return buf
+
+ def _flush(self, buf, prefix):
+ width = 16
+ hexwidth = width * 3 - 1
+
+ count = 0
+ buf.seek(0)
+
+ while True:
+ line = buf.read(16)
+
+ if not line:
+ if count:
+ self._out.write("\n") # separate the output block with a newline
+ return
+
+ self._out.write("%s %04X:\t" % (prefix, count))
+ self._out.write("%*s\t" % (-hexwidth, line.hex(" ")))
+ self._out.write(line.translate(self._translation_map).decode("ascii"))
+ self._out.write("\n")
+
+ count += 16
+
+ def print_debug(self, obj, *, prefix=""):
+ contents = ""
+ if obj is not None:
+ contents = str(obj)
+
+ for line in contents.splitlines():
+ self._out.write("%s%s\n" % (prefix, line))
+
+ self._out.write("\n")
+
+ def flush_debug(self, *, prefix=""):
+ self._flush(self._rbuf, prefix + "<")
+ self._rbuf = io.BytesIO()
+
+ self._flush(self._wbuf, prefix + ">")
+ self._wbuf = io.BytesIO()
+
+ def end_packet(self, pkt, *, read=False, prefix="", indent=" "):
+ """
+ Marks the end of a logical "packet" of data. A string representation of
+ pkt will be printed, and the debug buffers will be flushed with an
+ indent. All lines can be optionally prefixed.
+
+ If read is True, the packet representation is written after the debug
+ buffers; otherwise the default of False (meaning write) causes the
+ packet representation to be dumped first. This is meant to capture the
+ logical flow of layer translation.
+ """
+ write = not read
+
+ if write:
+ self.print_debug(pkt, prefix=prefix + "> ")
+
+ self.flush_debug(prefix=prefix + indent)
+
+ if read:
+ self.print_debug(pkt, prefix=prefix + "< ")
+
+
+@contextlib.contextmanager
+def wrap(socket, *, debug_stream=None):
+ """
+ Transforms a raw socket into a connection that can be used for Construct
+ building and parsing. The return value is a context manager and can be used
+ in a with statement.
+ """
+ # It is critical that buffering be disabled here, so that we can still
+ # manipulate the raw socket without desyncing the stream.
+ with socket.makefile("rwb", buffering=0) as sfile:
+ # Expose the original socket's recv() on the SocketIO object we return.
+ def recv(self, *args):
+ return socket.recv(*args)
+
+ sfile.recv = recv.__get__(sfile)
+
+ conn = sfile
+ if debug_stream:
+ conn = _DebugStream(conn, debug_stream)
+
+ try:
+ yield conn
+ finally:
+ if debug_stream:
+ conn.flush_debug(prefix="? ")
+
+
+def _send(stream, cls, obj):
+ debugging = hasattr(stream, "flush_debug")
+ out = io.BytesIO()
+
+ # Ideally we would build directly to the passed stream, but because we need
+ # to reparse the generated output for the debugging case, build to an
+ # intermediate BytesIO and send it instead.
+ cls.build_stream(obj, out)
+ buf = out.getvalue()
+
+ stream.write(buf)
+ if debugging:
+ pkt = cls.parse(buf)
+ stream.end_packet(pkt)
+
+ stream.flush()
+
+
+def send(stream, packet_type, payload_data=None, **payloadkw):
+ """
+ Sends a packet on the given pq3 connection. type is the pq3.types member
+ that should be assigned to the packet. If payload_data is given, it will be
+ used as the packet payload; otherwise the key/value pairs in payloadkw will
+ be the payload contents.
+ """
+ data = payloadkw
+
+ if payload_data is not None:
+ if payloadkw:
+ raise ValueError(
+ "payload_data and payload keywords may not be used simultaneously"
+ )
+
+ data = payload_data
+
+ _send(stream, Pq3, dict(type=packet_type, payload=data))
+
+
+def send_startup(stream, proto=None, **kwargs):
+ """
+ Sends a startup packet on the given pq3 connection. In most cases you should
+ use the handshake functions instead, which will do this for you.
+
+ By default, a protocol version 3 packet will be sent. This can be overridden
+ with the proto parameter.
+ """
+ pkt = {}
+
+ if proto is not None:
+ pkt["proto"] = proto
+ if kwargs:
+ pkt["payload"] = kwargs
+
+ _send(stream, Startup, pkt)
+
+
+def recv1(stream, *, cls=Pq3):
+ """
+ Receives a single pq3 packet from the given stream and returns it.
+ """
+ resp = cls.parse_stream(stream)
+
+ debugging = hasattr(stream, "flush_debug")
+ if debugging:
+ stream.end_packet(resp, read=True)
+
+ return resp
+
+
+def handshake(stream, **kwargs):
+ """
+ Performs a libpq v3 startup handshake. kwargs should contain the key/value
+ parameters to send to the server in the startup packet.
+ """
+ # Send our startup parameters.
+ send_startup(stream, **kwargs)
+
+ # Receive and dump packets until the server indicates it's ready for our
+ # first query.
+ while True:
+ resp = recv1(stream)
+ if resp is None:
+ raise RuntimeError("server closed connection during handshake")
+
+ if resp.type == types.ReadyForQuery:
+ return
+ elif resp.type == types.ErrorResponse:
+ raise RuntimeError(
+ f"received error response from peer: {resp.payload.fields!r}"
+ )
+
+
+# TLS
+
+
+class _TLSStream(object):
+ """
+ A file-like object that performs TLS encryption/decryption on a wrapped
+ stream. Differs from ssl.SSLSocket in that we have full visibility and
+ control over the TLS layer.
+ """
+
+ def __init__(self, stream, context):
+ self._stream = stream
+ self._debugging = hasattr(stream, "flush_debug")
+
+ self._in = ssl.MemoryBIO()
+ self._out = ssl.MemoryBIO()
+ self._ssl = context.wrap_bio(self._in, self._out)
+
+ def handshake(self):
+ try:
+ self._pump(lambda: self._ssl.do_handshake())
+ finally:
+ self._flush_debug(prefix="? ")
+
+ def read(self, *args):
+ return self._pump(lambda: self._ssl.read(*args))
+
+ def write(self, *args):
+ return self._pump(lambda: self._ssl.write(*args))
+
+ def _decode(self, buf):
+ """
+ Attempts to decode a buffer of TLS data into a packet representation
+ that can be printed.
+
+ TODO: handle buffers (and record fragments) that don't align with packet
+ boundaries.
+ """
+ end = len(buf)
+ bio = io.BytesIO(buf)
+
+ ret = io.StringIO()
+
+ while bio.tell() < end:
+ record = tls.Plaintext.parse_stream(bio)
+
+ if ret.tell() > 0:
+ ret.write("\n")
+ ret.write("[Record] ")
+ ret.write(str(record))
+ ret.write("\n")
+
+ if record.type == tls.ContentType.handshake:
+ record_cls = tls.Handshake
+ else:
+ continue
+
+ innerlen = len(record.fragment)
+ inner = io.BytesIO(record.fragment)
+
+ while inner.tell() < innerlen:
+ msg = record_cls.parse_stream(inner)
+
+ indented = "[Message] " + str(msg)
+ indented = textwrap.indent(indented, " ")
+
+ ret.write("\n")
+ ret.write(indented)
+ ret.write("\n")
+
+ return ret.getvalue()
+
+ def flush(self):
+ if not self._out.pending:
+ self._stream.flush()
+ return
+
+ buf = self._out.read()
+ self._stream.write(buf)
+
+ if self._debugging:
+ pkt = self._decode(buf)
+ self._stream.end_packet(pkt, prefix=" ")
+
+ self._stream.flush()
+
+ def _pump(self, operation):
+ while True:
+ try:
+ return operation()
+ except (ssl.SSLWantReadError, ssl.SSLWantWriteError) as e:
+ want = e
+ self._read_write(want)
+
+ def _recv(self, maxsize):
+ buf = self._stream.recv(4096)
+ if not buf:
+ self._in.write_eof()
+ return
+
+ self._in.write(buf)
+
+ if not self._debugging:
+ return
+
+ pkt = self._decode(buf)
+ self._stream.end_packet(pkt, read=True, prefix=" ")
+
+ def _read_write(self, want):
+ # XXX This needs work. So many corner cases yet to handle. For one,
+ # doing blocking writes in flush may lead to distributed deadlock if the
+ # peer is already blocking on its writes.
+
+ if isinstance(want, ssl.SSLWantWriteError):
+ assert self._out.pending, "SSL backend wants write without data"
+
+ self.flush()
+
+ if isinstance(want, ssl.SSLWantReadError):
+ self._recv(4096)
+
+ def _flush_debug(self, prefix):
+ if not self._debugging:
+ return
+
+ self._stream.flush_debug(prefix=prefix)
+
+
+@contextlib.contextmanager
+def tls_handshake(stream, context):
+ """
+ Performs a TLS handshake over the given stream (which must have been created
+ via a call to wrap()), and returns a new stream which transparently tunnels
+ data over the TLS connection.
+
+ If the passed stream has debugging enabled, the returned stream will also
+ have debugging, using the same output IO.
+ """
+ debugging = hasattr(stream, "flush_debug")
+
+ # Send our startup parameters.
+ send_startup(stream, proto=protocol(1234, 5679))
+
+ # Look at the SSL response.
+ resp = stream.read(1)
+ if debugging:
+ stream.flush_debug(prefix=" ")
+
+ if resp == b"N":
+ raise RuntimeError("server does not support SSLRequest")
+ if resp != b"S":
+ raise RuntimeError(f"unexpected response of type {resp!r} during TLS startup")
+
+ tls = _TLSStream(stream, context)
+ tls.handshake()
+
+ if debugging:
+ tls = _DebugStream(tls, stream._out)
+
+ try:
+ yield tls
+ # TODO: teardown/unwrap the connection?
+ finally:
+ if debugging:
+ tls.flush_debug(prefix="? ")
diff --git a/src/test/python/pytest.ini b/src/test/python/pytest.ini
new file mode 100644
index 0000000000..ab7a6e7fb9
--- /dev/null
+++ b/src/test/python/pytest.ini
@@ -0,0 +1,4 @@
+[pytest]
+
+markers =
+ slow: mark test as slow
diff --git a/src/test/python/requirements.txt b/src/test/python/requirements.txt
new file mode 100644
index 0000000000..32f105ea84
--- /dev/null
+++ b/src/test/python/requirements.txt
@@ -0,0 +1,7 @@
+black
+cryptography~=3.4.6
+construct~=2.10.61
+isort~=5.6
+psycopg2~=2.8.6
+pytest~=6.1
+pytest-asyncio~=0.14.0
diff --git a/src/test/python/server/__init__.py b/src/test/python/server/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/src/test/python/server/conftest.py b/src/test/python/server/conftest.py
new file mode 100644
index 0000000000..ba7342a453
--- /dev/null
+++ b/src/test/python/server/conftest.py
@@ -0,0 +1,45 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import socket
+import sys
+
+import pytest
+
+import pq3
+
+
+@pytest.fixture
+def connect():
+ """
+ A factory fixture that, when called, returns a socket connected to a
+ Postgres server, wrapped in a pq3 connection. The calling test will be
+ skipped automatically if a server is not running at PGHOST:PGPORT, so it's
+ best to connect as soon as possible after the test case begins, to avoid
+ doing unnecessary work.
+ """
+ # Set up an ExitStack to handle safe cleanup of all of the moving pieces.
+ with contextlib.ExitStack() as stack:
+
+ def conn_factory():
+ addr = (pq3.pghost(), pq3.pgport())
+
+ try:
+ sock = socket.create_connection(addr, timeout=2)
+ except ConnectionError as e:
+ pytest.skip(f"unable to connect to {addr}: {e}")
+
+ # Have ExitStack close our socket.
+ stack.enter_context(sock)
+
+ # Wrap the connection in a pq3 layer and have ExitStack clean it up
+ # too.
+ wrap_ctx = pq3.wrap(sock, debug_stream=sys.stdout)
+ conn = stack.enter_context(wrap_ctx)
+
+ return conn
+
+ yield conn_factory
diff --git a/src/test/python/server/test_oauth.py b/src/test/python/server/test_oauth.py
new file mode 100644
index 0000000000..cb5ca7fa23
--- /dev/null
+++ b/src/test/python/server/test_oauth.py
@@ -0,0 +1,1012 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import contextlib
+import json
+import os
+import pathlib
+import secrets
+import shlex
+import shutil
+import socket
+import struct
+from multiprocessing import shared_memory
+
+import psycopg2
+import pytest
+from psycopg2 import sql
+
+import pq3
+
+MAX_SASL_MESSAGE_LENGTH = 65535
+
+INVALID_AUTHORIZATION_ERRCODE = b"28000"
+PROTOCOL_VIOLATION_ERRCODE = b"08P01"
+FEATURE_NOT_SUPPORTED_ERRCODE = b"0A000"
+
+SHARED_MEM_NAME = "oauth-pytest"
+MAX_TOKEN_SIZE = 4096
+MAX_UINT16 = 2 ** 16 - 1
+
+
+def skip_if_no_postgres():
+ """
+ Used by the oauth_ctx fixture to skip this test module if no Postgres server
+ is running.
+
+ This logic is nearly duplicated with the conn fixture. Ideally oauth_ctx
+ would depend on that, but a module-scope fixture can't depend on a
+ test-scope fixture, and we haven't reached the rule of three yet.
+ """
+ addr = (pq3.pghost(), pq3.pgport())
+
+ try:
+ with socket.create_connection(addr, timeout=2):
+ pass
+ except ConnectionError as e:
+ pytest.skip(f"unable to connect to {addr}: {e}")
+
+
+@contextlib.contextmanager
+def prepend_file(path, lines):
+ """
+ A context manager that prepends a file on disk with the desired lines of
+ text. When the context manager is exited, the file will be restored to its
+ original contents.
+ """
+ # First make a backup of the original file.
+ bak = path + ".bak"
+ shutil.copy2(path, bak)
+
+ try:
+ # Write the new lines, followed by the original file content.
+ with open(path, "w") as new, open(bak, "r") as orig:
+ new.writelines(lines)
+ shutil.copyfileobj(orig, new)
+
+ # Return control to the calling code.
+ yield
+
+ finally:
+ # Put the backup back into place.
+ os.replace(bak, path)
+
+
+@pytest.fixture(scope="module")
+def oauth_ctx():
+ """
+ Creates a database and user that use the oauth auth method. The context
+ object contains the dbname and user attributes as strings to be used during
+ connection, as well as the issuer and scope that have been set in the HBA
+ configuration.
+
+ This fixture assumes that the standard PG* environment variables point to a
+ server running on a local machine, and that the PGUSER has rights to create
+ databases and roles.
+ """
+ skip_if_no_postgres() # don't bother running these tests without a server
+
+ id = secrets.token_hex(4)
+
+ class Context:
+ dbname = "oauth_test_" + id
+
+ user = "oauth_user_" + id
+ map_user = "oauth_map_user_" + id
+ authz_user = "oauth_authz_user_" + id
+
+ issuer = "https://example.com/" + id
+ scope = "openid " + id
+
+ ctx = Context()
+ hba_lines = (
+ f'host {ctx.dbname} {ctx.map_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" map=oauth\n',
+ f'host {ctx.dbname} {ctx.authz_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" trust_validator_authz=1\n',
+ f'host {ctx.dbname} all samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}"\n',
+ )
+ ident_lines = (r"oauth /^(.*)@example\.com$ \1",)
+
+ conn = psycopg2.connect("")
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+
+ # Create our roles and database.
+ user = sql.Identifier(ctx.user)
+ map_user = sql.Identifier(ctx.map_user)
+ authz_user = sql.Identifier(ctx.authz_user)
+ dbname = sql.Identifier(ctx.dbname)
+
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(map_user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(authz_user))
+ c.execute(sql.SQL("CREATE DATABASE {};").format(dbname))
+
+ # Make this test script the server's oauth_validator.
+ path = pathlib.Path(__file__).parent / "validate_bearer.py"
+ path = str(path.absolute())
+
+ cmd = f"{shlex.quote(path)} {SHARED_MEM_NAME} <&%f"
+ c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (cmd,))
+
+ # Replace pg_hba and pg_ident.
+ c.execute("SHOW hba_file;")
+ hba = c.fetchone()[0]
+
+ c.execute("SHOW ident_file;")
+ ident = c.fetchone()[0]
+
+ with prepend_file(hba, hba_lines), prepend_file(ident, ident_lines):
+ c.execute("SELECT pg_reload_conf();")
+
+ # Use the new database and user.
+ yield ctx
+
+ # Put things back the way they were.
+ c.execute("SELECT pg_reload_conf();")
+
+ c.execute("ALTER SYSTEM RESET oauth_validator_command;")
+ c.execute(sql.SQL("DROP DATABASE {};").format(dbname))
+ c.execute(sql.SQL("DROP ROLE {};").format(authz_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(map_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(user))
+
+
+@pytest.fixture()
+def conn(oauth_ctx, connect):
+ """
+ A convenience wrapper for connect(). The main purpose of this fixture is to
+ make sure oauth_ctx runs its setup code before the connection is made.
+ """
+ return connect()
+
+
+@pytest.fixture(scope="module", autouse=True)
+def authn_id_extension(oauth_ctx):
+ """
+ Performs a `CREATE EXTENSION authn_id` in the test database. This fixture is
+ autoused, so tests don't need to rely on it.
+ """
+ conn = psycopg2.connect(database=oauth_ctx.dbname)
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+ c.execute("CREATE EXTENSION authn_id;")
+
+
+@pytest.fixture(scope="session")
+def shared_mem():
+ """
+ Yields a shared memory segment that can be used for communication between
+ the bearer_token fixture and ./validate_bearer.py.
+ """
+ size = MAX_TOKEN_SIZE + 2 # two byte length prefix
+ mem = shared_memory.SharedMemory(SHARED_MEM_NAME, create=True, size=size)
+
+ try:
+ with contextlib.closing(mem):
+ yield mem
+ finally:
+ mem.unlink()
+
+
+@pytest.fixture()
+def bearer_token(shared_mem):
+ """
+ Returns a factory function that, when called, will store a Bearer token in
+ shared_mem. If token is None (the default), a new token will be generated
+ using secrets.token_urlsafe() and returned; otherwise the passed token will
+ be used as-is.
+
+ When token is None, the generated token size in bytes may be specified as an
+ argument; if unset, a small 16-byte token will be generated. The token size
+ may not exceed MAX_TOKEN_SIZE in any case.
+
+ The return value is the token, converted to a bytes object.
+
+ As a special case for testing failure modes, accept_any may be set to True.
+ This signals to the validator command that any bearer token should be
+ accepted. The returned token in this case may be used or discarded as needed
+ by the test.
+ """
+
+ def set_token(token=None, *, size=16, accept_any=False):
+ if token is not None:
+ size = len(token)
+
+ if size > MAX_TOKEN_SIZE:
+ raise ValueError(f"token size {size} exceeds maximum size {MAX_TOKEN_SIZE}")
+
+ if token is None:
+ if size % 4:
+ raise ValueError(f"requested token size {size} is not a multiple of 4")
+
+ token = secrets.token_urlsafe(size // 4 * 3)
+ assert len(token) == size
+
+ try:
+ token = token.encode("ascii")
+ except AttributeError:
+ pass # already encoded
+
+ if accept_any:
+ # Two-byte magic value.
+ shared_mem.buf[:2] = struct.pack("H", MAX_UINT16)
+ else:
+ # Two-byte length prefix, then the token data.
+ shared_mem.buf[:2] = struct.pack("H", len(token))
+ shared_mem.buf[2 : size + 2] = token
+
+ return token
+
+ return set_token
+
+
+def begin_oauth_handshake(conn, oauth_ctx, *, user=None):
+ if user is None:
+ user = oauth_ctx.authz_user
+
+ pq3.send_startup(conn, user=user, database=oauth_ctx.dbname)
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.AuthnRequest
+
+ # The server should advertise exactly one mechanism.
+ assert resp.payload.type == pq3.authn.SASL
+ assert resp.payload.body == [b"OAUTHBEARER", b""]
+
+
+def send_initial_response(conn, *, auth=None, bearer=None):
+ """
+ Sends the OAUTHBEARER initial response on the connection, using the given
+ bearer token. Alternatively to a bearer token, the initial response's auth
+ field may be explicitly specified to test corner cases.
+ """
+ if bearer is not None and auth is not None:
+ raise ValueError("exactly one of the auth and bearer kwargs must be set")
+
+ if bearer is not None:
+ auth = b"Bearer " + bearer
+
+ if auth is None:
+ raise ValueError("exactly one of the auth and bearer kwargs must be set")
+
+ initial = pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"n,,\x01auth=" + auth + b"\x01\x01",
+ )
+ )
+ pq3.send(conn, pq3.types.PasswordMessage, initial)
+
+
+def expect_handshake_success(conn):
+ """
+ Validates that the server responds with an AuthnOK message, and then drains
+ the connection until a ReadyForQuery message is received.
+ """
+ resp = pq3.recv1(conn)
+
+ assert resp.type == pq3.types.AuthnRequest
+ assert resp.payload.type == pq3.authn.OK
+ assert not resp.payload.body
+
+ receive_until(conn, pq3.types.ReadyForQuery)
+
+
+def expect_handshake_failure(conn, oauth_ctx):
+ """
+ Performs the OAUTHBEARER SASL failure "handshake" and validates the server's
+ side of the conversation, including the final ErrorResponse.
+ """
+
+ # We expect a discovery "challenge" back from the server before the authn
+ # failure message.
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.AuthnRequest
+
+ req = resp.payload
+ assert req.type == pq3.authn.SASLContinue
+
+ body = json.loads(req.body)
+ assert body["status"] == "invalid_token"
+ assert body["scope"] == oauth_ctx.scope
+
+ expected_config = oauth_ctx.issuer + "/.well-known/openid-configuration"
+ assert body["openid-configuration"] == expected_config
+
+ # Send the dummy response to complete the failed handshake.
+ pq3.send(conn, pq3.types.PasswordMessage, b"\x01")
+ resp = pq3.recv1(conn)
+
+ err = ExpectedError(INVALID_AUTHORIZATION_ERRCODE, "bearer authentication failed")
+ err.match(resp)
+
+
+def receive_until(conn, type):
+ """
+ receive_until pulls packets off the pq3 connection until a packet with the
+ desired type is found, or an error response is received.
+ """
+ while True:
+ pkt = pq3.recv1(conn)
+
+ if pkt.type == type:
+ return pkt
+ elif pkt.type == pq3.types.ErrorResponse:
+ raise RuntimeError(
+ f"received error response from peer: {pkt.payload.fields!r}"
+ )
+
+
+@pytest.mark.parametrize("token_len", [16, 1024, 4096])
+@pytest.mark.parametrize(
+ "auth_prefix",
+ [
+ b"Bearer ",
+ b"bearer ",
+ b"Bearer ",
+ ],
+)
+def test_oauth(conn, oauth_ctx, bearer_token, auth_prefix, token_len):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Generate our bearer token with the desired length.
+ token = bearer_token(size=token_len)
+ auth = auth_prefix + token
+
+ send_initial_response(conn, auth=auth)
+ expect_handshake_success(conn)
+
+ # Make sure that the server has not set an authenticated ID.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT authn_id();")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ assert row.columns == [None]
+
+
+@pytest.mark.parametrize(
+ "token_value",
+ [
+ "abcdzA==",
+ "123456M=",
+ "x-._~+/x",
+ ],
+)
+def test_oauth_bearer_corner_cases(conn, oauth_ctx, bearer_token, token_value):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ send_initial_response(conn, bearer=bearer_token(token_value))
+
+ expect_handshake_success(conn)
+
+
+@pytest.mark.parametrize(
+ "user,authn_id,should_succeed",
+ [
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: ctx.user,
+ True,
+ id="validator authn: succeeds when authn_id == username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: None,
+ False,
+ id="validator authn: fails when authn_id is not set",
+ ),
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: ctx.authz_user,
+ False,
+ id="validator authn: fails when authn_id != username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: ctx.map_user + "@example.com",
+ True,
+ id="validator with map: succeeds when authn_id matches map",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: None,
+ False,
+ id="validator with map: fails when authn_id is not set",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: ctx.map_user + "@example.net",
+ False,
+ id="validator with map: fails when authn_id doesn't match map",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: None,
+ True,
+ id="validator authz: succeeds with no authn_id",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "",
+ True,
+ id="validator authz: succeeds with empty authn_id",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "postgres",
+ True,
+ id="validator authz: succeeds with basic username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "me@example.com",
+ True,
+ id="validator authz: succeeds with email address",
+ ),
+ ],
+)
+def test_oauth_authn_id(conn, oauth_ctx, bearer_token, user, authn_id, should_succeed):
+ token = None
+
+ authn_id = authn_id(oauth_ctx)
+ if authn_id is not None:
+ authn_id = authn_id.encode("ascii")
+
+ # As a hack to get the validator to reflect arbitrary output from this
+ # test, encode the desired output as a base64 token. The validator will
+ # key on the leading "output=" to differentiate this from the random
+ # tokens generated by secrets.token_urlsafe().
+ output = b"output=" + authn_id + b"\n"
+ token = base64.urlsafe_b64encode(output)
+
+ token = bearer_token(token)
+ username = user(oauth_ctx)
+
+ begin_oauth_handshake(conn, oauth_ctx, user=username)
+ send_initial_response(conn, bearer=token)
+
+ if not should_succeed:
+ expect_handshake_failure(conn, oauth_ctx)
+ return
+
+ expect_handshake_success(conn)
+
+ # Check the reported authn_id.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT authn_id();")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ assert row.columns == [authn_id]
+
+
+class ExpectedError(object):
+ def __init__(self, code, msg=None, detail=None):
+ self.code = code
+ self.msg = msg
+ self.detail = detail
+
+ # Protect against the footgun of an accidental empty string, which will
+ # "match" anything. If you don't want to match message or detail, just
+ # don't pass them.
+ if self.msg == "":
+ raise ValueError("msg must be non-empty or None")
+ if self.detail == "":
+ raise ValueError("detail must be non-empty or None")
+
+ def _getfield(self, resp, type):
+ """
+ Searches an ErrorResponse for a single field of the given type (e.g.
+ "M", "C", "D") and returns its value. Asserts if it doesn't find exactly
+ one field.
+ """
+ prefix = type.encode("ascii")
+ fields = [f for f in resp.payload.fields if f.startswith(prefix)]
+
+ assert len(fields) == 1
+ return fields[0][1:] # strip off the type byte
+
+ def match(self, resp):
+ """
+ Checks that the given response matches the expected code, message, and
+ detail (if given). The error code must match exactly. The expected
+ message and detail must be contained within the actual strings.
+ """
+ assert resp.type == pq3.types.ErrorResponse
+
+ code = self._getfield(resp, "C")
+ assert code == self.code
+
+ if self.msg:
+ msg = self._getfield(resp, "M")
+ expected = self.msg.encode("utf-8")
+ assert expected in msg
+
+ if self.detail:
+ detail = self._getfield(resp, "D")
+ expected = self.detail.encode("utf-8")
+ assert expected in detail
+
+
+def test_oauth_rejected_bearer(conn, oauth_ctx, bearer_token):
+ # Generate a new bearer token, which we will proceed not to use.
+ _ = bearer_token()
+
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send a bearer token that doesn't match what the validator expects. It
+ # should fail the connection.
+ send_initial_response(conn, bearer=b"xxxxxx")
+
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.parametrize(
+ "bad_bearer",
+ [
+ b"Bearer ",
+ b"Bearer a===b",
+ b"Bearer hello!",
+ b"Bearer me@example.com",
+ b'OAuth realm="Example"',
+ b"",
+ ],
+)
+def test_oauth_invalid_bearer(conn, oauth_ctx, bearer_token, bad_bearer):
+ # Tell the validator to accept any token. This ensures that the invalid
+ # bearer tokens are rejected before the validation step.
+ _ = bearer_token(accept_any=True)
+
+ begin_oauth_handshake(conn, oauth_ctx)
+ send_initial_response(conn, auth=bad_bearer)
+
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.slow
+@pytest.mark.parametrize(
+ "resp_type,resp,err",
+ [
+ pytest.param(
+ None,
+ None,
+ None,
+ marks=pytest.mark.slow,
+ id="no response (expect timeout)",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"hello",
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not send a kvsep response",
+ ),
+ id="bad dummy response",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"\x01\x01",
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not send a kvsep response",
+ ),
+ id="multiple kvseps",
+ ),
+ pytest.param(
+ pq3.types.Query,
+ dict(query=b""),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "expected SASL response"),
+ id="bad response message type",
+ ),
+ ],
+)
+def test_oauth_bad_response_to_error_challenge(conn, oauth_ctx, resp_type, resp, err):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send an empty auth initial response, which will force an authn failure.
+ send_initial_response(conn, auth=b"")
+
+ # We expect a discovery "challenge" back from the server before the authn
+ # failure message.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.AuthnRequest
+
+ req = pkt.payload
+ assert req.type == pq3.authn.SASLContinue
+
+ body = json.loads(req.body)
+ assert body["status"] == "invalid_token"
+
+ if resp_type is None:
+ # Do not send the dummy response. We should time out and not get a
+ # response from the server.
+ with pytest.raises(socket.timeout):
+ conn.read(1)
+
+ # Done with the test.
+ return
+
+ # Send the bad response.
+ pq3.send(conn, resp_type, resp)
+
+ # Make sure the server fails the connection correctly.
+ pkt = pq3.recv1(conn)
+ err.match(pkt)
+
+
+@pytest.mark.parametrize(
+ "type,payload,err",
+ [
+ pytest.param(
+ pq3.types.ErrorResponse,
+ dict(fields=[b""]),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "expected SASL response"),
+ id="error response in initial message",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"x" * (MAX_SASL_MESSAGE_LENGTH + 1),
+ ExpectedError(
+ INVALID_AUTHORIZATION_ERRCODE, "bearer authentication failed"
+ ),
+ id="overlong initial response data",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"SCRAM-SHA-256")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE, "invalid SASL authentication mechanism"
+ ),
+ id="bad SASL mechanism selection",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", len=2, data=b"x")),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "insufficient data"),
+ id="SASL data underflow",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", len=0, data=b"x")),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "invalid message format"),
+ id="SASL data overflow",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "message is empty",
+ ),
+ id="empty",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"n,,\x01auth=\x01\x01\0")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "length does not match input length",
+ ),
+ id="contains null byte",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected channel-binding flag", # XXX this is a bit strange
+ ),
+ id="initial error response",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"p=tls-server-end-point,,\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "server does not support channel binding",
+ ),
+ id="uses channel binding",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"x,,\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected channel-binding flag",
+ ),
+ id="invalid channel binding specifier",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Comma expected",
+ ),
+ id="bad GS2 header: missing channel binding terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,a")),
+ ExpectedError(
+ FEATURE_NOT_SUPPORTED_ERRCODE,
+ "client uses authorization identity",
+ ),
+ id="bad GS2 header: authzid in use",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,b,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected attribute",
+ ),
+ id="bad GS2 header: extra attribute",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected attribute 0x00", # XXX this is a bit strange
+ ),
+ id="bad GS2 header: missing authzid terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Key-value separator expected",
+ ),
+ id="missing initial kvsep",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Key-value separator expected",
+ ),
+ id="missing initial kvsep",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "does not contain an auth value",
+ ),
+ id="missing auth value: empty key-value list",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01host=example.com\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "does not contain an auth value",
+ ),
+ id="missing auth value: other keys present",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01host=example.com")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "unterminated key/value pair",
+ ),
+ id="missing value terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not contain a final terminator",
+ ),
+ id="missing list terminator: empty list",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01auth=Bearer 0\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not contain a final terminator",
+ ),
+ id="missing list terminator: with auth value",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01auth=Bearer 0\x01\x01blah")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "additional data after the final terminator",
+ ),
+ id="additional key after terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01key\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "key without a value",
+ ),
+ id="key without value",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"y,,\x01auth=Bearer 0\x01auth=Bearer 1\x01\x01",
+ )
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "contains multiple auth values",
+ ),
+ id="multiple auth values",
+ ),
+ ],
+)
+def test_oauth_bad_initial_response(conn, oauth_ctx, type, payload, err):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # The server expects a SASL response; give it something else instead.
+ if not isinstance(payload, dict):
+ payload = dict(payload_data=payload)
+ pq3.send(conn, type, **payload)
+
+ resp = pq3.recv1(conn)
+ err.match(resp)
+
+
+def test_oauth_empty_initial_response(conn, oauth_ctx, bearer_token):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send an initial response without data.
+ initial = pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER"))
+ pq3.send(conn, pq3.types.PasswordMessage, initial)
+
+ # The server should respond with an empty challenge so we can send the data
+ # it wants.
+ pkt = pq3.recv1(conn)
+
+ assert pkt.type == pq3.types.AuthnRequest
+ assert pkt.payload.type == pq3.authn.SASLContinue
+ assert not pkt.payload.body
+
+ # Now send the initial data.
+ data = b"n,,\x01auth=Bearer " + bearer_token() + b"\x01\x01"
+ pq3.send(conn, pq3.types.PasswordMessage, data)
+
+ # Server should now complete the handshake.
+ expect_handshake_success(conn)
+
+
+@pytest.fixture()
+def set_validator():
+ """
+ A per-test fixture that allows a test to override the setting of
+ oauth_validator_command for the cluster. The setting will be reverted during
+ teardown.
+
+ Passing None will perform an ALTER SYSTEM RESET.
+ """
+ conn = psycopg2.connect("")
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+
+ # Save the previous value.
+ c.execute("SHOW oauth_validator_command;")
+ prev_cmd = c.fetchone()[0]
+
+ def setter(cmd):
+ c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (cmd,))
+ c.execute("SELECT pg_reload_conf();")
+
+ yield setter
+
+ # Restore the previous value.
+ c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (prev_cmd,))
+ c.execute("SELECT pg_reload_conf();")
+
+
+def test_oauth_no_validator(oauth_ctx, set_validator, connect, bearer_token):
+ # Clear out our validator command, then establish a new connection.
+ set_validator("")
+ conn = connect()
+
+ begin_oauth_handshake(conn, oauth_ctx)
+ send_initial_response(conn, bearer=bearer_token())
+
+ # The server should fail the connection.
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+def test_oauth_validator_role(oauth_ctx, set_validator, connect):
+ # Switch the validator implementation. This validator will reflect the
+ # PGUSER as the authenticated identity.
+ path = pathlib.Path(__file__).parent / "validate_reflect.py"
+ path = str(path.absolute())
+
+ set_validator(f"{shlex.quote(path)} '%r' <&%f")
+ conn = connect()
+
+ # Log in. Note that the reflection validator ignores the bearer token.
+ begin_oauth_handshake(conn, oauth_ctx, user=oauth_ctx.user)
+ send_initial_response(conn, bearer=b"dontcare")
+ expect_handshake_success(conn)
+
+ # Check the user identity.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT authn_id();")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ expected = oauth_ctx.user.encode("utf-8")
+ assert row.columns == [expected]
+
+
+def test_oauth_role_with_shell_unsafe_characters(oauth_ctx, set_validator, connect):
+ """
+ XXX This test pins undesirable behavior. We should be able to handle any
+ valid Postgres role name.
+ """
+ # Switch the validator implementation. This validator will reflect the
+ # PGUSER as the authenticated identity.
+ path = pathlib.Path(__file__).parent / "validate_reflect.py"
+ path = str(path.absolute())
+
+ set_validator(f"{shlex.quote(path)} '%r' <&%f")
+ conn = connect()
+
+ unsafe_username = "hello'there"
+ begin_oauth_handshake(conn, oauth_ctx, user=unsafe_username)
+
+ # The server should reject the handshake.
+ send_initial_response(conn, bearer=b"dontcare")
+ expect_handshake_failure(conn, oauth_ctx)
diff --git a/src/test/python/server/test_server.py b/src/test/python/server/test_server.py
new file mode 100644
index 0000000000..02126dba79
--- /dev/null
+++ b/src/test/python/server/test_server.py
@@ -0,0 +1,21 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import pq3
+
+
+def test_handshake(connect):
+ """Basic sanity check."""
+ conn = connect()
+
+ pq3.handshake(conn, user=pq3.pguser(), database=pq3.pgdatabase())
+
+ pq3.send(conn, pq3.types.Query, query=b"")
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.EmptyQueryResponse
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.ReadyForQuery
diff --git a/src/test/python/server/validate_bearer.py b/src/test/python/server/validate_bearer.py
new file mode 100755
index 0000000000..2cc73ff154
--- /dev/null
+++ b/src/test/python/server/validate_bearer.py
@@ -0,0 +1,101 @@
+#! /usr/bin/env python3
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+# DO NOT USE THIS OAUTH VALIDATOR IN PRODUCTION. It doesn't actually validate
+# anything, and it logs the bearer token data, which is sensitive.
+#
+# This executable is used as an oauth_validator_command in concert with
+# test_oauth.py. Memory is shared and communicated from that test module's
+# bearer_token() fixture.
+#
+# This script must run under the Postgres server environment; keep the
+# dependency list fairly standard.
+
+import base64
+import binascii
+import contextlib
+import struct
+import sys
+from multiprocessing import shared_memory
+
+MAX_UINT16 = 2 ** 16 - 1
+
+
+def remove_shm_from_resource_tracker():
+ """
+ Monkey-patch multiprocessing.resource_tracker so SharedMemory won't be
+ tracked. Pulled from this thread, where there are more details:
+
+ https://bugs.python.org/issue38119
+
+ TL;DR: all clients of shared memory segments automatically destroy them on
+ process exit, which makes shared memory segments much less useful. This
+ monkeypatch removes that behavior so that we can defer to the test to manage
+ the segment lifetime.
+
+ Ideally a future Python patch will pull in this fix and then the entire
+ function can go away.
+ """
+ from multiprocessing import resource_tracker
+
+ def fix_register(name, rtype):
+ if rtype == "shared_memory":
+ return
+ return resource_tracker._resource_tracker.register(self, name, rtype)
+
+ resource_tracker.register = fix_register
+
+ def fix_unregister(name, rtype):
+ if rtype == "shared_memory":
+ return
+ return resource_tracker._resource_tracker.unregister(self, name, rtype)
+
+ resource_tracker.unregister = fix_unregister
+
+ if "shared_memory" in resource_tracker._CLEANUP_FUNCS:
+ del resource_tracker._CLEANUP_FUNCS["shared_memory"]
+
+
+def main(args):
+ remove_shm_from_resource_tracker() # XXX remove some day
+
+ # Get the expected token from the currently running test.
+ shared_mem_name = args[0]
+
+ mem = shared_memory.SharedMemory(shared_mem_name)
+ with contextlib.closing(mem):
+ # First two bytes are the token length.
+ size = struct.unpack("H", mem.buf[:2])[0]
+
+ if size == MAX_UINT16:
+ # Special case: the test wants us to accept any token.
+ sys.stderr.write("accepting token without validation\n")
+ return
+
+ # The remainder of the buffer contains the expected token.
+ assert size <= (mem.size - 2)
+ expected_token = mem.buf[2 : size + 2].tobytes()
+
+ mem.buf[:] = b"\0" * mem.size # scribble over the token
+
+ token = sys.stdin.buffer.read()
+ if token != expected_token:
+ sys.exit(f"failed to match Bearer token ({token!r} != {expected_token!r})")
+
+ # See if the test wants us to print anything. If so, it will have encoded
+ # the desired output in the token with an "output=" prefix.
+ try:
+ # altchars="-_" corresponds to the urlsafe alphabet.
+ data = base64.b64decode(token, altchars="-_", validate=True)
+
+ if data.startswith(b"output="):
+ sys.stdout.buffer.write(data[7:])
+
+ except binascii.Error:
+ pass
+
+
+if __name__ == "__main__":
+ main(sys.argv[1:])
diff --git a/src/test/python/server/validate_reflect.py b/src/test/python/server/validate_reflect.py
new file mode 100755
index 0000000000..24c3a7e715
--- /dev/null
+++ b/src/test/python/server/validate_reflect.py
@@ -0,0 +1,34 @@
+#! /usr/bin/env python3
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+# DO NOT USE THIS OAUTH VALIDATOR IN PRODUCTION. It ignores the bearer token
+# entirely and automatically logs the user in.
+#
+# This executable is used as an oauth_validator_command in concert with
+# test_oauth.py. It expects the user's desired role name as an argument; the
+# actual token will be discarded and the user will be logged in with the role
+# name as the authenticated identity.
+#
+# This script must run under the Postgres server environment; keep the
+# dependency list fairly standard.
+
+import sys
+
+
+def main(args):
+ # We have to read the entire token as our first action to unblock the
+ # server, but we won't actually use it.
+ _ = sys.stdin.buffer.read()
+
+ if len(args) != 1:
+ sys.exit("usage: ./validate_reflect.py ROLE")
+
+ # Log the user in as the provided role.
+ role = args[0]
+ print(role)
+
+
+if __name__ == "__main__":
+ main(sys.argv[1:])
diff --git a/src/test/python/test_internals.py b/src/test/python/test_internals.py
new file mode 100644
index 0000000000..dee4855fc0
--- /dev/null
+++ b/src/test/python/test_internals.py
@@ -0,0 +1,138 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import io
+
+from pq3 import _DebugStream
+
+
+def test_DebugStream_read():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+
+ stream = _DebugStream(under, out)
+
+ res = stream.read(5)
+ assert res == b"abcde"
+
+ res = stream.read(16)
+ assert res == b"fghijklmnopqrstu"
+
+ stream.flush_debug()
+
+ res = stream.read()
+ assert res == b"vwxyz"
+
+ stream.flush_debug()
+
+ expected = (
+ "< 0000:\t61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 70\tabcdefghijklmnop\n"
+ "< 0010:\t71 72 73 74 75 \tqrstu\n"
+ "\n"
+ "< 0000:\t76 77 78 79 7a \tvwxyz\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_write():
+ under = io.BytesIO()
+ out = io.StringIO()
+
+ stream = _DebugStream(under, out)
+
+ stream.write(b"\x00\x01\x02")
+ stream.flush()
+
+ assert under.getvalue() == b"\x00\x01\x02"
+
+ stream.write(b"\xc0\xc1\xc2")
+ stream.flush()
+
+ assert under.getvalue() == b"\x00\x01\x02\xc0\xc1\xc2"
+
+ stream.flush_debug()
+
+ expected = "> 0000:\t00 01 02 c0 c1 c2 \t......\n\n"
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_read_write():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+ stream = _DebugStream(under, out)
+
+ res = stream.read(5)
+ assert res == b"abcde"
+
+ stream.write(b"xxxxx")
+ stream.flush()
+
+ assert under.getvalue() == b"abcdexxxxxklmnopqrstuvwxyz"
+
+ res = stream.read(5)
+ assert res == b"klmno"
+
+ stream.write(b"xxxxx")
+ stream.flush()
+
+ assert under.getvalue() == b"abcdexxxxxklmnoxxxxxuvwxyz"
+
+ stream.flush_debug()
+
+ expected = (
+ "< 0000:\t61 62 63 64 65 6b 6c 6d 6e 6f \tabcdeklmno\n"
+ "\n"
+ "> 0000:\t78 78 78 78 78 78 78 78 78 78 \txxxxxxxxxx\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_end_packet():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+ stream = _DebugStream(under, out)
+
+ stream.read(5)
+ stream.end_packet("read description", read=True, indent=" ")
+
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("write description", indent=" ")
+
+ stream.read(5)
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("read/write combo for read", read=True, indent=" ")
+
+ stream.read(5)
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("read/write combo for write", indent=" ")
+
+ expected = (
+ " < 0000:\t61 62 63 64 65 \tabcde\n"
+ "\n"
+ "< read description\n"
+ "\n"
+ "> write description\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ " < 0000:\t6b 6c 6d 6e 6f \tklmno\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ "< read/write combo for read\n"
+ "\n"
+ "> read/write combo for write\n"
+ "\n"
+ " < 0000:\t75 76 77 78 79 \tuvwxy\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
diff --git a/src/test/python/test_pq3.py b/src/test/python/test_pq3.py
new file mode 100644
index 0000000000..e0c0e0568d
--- /dev/null
+++ b/src/test/python/test_pq3.py
@@ -0,0 +1,558 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import getpass
+import io
+import struct
+import sys
+
+import pytest
+from construct import Container, PaddingError, StreamError, TerminatedError
+
+import pq3
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"\x00\x00\x00\x10\x00\x04\x00\x00abcdefgh",
+ Container(len=16, proto=0x40000, payload=b"abcdefgh"),
+ b"",
+ id="8-byte payload",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x08\x00\x04\x00\x00",
+ Container(len=8, proto=0x40000, payload=b""),
+ b"",
+ id="no payload",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x09\x00\x04\x00\x00abcde",
+ Container(len=9, proto=0x40000, payload=b"a"),
+ b"bcde",
+ id="1-byte payload and extra padding",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x0B\x00\x03\x00\x00hi\x00",
+ Container(len=11, proto=pq3.protocol(3, 0), payload=[b"hi"]),
+ b"",
+ id="implied parameter list when using proto version 3.0",
+ ),
+ ],
+)
+def test_Startup_parse(raw, expected, extra):
+ with io.BytesIO(raw) as stream:
+ actual = pq3.Startup.parse_stream(stream)
+
+ assert actual == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "packet,expected_bytes",
+ [
+ pytest.param(
+ dict(),
+ b"\x00\x00\x00\x08\x00\x00\x00\x00",
+ id="nothing set",
+ ),
+ pytest.param(
+ dict(len=10, proto=0x12345678),
+ b"\x00\x00\x00\x0A\x12\x34\x56\x78\x00\x00",
+ id="len and proto set explicitly",
+ ),
+ pytest.param(
+ dict(proto=0x12345678),
+ b"\x00\x00\x00\x08\x12\x34\x56\x78",
+ id="implied len with no payload",
+ ),
+ pytest.param(
+ dict(proto=0x12345678, payload=b"abcd"),
+ b"\x00\x00\x00\x0C\x12\x34\x56\x78abcd",
+ id="implied len with payload",
+ ),
+ pytest.param(
+ dict(payload=[b""]),
+ b"\x00\x00\x00\x09\x00\x03\x00\x00\x00",
+ id="implied proto version 3 when sending parameters",
+ ),
+ pytest.param(
+ dict(payload=[b"hi", b""]),
+ b"\x00\x00\x00\x0C\x00\x03\x00\x00hi\x00\x00",
+ id="implied proto version 3 and len when sending more than one parameter",
+ ),
+ pytest.param(
+ dict(payload=dict(user="jsmith", database="postgres")),
+ b"\x00\x00\x00\x27\x00\x03\x00\x00user\x00jsmith\x00database\x00postgres\x00\x00",
+ id="auto-serialization of dict parameters",
+ ),
+ ],
+)
+def test_Startup_build(packet, expected_bytes):
+ actual = pq3.Startup.build(packet)
+ assert actual == expected_bytes
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"*\x00\x00\x00\x08abcd",
+ dict(type=b"*", len=8, payload=b"abcd"),
+ b"",
+ id="4-byte payload",
+ ),
+ pytest.param(
+ b"*\x00\x00\x00\x04",
+ dict(type=b"*", len=4, payload=b""),
+ b"",
+ id="no payload",
+ ),
+ pytest.param(
+ b"*\x00\x00\x00\x05xabcd",
+ dict(type=b"*", len=5, payload=b"x"),
+ b"abcd",
+ id="1-byte payload with extra padding",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x08\x00\x00\x00\x00",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=8,
+ payload=dict(type=pq3.authn.OK, body=None),
+ ),
+ b"",
+ id="AuthenticationOk",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x12\x00\x00\x00\x0AEXTERNAL\x00\x00",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=18,
+ payload=dict(type=pq3.authn.SASL, body=[b"EXTERNAL", b""]),
+ ),
+ b"",
+ id="AuthenticationSASL",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0B12345",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=13,
+ payload=dict(type=pq3.authn.SASLContinue, body=b"12345"),
+ ),
+ b"",
+ id="AuthenticationSASLContinue",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0C12345",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=13,
+ payload=dict(type=pq3.authn.SASLFinal, body=b"12345"),
+ ),
+ b"",
+ id="AuthenticationSASLFinal",
+ ),
+ pytest.param(
+ b"p\x00\x00\x00\x0Bhunter2",
+ dict(
+ type=pq3.types.PasswordMessage,
+ len=11,
+ payload=b"hunter2",
+ ),
+ b"",
+ id="PasswordMessage",
+ ),
+ pytest.param(
+ b"K\x00\x00\x00\x0C\x00\x00\x00\x00\x12\x34\x56\x78",
+ dict(
+ type=pq3.types.BackendKeyData,
+ len=12,
+ payload=dict(pid=0, key=0x12345678),
+ ),
+ b"",
+ id="BackendKeyData",
+ ),
+ pytest.param(
+ b"C\x00\x00\x00\x08SET\x00",
+ dict(
+ type=pq3.types.CommandComplete,
+ len=8,
+ payload=dict(tag=b"SET"),
+ ),
+ b"",
+ id="CommandComplete",
+ ),
+ pytest.param(
+ b"E\x00\x00\x00\x11Mbad!\x00Mdog!\x00\x00",
+ dict(type=b"E", len=17, payload=dict(fields=[b"Mbad!", b"Mdog!", b""])),
+ b"",
+ id="ErrorResponse",
+ ),
+ pytest.param(
+ b"S\x00\x00\x00\x08a\x00b\x00",
+ dict(
+ type=pq3.types.ParameterStatus,
+ len=8,
+ payload=dict(name=b"a", value=b"b"),
+ ),
+ b"",
+ id="ParameterStatus",
+ ),
+ pytest.param(
+ b"Z\x00\x00\x00\x05x",
+ dict(type=b"Z", len=5, payload=dict(status=b"x")),
+ b"",
+ id="ReadyForQuery",
+ ),
+ pytest.param(
+ b"Q\x00\x00\x00\x06!\x00",
+ dict(type=pq3.types.Query, len=6, payload=dict(query=b"!")),
+ b"",
+ id="Query",
+ ),
+ pytest.param(
+ b"D\x00\x00\x00\x0B\x00\x01\x00\x00\x00\x01!",
+ dict(type=pq3.types.DataRow, len=11, payload=dict(columns=[b"!"])),
+ b"",
+ id="DataRow",
+ ),
+ pytest.param(
+ b"D\x00\x00\x00\x06\x00\x00extra",
+ dict(type=pq3.types.DataRow, len=6, payload=dict(columns=[])),
+ b"extra",
+ id="DataRow with extra data",
+ ),
+ pytest.param(
+ b"I\x00\x00\x00\x04",
+ dict(type=pq3.types.EmptyQueryResponse, len=4, payload=None),
+ b"",
+ id="EmptyQueryResponse",
+ ),
+ pytest.param(
+ b"I\x00\x00\x00\x04\xFF",
+ dict(type=b"I", len=4, payload=None),
+ b"\xFF",
+ id="EmptyQueryResponse with extra bytes",
+ ),
+ pytest.param(
+ b"X\x00\x00\x00\x04",
+ dict(type=pq3.types.Terminate, len=4, payload=None),
+ b"",
+ id="Terminate",
+ ),
+ ],
+)
+def test_Pq3_parse(raw, expected, extra):
+ with io.BytesIO(raw) as stream:
+ actual = pq3.Pq3.parse_stream(stream)
+
+ assert actual == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(type=b"*", len=5),
+ b"*\x00\x00\x00\x05\x00",
+ id="type and len set explicitly",
+ ),
+ pytest.param(
+ dict(type=b"*"),
+ b"*\x00\x00\x00\x04",
+ id="implied len with no payload",
+ ),
+ pytest.param(
+ dict(type=b"*", payload=b"1234"),
+ b"*\x00\x00\x00\x081234",
+ id="implied len with payload",
+ ),
+ pytest.param(
+ dict(type=pq3.types.AuthnRequest, payload=dict(type=pq3.authn.OK)),
+ b"R\x00\x00\x00\x08\x00\x00\x00\x00",
+ id="implied len/type for AuthenticationOK",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(
+ type=pq3.authn.SASL,
+ body=[b"SCRAM-SHA-256-PLUS", b"SCRAM-SHA-256", b""],
+ ),
+ ),
+ b"R\x00\x00\x00\x2A\x00\x00\x00\x0ASCRAM-SHA-256-PLUS\x00SCRAM-SHA-256\x00\x00",
+ id="implied len/type for AuthenticationSASL",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(type=pq3.authn.SASLContinue, body=b"12345"),
+ ),
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0B12345",
+ id="implied len/type for AuthenticationSASLContinue",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(type=pq3.authn.SASLFinal, body=b"12345"),
+ ),
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0C12345",
+ id="implied len/type for AuthenticationSASLFinal",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.PasswordMessage,
+ payload=b"hunter2",
+ ),
+ b"p\x00\x00\x00\x0Bhunter2",
+ id="implied len/type for PasswordMessage",
+ ),
+ pytest.param(
+ dict(type=pq3.types.BackendKeyData, payload=dict(pid=1, key=7)),
+ b"K\x00\x00\x00\x0C\x00\x00\x00\x01\x00\x00\x00\x07",
+ id="implied len/type for BackendKeyData",
+ ),
+ pytest.param(
+ dict(type=pq3.types.CommandComplete, payload=dict(tag=b"SET")),
+ b"C\x00\x00\x00\x08SET\x00",
+ id="implied len/type for CommandComplete",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ErrorResponse, payload=dict(fields=[b"error", b""])),
+ b"E\x00\x00\x00\x0Berror\x00\x00",
+ id="implied len/type for ErrorResponse",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ParameterStatus, payload=dict(name=b"a", value=b"b")),
+ b"S\x00\x00\x00\x08a\x00b\x00",
+ id="implied len/type for ParameterStatus",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ReadyForQuery, payload=dict(status=b"I")),
+ b"Z\x00\x00\x00\x05I",
+ id="implied len/type for ReadyForQuery",
+ ),
+ pytest.param(
+ dict(type=pq3.types.Query, payload=dict(query=b"SELECT 1;")),
+ b"Q\x00\x00\x00\x0eSELECT 1;\x00",
+ id="implied len/type for Query",
+ ),
+ pytest.param(
+ dict(type=pq3.types.DataRow, payload=dict(columns=[b"abcd"])),
+ b"D\x00\x00\x00\x0E\x00\x01\x00\x00\x00\x04abcd",
+ id="implied len/type for DataRow",
+ ),
+ pytest.param(
+ dict(type=pq3.types.EmptyQueryResponse),
+ b"I\x00\x00\x00\x04",
+ id="implied len for EmptyQueryResponse",
+ ),
+ pytest.param(
+ dict(type=pq3.types.Terminate),
+ b"X\x00\x00\x00\x04",
+ id="implied len for Terminate",
+ ),
+ ],
+)
+def test_Pq3_build(fields, expected):
+ actual = pq3.Pq3.build(fields)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"\x00\x00",
+ dict(columns=[]),
+ b"",
+ id="no columns",
+ ),
+ pytest.param(
+ b"\x00\x01\x00\x00\x00\x04abcd",
+ dict(columns=[b"abcd"]),
+ b"",
+ id="one column",
+ ),
+ pytest.param(
+ b"\x00\x02\x00\x00\x00\x04abcd\x00\x00\x00\x01x",
+ dict(columns=[b"abcd", b"x"]),
+ b"",
+ id="multiple columns",
+ ),
+ pytest.param(
+ b"\x00\x02\x00\x00\x00\x00\x00\x00\x00\x01x",
+ dict(columns=[b"", b"x"]),
+ b"",
+ id="empty column value",
+ ),
+ pytest.param(
+ b"\x00\x02\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF",
+ dict(columns=[None, None]),
+ b"",
+ id="null columns",
+ ),
+ ],
+)
+def test_DataRow_parse(raw, expected, extra):
+ pkt = b"D" + struct.pack("!i", len(raw) + 4) + raw
+ with io.BytesIO(pkt) as stream:
+ actual = pq3.Pq3.parse_stream(stream)
+
+ assert actual.type == pq3.types.DataRow
+ assert actual.payload == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(),
+ b"\x00\x00",
+ id="no columns",
+ ),
+ pytest.param(
+ dict(columns=[None, None]),
+ b"\x00\x02\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF",
+ id="null columns",
+ ),
+ ],
+)
+def test_DataRow_build(fields, expected):
+ actual = pq3.Pq3.build(dict(type=pq3.types.DataRow, payload=fields))
+
+ expected = b"D" + struct.pack("!i", len(expected) + 4) + expected
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "raw,expected,exception",
+ [
+ pytest.param(
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ dict(name=b"EXTERNAL", len=-1, data=None),
+ None,
+ id="no initial response",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\x02me",
+ dict(name=b"EXTERNAL", len=2, data=b"me"),
+ None,
+ id="initial response",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\x02meextra",
+ None,
+ TerminatedError,
+ id="extra data",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\xFFme",
+ None,
+ StreamError,
+ id="underflow",
+ ),
+ ],
+)
+def test_SASLInitialResponse_parse(raw, expected, exception):
+ ctx = contextlib.nullcontext()
+ if exception:
+ ctx = pytest.raises(exception)
+
+ with ctx:
+ actual = pq3.SASLInitialResponse.parse(raw)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(name=b"EXTERNAL"),
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ id="no initial response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=None),
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ id="no initial response (explicit None)",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=b""),
+ b"EXTERNAL\x00\x00\x00\x00\x00",
+ id="empty response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=b"me@example.com"),
+ b"EXTERNAL\x00\x00\x00\x00\x0Eme@example.com",
+ id="initial response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", len=2, data=b"me@example.com"),
+ b"EXTERNAL\x00\x00\x00\x00\x02me@example.com",
+ id="data overflow",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", len=14, data=b"me"),
+ b"EXTERNAL\x00\x00\x00\x00\x0Eme",
+ id="data underflow",
+ ),
+ ],
+)
+def test_SASLInitialResponse_build(fields, expected):
+ actual = pq3.SASLInitialResponse.build(fields)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "version,expected_bytes",
+ [
+ pytest.param((3, 0), b"\x00\x03\x00\x00", id="version 3"),
+ pytest.param((1234, 5679), b"\x04\xd2\x16\x2f", id="SSLRequest"),
+ ],
+)
+def test_protocol(version, expected_bytes):
+ # Make sure the integer returned by protocol is correctly serialized on the
+ # wire.
+ assert struct.pack("!i", pq3.protocol(*version)) == expected_bytes
+
+
+@pytest.mark.parametrize(
+ "envvar,func,expected",
+ [
+ ("PGHOST", pq3.pghost, "localhost"),
+ ("PGPORT", pq3.pgport, 5432),
+ ("PGUSER", pq3.pguser, getpass.getuser()),
+ ("PGDATABASE", pq3.pgdatabase, "postgres"),
+ ],
+)
+def test_env_defaults(monkeypatch, envvar, func, expected):
+ monkeypatch.delenv(envvar, raising=False)
+
+ actual = func()
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "envvars,func,expected",
+ [
+ (dict(PGHOST="otherhost"), pq3.pghost, "otherhost"),
+ (dict(PGPORT="6789"), pq3.pgport, 6789),
+ (dict(PGUSER="postgres"), pq3.pguser, "postgres"),
+ (dict(PGDATABASE="template1"), pq3.pgdatabase, "template1"),
+ ],
+)
+def test_env(monkeypatch, envvars, func, expected):
+ for k, v in envvars.items():
+ monkeypatch.setenv(k, v)
+
+ actual = func()
+ assert actual == expected
diff --git a/src/test/python/tls.py b/src/test/python/tls.py
new file mode 100644
index 0000000000..075c02c1ca
--- /dev/null
+++ b/src/test/python/tls.py
@@ -0,0 +1,195 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+from construct import *
+
+#
+# TLS 1.3
+#
+# Most of the types below are transcribed from RFC 8446:
+#
+# https://tools.ietf.org/html/rfc8446
+#
+
+
+def _Vector(size_field, element):
+ return Prefixed(size_field, GreedyRange(element))
+
+
+# Alerts
+
+AlertLevel = Enum(
+ Byte,
+ warning=1,
+ fatal=2,
+)
+
+AlertDescription = Enum(
+ Byte,
+ close_notify=0,
+ unexpected_message=10,
+ bad_record_mac=20,
+ decryption_failed_RESERVED=21,
+ record_overflow=22,
+ decompression_failure=30,
+ handshake_failure=40,
+ no_certificate_RESERVED=41,
+ bad_certificate=42,
+ unsupported_certificate=43,
+ certificate_revoked=44,
+ certificate_expired=45,
+ certificate_unknown=46,
+ illegal_parameter=47,
+ unknown_ca=48,
+ access_denied=49,
+ decode_error=50,
+ decrypt_error=51,
+ export_restriction_RESERVED=60,
+ protocol_version=70,
+ insufficient_security=71,
+ internal_error=80,
+ user_canceled=90,
+ no_renegotiation=100,
+ unsupported_extension=110,
+)
+
+Alert = Struct(
+ "level" / AlertLevel,
+ "description" / AlertDescription,
+)
+
+
+# Extensions
+
+ExtensionType = Enum(
+ Int16ub,
+ server_name=0,
+ max_fragment_length=1,
+ status_request=5,
+ supported_groups=10,
+ signature_algorithms=13,
+ use_srtp=14,
+ heartbeat=15,
+ application_layer_protocol_negotiation=16,
+ signed_certificate_timestamp=18,
+ client_certificate_type=19,
+ server_certificate_type=20,
+ padding=21,
+ pre_shared_key=41,
+ early_data=42,
+ supported_versions=43,
+ cookie=44,
+ psk_key_exchange_modes=45,
+ certificate_authorities=47,
+ oid_filters=48,
+ post_handshake_auth=49,
+ signature_algorithms_cert=50,
+ key_share=51,
+)
+
+Extension = Struct(
+ "extension_type" / ExtensionType,
+ "extension_data" / Prefixed(Int16ub, GreedyBytes),
+)
+
+
+# ClientHello
+
+
+class _CipherSuiteAdapter(Adapter):
+ class _hextuple(tuple):
+ def __repr__(self):
+ return f"(0x{self[0]:02X}, 0x{self[1]:02X})"
+
+ def _encode(self, obj, context, path):
+ return bytes(obj)
+
+ def _decode(self, obj, context, path):
+ assert len(obj) == 2
+ return self._hextuple(obj)
+
+
+ProtocolVersion = Hex(Int16ub)
+
+Random = Hex(Bytes(32))
+
+CipherSuite = _CipherSuiteAdapter(Byte[2])
+
+ClientHello = Struct(
+ "legacy_version" / ProtocolVersion,
+ "random" / Random,
+ "legacy_session_id" / Prefixed(Byte, Hex(GreedyBytes)),
+ "cipher_suites" / _Vector(Int16ub, CipherSuite),
+ "legacy_compression_methods" / Prefixed(Byte, GreedyBytes),
+ "extensions" / _Vector(Int16ub, Extension),
+)
+
+# ServerHello
+
+ServerHello = Struct(
+ "legacy_version" / ProtocolVersion,
+ "random" / Random,
+ "legacy_session_id_echo" / Prefixed(Byte, Hex(GreedyBytes)),
+ "cipher_suite" / CipherSuite,
+ "legacy_compression_method" / Hex(Byte),
+ "extensions" / _Vector(Int16ub, Extension),
+)
+
+# Handshake
+
+HandshakeType = Enum(
+ Byte,
+ client_hello=1,
+ server_hello=2,
+ new_session_ticket=4,
+ end_of_early_data=5,
+ encrypted_extensions=8,
+ certificate=11,
+ certificate_request=13,
+ certificate_verify=15,
+ finished=20,
+ key_update=24,
+ message_hash=254,
+)
+
+Handshake = Struct(
+ "msg_type" / HandshakeType,
+ "length" / Int24ub,
+ "payload"
+ / Switch(
+ this.msg_type,
+ {
+ HandshakeType.client_hello: ClientHello,
+ HandshakeType.server_hello: ServerHello,
+ # HandshakeType.end_of_early_data: EndOfEarlyData,
+ # HandshakeType.encrypted_extensions: EncryptedExtensions,
+ # HandshakeType.certificate_request: CertificateRequest,
+ # HandshakeType.certificate: Certificate,
+ # HandshakeType.certificate_verify: CertificateVerify,
+ # HandshakeType.finished: Finished,
+ # HandshakeType.new_session_ticket: NewSessionTicket,
+ # HandshakeType.key_update: KeyUpdate,
+ },
+ default=FixedSized(this.length, GreedyBytes),
+ ),
+)
+
+# Records
+
+ContentType = Enum(
+ Byte,
+ invalid=0,
+ change_cipher_spec=20,
+ alert=21,
+ handshake=22,
+ application_data=23,
+)
+
+Plaintext = Struct(
+ "type" / ContentType,
+ "legacy_record_version" / ProtocolVersion,
+ "length" / Int16ub,
+ "fragment" / FixedSized(this.length, GreedyBytes),
+)
--
2.25.1
v4-0010-contrib-oauth-switch-to-pluggable-auth-API.patch (text/x-patch, 17.1K) download
From f520c08e1aee5239051e304c8a8faf5cb25bdbf2 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Fri, 25 Mar 2022 16:35:30 -0700
Subject: [PATCH v4 10/10] contrib/oauth: switch to pluggable auth API
Move the core server implementation to contrib/oauth as a pluggable
provider, using the RegisterAuthProvider() API. oauth_validator_command
has been moved from core to a custom GUC. HBA options are handled
using the new hook. Tests have been updated to handle the new
implementations.
One server modification remains: allowing custom SASL mechanisms to
declare their own maximum message length.
This patch is optional; you can apply/revert it to compare the two
approaches.
---
contrib/oauth/Makefile | 16 ++++
.../auth-oauth.c => contrib/oauth/oauth.c | 88 ++++++++++++++++---
src/backend/libpq/Makefile | 1 -
src/backend/libpq/auth.c | 7 --
src/backend/libpq/hba.c | 27 +-----
src/backend/utils/misc/guc.c | 12 ---
src/include/libpq/hba.h | 6 +-
src/include/libpq/oauth.h | 24 -----
src/test/python/README | 3 +-
src/test/python/server/test_oauth.py | 20 ++---
10 files changed, 104 insertions(+), 100 deletions(-)
create mode 100644 contrib/oauth/Makefile
rename src/backend/libpq/auth-oauth.c => contrib/oauth/oauth.c (90%)
delete mode 100644 src/include/libpq/oauth.h
diff --git a/contrib/oauth/Makefile b/contrib/oauth/Makefile
new file mode 100644
index 0000000000..880bc1fef3
--- /dev/null
+++ b/contrib/oauth/Makefile
@@ -0,0 +1,16 @@
+# contrib/oauth/Makefile
+
+MODULE_big = oauth
+OBJS = oauth.o
+PGFILEDESC = "oauth - auth provider supporting OAuth 2.0/OIDC"
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/oauth
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/backend/libpq/auth-oauth.c b/contrib/oauth/oauth.c
similarity index 90%
rename from src/backend/libpq/auth-oauth.c
rename to contrib/oauth/oauth.c
index c1232a31a0..e83f3c5d99 100644
--- a/src/backend/libpq/auth-oauth.c
+++ b/contrib/oauth/oauth.c
@@ -1,33 +1,39 @@
-/*-------------------------------------------------------------------------
+/* -------------------------------------------------------------------------
*
- * auth-oauth.c
+ * oauth.c
* Server-side implementation of the SASL OAUTHBEARER mechanism.
*
* See the following RFC for more details:
* - RFC 7628: https://tools.ietf.org/html/rfc7628
*
- * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
- * src/backend/libpq/auth-oauth.c
+ * contrib/oauth/oauth.c
*
- *-------------------------------------------------------------------------
+ * -------------------------------------------------------------------------
*/
+
#include "postgres.h"
#include <unistd.h>
#include <fcntl.h>
#include "common/oauth-common.h"
+#include "fmgr.h"
#include "lib/stringinfo.h"
#include "libpq/auth.h"
#include "libpq/hba.h"
-#include "libpq/oauth.h"
#include "libpq/sasl.h"
#include "storage/fd.h"
+#include "utils/guc.h"
+
+PG_MODULE_MAGIC;
+
+void _PG_init(void);
/* GUC */
-char *oauth_validator_command;
+static char *oauth_validator_command;
static void oauth_get_mechanisms(Port *port, StringInfo buf);
static void *oauth_init(Port *port, const char *selected_mech, const char *shadow_pass);
@@ -35,7 +41,7 @@ static int oauth_exchange(void *opaq, const char *input, int inputlen,
char **output, int *outputlen, const char **logdetail);
/* Mechanism declaration */
-const pg_be_sasl_mech pg_be_oauth_mech = {
+static const pg_be_sasl_mech oauth_mech = {
oauth_get_mechanisms,
oauth_init,
oauth_exchange,
@@ -57,12 +63,13 @@ struct oauth_ctx
Port *port;
const char *issuer;
const char *scope;
+ bool skip_usermap;
};
static char *sanitize_char(char c);
static char *parse_kvpairs_for_auth(char **input);
static void generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen);
-static bool validate(Port *port, const char *auth, const char **logdetail);
+static bool validate(struct oauth_ctx *ctx, const char *auth, const char **logdetail);
static bool run_validator_command(Port *port, const char *token);
static bool check_exit(FILE **fh, const char *command);
static bool unset_cloexec(int fd);
@@ -84,6 +91,7 @@ static void *
oauth_init(Port *port, const char *selected_mech, const char *shadow_pass)
{
struct oauth_ctx *ctx;
+ ListCell *lc;
if (strcmp(selected_mech, OAUTHBEARER_NAME))
ereport(ERROR,
@@ -96,8 +104,21 @@ oauth_init(Port *port, const char *selected_mech, const char *shadow_pass)
ctx->port = port;
Assert(port->hba);
- ctx->issuer = port->hba->oauth_issuer;
- ctx->scope = port->hba->oauth_scope;
+
+ foreach (lc, port->hba->custom_auth_options)
+ {
+ CustomOption *option = lfirst(lc);
+
+ if (strcmp(option->name, "issuer") == 0)
+ ctx->issuer = option->value;
+ else if (strcmp(option->name, "scope") == 0)
+ ctx->scope = option->value;
+ else if (strcmp(option->name, "trust_validator_authz") == 0)
+ {
+ if (strcmp(option->value, "1") == 0)
+ ctx->skip_usermap = true;
+ }
+ }
return ctx;
}
@@ -248,7 +269,7 @@ oauth_exchange(void *opaq, const char *input, int inputlen,
errmsg("malformed OAUTHBEARER message"),
errdetail("Message contains additional data after the final terminator.")));
- if (!validate(ctx->port, auth, logdetail))
+ if (!validate(ctx, auth, logdetail))
{
generate_error_response(ctx, output, outputlen);
@@ -415,12 +436,13 @@ generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
}
static bool
-validate(Port *port, const char *auth, const char **logdetail)
+validate(struct oauth_ctx *ctx, const char *auth, const char **logdetail)
{
static const char * const b64_set = "abcdefghijklmnopqrstuvwxyz"
"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
"0123456789-._~+/";
+ Port *port = ctx->port;
const char *token;
size_t span;
int ret;
@@ -497,7 +519,7 @@ validate(Port *port, const char *auth, const char **logdetail)
if (!run_validator_command(port, token))
return false;
- if (port->hba->oauth_skip_usermap)
+ if (ctx->skip_usermap)
{
/*
* If the validator is our authorization authority, we're done.
@@ -795,3 +817,41 @@ username_ok_for_shell(const char *username)
return true;
}
+
+static int CheckOAuth(Port *port)
+{
+ return CheckSASLAuth(&oauth_mech, port, NULL, NULL);
+}
+
+static const char *OAuthError(Port *port)
+{
+ return psprintf("OAuth bearer authentication failed for user \"%s\"",
+ port->user_name);
+}
+
+static bool OAuthCheckOption(char *name, char *val,
+ struct HbaLine *hbaline, char **errmsg)
+{
+ if (!strcmp(name, "issuer"))
+ return true;
+ if (!strcmp(name, "scope"))
+ return true;
+ if (!strcmp(name, "trust_validator_authz"))
+ return true;
+
+ return false;
+}
+
+void
+_PG_init(void)
+{
+ RegisterAuthProvider("oauth", CheckOAuth, OAuthError, OAuthCheckOption);
+
+ DefineCustomStringVariable("oauth.validator_command",
+ gettext_noop("Command to validate OAuth v2 bearer tokens."),
+ NULL,
+ &oauth_validator_command,
+ "",
+ PGC_SIGHUP, GUC_SUPERUSER_ONLY,
+ NULL, NULL, NULL);
+}
diff --git a/src/backend/libpq/Makefile b/src/backend/libpq/Makefile
index 98eb2a8242..6d385fd6a4 100644
--- a/src/backend/libpq/Makefile
+++ b/src/backend/libpq/Makefile
@@ -15,7 +15,6 @@ include $(top_builddir)/src/Makefile.global
# be-fsstubs is here for historical reasons, probably belongs elsewhere
OBJS = \
- auth-oauth.o \
auth-sasl.o \
auth-scram.o \
auth.o \
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 17042d84ad..4a8a63922a 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -30,7 +30,6 @@
#include "libpq/auth.h"
#include "libpq/crypt.h"
#include "libpq/libpq.h"
-#include "libpq/oauth.h"
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
#include "libpq/scram.h"
@@ -299,9 +298,6 @@ auth_failed(Port *port, int status, const char *logdetail)
case uaRADIUS:
errstr = gettext_noop("RADIUS authentication failed for user \"%s\"");
break;
- case uaOAuth:
- errstr = gettext_noop("OAuth bearer authentication failed for user \"%s\"");
- break;
case uaCustom:
{
CustomAuthProvider *provider = get_provider_by_name(port->hba->custom_provider);
@@ -630,9 +626,6 @@ ClientAuthentication(Port *port)
case uaTrust:
status = STATUS_OK;
break;
- case uaOAuth:
- status = CheckSASLAuth(&pg_be_oauth_mech, port, NULL, NULL);
- break;
case uaCustom:
{
CustomAuthProvider *provider = get_provider_by_name(port->hba->custom_provider);
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index cd3b1cc140..6bf986d5b3 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -137,7 +137,6 @@ static const char *const UserAuthName[] =
"radius",
"custom",
"peer",
- "oauth",
};
@@ -1402,8 +1401,6 @@ parse_hba_line(TokenizedLine *tok_line, int elevel)
#endif
else if (strcmp(token->string, "radius") == 0)
parsedline->auth_method = uaRADIUS;
- else if (strcmp(token->string, "oauth") == 0)
- parsedline->auth_method = uaOAuth;
else if (strcmp(token->string, "custom") == 0)
parsedline->auth_method = uaCustom;
else
@@ -1733,9 +1730,8 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->auth_method != uaGSS &&
hbaline->auth_method != uaSSPI &&
hbaline->auth_method != uaCert &&
- hbaline->auth_method != uaOAuth &&
hbaline->auth_method != uaCustom)
- INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, cert, oauth, and custom"));
+ INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, cert, and custom"));
hbaline->usermap = pstrdup(val);
}
else if (strcmp(name, "clientcert") == 0)
@@ -2119,27 +2115,6 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->radiusidentifiers = parsed_identifiers;
hbaline->radiusidentifiers_s = pstrdup(val);
}
- else if (strcmp(name, "issuer") == 0)
- {
- if (hbaline->auth_method != uaOAuth)
- INVALID_AUTH_OPTION("issuer", gettext_noop("oauth"));
- hbaline->oauth_issuer = pstrdup(val);
- }
- else if (strcmp(name, "scope") == 0)
- {
- if (hbaline->auth_method != uaOAuth)
- INVALID_AUTH_OPTION("scope", gettext_noop("oauth"));
- hbaline->oauth_scope = pstrdup(val);
- }
- else if (strcmp(name, "trust_validator_authz") == 0)
- {
- if (hbaline->auth_method != uaOAuth)
- INVALID_AUTH_OPTION("trust_validator_authz", gettext_noop("oauth"));
- if (strcmp(val, "1") == 0)
- hbaline->oauth_skip_usermap = true;
- else
- hbaline->oauth_skip_usermap = false;
- }
else if (strcmp(name, "provider") == 0)
{
REQUIRE_AUTH_OPTION(uaCustom, "provider", "custom");
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 9a5b2aa496..f70f7f5c01 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -59,7 +59,6 @@
#include "libpq/auth.h"
#include "libpq/libpq.h"
#include "libpq/pqformat.h"
-#include "libpq/oauth.h"
#include "miscadmin.h"
#include "optimizer/cost.h"
#include "optimizer/geqo.h"
@@ -4667,17 +4666,6 @@ static struct config_string ConfigureNamesString[] =
check_backtrace_functions, assign_backtrace_functions, NULL
},
- {
- {"oauth_validator_command", PGC_SIGHUP, CONN_AUTH_AUTH,
- gettext_noop("Command to validate OAuth v2 bearer tokens."),
- NULL,
- GUC_SUPERUSER_ONLY
- },
- &oauth_validator_command,
- "",
- NULL, NULL, NULL
- },
-
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, NULL, NULL, NULL, NULL
diff --git a/src/include/libpq/hba.h b/src/include/libpq/hba.h
index e405103a2e..bbc94363cb 100644
--- a/src/include/libpq/hba.h
+++ b/src/include/libpq/hba.h
@@ -40,8 +40,7 @@ typedef enum UserAuth
uaRADIUS,
uaCustom,
uaPeer,
- uaOAuth
-#define USER_AUTH_LAST uaOAuth /* Must be last value of this enum */
+#define USER_AUTH_LAST uaPeer /* Must be last value of this enum */
} UserAuth;
/*
@@ -129,9 +128,6 @@ typedef struct HbaLine
char *radiusidentifiers_s;
List *radiusports;
char *radiusports_s;
- char *oauth_issuer;
- char *oauth_scope;
- bool oauth_skip_usermap;
char *custom_provider;
List *custom_auth_options;
} HbaLine;
diff --git a/src/include/libpq/oauth.h b/src/include/libpq/oauth.h
deleted file mode 100644
index 870e426af1..0000000000
--- a/src/include/libpq/oauth.h
+++ /dev/null
@@ -1,24 +0,0 @@
-/*-------------------------------------------------------------------------
- *
- * oauth.h
- * Interface to libpq/auth-oauth.c
- *
- * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
- * Portions Copyright (c) 1994, Regents of the University of California
- *
- * src/include/libpq/oauth.h
- *
- *-------------------------------------------------------------------------
- */
-#ifndef PG_OAUTH_H
-#define PG_OAUTH_H
-
-#include "libpq/libpq-be.h"
-#include "libpq/sasl.h"
-
-extern char *oauth_validator_command;
-
-/* Implementation */
-extern const pg_be_sasl_mech pg_be_oauth_mech;
-
-#endif /* PG_OAUTH_H */
diff --git a/src/test/python/README b/src/test/python/README
index 0bda582c4b..0fbc1046cf 100644
--- a/src/test/python/README
+++ b/src/test/python/README
@@ -13,7 +13,8 @@ but you can adjust as needed for your setup.
## Requirements
-A supported version (3.6+) of Python.
+- A supported version (3.6+) of Python.
+- The oauth extension must be installed and loaded via shared_preload_libraries.
The first run of
diff --git a/src/test/python/server/test_oauth.py b/src/test/python/server/test_oauth.py
index cb5ca7fa23..07fc25edc2 100644
--- a/src/test/python/server/test_oauth.py
+++ b/src/test/python/server/test_oauth.py
@@ -103,9 +103,9 @@ def oauth_ctx():
ctx = Context()
hba_lines = (
- f'host {ctx.dbname} {ctx.map_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" map=oauth\n',
- f'host {ctx.dbname} {ctx.authz_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" trust_validator_authz=1\n',
- f'host {ctx.dbname} all samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}"\n',
+ f'host {ctx.dbname} {ctx.map_user} samehost custom provider=oauth issuer="{ctx.issuer}" scope="{ctx.scope}" map=oauth\n',
+ f'host {ctx.dbname} {ctx.authz_user} samehost custom provider=oauth issuer="{ctx.issuer}" scope="{ctx.scope}" trust_validator_authz=1\n',
+ f'host {ctx.dbname} all samehost custom provider=oauth issuer="{ctx.issuer}" scope="{ctx.scope}"\n',
)
ident_lines = (r"oauth /^(.*)@example\.com$ \1",)
@@ -126,12 +126,12 @@ def oauth_ctx():
c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(authz_user))
c.execute(sql.SQL("CREATE DATABASE {};").format(dbname))
- # Make this test script the server's oauth_validator.
+ # Make this test script the server's oauth validator.
path = pathlib.Path(__file__).parent / "validate_bearer.py"
path = str(path.absolute())
cmd = f"{shlex.quote(path)} {SHARED_MEM_NAME} <&%f"
- c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (cmd,))
+ c.execute("ALTER SYSTEM SET oauth.validator_command TO %s;", (cmd,))
# Replace pg_hba and pg_ident.
c.execute("SHOW hba_file;")
@@ -149,7 +149,7 @@ def oauth_ctx():
# Put things back the way they were.
c.execute("SELECT pg_reload_conf();")
- c.execute("ALTER SYSTEM RESET oauth_validator_command;")
+ c.execute("ALTER SYSTEM RESET oauth.validator_command;")
c.execute(sql.SQL("DROP DATABASE {};").format(dbname))
c.execute(sql.SQL("DROP ROLE {};").format(authz_user))
c.execute(sql.SQL("DROP ROLE {};").format(map_user))
@@ -930,7 +930,7 @@ def test_oauth_empty_initial_response(conn, oauth_ctx, bearer_token):
def set_validator():
"""
A per-test fixture that allows a test to override the setting of
- oauth_validator_command for the cluster. The setting will be reverted during
+ oauth.validator_command for the cluster. The setting will be reverted during
teardown.
Passing None will perform an ALTER SYSTEM RESET.
@@ -942,17 +942,17 @@ def set_validator():
c = conn.cursor()
# Save the previous value.
- c.execute("SHOW oauth_validator_command;")
+ c.execute("SHOW oauth.validator_command;")
prev_cmd = c.fetchone()[0]
def setter(cmd):
- c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (cmd,))
+ c.execute("ALTER SYSTEM SET oauth.validator_command TO %s;", (cmd,))
c.execute("SELECT pg_reload_conf();")
yield setter
# Restore the previous value.
- c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (prev_cmd,))
+ c.execute("ALTER SYSTEM SET oauth.validator_command TO %s;", (prev_cmd,))
c.execute("SELECT pg_reload_conf();")
--
2.25.1
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-09-20 05:03 mahendrakar s <mahendrakarforpg@gmail.com>
parent: Jacob Champion <pchampion@vmware.com>
2 siblings, 1 reply; 424+ messages in thread
From: mahendrakar s @ 2022-09-20 05:03 UTC (permalink / raw)
To: pgsql-hackers; Jacob Champion <jchampion@timescale.com>; +Cc: smilingsamay@gmail.com <smilingsamay@gmail.com>; andres@anarazel.de <andres@anarazel.de>; anchudno@microsoft.com; mahendrakars@microsoft.com
Hi Hackers,
We are trying to implement AAD(Azure AD) support in PostgreSQL and it
can be achieved with support of the OAuth method. To support AAD on
top of OAuth in a generic fashion (i.e for all other OAuth providers),
we are proposing this patch. It basically exposes two new hooks (one
for error reporting and one for OAuth provider specific token
validation) and passing OAuth bearer token to backend. It also adds
support for client credentials flow of OAuth additional to device code
flow which Jacob has proposed.
The changes for each component are summarized below.
1. Provider-specific extension:
Each OAuth provider implements their own token validator as an
extension. Extension registers an OAuth provider hook which is matched
to a line in the HBA file.
2. Add support to pass on the OAuth bearer token. In this
obtaining the bearer token is left to 3rd party application or user.
./psql -U <username> -d 'dbname=postgres
oauth_client_id=<client_id> oauth_bearer_token=<token>
3. HBA: An additional param ‘provider’ is added for the oauth method.
Defining "oauth" as method + passing provider, issuer endpoint
and expected audience
* * * * oauth provider=<token validation extension>
issuer=.... scope=....
4. Engine Backend:
Support for generic OAUTHBEARER type, requesting client to
provide token and passing to token for provider-specific extension.
5. Engine Frontend: Two-tiered approach.
a) libpq transparently passes on the token received
from 3rd party client as is to the backend.
b) libpq optionally compiled for the clients which
explicitly need libpq to orchestrate OAuth communication with the
issuer (it depends heavily on 3rd party library iddawc as Jacob
already pointed out. The library seems to be supporting all the OAuth
flows.)
Please let us know your thoughts as the proposed method supports
different OAuth flows with the use of provider specific hooks. We
think that the proposal would be useful for various OAuth providers.
Thanks,
Mahendrakar.
On Tue, 20 Sept 2022 at 10:18, Jacob Champion <pchampion@vmware.com> wrote:
>
> On Tue, 2021-06-22 at 23:22 +0000, Jacob Champion wrote:
> > On Fri, 2021-06-18 at 11:31 +0300, Heikki Linnakangas wrote:
> > >
> > > A few small things caught my eye in the backend oauth_exchange function:
> > >
> > > > + /* Handle the client's initial message. */
> > > > + p = strdup(input);
> > >
> > > this strdup() should be pstrdup().
> >
> > Thanks, I'll fix that in the next re-roll.
> >
> > > In the same function, there are a bunch of reports like this:
> > >
> > > > ereport(ERROR,
> > > > + (errcode(ERRCODE_PROTOCOL_VIOLATION),
> > > > + errmsg("malformed OAUTHBEARER message"),
> > > > + errdetail("Comma expected, but found character \"%s\".",
> > > > + sanitize_char(*p))));
> > >
> > > I don't think the double quotes are needed here, because sanitize_char
> > > will return quotes if it's a single character. So it would end up
> > > looking like this: ... found character "'x'".
> >
> > I'll fix this too. Thanks!
>
> v2, attached, incorporates Heikki's suggested fixes and also rebases on
> top of latest HEAD, which had the SASL refactoring changes committed
> last month.
>
> The biggest change from the last patchset is 0001, an attempt at
> enabling jsonapi in the frontend without the use of palloc(), based on
> suggestions by Michael and Tom from last commitfest. I've also made
> some improvements to the pytest suite. No major changes to the OAuth
> implementation yet.
>
> --Jacob
Attachments:
v1-0001-oauth-provider-support.patch (application/x-patch, 17.5K) download
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
index c47211132c..86f820482b 100644
--- a/src/backend/libpq/auth-oauth.c
+++ b/src/backend/libpq/auth-oauth.c
@@ -24,7 +24,9 @@
#include "libpq/hba.h"
#include "libpq/oauth.h"
#include "libpq/sasl.h"
+#include "miscadmin.h"
#include "storage/fd.h"
+#include "utils/memutils.h"
/* GUC */
char *oauth_validator_command;
@@ -34,6 +36,13 @@ static void *oauth_init(Port *port, const char *selected_mech, const char *shado
static int oauth_exchange(void *opaq, const char *input, int inputlen,
char **output, int *outputlen, char **logdetail);
+/*----------------------------------------------------------------
+ * OAuth Authentication
+ *----------------------------------------------------------------
+ */
+static List *oauth_providers = NIL;
+static OAuthProvider* oauth_provider = NULL;
+
/* Mechanism declaration */
const pg_be_sasl_mech pg_be_oauth_mech = {
oauth_get_mechanisms,
@@ -63,15 +72,90 @@ static char *sanitize_char(char c);
static char *parse_kvpairs_for_auth(char **input);
static void generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen);
static bool validate(Port *port, const char *auth, char **logdetail);
-static bool run_validator_command(Port *port, const char *token);
+static const char* run_validator_command(Port *port, const char *token);
static bool check_exit(FILE **fh, const char *command);
static bool unset_cloexec(int fd);
-static bool username_ok_for_shell(const char *username);
#define KVSEP 0x01
#define AUTH_KEY "auth"
#define BEARER_SCHEME "Bearer "
+/*----------------------------------------------------------------
+ * OAuth Token Validator
+ *----------------------------------------------------------------
+ */
+
+/*
+ * RegisterOAuthProvider registers a OAuth Token Validator to be
+ * used for oauth token validation. It validates the token and adds the valiator
+ * name and it's hooks to a list of loaded token validator. The right validator's
+ * hooks can then be called based on the validator name specified in
+ * pg_hba.conf.
+ *
+ * This function should be called in _PG_init() by any extension looking to
+ * add a custom authentication method.
+ */
+void
+RegisterOAuthProvider(
+ const char *provider_name,
+ OAuthProviderCheck_hook_type OAuthProviderCheck_hook,
+ OAuthProviderError_hook_type OAuthProviderError_hook
+)
+{
+ if (!process_shared_preload_libraries_in_progress)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("RegisterOAuthProvider can only be called by a shared_preload_library")));
+ return;
+ }
+
+ MemoryContext oldcxt;
+ if (oauth_provider == NULL)
+ {
+ oldcxt = MemoryContextSwitchTo(TopMemoryContext);
+ oauth_provider = palloc(sizeof(OAuthProvider));
+ oauth_provider->name = pstrdup(provider_name);
+ oauth_provider->oauth_provider_hook = OAuthProviderCheck_hook;
+ oauth_provider->oauth_error_hook = OAuthProviderError_hook;
+ oauth_providers = lappend(oauth_providers, oauth_provider);
+ MemoryContextSwitchTo(oldcxt);
+ }
+ else
+ {
+ if (oauth_provider && oauth_provider->name)
+ {
+ ereport(ERROR,
+ (errmsg("OAuth provider \"%s\" is already loaded.",
+ oauth_provider->name)));
+ }
+ else
+ {
+ ereport(ERROR,
+ (errmsg("OAuth provider is already loaded.")));
+ }
+ }
+}
+
+/*
+ * Returns the oauth provider (which includes it's
+ * callback functions) based on name specified.
+ */
+OAuthProvider *get_provider_by_name(const char *name)
+{
+ ListCell *lc;
+ foreach(lc, oauth_providers)
+ {
+ OAuthProvider *provider = (OAuthProvider *) lfirst(lc);
+ if (strcmp(provider->name, name) == 0)
+ {
+ return provider;
+ }
+ }
+
+ return NULL;
+}
+
static void
oauth_get_mechanisms(Port *port, StringInfo buf)
{
@@ -494,17 +578,17 @@ validate(Port *port, const char *auth, char **logdetail)
}
/* Have the validator check the token. */
- if (!run_validator_command(port, token))
+ if (run_validator_command(port, token) == NULL)
return false;
-
+
if (port->hba->oauth_skip_usermap)
{
/*
- * If the validator is our authorization authority, we're done.
- * Authentication may or may not have been performed depending on the
- * validator implementation; all that matters is that the validator says
- * the user can log in with the target role.
- */
+ * If the validator is our authorization authority, we're done.
+ * Authentication may or may not have been performed depending on the
+ * validator implementation; all that matters is that the validator says
+ * the user can log in with the target role.
+ */
return true;
}
@@ -524,193 +608,26 @@ validate(Port *port, const char *auth, char **logdetail)
return (ret == STATUS_OK);
}
-static bool
+static const char*
run_validator_command(Port *port, const char *token)
{
- bool success = false;
- int rc;
- int pipefd[2];
- int rfd = -1;
- int wfd = -1;
-
- StringInfoData command = { 0 };
- char *p;
- FILE *fh = NULL;
-
- ssize_t written;
- char *line = NULL;
- size_t size = 0;
- ssize_t len;
-
- Assert(oauth_validator_command);
-
- if (!oauth_validator_command[0])
- {
- ereport(COMMERROR,
- (errmsg("oauth_validator_command is not set"),
- errhint("To allow OAuth authenticated connections, set "
- "oauth_validator_command in postgresql.conf.")));
- return false;
- }
-
- /*
- * Since popen() is unidirectional, open up a pipe for the other direction.
- * Use CLOEXEC to ensure that our write end doesn't accidentally get copied
- * into child processes, which would prevent us from closing it cleanly.
- *
- * XXX this is ugly. We should just read from the child process's stdout,
- * but that's a lot more code.
- * XXX by bypassing the popen API, we open the potential of process
- * deadlock. Clearly document child process requirements (i.e. the child
- * MUST read all data off of the pipe before writing anything).
- * TODO: port to Windows using _pipe().
- */
- rc = pipe2(pipefd, O_CLOEXEC);
- if (rc < 0)
+ if(oauth_provider->oauth_provider_hook == NULL)
{
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not create child pipe: %m")));
return false;
}
- rfd = pipefd[0];
- wfd = pipefd[1];
-
- /* Allow the read pipe be passed to the child. */
- if (!unset_cloexec(rfd))
+ char *id = oauth_provider->
+ oauth_provider_hook(port, token);
+ if(id == NULL)
{
- /* error message was already logged */
- goto cleanup;
- }
-
- /*
- * Construct the command, substituting any recognized %-specifiers:
- *
- * %f: the file descriptor of the input pipe
- * %r: the role that the client wants to assume (port->user_name)
- * %%: a literal '%'
- */
- initStringInfo(&command);
-
- for (p = oauth_validator_command; *p; p++)
- {
- if (p[0] == '%')
- {
- switch (p[1])
- {
- case 'f':
- appendStringInfo(&command, "%d", rfd);
- p++;
- break;
- case 'r':
- /*
- * TODO: decide how this string should be escaped. The role
- * is controlled by the client, so if we don't escape it,
- * command injections are inevitable.
- *
- * This is probably an indication that the role name needs
- * to be communicated to the validator process in some other
- * way. For this proof of concept, just be incredibly strict
- * about the characters that are allowed in user names.
- */
- if (!username_ok_for_shell(port->user_name))
- goto cleanup;
-
- appendStringInfoString(&command, port->user_name);
- p++;
- break;
- case '%':
- appendStringInfoChar(&command, '%');
- p++;
- break;
- default:
- appendStringInfoChar(&command, p[0]);
- }
- }
- else
- appendStringInfoChar(&command, p[0]);
- }
-
- /* Execute the command. */
- fh = OpenPipeStream(command.data, "re");
- /* TODO: handle failures */
-
- /* We don't need the read end of the pipe anymore. */
- close(rfd);
- rfd = -1;
-
- /* Give the command the token to validate. */
- written = write(wfd, token, strlen(token));
- if (written != strlen(token))
- {
- /* TODO must loop for short writes, EINTR et al */
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not write token to child pipe: %m")));
- goto cleanup;
- }
-
- close(wfd);
- wfd = -1;
-
- /*
- * Read the command's response.
- *
- * TODO: getline() is probably too new to use, unfortunately.
- * TODO: loop over all lines
- */
- if ((len = getline(&line, &size, fh)) >= 0)
- {
- /* TODO: fail if the authn_id doesn't end with a newline */
- if (len > 0)
- line[len - 1] = '\0';
-
- set_authn_id(port, line);
- }
- else if (ferror(fh))
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not read from command \"%s\": %m",
- command.data)));
- goto cleanup;
- }
-
- /* Make sure the command exits cleanly. */
- if (!check_exit(&fh, command.data))
- {
- /* error message already logged */
- goto cleanup;
- }
-
- /* Done. */
- success = true;
-
-cleanup:
- if (line)
- free(line);
-
- /*
- * In the successful case, the pipe fds are already closed. For the error
- * case, always close out the pipe before waiting for the command, to
- * prevent deadlock.
- */
- if (rfd >= 0)
- close(rfd);
- if (wfd >= 0)
- close(wfd);
-
- if (fh)
- {
- Assert(!success);
- check_exit(&fh, command.data);
+ ereport(LOG,
+ (errmsg("OAuth bearer token validation failed" )));
+ return NULL;
}
- if (command.data)
- pfree(command.data);
-
- return success;
+ set_authn_id(port, id);
+
+ return id;
}
static bool
@@ -769,29 +686,3 @@ unset_cloexec(int fd)
return true;
}
-
-/*
- * XXX This should go away eventually and be replaced with either a proper
- * escape or a different strategy for communication with the validator command.
- */
-static bool
-username_ok_for_shell(const char *username)
-{
- /* This set is borrowed from fe_utils' appendShellStringNoError(). */
- static const char * const allowed = "abcdefghijklmnopqrstuvwxyz"
- "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
- "0123456789-_./:";
- size_t span;
-
- Assert(username && username[0]); /* should have already been checked */
-
- span = strspn(username, allowed);
- if (username[span] != '\0')
- {
- ereport(COMMERROR,
- (errmsg("PostgreSQL user name contains unsafe characters and cannot be passed to the OAuth validator")));
- return false;
- }
-
- return true;
-}
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 333051ad3c..0bbcf231d2 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -296,8 +296,14 @@ auth_failed(Port *port, int status, const char *logdetail)
errstr = gettext_noop("RADIUS authentication failed for user \"%s\"");
break;
case uaOAuth:
- errstr = gettext_noop("OAuth bearer authentication failed for user \"%s\"");
- break;
+ {
+ OAuthProvider *provider = get_provider_by_name(port->hba->oauth_provider);
+ if(provider->oauth_error_hook)
+ errstr = provider->oauth_error_hook(port);
+ else
+ errstr = gettext_noop("OAuth bearer authentication failed for user \"%s\"");
+ break;
+ }
default:
errstr = gettext_noop("authentication failed for user \"%s\": invalid authentication method");
break;
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index 943e78ddff..94fb5d434d 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -1663,6 +1663,14 @@ parse_hba_line(TokenizedAuthLine *tok_line, int elevel)
parsedline->clientcert = clientCertFull;
}
+ /*
+ * Ensure that the token validation provider name is specified as provider for oauth method.
+ */
+ if (parsedline->auth_method == uaOAuth)
+ {
+ MANDATORY_AUTH_ARG(parsedline->oauth_provider, "provider", "oauth");
+ }
+
return parsedline;
}
@@ -2095,6 +2103,31 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
else
hbaline->oauth_skip_usermap = false;
}
+ else if (strcmp(name, "provider") == 0)
+ {
+ REQUIRE_AUTH_OPTION(uaOAuth, "provider", "oauth");
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("provider", gettext_noop("oauth"));
+ /*
+ * Verify that the token validation mentioned is loaded via shared_preload_libraries.
+ */
+ if (get_provider_by_name(val) == NULL)
+ {
+ ereport(elevel,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ errmsg("cannot use oauth provider %s",val),
+ errhint("Load provider token validation via shared_preload_libraries."),
+ errcontext("line %d of configuration file \"%s\"",
+ line_num, HbaFileName)));
+ *err_msg = psprintf("cannot use oauth provider %s", val);
+
+ return false;
+ }
+ else
+ {
+ hbaline->oauth_provider = pstrdup(val);
+ }
+ }
else
{
ereport(elevel,
diff --git a/src/include/libpq/auth.h b/src/include/libpq/auth.h
index 485e48970e..938ac399dc 100644
--- a/src/include/libpq/auth.h
+++ b/src/include/libpq/auth.h
@@ -44,4 +44,29 @@ extern void set_authn_id(Port *port, const char *id);
typedef void (*ClientAuthentication_hook_type) (Port *, int);
extern PGDLLIMPORT ClientAuthentication_hook_type ClientAuthentication_hook;
+/* Declarations for oAuth authentication providers */
+typedef const char* (*OAuthProviderCheck_hook_type) (Port *, const char*);
+
+/* Hook for plugins to report error messages in validation_failed() */
+typedef const char * (*OAuthProviderError_hook_type) (Port *);
+
+/* Hook for plugins to validate oauth provider options */
+typedef bool (*OAuthProviderValidateOptions_hook_type)
+ (char *, char *, HbaLine *, char **);
+
+typedef struct OAuthProvider
+{
+ const char *name;
+ OAuthProviderCheck_hook_type oauth_provider_hook;
+ OAuthProviderError_hook_type oauth_error_hook;
+} OAuthProvider;
+
+extern void RegisterOAuthProvider
+ (const char *provider_name,
+ OAuthProviderCheck_hook_type OAuthProviderCheck_hook,
+ OAuthProviderError_hook_type OAuthProviderError_hook
+ );
+
+extern OAuthProvider *get_provider_by_name(const char *name);
+
#endif /* AUTH_H */
diff --git a/src/include/libpq/hba.h b/src/include/libpq/hba.h
index c1b1313989..d65395cc22 100644
--- a/src/include/libpq/hba.h
+++ b/src/include/libpq/hba.h
@@ -123,6 +123,7 @@ typedef struct HbaLine
char *radiusports_s;
char *oauth_issuer;
char *oauth_scope;
+ char *oauth_provider;
bool oauth_skip_usermap;
} HbaLine;
diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
index 91d2c69f16..61a0b80b7e 100644
--- a/src/interfaces/libpq/fe-auth-oauth.c
+++ b/src/interfaces/libpq/fe-auth-oauth.c
@@ -174,6 +174,16 @@ get_auth_token(PGconn *conn)
if (!token_buf)
goto cleanup;
+ if(conn->oauth_bearer_token)
+ {
+ appendPQExpBufferStr(token_buf, "Bearer ");
+ appendPQExpBufferStr(token_buf, conn->oauth_bearer_token);
+ if (PQExpBufferBroken(token_buf))
+ goto cleanup;
+ token = strdup(token_buf->data);
+ goto cleanup;
+ }
+
err = i_set_str_parameter(&session, I_OPT_OPENID_CONFIG_ENDPOINT, conn->oauth_discovery_uri);
if (err)
{
@@ -201,18 +211,22 @@ get_auth_token(PGconn *conn)
libpq_gettext("issuer does not support device authorization"));
goto cleanup;
}
+
+ //default device flow
+ int session_response_type = I_RESPONSE_TYPE_DEVICE_CODE;
+ auth_method = I_TOKEN_AUTH_METHOD_NONE;
+ if (conn->oauth_client_secret && *conn->oauth_client_secret)
+ {
+ auth_method = I_TOKEN_AUTH_METHOD_SECRET_BASIC;
+ }
- err = i_set_response_type(&session, I_RESPONSE_TYPE_DEVICE_CODE);
+ err = i_set_response_type(&session, session_response_type);
if (err)
{
iddawc_error(conn, err, "failed to set device code response type");
goto cleanup;
}
- auth_method = I_TOKEN_AUTH_METHOD_NONE;
- if (conn->oauth_client_secret && *conn->oauth_client_secret)
- auth_method = I_TOKEN_AUTH_METHOD_SECRET_BASIC;
-
err = i_set_parameter_list(&session,
I_OPT_CLIENT_ID, conn->oauth_client_id,
I_OPT_CLIENT_SECRET, conn->oauth_client_secret,
@@ -250,6 +264,18 @@ get_auth_token(PGconn *conn)
goto cleanup;
}
+ if (conn->oauth_client_secret && *conn->oauth_client_secret)
+ {
+ session_response_type = I_RESPONSE_TYPE_CLIENT_CREDENTIALS;
+ }
+
+ err = i_set_response_type(&session, session_response_type);
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set session response type");
+ goto cleanup;
+ }
+
/*
* Poll the token endpoint until either the user logs in and authorizes the
* use of a token, or a hard failure occurs. We perform one ping _before_
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 2ff450ce05..5d804c8c0d 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -361,6 +361,10 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"OAuth-Scope", "", 15,
offsetof(struct pg_conn, oauth_scope)},
+ {"oauth_bearer_token", NULL, NULL, NULL,
+ "OAuth-Bearer", "", 20,
+ offsetof(struct pg_conn, oauth_bearer_token)},
+
/* Terminating entry --- MUST BE LAST */
{NULL, NULL, NULL, NULL,
NULL, NULL, 0}
@@ -4200,6 +4204,8 @@ freePGconn(PGconn *conn)
free(conn->oauth_discovery_uri);
if (conn->oauth_client_id)
free(conn->oauth_client_id);
+ if(conn->oauth_bearer_token)
+ free(conn->oauth_bearer_token);
if (conn->oauth_client_secret)
free(conn->oauth_client_secret);
if (conn->oauth_scope)
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 1b4de3dff0..91e71afe14 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -402,6 +402,7 @@ struct pg_conn
char *oauth_client_id; /* client identifier */
char *oauth_client_secret; /* client secret */
char *oauth_scope; /* access token scope */
+ char *oauth_bearer_token; /* oauth token */
bool oauth_want_retry; /* should we retry on failure? */
/* Optional file to write trace info to */
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-09-20 23:19 Jacob Champion <jchampion@timescale.com>
parent: mahendrakar s <mahendrakarforpg@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2022-09-20 23:19 UTC (permalink / raw)
To: mahendrakar s <mahendrakarforpg@gmail.com>; +Cc: pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>; andres@anarazel.de <andres@anarazel.de>; anchudno@microsoft.com; mahendrakars@microsoft.com
Hi Mahendrakar, thanks for your interest and for the patch!
On Mon, Sep 19, 2022 at 10:03 PM mahendrakar s
<mahendrakarforpg@gmail.com> wrote:
> The changes for each component are summarized below.
>
> 1. Provider-specific extension:
> Each OAuth provider implements their own token validator as an
> extension. Extension registers an OAuth provider hook which is matched
> to a line in the HBA file.
How easy is it to write a Bearer validator using C? My limited
understanding was that most providers were publishing libraries in
higher-level languages.
Along those lines, sample validators will need to be provided, both to
help in review and to get the pytest suite green again. (And coverage
for the new code is important, too.)
> 2. Add support to pass on the OAuth bearer token. In this
> obtaining the bearer token is left to 3rd party application or user.
>
> ./psql -U <username> -d 'dbname=postgres
> oauth_client_id=<client_id> oauth_bearer_token=<token>
This hurts, but I think people are definitely going to ask for it, given
the frightening practice of copy-pasting these (incredibly sensitive
secret) tokens all over the place... Ideally I'd like to implement
sender constraints for the Bearer token, to *prevent* copy-pasting (or,
you know, outright theft). But I'm not sure that sender constraints are
well-implemented yet for the major providers.
> 3. HBA: An additional param ‘provider’ is added for the oauth method.
> Defining "oauth" as method + passing provider, issuer endpoint
> and expected audience
>
> * * * * oauth provider=<token validation extension>
> issuer=.... scope=....
Naming aside (this conflicts with Samay's previous proposal, I think), I
have concerns about the implementation. There's this code:
> + if (oauth_provider && oauth_provider->name)
> + {
> + ereport(ERROR,
> + (errmsg("OAuth provider \"%s\" is already loaded.",
> + oauth_provider->name)));
> + }
which appears to prevent loading more than one global provider. But
there's also code that deals with a provider list? (Again, it'd help to
have test code covering the new stuff.)
> b) libpq optionally compiled for the clients which
> explicitly need libpq to orchestrate OAuth communication with the
> issuer (it depends heavily on 3rd party library iddawc as Jacob
> already pointed out. The library seems to be supporting all the OAuth
> flows.)
Speaking of iddawc, I don't think it's a dependency we should choose to
rely on. For all the code that it has, it doesn't seem to provide
compatibility with several real-world providers.
Google, for one, chose not to follow the IETF spec it helped author, and
iddawc doesn't support its flavor of Device Authorization. At another
point, I think iddawc tried to decode Azure's Bearer tokens, which is
incorrect...
I haven't been able to check if those problems have been fixed in a
recent version, but if we're going to tie ourselves to a huge
dependency, I'd at least like to believe that said dependency is
battle-tested and solid, and personally I don't feel like iddawc is.
> - auth_method = I_TOKEN_AUTH_METHOD_NONE;
> - if (conn->oauth_client_secret && *conn->oauth_client_secret)
> - auth_method = I_TOKEN_AUTH_METHOD_SECRET_BASIC;
This code got moved, but I'm not sure why? It doesn't appear to have
made a change to the logic.
> + if (conn->oauth_client_secret && *conn->oauth_client_secret)
> + {
> + session_response_type = I_RESPONSE_TYPE_CLIENT_CREDENTIALS;
> + }
Is this an Azure-specific requirement? Ideally a public client (which
psql is) shouldn't have to provide a secret to begin with, if I
understand that bit of the protocol correctly. I think Google also
required provider-specific changes in this part of the code, and
unfortunately I don't think they looked the same as yours.
We'll have to figure all that out... Standards are great; everyone has
one of their own. :)
Thanks,
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-09-21 16:03 Jacob Champion <jchampion@timescale.com>
parent: Jacob Champion <jchampion@timescale.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2022-09-21 16:03 UTC (permalink / raw)
To: mahendrakar s <mahendrakarforpg@gmail.com>; +Cc: pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>; andres@anarazel.de <andres@anarazel.de>; anchudno@microsoft.com; mahendrakars@microsoft.com
On Tue, Sep 20, 2022 at 4:19 PM Jacob Champion <jchampion@timescale.com> wrote:
> > 2. Add support to pass on the OAuth bearer token. In this
> > obtaining the bearer token is left to 3rd party application or user.
> >
> > ./psql -U <username> -d 'dbname=postgres
> > oauth_client_id=<client_id> oauth_bearer_token=<token>
>
> This hurts, but I think people are definitely going to ask for it, given
> the frightening practice of copy-pasting these (incredibly sensitive
> secret) tokens all over the place...
After some further thought -- in this case, you already have an opaque
Bearer token (and therefore you already know, out of band, which
provider needs to be used), you're willing to copy-paste it from
whatever service you got it from, and you have an extension plugged
into Postgres on the backend that verifies this Bearer blob using some
procedure that Postgres knows nothing about.
Why do you need the OAUTHBEARER mechanism logic at that point? Isn't
that identical to a custom password scheme? It seems like that could
be handled completely by Samay's pluggable auth proposal.
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* RE: [EXTERNAL] Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-09-21 22:10 Andrey Chudnovskiy <Andrey.Chudnovskiy@microsoft.com>
parent: Jacob Champion <jchampion@timescale.com>
0 siblings, 1 reply; 424+ messages in thread
From: Andrey Chudnovskiy @ 2022-09-21 22:10 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; mahendrakar s <mahendrakarforpg@gmail.com>; +Cc: pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>; andres@anarazel.de <andres@anarazel.de>; Mahendrakar Srinivasarao <mahendrakars@microsoft.com>
We can support both passing the token from an upstream client and libpq implementing OAUTH2 protocol to obtaining one.
Libpq implementing OAUTHBEARER is needed for community/3rd party tools to have user-friendly authentication experience:
1. For community client tools, like pg_admin, psql etc.
Example experience: pg_admin would be able to open a popup dialog to authenticate customer and keep refresh token to avoid asking the user frequently.
2. For 3rd party connectors supporting generic OAUTH with any provider. Useful for datawiz clients, like Tableau or ETL tools. Those can support both user and client OAUTH flows.
Libpq passing toked directly from an upstream client is useful in other scenarios:
1. Enterprise clients, built with .Net / Java and using provider-specific authentication libraries, like MSAL for AAD. Those can also support more advance provider-specific token acquisition flows.
2. Resource-tight (like IoT) clients. Those can be compiled without optional libpq flag not including the iddawc or other dependency.
Thanks!
Andrey.
-----Original Message-----
From: Jacob Champion <jchampion@timescale.com>
Sent: Wednesday, September 21, 2022 9:03 AM
To: mahendrakar s <mahendrakarforpg@gmail.com>
Cc: pgsql-hackers@postgresql.org; smilingsamay@gmail.com; andres@anarazel.de; Andrey Chudnovskiy <Andrey.Chudnovskiy@microsoft.com>; Mahendrakar Srinivasarao <mahendrakars@microsoft.com>
Subject: [EXTERNAL] Re: [PoC] Federated Authn/z with OAUTHBEARER
[You don't often get email from jchampion@timescale.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
On Tue, Sep 20, 2022 at 4:19 PM Jacob Champion <jchampion@timescale.com> wrote:
> > 2. Add support to pass on the OAuth bearer token. In this
> > obtaining the bearer token is left to 3rd party application or user.
> >
> > ./psql -U <username> -d 'dbname=postgres
> > oauth_client_id=<client_id> oauth_bearer_token=<token>
>
> This hurts, but I think people are definitely going to ask for it,
> given the frightening practice of copy-pasting these (incredibly
> sensitive
> secret) tokens all over the place...
After some further thought -- in this case, you already have an opaque Bearer token (and therefore you already know, out of band, which provider needs to be used), you're willing to copy-paste it from whatever service you got it from, and you have an extension plugged into Postgres on the backend that verifies this Bearer blob using some procedure that Postgres knows nothing about.
Why do you need the OAUTHBEARER mechanism logic at that point? Isn't that identical to a custom password scheme? It seems like that could be handled completely by Samay's pluggable auth proposal.
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [EXTERNAL] Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-09-21 22:31 Jacob Champion <jchampion@timescale.com>
parent: Andrey Chudnovskiy <Andrey.Chudnovskiy@microsoft.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2022-09-21 22:31 UTC (permalink / raw)
To: Andrey Chudnovskiy <Andrey.Chudnovskiy@microsoft.com>; +Cc: mahendrakar s <mahendrakarforpg@gmail.com>; pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>; andres@anarazel.de <andres@anarazel.de>; Mahendrakar Srinivasarao <mahendrakars@microsoft.com>
On Wed, Sep 21, 2022 at 3:10 PM Andrey Chudnovskiy
<Andrey.Chudnovskiy@microsoft.com> wrote:
> We can support both passing the token from an upstream client and libpq implementing OAUTH2 protocol to obtaining one.
Right, I agree that we could potentially do both.
> Libpq passing toked directly from an upstream client is useful in other scenarios:
> 1. Enterprise clients, built with .Net / Java and using provider-specific authentication libraries, like MSAL for AAD. Those can also support more advance provider-specific token acquisition flows.
> 2. Resource-tight (like IoT) clients. Those can be compiled without optional libpq flag not including the iddawc or other dependency.
What I don't understand is how the OAUTHBEARER mechanism helps you in
this case. You're short-circuiting the negotiation where the server
tells the client what provider to use and what scopes to request, and
instead you're saying "here's a secret string, just take it and
validate it with magic."
I realize the ability to pass an opaque token may be useful, but from
the server's perspective, I don't see what differentiates it from the
password auth method plus a custom authenticator plugin. Why pay for
the additional complexity of OAUTHBEARER if you're not going to use
it?
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [EXTERNAL] Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-09-22 04:55 Andrey Chudnovsky <achudnovskij@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
0 siblings, 1 reply; 424+ messages in thread
From: Andrey Chudnovsky @ 2022-09-22 04:55 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: Andrey Chudnovskiy <Andrey.Chudnovskiy@microsoft.com>; mahendrakar s <mahendrakarforpg@gmail.com>; pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>; andres@anarazel.de <andres@anarazel.de>; Mahendrakar Srinivasarao <mahendrakars@microsoft.com>
First, My message from corp email wasn't displayed in the thread,
That is what Jacob replied to, let me post it here for context:
> We can support both passing the token from an upstream client and libpq implementing OAUTH2 protocol to obtain one.
>
> Libpq implementing OAUTHBEARER is needed for community/3rd party tools to have user-friendly authentication experience:
>
> 1. For community client tools, like pg_admin, psql etc.
> Example experience: pg_admin would be able to open a popup dialog to authenticate customers and keep refresh tokens to avoid asking the user frequently.
> 2. For 3rd party connectors supporting generic OAUTH with any provider. Useful for datawiz clients, like Tableau or ETL tools. Those can support both user and client OAUTH flows.
>
> Libpq passing toked directly from an upstream client is useful in other scenarios:
> 1. Enterprise clients, built with .Net / Java and using provider-specific authentication libraries, like MSAL for AAD. Those can also support more advanced provider-specific token acquisition flows.
> 2. Resource-tight (like IoT) clients. Those can be compiled without the optional libpq flag not including the iddawc or other dependency.
-----------------------------------------------------------------------------------------------------
On this:
> What I don't understand is how the OAUTHBEARER mechanism helps you in
> this case. You're short-circuiting the negotiation where the server
> tells the client what provider to use and what scopes to request, and
> instead you're saying "here's a secret string, just take it and
> validate it with magic."
>
> I realize the ability to pass an opaque token may be useful, but from
> the server's perspective, I don't see what differentiates it from the
> password auth method plus a custom authenticator plugin. Why pay for
> the additional complexity of OAUTHBEARER if you're not going to use
> it?
Yes, passing a token as a new auth method won't make much sense in
isolation. However:
1. Since OAUTHBEARER is supported in the ecosystem, passing a token as
a way to authenticate with OAUTHBEARER is more consistent (IMO), then
passing it as a password.
2. Validation on the backend side doesn't depend on whether the token
is obtained by libpq or transparently passed by the upstream client.
3. Single OAUTH auth method on the server side for both scenarios,
would allow both enterprise clients with their own Token acquisition
and community clients using libpq flows to connect as the same PG
users/roles.
On Wed, Sep 21, 2022 at 8:36 PM Jacob Champion <jchampion@timescale.com> wrote:
>
> On Wed, Sep 21, 2022 at 3:10 PM Andrey Chudnovskiy
> <Andrey.Chudnovskiy@microsoft.com> wrote:
> > We can support both passing the token from an upstream client and libpq implementing OAUTH2 protocol to obtaining one.
>
> Right, I agree that we could potentially do both.
>
> > Libpq passing toked directly from an upstream client is useful in other scenarios:
> > 1. Enterprise clients, built with .Net / Java and using provider-specific authentication libraries, like MSAL for AAD. Those can also support more advance provider-specific token acquisition flows.
> > 2. Resource-tight (like IoT) clients. Those can be compiled without optional libpq flag not including the iddawc or other dependency.
>
> What I don't understand is how the OAUTHBEARER mechanism helps you in
> this case. You're short-circuiting the negotiation where the server
> tells the client what provider to use and what scopes to request, and
> instead you're saying "here's a secret string, just take it and
> validate it with magic."
>
> I realize the ability to pass an opaque token may be useful, but from
> the server's perspective, I don't see what differentiates it from the
> password auth method plus a custom authenticator plugin. Why pay for
> the additional complexity of OAUTHBEARER if you're not going to use
> it?
>
> --Jacob
>
>
>
>
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [EXTERNAL] Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-09-22 21:53 Jacob Champion <jchampion@timescale.com>
parent: Andrey Chudnovsky <achudnovskij@gmail.com>
0 siblings, 0 replies; 424+ messages in thread
From: Jacob Champion @ 2022-09-22 21:53 UTC (permalink / raw)
To: Andrey Chudnovsky <achudnovskij@gmail.com>; +Cc: Andrey Chudnovskiy <Andrey.Chudnovskiy@microsoft.com>; mahendrakar s <mahendrakarforpg@gmail.com>; pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>; andres@anarazel.de <andres@anarazel.de>; Mahendrakar Srinivasarao <mahendrakars@microsoft.com>
On 9/21/22 21:55, Andrey Chudnovsky wrote:
> First, My message from corp email wasn't displayed in the thread,
I see it on the public archives [1]. Your client is choosing some pretty
confusing quoting tactics, though, which you may want to adjust. :D
I have what I'll call some "skeptical curiosity" here -- you don't need
to defend your use cases to me by any means, but I'd love to understand
more about them.
> Yes, passing a token as a new auth method won't make much sense in
> isolation. However:
> 1. Since OAUTHBEARER is supported in the ecosystem, passing a token as
> a way to authenticate with OAUTHBEARER is more consistent (IMO), then
> passing it as a password.
Agreed. It's probably not a very strong argument for the new mechanism,
though, especially if you're not using the most expensive code inside it.
> 2. Validation on the backend side doesn't depend on whether the token
> is obtained by libpq or transparently passed by the upstream client.
Sure.
> 3. Single OAUTH auth method on the server side for both scenarios,
> would allow both enterprise clients with their own Token acquisition
> and community clients using libpq flows to connect as the same PG
> users/roles.
Okay, this is a stronger argument. With that in mind, I want to revisit
your examples and maybe provide some counterproposals:
>> Libpq passing toked directly from an upstream client is useful in other scenarios:
>> 1. Enterprise clients, built with .Net / Java and using provider-specific authentication libraries, like MSAL for AAD. Those can also support more advanced provider-specific token acquisition flows.
I can see that providing a token directly would help you work around
limitations in libpq's "standard" OAuth flows, whether we use iddawc or
not. And it's cheap in terms of implementation. But I have a feeling it
would fall apart rapidly with error cases, where the server is giving
libpq information via the OAUTHBEARER mechanism, but libpq can only
communicate to your wrapper through human-readable error messages on stderr.
This seems like clear motivation for client-side SASL plugins (which
were also discussed on Samay's proposal thread). That's a lot more
expensive to implement in libpq, but if it were hypothetically
available, wouldn't you rather your provider-specific code be able to
speak OAUTHBEARER directly with the server?
>> 2. Resource-tight (like IoT) clients. Those can be compiled without the optional libpq flag not including the iddawc or other dependency.
I want to dig into this much more; resource-constrained systems are near
and dear to me. I can see two cases here:
Case 1: The device is an IoT client that wants to connect on its own
behalf. Why would you want to use OAuth in that case? And how would the
IoT device get its Bearer token to begin with? I'm much more used to
architectures that provision high-entropy secrets for this, whether
they're incredibly long passwords per device (in which case,
channel-bound SCRAM should be a fairly strong choice?) or client certs
(which can be better decentralized, but make for a lot of bookkeeping).
If the answer to that is, "we want an IoT client to be able to connect
using the same role as a person", then I think that illustrates a clear
need for SASL negotiation. That would let the IoT client choose
SCRAM-*-PLUS or EXTERNAL, and the person at the keyboard can choose
OAUTHBEARER. Then we have incredible flexibility, because you don't have
to engineer one mechanism to handle them all.
Case 2: The constrained device is being used as a jump point. So there's
an actual person at a keyboard, trying to get into a backend server
(maybe behind a firewall layer, etc.), and the middlebox is either not
web-connected or is incredibly tiny for some reason. That might be a
good use case for a copy-pasted Bearer token, but is there actual demand
for that use case? What motivation would you (or your end user) have for
choosing a fairly heavy, web-centric authentication method in such a
constrained environment?
Are there other resource-constrained use cases I've missed?
Thanks,
--Jacob
[1]
https://www.postgresql.org/message-id/MN0PR21MB31694BAC193ECE1807FD45358F4F9%40MN0PR21MB3169.namprd2...
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-09-23 22:39 Jacob Champion <jchampion@timescale.com>
parent: Jacob Champion <pchampion@vmware.com>
0 siblings, 2 replies; 424+ messages in thread
From: Jacob Champion @ 2022-09-23 22:39 UTC (permalink / raw)
To: pgsql-hackers; +Cc: hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>
On Fri, Mar 25, 2022 at 5:00 PM Jacob Champion <pchampion@vmware.com> wrote:
> v4 rebases over the latest version of the pluggable auth patchset
> (included as 0001-4). Note that there's a recent conflict as
> of d4781d887; use an older commit as the base (or wait for the other
> thread to be updated).
Here's a newly rebased v5. (They're all zipped now, which I probably
should have done a while back, sorry.)
- As before, 0001-4 are the pluggable auth set; they've now diverged
from the official version over on the other thread [1].
- I'm not sure that 0005 is still completely coherent after the
rebase, given the recent changes to jsonapi.c. But for now, the tests
are green, and that should be enough to keep the conversation going.
- 0008 will hopefully be obsoleted when the SYSTEM_USER proposal [2] lands.
Thanks,
--Jacob
[1] https://www.postgresql.org/message-id/CAJxrbyxgFzfqby%2BVRCkeAhJnwVZE50%2BZLPx0JT2TDg9LbZtkCg%40mail...
[2] https://www.postgresql.org/message-id/flat/7e692b8c-0b11-45db-1cad-3afc5b57409f@amazon.com
Attachments:
v5-0004-Add-support-for-map-and-custom-auth-options.patch.gz (application/gzip, 3.7K) download
v5-0001-Add-support-for-custom-authentication-methods.patch.gz (application/gzip, 3.9K) download
v5-0002-Add-sample-extension-to-test-custom-auth-provider.patch.gz (application/gzip, 1.6K) download
v5-0005-common-jsonapi-support-FRONTEND-clients.patch.gz (application/gzip, 6.3K) download
v5-0003-Add-tests-for-test_auth_provider-extension.patch.gz (application/gzip, 2.2K) download
v5-0006-libpq-add-OAUTHBEARER-SASL-mechanism.patch.gz (application/gzip, 10.4K) download
v5-0009-Add-pytest-suite-for-OAuth.patch.gz (application/gzip, 28.5K) download
v5-0010-contrib-oauth-switch-to-pluggable-auth-API.patch.gz (application/gzip, 5.3K) download
v5-0008-Add-a-very-simple-authn_id-extension.patch.gz (application/gzip, 1.2K) download
v5-0007-backend-add-OAUTHBEARER-SASL-mechanism.patch.gz (application/gzip, 11.4K) download
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-09-27 01:39 Andrey Chudnovsky <achudnovskij@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
1 sibling, 1 reply; 424+ messages in thread
From: Andrey Chudnovsky @ 2022-09-27 01:39 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: pgsql-hackers; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>
>>> Libpq passing toked directly from an upstream client is useful in other scenarios:
>>> 1. Enterprise clients, built with .Net / Java and using provider-specific authentication libraries, like MSAL for AAD. Those can also support more advanced provider-specific token acquisition flows.
> I can see that providing a token directly would help you work around
> limitations in libpq's "standard" OAuth flows, whether we use iddawc or
> not. And it's cheap in terms of implementation. But I have a feeling it
> would fall apart rapidly with error cases, where the server is giving
> libpq information via the OAUTHBEARER mechanism, but libpq can only
> communicate to your wrapper through human-readable error messages on stderr.
For the providing token directly, that would be primarily used for
scenarios where the same party controls both the server and the client
side wrapper.
I.e. The client knows how to get a token for a particular principal
and doesn't need any additional information other than human readable
messages.
Please clarify the scenarios where you see this falling apart.
I can provide an example in the cloud world. We (Azure) as well as
other providers offer ways to obtain OAUTH tokens for
Service-to-Service communication at IAAS / PAAS level.
on Azure "Managed Identity" feature integrated in Compute VM allows a
client to make a local http call to get a token. VM itself manages the
certificate livecycle, as well as implements the corresponding OAUTH
flow.
This capability is used by both our 1st party PAAS offerings, as well
as 3rd party services deploying on VMs or managed K8S clusters.
Here, the client doesn't need libpq assistance in obtaining the token.
> This seems like clear motivation for client-side SASL plugins (which
> were also discussed on Samay's proposal thread). That's a lot more
> expensive to implement in libpq, but if it were hypothetically
> available, wouldn't you rather your provider-specific code be able to
> speak OAUTHBEARER directly with the server?
I generally agree that pluggable auth layers in libpq could be
beneficial. However, as you pointed out in Samay's thread, that would
require a new distribution model for libpq / clients to optionally
include provider-specific logic.
My optimistic plan here would be to implement several core OAUTH flows
in libpq core which would be generic enough to support major
enterprise OAUTH providers:
1. Client Credentials flow (Client_id + Client_secret) for backend applications.
2. Authorization Code Flow with PKCE and/or Device code flow for GUI
applications.
(2.) above would require a protocol between libpq and upstream clients
to exchange several messages.
Your patch includes a way for libpq to deliver to the client a message
about the next authentication steps, so planned to build on top of
that.
A little about scenarios, we look at.
What we're trying to achieve here is an easy integration path for
multiple players in the ecosystem:
- Managed PaaS Postgres providers (both us and multi-cloud solutions)
- SaaS providers deploying postgres on IaaS/PaaS providers' clouds
- Tools - pg_admin, psql and other ones.
- BI, ETL, Federation and other scenarios where postgres is used as
the data source.
If we can offer a provider agnostic solution for Backend <=> libpq <=>
Upstreal client path, we can have all players above build support for
OAUTH credentials, managed by the cloud provider of their choice.
For us, that would mean:
- Better administrator experience with pg_admin / psql handling of the
AAD (Azure Active Directory) authentication flows.
- Path for integration solutions using Postgres to build AAD
authentication in their management experience.
- Ability to use AAD identity provider for any Postgres deployments
other than our 1st party PaaS offering.
- Ability to offer github as the identity provider for PaaS Postgres offering.
Other players in the ecosystem above would be able to get the same benefits.
Does that make sense and possible without provider specific libpq plugin?
-------------------------
On resource constrained scenarios.
> I want to dig into this much more; resource-constrained systems are near
> and dear to me. I can see two cases here:
I just referred to the ability to compile libpq without extra
dependencies to save some kilobytes.
Not sure if OAUTH is widely used in those cases. It involves overhead
anyway, and requires the device to talk to an additional party (OAUTH
provider).
Likely Cert authentication is easier.
If needed, it can get libpq with full OAUTH support and use a client
code. But I didn't think about this scenario.
On Fri, Sep 23, 2022 at 3:39 PM Jacob Champion <jchampion@timescale.com> wrote:
>
> On Fri, Mar 25, 2022 at 5:00 PM Jacob Champion <pchampion@vmware.com> wrote:
> > v4 rebases over the latest version of the pluggable auth patchset
> > (included as 0001-4). Note that there's a recent conflict as
> > of d4781d887; use an older commit as the base (or wait for the other
> > thread to be updated).
>
> Here's a newly rebased v5. (They're all zipped now, which I probably
> should have done a while back, sorry.)
>
> - As before, 0001-4 are the pluggable auth set; they've now diverged
> from the official version over on the other thread [1].
> - I'm not sure that 0005 is still completely coherent after the
> rebase, given the recent changes to jsonapi.c. But for now, the tests
> are green, and that should be enough to keep the conversation going.
> - 0008 will hopefully be obsoleted when the SYSTEM_USER proposal [2] lands.
>
> Thanks,
> --Jacob
>
> [1] https://www.postgresql.org/message-id/CAJxrbyxgFzfqby%2BVRCkeAhJnwVZE50%2BZLPx0JT2TDg9LbZtkCg%40mail...
> [2] https://www.postgresql.org/message-id/flat/7e692b8c-0b11-45db-1cad-3afc5b57409f@amazon.com
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-09-27 21:45 Jacob Champion <jchampion@timescale.com>
parent: Andrey Chudnovsky <achudnovskij@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2022-09-27 21:45 UTC (permalink / raw)
To: Andrey Chudnovsky <achudnovskij@gmail.com>; +Cc: pgsql-hackers; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>
On Mon, Sep 26, 2022 at 6:39 PM Andrey Chudnovsky
<achudnovskij@gmail.com> wrote:
> For the providing token directly, that would be primarily used for
> scenarios where the same party controls both the server and the client
> side wrapper.
> I.e. The client knows how to get a token for a particular principal
> and doesn't need any additional information other than human readable
> messages.
> Please clarify the scenarios where you see this falling apart.
The most concrete example I can see is with the OAUTHBEARER error
response. If you want to eventually handle differing scopes per role,
or different error statuses (which the proof-of-concept currently
hardcodes as `invalid_token`), then the client can't assume it knows
what the server is going to say there. I think that's true even if you
control both sides and are hardcoding the provider.
How should we communicate those pieces to a custom client when it's
passing a token directly? The easiest way I can see is for the custom
client to speak the OAUTHBEARER protocol directly (e.g. SASL plugin).
If you had to parse the libpq error message, I don't think that'd be
particularly maintainable.
> I can provide an example in the cloud world. We (Azure) as well as
> other providers offer ways to obtain OAUTH tokens for
> Service-to-Service communication at IAAS / PAAS level.
> on Azure "Managed Identity" feature integrated in Compute VM allows a
> client to make a local http call to get a token. VM itself manages the
> certificate livecycle, as well as implements the corresponding OAUTH
> flow.
> This capability is used by both our 1st party PAAS offerings, as well
> as 3rd party services deploying on VMs or managed K8S clusters.
> Here, the client doesn't need libpq assistance in obtaining the token.
Cool. To me that's the strongest argument yet for directly providing
tokens to libpq.
> My optimistic plan here would be to implement several core OAUTH flows
> in libpq core which would be generic enough to support major
> enterprise OAUTH providers:
> 1. Client Credentials flow (Client_id + Client_secret) for backend applications.
> 2. Authorization Code Flow with PKCE and/or Device code flow for GUI
> applications.
As long as it's clear to DBAs when to use which flow (because existing
documentation for that is hit-and-miss), I think it's reasonable to
eventually support multiple flows. Personally my preference would be
to start with one or two core flows, and expand outward once we're
sure that we do those perfectly. Otherwise the explosion of knobs and
buttons might be overwhelming, both to users and devs.
Related to the question of flows is the client implementation library.
I've mentioned that I don't think iddawc is production-ready. As far
as I'm aware, there is only one certified OpenID relying party written
in C, and that's... an Apache server plugin. That leaves us either
choosing an untested library, scouring the web for a "tested" library
(and hoping we're right in our assessment), or implementing our own
(which is going to tamp down enthusiasm for supporting many flows,
though that has its own set of benefits). If you know of any reliable
implementations with a C API, please let me know.
> (2.) above would require a protocol between libpq and upstream clients
> to exchange several messages.
> Your patch includes a way for libpq to deliver to the client a message
> about the next authentication steps, so planned to build on top of
> that.
Specifically it delivers that message to an end user. If you want a
generic machine client to be able to use that, then we'll need to talk
about how.
> A little about scenarios, we look at.
> What we're trying to achieve here is an easy integration path for
> multiple players in the ecosystem:
> - Managed PaaS Postgres providers (both us and multi-cloud solutions)
> - SaaS providers deploying postgres on IaaS/PaaS providers' clouds
> - Tools - pg_admin, psql and other ones.
> - BI, ETL, Federation and other scenarios where postgres is used as
> the data source.
>
> If we can offer a provider agnostic solution for Backend <=> libpq <=>
> Upstreal client path, we can have all players above build support for
> OAUTH credentials, managed by the cloud provider of their choice.
Well... I don't quite understand why we'd go to the trouble of
providing a provider-agnostic communication solution only to have
everyone write their own provider-specific client support. Unless
you're saying Microsoft would provide an officially blessed plugin for
the *server* side only, and Google would provide one of their own, and
so on.
The server side authorization is the only place where I think it makes
sense to specialize by default. libpq should remain agnostic, with the
understanding that we'll need to make hard decisions when a major
provider decides not to follow a spec.
> For us, that would mean:
> - Better administrator experience with pg_admin / psql handling of the
> AAD (Azure Active Directory) authentication flows.
> - Path for integration solutions using Postgres to build AAD
> authentication in their management experience.
> - Ability to use AAD identity provider for any Postgres deployments
> other than our 1st party PaaS offering.
> - Ability to offer github as the identity provider for PaaS Postgres offering.
GitHub is unfortunately a bit tricky, unless they've started
supporting OpenID recently?
> Other players in the ecosystem above would be able to get the same benefits.
>
> Does that make sense and possible without provider specific libpq plugin?
If the players involved implement the flows and follow the specs, yes.
That's a big "if", unfortunately. I think GitHub and Google are two
major players who are currently doing things their own way.
> I just referred to the ability to compile libpq without extra
> dependencies to save some kilobytes.
> Not sure if OAUTH is widely used in those cases. It involves overhead
> anyway, and requires the device to talk to an additional party (OAUTH
> provider).
> Likely Cert authentication is easier.
> If needed, it can get libpq with full OAUTH support and use a client
> code. But I didn't think about this scenario.
Makes sense. Thanks!
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-09-30 14:47 Andrey Chudnovsky <achudnovskij@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
0 siblings, 1 reply; 424+ messages in thread
From: Andrey Chudnovsky @ 2022-09-30 14:47 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: hlinnaka@iki.fi <hlinnaka@iki.fi>; mahendrakar s <mahendrakarforpg@gmail.com>; michael@paquier.xyz <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>
> The most concrete example I can see is with the OAUTHBEARER error
> response. If you want to eventually handle differing scopes per role,
> or different error statuses (which the proof-of-concept currently
> hardcodes as `invalid_token`), then the client can't assume it knows
> what the server is going to say there. I think that's true even if you
> control both sides and are hardcoding the provider.
Ok, I see the point. It's related to the topic of communication
between libpq and the upstream client.
> How should we communicate those pieces to a custom client when it's
> passing a token directly? The easiest way I can see is for the custom
> client to speak the OAUTHBEARER protocol directly (e.g. SASL plugin).
> If you had to parse the libpq error message, I don't think that'd be
> particularly maintainable.
I agree that parsing the message is not a sustainable way.
Could you provide more details on the SASL plugin approach you propose?
Specifically, is this basically a set of extension hooks for the client
side?
With the need for the client to be compiled with the plugins based on
the set of providers it needs.
> Well... I don't quite understand why we'd go to the trouble of
> providing a provider-agnostic communication solution only to have
> everyone write their own provider-specific client support. Unless
> you're saying Microsoft would provide an officially blessed plugin for
> the *server* side only, and Google would provide one of their own, and
> so on.
Yes, via extensions. Identity providers can open source extensions to
use their auth services outside of first party PaaS offerings.
For 3rd party Postgres PaaS or on premise deployments.
> The server side authorization is the only place where I think it makes
> sense to specialize by default. libpq should remain agnostic, with the
> understanding that we'll need to make hard decisions when a major
> provider decides not to follow a spec.
Completely agree with agnostic libpq. Though needs validation with
several major providers to know if this is possible.
> Specifically it delivers that message to an end user. If you want a
> generic machine client to be able to use that, then we'll need to talk
> about how.
Yes, that's what needs to be decided.
In both Device code and Authorization code scenarios, libpq and the
client would need to exchange a couple of pieces of metadata.
Plus, after success, the client should be able to access a refresh token
for further use.
Can we implement a generic protocol like for this between libpq and the
clients?
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-09-30 20:45 Jacob Champion <jchampion@timescale.com>
parent: Andrey Chudnovsky <achudnovskij@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2022-09-30 20:45 UTC (permalink / raw)
To: Andrey Chudnovsky <achudnovskij@gmail.com>; +Cc: hlinnaka@iki.fi <hlinnaka@iki.fi>; mahendrakar s <mahendrakarforpg@gmail.com>; michael@paquier.xyz <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>
On Fri, Sep 30, 2022 at 7:47 AM Andrey Chudnovsky
<achudnovskij@gmail.com> wrote:
> > How should we communicate those pieces to a custom client when it's
> > passing a token directly? The easiest way I can see is for the custom
> > client to speak the OAUTHBEARER protocol directly (e.g. SASL plugin).
> > If you had to parse the libpq error message, I don't think that'd be
> > particularly maintainable.
>
> I agree that parsing the message is not a sustainable way.
> Could you provide more details on the SASL plugin approach you propose?
>
> Specifically, is this basically a set of extension hooks for the client side?
> With the need for the client to be compiled with the plugins based on
> the set of providers it needs.
That's a good question. I can see two broad approaches, with maybe
some ability to combine them into a hybrid:
1. If there turns out to be serious interest in having libpq itself
handle OAuth natively (with all of the web-facing code that implies,
and all of the questions still left to answer), then we might be able
to provide a "token hook" in the same way that we currently provide a
passphrase hook for OpenSSL keys. By default, libpq would use its
internal machinery to take the provider details, navigate its builtin
flow, and return the Bearer token. If you wanted to override that
behavior as a client, you could replace the builtin flow with your
own, by registering a set of callbacks.
2. Alternatively, OAuth support could be provided via a mechanism
plugin for some third-party SASL library (GNU libgsasl, Cyrus
libsasl2). We could provide an OAuth plugin in contrib that handles
the default flow. Other providers could publish their alternative
plugins to completely replace the OAUTHBEARER mechanism handling.
Approach (2) would make for some duplicated effort since every
provider has to write code to speak the OAUTHBEARER protocol. It might
simplify provider-specific distribution, since (at least for Cyrus) I
think you could build a single plugin that supports both the client
and server side. But it would be a lot easier to unknowingly (or
knowingly) break the spec, since you'd control both the client and
server sides. There would be less incentive to interoperate.
Finally, we could potentially take pieces from both, by having an
official OAuth mechanism plugin that provides a client-side hook to
override the flow. I have no idea if the benefits would offset the
costs of a plugin-for-a-plugin style architecture. And providers would
still be free to ignore it and just provide a full mechanism plugin
anyway.
> > Well... I don't quite understand why we'd go to the trouble of
> > providing a provider-agnostic communication solution only to have
> > everyone write their own provider-specific client support. Unless
> > you're saying Microsoft would provide an officially blessed plugin for
> > the *server* side only, and Google would provide one of their own, and
> > so on.
>
> Yes, via extensions. Identity providers can open source extensions to
> use their auth services outside of first party PaaS offerings.
> For 3rd party Postgres PaaS or on premise deployments.
Sounds reasonable.
> > The server side authorization is the only place where I think it makes
> > sense to specialize by default. libpq should remain agnostic, with the
> > understanding that we'll need to make hard decisions when a major
> > provider decides not to follow a spec.
>
> Completely agree with agnostic libpq. Though needs validation with
> several major providers to know if this is possible.
Agreed.
> > Specifically it delivers that message to an end user. If you want a
> > generic machine client to be able to use that, then we'll need to talk
> > about how.
>
> Yes, that's what needs to be decided.
> In both Device code and Authorization code scenarios, libpq and the
> client would need to exchange a couple of pieces of metadata.
> Plus, after success, the client should be able to access a refresh token for further use.
>
> Can we implement a generic protocol like for this between libpq and the clients?
I think we can probably prototype a callback hook for approach (1)
pretty quickly. (2) is a lot more work and investigation, but it's
work that I'm interested in doing (when I get the time). I think there
are other very good reasons to consider a third-party SASL library,
and some good lessons to be learned, even if the community decides not
to go down that road.
Thanks,
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-10-03 18:04 Andrey Chudnovsky <achudnovskij@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
0 siblings, 1 reply; 424+ messages in thread
From: Andrey Chudnovsky @ 2022-10-03 18:04 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: hlinnaka@iki.fi <hlinnaka@iki.fi>; mahendrakar s <mahendrakarforpg@gmail.com>; michael@paquier.xyz <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>
> I think we can probably prototype a callback hook for approach (1)
> pretty quickly. (2) is a lot more work and investigation, but it's
> work that I'm interested in doing (when I get the time). I think there
> are other very good reasons to consider a third-party SASL library,
> and some good lessons to be learned, even if the community decides not
> to go down that road.
Makes sense. We will work on (1.) and do some check if there are any
blockers for a shared solution to support github and google.
On Fri, Sep 30, 2022 at 1:45 PM Jacob Champion <jchampion@timescale.com> wrote:
>
> On Fri, Sep 30, 2022 at 7:47 AM Andrey Chudnovsky
> <achudnovskij@gmail.com> wrote:
> > > How should we communicate those pieces to a custom client when it's
> > > passing a token directly? The easiest way I can see is for the custom
> > > client to speak the OAUTHBEARER protocol directly (e.g. SASL plugin).
> > > If you had to parse the libpq error message, I don't think that'd be
> > > particularly maintainable.
> >
> > I agree that parsing the message is not a sustainable way.
> > Could you provide more details on the SASL plugin approach you propose?
> >
> > Specifically, is this basically a set of extension hooks for the client side?
> > With the need for the client to be compiled with the plugins based on
> > the set of providers it needs.
>
> That's a good question. I can see two broad approaches, with maybe
> some ability to combine them into a hybrid:
>
> 1. If there turns out to be serious interest in having libpq itself
> handle OAuth natively (with all of the web-facing code that implies,
> and all of the questions still left to answer), then we might be able
> to provide a "token hook" in the same way that we currently provide a
> passphrase hook for OpenSSL keys. By default, libpq would use its
> internal machinery to take the provider details, navigate its builtin
> flow, and return the Bearer token. If you wanted to override that
> behavior as a client, you could replace the builtin flow with your
> own, by registering a set of callbacks.
>
> 2. Alternatively, OAuth support could be provided via a mechanism
> plugin for some third-party SASL library (GNU libgsasl, Cyrus
> libsasl2). We could provide an OAuth plugin in contrib that handles
> the default flow. Other providers could publish their alternative
> plugins to completely replace the OAUTHBEARER mechanism handling.
>
> Approach (2) would make for some duplicated effort since every
> provider has to write code to speak the OAUTHBEARER protocol. It might
> simplify provider-specific distribution, since (at least for Cyrus) I
> think you could build a single plugin that supports both the client
> and server side. But it would be a lot easier to unknowingly (or
> knowingly) break the spec, since you'd control both the client and
> server sides. There would be less incentive to interoperate.
>
> Finally, we could potentially take pieces from both, by having an
> official OAuth mechanism plugin that provides a client-side hook to
> override the flow. I have no idea if the benefits would offset the
> costs of a plugin-for-a-plugin style architecture. And providers would
> still be free to ignore it and just provide a full mechanism plugin
> anyway.
>
> > > Well... I don't quite understand why we'd go to the trouble of
> > > providing a provider-agnostic communication solution only to have
> > > everyone write their own provider-specific client support. Unless
> > > you're saying Microsoft would provide an officially blessed plugin for
> > > the *server* side only, and Google would provide one of their own, and
> > > so on.
> >
> > Yes, via extensions. Identity providers can open source extensions to
> > use their auth services outside of first party PaaS offerings.
> > For 3rd party Postgres PaaS or on premise deployments.
>
> Sounds reasonable.
>
> > > The server side authorization is the only place where I think it makes
> > > sense to specialize by default. libpq should remain agnostic, with the
> > > understanding that we'll need to make hard decisions when a major
> > > provider decides not to follow a spec.
> >
> > Completely agree with agnostic libpq. Though needs validation with
> > several major providers to know if this is possible.
>
> Agreed.
>
> > > Specifically it delivers that message to an end user. If you want a
> > > generic machine client to be able to use that, then we'll need to talk
> > > about how.
> >
> > Yes, that's what needs to be decided.
> > In both Device code and Authorization code scenarios, libpq and the
> > client would need to exchange a couple of pieces of metadata.
> > Plus, after success, the client should be able to access a refresh token for further use.
> >
> > Can we implement a generic protocol like for this between libpq and the clients?
>
> I think we can probably prototype a callback hook for approach (1)
> pretty quickly. (2) is a lot more work and investigation, but it's
> work that I'm interested in doing (when I get the time). I think there
> are other very good reasons to consider a third-party SASL library,
> and some good lessons to be learned, even if the community decides not
> to go down that road.
>
> Thanks,
> --Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-11-23 09:58 mahendrakar s <mahendrakarforpg@gmail.com>
parent: Andrey Chudnovsky <achudnovskij@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: mahendrakar s @ 2022-11-23 09:58 UTC (permalink / raw)
To: Andrey Chudnovsky <achudnovskij@gmail.com>; +Cc: Jacob Champion <jchampion@timescale.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>
Hi,
We validated on libpq handling OAuth natively with different flows
with different OIDC certified providers.
Flows: Device Code, Client Credentials and Refresh Token.
Providers: Microsoft, Google and Okta.
Also validated with OAuth provider Github.
We propose using OpenID Connect (OIDC) as the protocol, instead of
OAuth, as it is:
- Discovery mechanism to bridge the differences and provide metadata.
- Stricter protocol and certification process to reliably identify
which providers can be supported.
- OIDC is designed for authentication, while the main purpose of OAUTH is to
authorize applications on behalf of the user.
Github is not OIDC certified, so won’t be supported with this proposal.
However, it may be supported in the future through the ability for the
extension to provide custom discovery document content.
OpenID configuration has a well-known discovery mechanism
for the provider configuration URI which is
defined in OpenID Connect. It allows libpq to fetch
metadata about provider (i.e endpoints, supported grants, response types, etc).
In the attached patch (based on V2 patch in the thread and does not
contain Samay's changes):
- Provider can configure issuer url and scope through the options hook.)
- Server passes on an open discovery url and scope to libpq.
- Libpq handles OAuth flow based on the flow_type sent in the
connection string [1].
- Added callbacks to notify a structure to client tools if OAuth flow
requires user interaction.
- Pg backend uses hooks to validate bearer token.
Note that authentication code flow with PKCE for GUI clients is not
implemented yet.
Proposed next steps:
- Broaden discussion to reach agreement on the approach.
- Implement libpq changes without iddawc
- Prototype GUI flow with pgAdmin
Thanks,
Mahendrakar.
[1]:
connection string for refresh token flow:
./psql -U <user> -d 'dbname=postgres oauth_client_id=<client_id>
oauth_flow_type=<flowtype> oauth_refresh_token=<refresh token>'
On Mon, 3 Oct 2022 at 23:34, Andrey Chudnovsky <achudnovskij@gmail.com> wrote:
>
> > I think we can probably prototype a callback hook for approach (1)
> > pretty quickly. (2) is a lot more work and investigation, but it's
> > work that I'm interested in doing (when I get the time). I think there
> > are other very good reasons to consider a third-party SASL library,
> > and some good lessons to be learned, even if the community decides not
> > to go down that road.
>
> Makes sense. We will work on (1.) and do some check if there are any
> blockers for a shared solution to support github and google.
>
> On Fri, Sep 30, 2022 at 1:45 PM Jacob Champion <jchampion@timescale.com> wrote:
> >
> > On Fri, Sep 30, 2022 at 7:47 AM Andrey Chudnovsky
> > <achudnovskij@gmail.com> wrote:
> > > > How should we communicate those pieces to a custom client when it's
> > > > passing a token directly? The easiest way I can see is for the custom
> > > > client to speak the OAUTHBEARER protocol directly (e.g. SASL plugin).
> > > > If you had to parse the libpq error message, I don't think that'd be
> > > > particularly maintainable.
> > >
> > > I agree that parsing the message is not a sustainable way.
> > > Could you provide more details on the SASL plugin approach you propose?
> > >
> > > Specifically, is this basically a set of extension hooks for the client side?
> > > With the need for the client to be compiled with the plugins based on
> > > the set of providers it needs.
> >
> > That's a good question. I can see two broad approaches, with maybe
> > some ability to combine them into a hybrid:
> >
> > 1. If there turns out to be serious interest in having libpq itself
> > handle OAuth natively (with all of the web-facing code that implies,
> > and all of the questions still left to answer), then we might be able
> > to provide a "token hook" in the same way that we currently provide a
> > passphrase hook for OpenSSL keys. By default, libpq would use its
> > internal machinery to take the provider details, navigate its builtin
> > flow, and return the Bearer token. If you wanted to override that
> > behavior as a client, you could replace the builtin flow with your
> > own, by registering a set of callbacks.
> >
> > 2. Alternatively, OAuth support could be provided via a mechanism
> > plugin for some third-party SASL library (GNU libgsasl, Cyrus
> > libsasl2). We could provide an OAuth plugin in contrib that handles
> > the default flow. Other providers could publish their alternative
> > plugins to completely replace the OAUTHBEARER mechanism handling.
> >
> > Approach (2) would make for some duplicated effort since every
> > provider has to write code to speak the OAUTHBEARER protocol. It might
> > simplify provider-specific distribution, since (at least for Cyrus) I
> > think you could build a single plugin that supports both the client
> > and server side. But it would be a lot easier to unknowingly (or
> > knowingly) break the spec, since you'd control both the client and
> > server sides. There would be less incentive to interoperate.
> >
> > Finally, we could potentially take pieces from both, by having an
> > official OAuth mechanism plugin that provides a client-side hook to
> > override the flow. I have no idea if the benefits would offset the
> > costs of a plugin-for-a-plugin style architecture. And providers would
> > still be free to ignore it and just provide a full mechanism plugin
> > anyway.
> >
> > > > Well... I don't quite understand why we'd go to the trouble of
> > > > providing a provider-agnostic communication solution only to have
> > > > everyone write their own provider-specific client support. Unless
> > > > you're saying Microsoft would provide an officially blessed plugin for
> > > > the *server* side only, and Google would provide one of their own, and
> > > > so on.
> > >
> > > Yes, via extensions. Identity providers can open source extensions to
> > > use their auth services outside of first party PaaS offerings.
> > > For 3rd party Postgres PaaS or on premise deployments.
> >
> > Sounds reasonable.
> >
> > > > The server side authorization is the only place where I think it makes
> > > > sense to specialize by default. libpq should remain agnostic, with the
> > > > understanding that we'll need to make hard decisions when a major
> > > > provider decides not to follow a spec.
> > >
> > > Completely agree with agnostic libpq. Though needs validation with
> > > several major providers to know if this is possible.
> >
> > Agreed.
> >
> > > > Specifically it delivers that message to an end user. If you want a
> > > > generic machine client to be able to use that, then we'll need to talk
> > > > about how.
> > >
> > > Yes, that's what needs to be decided.
> > > In both Device code and Authorization code scenarios, libpq and the
> > > client would need to exchange a couple of pieces of metadata.
> > > Plus, after success, the client should be able to access a refresh token for further use.
> > >
> > > Can we implement a generic protocol like for this between libpq and the clients?
> >
> > I think we can probably prototype a callback hook for approach (1)
> > pretty quickly. (2) is a lot more work and investigation, but it's
> > work that I'm interested in doing (when I get the time). I think there
> > are other very good reasons to consider a third-party SASL library,
> > and some good lessons to be learned, even if the community decides not
> > to go down that road.
> >
> > Thanks,
> > --Jacob
Attachments:
v1-0001-oauth-flows-validation-hook-approach.patch (application/octet-stream, 31.2K) download
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
index 3a625847f3..f213a40b65 100644
--- a/src/backend/libpq/auth-oauth.c
+++ b/src/backend/libpq/auth-oauth.c
@@ -24,15 +24,23 @@
#include "libpq/hba.h"
#include "libpq/oauth.h"
#include "libpq/sasl.h"
+#include "miscadmin.h"
#include "storage/fd.h"
/* GUC */
char *oauth_validator_command;
+static OAuthProvider* oauth_provider = NULL;
+
+/*----------------------------------------------------------------
+ * OAuth Authentication
+ *----------------------------------------------------------------
+ */
+static List *oauth_providers = NIL;
static void oauth_get_mechanisms(Port *port, StringInfo buf);
static void *oauth_init(Port *port, const char *selected_mech, const char *shadow_pass);
static int oauth_exchange(void *opaq, const char *input, int inputlen,
- char **output, int *outputlen, char **logdetail);
+ char **output, int *outputlen, const char **logdetail);
/* Mechanism declaration */
const pg_be_sasl_mech pg_be_oauth_mech = {
@@ -43,7 +51,6 @@ const pg_be_sasl_mech pg_be_oauth_mech = {
PG_MAX_AUTH_TOKEN_LENGTH,
};
-
typedef enum
{
OAUTH_STATE_INIT = 0,
@@ -62,7 +69,7 @@ struct oauth_ctx
static char *sanitize_char(char c);
static char *parse_kvpairs_for_auth(char **input);
static void generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen);
-static bool validate(Port *port, const char *auth, char **logdetail);
+static bool validate(Port *port, const char *auth, const char **logdetail);
static bool run_validator_command(Port *port, const char *token);
static bool check_exit(FILE **fh, const char *command);
static bool unset_cloexec(int fd);
@@ -72,6 +79,86 @@ static bool username_ok_for_shell(const char *username);
#define AUTH_KEY "auth"
#define BEARER_SCHEME "Bearer "
+#include "utils/memutils.h"
+
+/*----------------------------------------------------------------
+ * OAuth Token Validator
+ *----------------------------------------------------------------
+ */
+
+/*
+ * RegistorOAuthProvider registers a OAuth Token Validator to be
+ * used for oauth token validation. It validates the token and adds the valiator
+ * name and it's hooks to a list of loaded token validator. The right validator's
+ * hooks can then be called based on the validator name specified in
+ * pg_hba.conf.
+ *
+ * This function should be called in _PG_init() by any extension looking to
+ * add a custom authentication method.
+ */
+void
+RegistorOAuthProvider(
+ const char *provider_name,
+ OAuthProviderCheck_hook_type OAuthProviderCheck_hook,
+ OAuthProviderError_hook_type OAuthProviderError_hook,
+ OAuthProviderOptions_hook_type OAuthProviderOptions_hook
+)
+{
+ if (!process_shared_preload_libraries_in_progress)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("RegistorOAuthProvider can only be called by a shared_preload_library")));
+ return;
+ }
+
+ MemoryContext oldcxt;
+ if (oauth_provider == NULL)
+ {
+ oldcxt = MemoryContextSwitchTo(TopMemoryContext);
+ oauth_provider = palloc(sizeof(OAuthProvider));
+ oauth_provider->name = pstrdup(provider_name);
+ oauth_provider->oauth_provider_hook = OAuthProviderCheck_hook;
+ oauth_provider->oauth_error_hook = OAuthProviderError_hook;
+ oauth_provider->oauth_options_hook = OAuthProviderOptions_hook;
+ oauth_providers = lappend(oauth_providers, oauth_provider);
+ MemoryContextSwitchTo(oldcxt);
+ }
+ else
+ {
+ if (oauth_provider && oauth_provider->name)
+ {
+ ereport(ERROR,
+ (errmsg("OAuth provider \"%s\" is already loaded.",
+ oauth_provider->name)));
+ }
+ else
+ {
+ ereport(ERROR,
+ (errmsg("OAuth provider is already loaded.")));
+ }
+ }
+}
+
+/*
+ * Returns the oauth provider (which includes it's
+ * callback functions) based on name specified.
+ */
+OAuthProvider *get_provider_by_name(const char *name)
+{
+ ListCell *lc;
+ foreach(lc, oauth_providers)
+ {
+ OAuthProvider *provider = (OAuthProvider *) lfirst(lc);
+ if (strcmp(provider->name, name) == 0)
+ {
+ return provider;
+ }
+ }
+
+ return NULL;
+}
+
static void
oauth_get_mechanisms(Port *port, StringInfo buf)
{
@@ -102,9 +189,32 @@ oauth_init(Port *port, const char *selected_mech, const char *shadow_pass)
return ctx;
}
+static void process_oauth_flow_type(pg_oauth_flow_type flow_type, struct oauth_ctx *ctx, char **output, int *outputlen)
+{
+ StringInfoData buf;
+ initStringInfo(&buf);
+
+ OAuthProviderOptions *oauth_options = oauth_provider->oauth_options_hook(flow_type);
+ ctx->scope = oauth_options->scope;
+ ctx->issuer = oauth_options->issuer_url;
+ appendStringInfo(&buf,
+ "{ "
+ "\"status\": \"invalid_token\", "
+ "\"openid-configuration\": \"%s/.well-known/openid-configuration\","
+ "\"scope\": \"%s\""
+ "}",
+ oauth_options->issuer_url,
+ oauth_options->scope);
+
+ *output = buf.data;
+ *outputlen = buf.len;
+
+ pfree(oauth_options);
+}
+
static int
oauth_exchange(void *opaq, const char *input, int inputlen,
- char **output, int *outputlen, char **logdetail)
+ char **output, int *outputlen, const char **logdetail)
{
char *p;
char cbind_flag;
@@ -247,11 +357,17 @@ oauth_exchange(void *opaq, const char *input, int inputlen,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
errmsg("malformed OAUTHBEARER message"),
errdetail("Message contains additional data after the final terminator.")));
-
- if (!validate(ctx->port, auth, logdetail))
+
+ /* if not Bearer, process flow_type*/
+ if (strncasecmp(auth, BEARER_SCHEME, strlen(BEARER_SCHEME)))
+ {
+ process_oauth_flow_type(atoi(auth), ctx, output, outputlen);
+ ctx->state = OAUTH_STATE_ERROR;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+ else if(!validate(ctx->port, auth, logdetail))
{
generate_error_response(ctx, output, outputlen);
-
ctx->state = OAUTH_STATE_ERROR;
return PG_SASL_EXCHANGE_CONTINUE;
}
@@ -415,7 +531,7 @@ generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
}
static bool
-validate(Port *port, const char *auth, char **logdetail)
+validate(Port *port, const char *auth, const char **logdetail)
{
static const char * const b64_set = "abcdefghijklmnopqrstuvwxyz"
"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
@@ -508,7 +624,7 @@ validate(Port *port, const char *auth, char **logdetail)
return true;
}
- /* Make sure the validator authenticated the user. */
+ /* Make sure the validator authenticated the user. */
if (!MyClientConnectionInfo.authn_id)
{
/* TODO: use logdetail; reduce message duplication */
@@ -518,199 +634,22 @@ validate(Port *port, const char *auth, char **logdetail)
return false;
}
- /* Finally, check the user map. */
- ret = check_usermap(port->hba->usermap, port->user_name,
- MyClientConnectionInfo.authn_id, false);
+ /* Finally, check the user map. */
+ ret = check_usermap(port->hba->usermap, port->user_name,
+ MyClientConnectionInfo.authn_id, false);
return (ret == STATUS_OK);
}
static bool
run_validator_command(Port *port, const char *token)
{
- bool success = false;
- int rc;
- int pipefd[2];
- int rfd = -1;
- int wfd = -1;
-
- StringInfoData command = { 0 };
- char *p;
- FILE *fh = NULL;
-
- ssize_t written;
- char *line = NULL;
- size_t size = 0;
- ssize_t len;
-
- Assert(oauth_validator_command);
-
- if (!oauth_validator_command[0])
- {
- ereport(COMMERROR,
- (errmsg("oauth_validator_command is not set"),
- errhint("To allow OAuth authenticated connections, set "
- "oauth_validator_command in postgresql.conf.")));
- return false;
- }
-
- /*
- * Since popen() is unidirectional, open up a pipe for the other direction.
- * Use CLOEXEC to ensure that our write end doesn't accidentally get copied
- * into child processes, which would prevent us from closing it cleanly.
- *
- * XXX this is ugly. We should just read from the child process's stdout,
- * but that's a lot more code.
- * XXX by bypassing the popen API, we open the potential of process
- * deadlock. Clearly document child process requirements (i.e. the child
- * MUST read all data off of the pipe before writing anything).
- * TODO: port to Windows using _pipe().
- */
- rc = pipe2(pipefd, O_CLOEXEC);
- if (rc < 0)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not create child pipe: %m")));
- return false;
- }
-
- rfd = pipefd[0];
- wfd = pipefd[1];
-
- /* Allow the read pipe be passed to the child. */
- if (!unset_cloexec(rfd))
- {
- /* error message was already logged */
- goto cleanup;
- }
-
- /*
- * Construct the command, substituting any recognized %-specifiers:
- *
- * %f: the file descriptor of the input pipe
- * %r: the role that the client wants to assume (port->user_name)
- * %%: a literal '%'
- */
- initStringInfo(&command);
-
- for (p = oauth_validator_command; *p; p++)
- {
- if (p[0] == '%')
- {
- switch (p[1])
- {
- case 'f':
- appendStringInfo(&command, "%d", rfd);
- p++;
- break;
- case 'r':
- /*
- * TODO: decide how this string should be escaped. The role
- * is controlled by the client, so if we don't escape it,
- * command injections are inevitable.
- *
- * This is probably an indication that the role name needs
- * to be communicated to the validator process in some other
- * way. For this proof of concept, just be incredibly strict
- * about the characters that are allowed in user names.
- */
- if (!username_ok_for_shell(port->user_name))
- goto cleanup;
-
- appendStringInfoString(&command, port->user_name);
- p++;
- break;
- case '%':
- appendStringInfoChar(&command, '%');
- p++;
- break;
- default:
- appendStringInfoChar(&command, p[0]);
- }
- }
- else
- appendStringInfoChar(&command, p[0]);
- }
-
- /* Execute the command. */
- fh = OpenPipeStream(command.data, "re");
- /* TODO: handle failures */
-
- /* We don't need the read end of the pipe anymore. */
- close(rfd);
- rfd = -1;
-
- /* Give the command the token to validate. */
- written = write(wfd, token, strlen(token));
- if (written != strlen(token))
- {
- /* TODO must loop for short writes, EINTR et al */
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not write token to child pipe: %m")));
- goto cleanup;
- }
-
- close(wfd);
- wfd = -1;
-
- /*
- * Read the command's response.
- *
- * TODO: getline() is probably too new to use, unfortunately.
- * TODO: loop over all lines
- */
- if ((len = getline(&line, &size, fh)) >= 0)
- {
- /* TODO: fail if the authn_id doesn't end with a newline */
- if (len > 0)
- line[len - 1] = '\0';
-
- set_authn_id(port, line);
- }
- else if (ferror(fh))
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not read from command \"%s\": %m",
- command.data)));
- goto cleanup;
- }
-
- /* Make sure the command exits cleanly. */
- if (!check_exit(&fh, command.data))
+ int result = oauth_provider->oauth_provider_hook(port, token);
+ if(result == STATUS_OK)
{
- /* error message already logged */
- goto cleanup;
- }
-
- /* Done. */
- success = true;
-
-cleanup:
- if (line)
- free(line);
-
- /*
- * In the successful case, the pipe fds are already closed. For the error
- * case, always close out the pipe before waiting for the command, to
- * prevent deadlock.
- */
- if (rfd >= 0)
- close(rfd);
- if (wfd >= 0)
- close(wfd);
-
- if (fh)
- {
- Assert(!success);
- check_exit(&fh, command.data);
+ set_authn_id(port, port->user_name);
+ return true;
}
-
- if (command.data)
- pfree(command.data);
-
- return success;
+ return false;
}
static bool
@@ -780,7 +719,7 @@ username_ok_for_shell(const char *username)
/* This set is borrowed from fe_utils' appendShellStringNoError(). */
static const char * const allowed = "abcdefghijklmnopqrstuvwxyz"
"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
- "0123456789-_./:";
+ "0123456789-_./@:";
size_t span;
Assert(username && username[0]); /* should have already been checked */
diff --git a/src/include/libpq/auth.h b/src/include/libpq/auth.h
index b62457d57c..7b7b6ff9aa 100644
--- a/src/include/libpq/auth.h
+++ b/src/include/libpq/auth.h
@@ -28,6 +28,41 @@ extern void set_authn_id(Port *port, const char *id);
/* Hook for plugins to get control in ClientAuthentication() */
typedef void (*ClientAuthentication_hook_type) (Port *, int);
extern PGDLLIMPORT ClientAuthentication_hook_type ClientAuthentication_hook;
+/* Declarations for oAuth authentication providers */
+typedef int (*OAuthProviderCheck_hook_type) (Port *, const char*);
+
+/* Hook for plugins to report error messages in validation_failed() */
+typedef const char * (*OAuthProviderError_hook_type) (Port *);
+
+/* Hook for plugins to validate oauth provider options */
+typedef bool (*OAuthProviderValidateOptions_hook_type)
+ (char *, char *, HbaLine *, char **);
+
+typedef struct OAuthProviderOptions
+{
+ char *issuer_url;
+ char *scope;
+} OAuthProviderOptions;
+
+/* Hook for plugins to get oauth params */
+typedef OAuthProviderOptions *(*OAuthProviderOptions_hook_type) (pg_oauth_flow_type);
+
+typedef struct OAuthProvider
+{
+ const char *name;
+ OAuthProviderCheck_hook_type oauth_provider_hook;
+ OAuthProviderError_hook_type oauth_error_hook;
+ OAuthProviderOptions_hook_type oauth_options_hook;
+} OAuthProvider;
+
+extern void RegistorOAuthProvider
+ (const char *provider_name,
+ OAuthProviderCheck_hook_type OAuthProviderCheck_hook,
+ OAuthProviderError_hook_type OAuthProviderError_hook,
+ OAuthProviderOptions_hook_type OAuthProviderParams_hook
+ );
+
+extern OAuthProvider *get_provider_by_name(const char *name);
#define PG_MAX_AUTH_TOKEN_LENGTH 65535
#endif /* AUTH_H */
diff --git a/src/include/libpq/libpq-be.h b/src/include/libpq/libpq-be.h
index 6d452ec6d9..f7bbb9dcf4 100644
--- a/src/include/libpq/libpq-be.h
+++ b/src/include/libpq/libpq-be.h
@@ -68,6 +68,17 @@ typedef enum CAC_state
CAC_TOOMANY
} CAC_state;
+/* OAuth flow types */
+typedef enum pg_oauth_flow_type
+{
+ OAUTH_DEVICE_CODE,
+ OAUTH_CLIENT_CREDENTIALS,
+ OAUTH_AUTH,
+ OAUTH_AUTH_PKCE,
+ OAUTH_REFRESH_TOKEN,
+ OAUTH_NONE
+} pg_oauth_flow_type;
+
/*
* GSSAPI specific state information
diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
index 91d2c69f16..1ba2e033c4 100644
--- a/src/interfaces/libpq/fe-auth-oauth.c
+++ b/src/interfaces/libpq/fe-auth-oauth.c
@@ -142,6 +142,43 @@ iddawc_request_error(PGconn *conn, struct _i_session *i, int err, const char *ms
appendPQExpBuffer(&conn->errorMessage, "(%s)\n", error_code);
}
+static pg_oauth_flow_type oauth_get_flow_type(const char *oauthflow)
+{
+ pg_oauth_flow_type flow_type;
+
+ if(!oauthflow)
+ {
+ return OAUTH_NONE;
+ }
+
+ /* client_secret, device_code, auth_code_pkce, refresh_token */
+ if(strcmp(oauthflow, "device_code") == 0)
+ {
+ flow_type = OAUTH_DEVICE_CODE;
+ }
+ else if(strcmp(oauthflow, "client_secret") == 0)
+ {
+ flow_type = OAUTH_CLIENT_CREDENTIALS;
+ }
+ else if(strcmp(oauthflow, "auth_code_pkce") == 0)
+ {
+ flow_type = OAUTH_AUTH_PKCE;
+ }
+ else if(strcmp(oauthflow, "refresh_token") == 0)
+ {
+ flow_type = OAUTH_REFRESH_TOKEN;
+ }
+ else if(strcmp(oauthflow, "auth_code"))
+ {
+ flow_type = OAUTH_AUTH_CODE;
+ }
+ else
+ {
+ flow_type = OAUTH_NONE;
+ }
+ return flow_type;
+}
+
static char *
get_auth_token(PGconn *conn)
{
@@ -150,29 +187,44 @@ get_auth_token(PGconn *conn)
int err;
int auth_method;
bool user_prompted = false;
- const char *verification_uri;
- const char *user_code;
- const char *access_token;
- const char *token_type;
- char *token = NULL;
-
+ char *verification_uri;
+ char *user_code;
+ char *access_token;
+ char *refresh_token;
+ char *token_type;
+ pg_oauth_flow_type flow_type;
+ char *token = NULL;
+ uint session_response_type;
+ PGOAuthMsgObj oauthMsgObj;
+
+ MemSet(&oauthMsgObj, 0x00, sizeof(PGOAuthMsgObj));
+
if (!conn->oauth_discovery_uri)
return strdup(""); /* ask the server for one */
- i_init_session(&session);
-
if (!conn->oauth_client_id)
{
/* We can't talk to a server without a client identifier. */
appendPQExpBufferStr(&conn->errorMessage,
libpq_gettext("no oauth_client_id is set for the connection"));
- goto cleanup;
+ return NULL;
}
- token_buf = createPQExpBuffer();
+ i_init_session(&session);
+ token_buf = createPQExpBuffer();
if (!token_buf)
goto cleanup;
+
+ if(conn->oauth_bearer_token)
+ {
+ appendPQExpBufferStr(token_buf, "Bearer ");
+ appendPQExpBufferStr(token_buf, conn->oauth_bearer_token);
+ if (PQExpBufferBroken(token_buf))
+ goto cleanup;
+ token = strdup(token_buf->data);
+ goto cleanup;
+ }
err = i_set_str_parameter(&session, I_OPT_OPENID_CONFIG_ENDPOINT, conn->oauth_discovery_uri);
if (err)
@@ -181,6 +233,8 @@ get_auth_token(PGconn *conn)
goto cleanup;
}
+ flow_type = oauth_get_flow_type(conn->oauth_flow_type);
+
err = i_get_openid_config(&session);
if (err)
{
@@ -201,18 +255,64 @@ get_auth_token(PGconn *conn)
libpq_gettext("issuer does not support device authorization"));
goto cleanup;
}
+ auth_method = I_TOKEN_AUTH_METHOD_NONE;
+
+ /* for refresh token flow, do not run auth request*/
+ if(flow_type == OAUTH_REFRESH_TOKEN && conn->oauth_refresh_token)
+ {
+ err = i_set_parameter_list(&session,
+ I_OPT_CLIENT_ID, conn->oauth_client_id,
+ I_OPT_REFRESH_TOKEN, conn->oauth_refresh_token,
+ I_OPT_RESPONSE_TYPE, I_RESPONSE_TYPE_REFRESH_TOKEN,
+ I_OPT_TOKEN_METHOD, auth_method,
+ I_OPT_CLIENT_SECRET, conn->oauth_client_secret,
+ I_OPT_SCOPE, conn->oauth_scope,
+ I_OPT_NONE
+ );
+
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set refresh token flow parameters");
+ goto cleanup;
+ }
- err = i_set_response_type(&session, I_RESPONSE_TYPE_DEVICE_CODE);
+ err = i_run_token_request(&session);
+ if (err)
+ {
+ iddawc_request_error(conn, &session, err,
+ "failed to obtain token authorization with refresh token flow");
+ goto cleanup;
+ }
+
+ access_token = i_get_str_parameter(&session, I_OPT_ACCESS_TOKEN);
+ token_type = i_get_str_parameter(&session, I_OPT_TOKEN_TYPE);
+
+ if (!access_token || !token_type || strcasecmp(token_type, "Bearer"))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer did not provide a bearer token"));
+ goto cleanup;
+ }
+
+ appendPQExpBufferStr(token_buf, "Bearer ");
+ appendPQExpBufferStr(token_buf, access_token);
+
+ if (PQExpBufferBroken(token_buf))
+ goto cleanup;
+
+ token = strdup(token_buf->data);
+ return token;
+ }
+
+ //default device flow
+ session_response_type = I_RESPONSE_TYPE_DEVICE_CODE;
+ err = i_set_response_type(&session, session_response_type);
if (err)
{
iddawc_error(conn, err, "failed to set device code response type");
goto cleanup;
}
- auth_method = I_TOKEN_AUTH_METHOD_NONE;
- if (conn->oauth_client_secret && *conn->oauth_client_secret)
- auth_method = I_TOKEN_AUTH_METHOD_SECRET_BASIC;
-
err = i_set_parameter_list(&session,
I_OPT_CLIENT_ID, conn->oauth_client_id,
I_OPT_CLIENT_SECRET, conn->oauth_client_secret,
@@ -225,7 +325,7 @@ get_auth_token(PGconn *conn)
iddawc_error(conn, err, "failed to set client identifier");
goto cleanup;
}
-
+
err = i_run_device_auth_request(&session);
if (err)
{
@@ -278,14 +378,15 @@ get_auth_token(PGconn *conn)
if (!user_prompted)
{
+ oauthMsgObj.verification_uri = verification_uri;
+ oauthMsgObj.user_code = user_code;
+ conn->oauthNoticeHooks.noticeRecArg = (void*) &oauthMsgObj;
+
/*
* Now that we know the token endpoint isn't broken, give the user
* the login instructions.
- */
- pqInternalNotice(&conn->noticeHooks,
- "Visit %s and enter the code: %s",
- verification_uri, user_code);
-
+ */
+ pqInternalOAuthNotice(&conn->oauthNoticeHooks, "");
user_prompted = true;
}
@@ -300,7 +401,7 @@ get_auth_token(PGconn *conn)
* A slow_down error requires us to permanently increase our retry
* interval by five seconds. RFC 8628, Sec. 3.5.
*/
- if (!strcmp(error_code, "slow_down"))
+ //if (!strcmp(error_code, "slow_down"))
{
interval += 5;
i_set_int_parameter(&session, I_OPT_DEVICE_AUTH_INTERVAL, interval);
@@ -323,6 +424,14 @@ get_auth_token(PGconn *conn)
access_token = i_get_str_parameter(&session, I_OPT_ACCESS_TOKEN);
token_type = i_get_str_parameter(&session, I_OPT_TOKEN_TYPE);
+ refresh_token = i_get_str_parameter(&session, I_OPT_REFRESH_TOKEN);
+
+ if(refresh_token)
+ {
+ MemSet(&oauthMsgObj, 0x00, sizeof(PGOAuthMsgObj));
+ oauthMsgObj.refresh_token = refresh_token;
+ pqInternalOAuthNotice(&conn->oauthNoticeHooks, "");
+ }
if (!access_token || !token_type || strcasecmp(token_type, "Bearer"))
{
@@ -358,6 +467,8 @@ client_initial_response(PGconn *conn)
PQExpBuffer discovery_buf = NULL;
char *token = NULL;
char *response = NULL;
+ pg_oauth_flow_type flow_type;
+ char oauth_flow_str[3];
token_buf = createPQExpBuffer();
if (!token_buf)
@@ -385,8 +496,26 @@ client_initial_response(PGconn *conn)
token = get_auth_token(conn);
if (!token)
goto cleanup;
-
+
+ if(strcmp(token, "") == 0)
+ {
+ flow_type = oauth_get_flow_type(conn->oauth_flow_type);
+ if(flow_type == OAUTH_NONE)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("value passed in oauth_flow_type is not valid."\
+ "supported flows: client_secret, device_code, auth_code_pkce, refresh_token\n"));
+ goto cleanup;
+ }
+ else
+ {
+ sprintf(oauth_flow_str, "%d", flow_type);
+ token = strdup(oauth_flow_str);
+ }
+ }
appendPQExpBuffer(token_buf, resp_format, token);
+// elog(INFO, "fe-flowtype: %s", token);
+
if (PQExpBufferBroken(token_buf))
goto cleanup;
@@ -406,6 +535,9 @@ cleanup:
#define ERROR_STATUS_FIELD "status"
#define ERROR_SCOPE_FIELD "scope"
#define ERROR_OPENID_CONFIGURATION_FIELD "openid-configuration"
+#define ERROR_ISSUER_URL_FIELD "issuer"
+#define ERROR_AUTH_ENDPOINT_FIELD "authorization_endpoint"
+#define ERROR_TOKEN_ENDPOINT_FIELD "token_endpoint"
struct json_ctx
{
@@ -420,6 +552,9 @@ struct json_ctx
char *status;
char *scope;
char *discovery_uri;
+ char *issuer_url;
+ char *auth_endpoint;
+ char *token_endpoint;
};
#define oauth_json_has_error(ctx) \
@@ -491,6 +626,21 @@ oauth_json_object_field_start(void *state, char *name, bool isnull)
ctx->target_field_name = ERROR_OPENID_CONFIGURATION_FIELD;
ctx->target_field = &ctx->discovery_uri;
}
+ else if(!strcmp(name, ERROR_ISSUER_URL_FIELD))
+ {
+ ctx->target_field_name = ERROR_ISSUER_URL_FIELD;
+ ctx->target_field = &ctx->issuer_url;
+ }
+ else if(!strcmp(name, ERROR_AUTH_ENDPOINT_FIELD))
+ {
+ ctx->target_field_name = ERROR_AUTH_ENDPOINT_FIELD;
+ ctx->target_field = &ctx->auth_endpoint;
+ }
+ else if(!strcmp(name, ERROR_TOKEN_ENDPOINT_FIELD))
+ {
+ ctx->target_field_name = ERROR_TOKEN_ENDPOINT_FIELD;
+ ctx->target_field = &ctx->token_endpoint;
+ }
}
free(name);
@@ -627,6 +777,15 @@ handle_oauth_sasl_error(PGconn *conn, char *msg, int msglen)
conn->oauth_scope = ctx.scope;
}
+
+ if(ctx.issuer_url)
+ {
+ if(conn->oauth_issuer)
+ free(conn->oauth_issuer);
+
+ conn->oauth_issuer = ctx.issuer_url;
+ }
+
/* TODO: missing error scope should clear any existing connection scope */
if (!ctx.status)
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 64f27fee18..e6e8dc48e2 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -358,6 +358,18 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"OAuth-Scope", "", 15,
offsetof(struct pg_conn, oauth_scope)},
+ {"oauth_bearer_token", NULL, NULL, NULL,
+ "OAuth-Bearer", "", 20,
+ offsetof(struct pg_conn, oauth_bearer_token)},
+
+ {"oauth_flow_type", NULL, NULL, NULL,
+ "OAuth-Flow-Type", "", 20,
+ offsetof(struct pg_conn, oauth_flow_type)},
+
+ {"oauth_refresh_token", NULL, NULL, NULL,
+ "OAuth-Refresh-Token", "", 40,
+ offsetof(struct pg_conn, oauth_refresh_token)},
+
/* Terminating entry --- MUST BE LAST */
{NULL, NULL, NULL, NULL,
NULL, NULL, 0}
@@ -427,6 +439,7 @@ static PQconninfoOption *conninfo_find(PQconninfoOption *connOptions,
const char *keyword);
static void defaultNoticeReceiver(void *arg, const PGresult *res);
static void defaultNoticeProcessor(void *arg, const char *message);
+static void OAuthMsgObjReceiver(void *arg, const PGresult *res);
static int parseServiceInfo(PQconninfoOption *options,
PQExpBuffer errorMessage);
static int parseServiceFile(const char *serviceFile,
@@ -3926,6 +3939,7 @@ makeEmptyPGconn(void)
/* install default notice hooks */
conn->noticeHooks.noticeRec = defaultNoticeReceiver;
conn->noticeHooks.noticeProc = defaultNoticeProcessor;
+ conn->oauthNoticeHooks.noticeRec = OAuthMsgObjReceiver;
conn->status = CONNECTION_BAD;
conn->asyncStatus = PGASYNC_IDLE;
@@ -4073,6 +4087,12 @@ freePGconn(PGconn *conn)
free(conn->oauth_client_secret);
if (conn->oauth_scope)
free(conn->oauth_scope);
+ if(conn->oauth_bearer_token)
+ free(conn->oauth_bearer_token);
+ if(conn->oauth_flow_type)
+ free(conn->oauth_flow_type);
+ if(conn->oauth_refresh_token)
+ free(conn->oauth_refresh_token);
termPQExpBuffer(&conn->errorMessage);
termPQExpBuffer(&conn->workBuffer);
@@ -6991,6 +7011,32 @@ defaultNoticeProcessor(void *arg, const char *message)
fprintf(stderr, "%s", message);
}
+static void
+OAuthMsgObjReceiver(void *arg, const PGresult *res)
+{
+ PGOAuthMsgObj *oauthMsg = (PGOAuthMsgObj *) arg;
+
+ if(oauthMsg->message)
+ {
+ fprintf(stderr, "%s\n", oauthMsg->message);
+ }
+
+ if(oauthMsg->verification_uri)
+ {
+ fprintf(stderr, "Visit: %s\n", oauthMsg->verification_uri);
+ }
+
+ if(oauthMsg->user_code)
+ {
+ fprintf(stderr, "Enter: %s\n", oauthMsg->user_code);
+ }
+
+ if(oauthMsg->refresh_token)
+ {
+ fprintf(stderr, "Refresh Token: %s\n", oauthMsg->refresh_token);
+ }
+}
+
/*
* returns a pointer to the next token or NULL if the current
* token doesn't match
diff --git a/src/interfaces/libpq/fe-exec.c b/src/interfaces/libpq/fe-exec.c
index da229d632a..4789c1a1fe 100644
--- a/src/interfaces/libpq/fe-exec.c
+++ b/src/interfaces/libpq/fe-exec.c
@@ -976,6 +976,58 @@ pqInternalNotice(const PGNoticeHooks *hooks, const char *fmt,...)
PQclear(res);
}
+/*
+ * pqInternalOAuthNotice - it is similar to pqInternalNotice
+ * except that OAuthNoticeHooks are invoked.
+ */
+void
+pqInternalOAuthNotice(const PGOAuthNoticeHooks *hooks, const char *fmt,...)
+{
+ char msgBuf[1024];
+ va_list args;
+ PGresult *res;
+
+ if (hooks->noticeRec == NULL)
+ return; /* nobody home to receive notice? */
+
+ /* Format the message */
+ va_start(args, fmt);
+ vsnprintf(msgBuf, sizeof(msgBuf), libpq_gettext(fmt), args);
+ va_end(args);
+ msgBuf[sizeof(msgBuf) - 1] = '\0'; /* make real sure it's terminated */
+
+ /* Make a PGresult to pass to the notice receiver */
+ res = PQmakeEmptyPGresult(NULL, PGRES_NONFATAL_ERROR);
+ if (!res)
+ return;
+ res->oauthNoticeHooks = *hooks;
+ res->oauthNoticeHooks.noticeRecArg = hooks->noticeRecArg;
+
+ /*
+ * Set up fields of notice.
+ */
+ pqSaveMessageField(res, PG_DIAG_MESSAGE_PRIMARY, msgBuf);
+ pqSaveMessageField(res, PG_DIAG_SEVERITY, libpq_gettext("NOTICE"));
+ pqSaveMessageField(res, PG_DIAG_SEVERITY_NONLOCALIZED, "NOTICE");
+ /* XXX should provide a SQLSTATE too? */
+
+ /*
+ * Result text is always just the primary message + newline. If we can't
+ * allocate it, substitute "out of memory", as in pqSetResultError.
+ */
+ res->errMsg = (char *) pqResultAlloc(res, strlen(msgBuf) + 2, false);
+ if (res->errMsg)
+ sprintf(res->errMsg, "%s\n", msgBuf);
+ else
+ res->errMsg = libpq_gettext("out of memory\n");
+
+ /*
+ * Pass to receiver, then free it.
+ */
+ res->oauthNoticeHooks.noticeRec(res->oauthNoticeHooks.noticeRecArg, res);
+ PQclear(res);
+}
+
/*
* pqAddTuple
* add a row pointer to the PGresult structure, growing it if necessary
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index b7df3224c0..ee5b2e2b59 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -197,6 +197,9 @@ typedef struct pgNotify
typedef void (*PQnoticeReceiver) (void *arg, const PGresult *res);
typedef void (*PQnoticeProcessor) (void *arg, const char *message);
+/* Function types for notice-handling callbacks */
+typedef void (*PQOAuthNoticeReceiver) (void *arg, const PGresult *res);
+
/* Print options for PQprint() */
typedef char pqbool;
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index ae76ae0e8f..3155d81e00 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -157,6 +157,24 @@ typedef struct
void *noticeProcArg;
} PGNoticeHooks;
+typedef struct
+{
+ char *verification_uri; /* URI the user should go to with the user_code in order to sign in */
+ char *user_code; /* used to identify the session on a secondary device */
+ char *refresh_token;
+ char *message; /* string with instructions for the user. */
+ char *response_error; /*JSON error response (400 Bad Request) */
+ uint expires_in; /* number of seconds before the device_code expire */
+ uint interval; /* number of seconds the client should wait between polling requests */
+} PGOAuthMsgObj;
+
+/* Fields needed for oauth callback handling */
+typedef struct
+{
+ PQOAuthNoticeReceiver noticeRec; /* OAuth notice message receiver */
+ void *noticeRecArg;
+} PGOAuthNoticeHooks;
+
typedef struct PGEvent
{
PGEventProc proc; /* the function to call on events */
@@ -186,6 +204,7 @@ struct pg_result
* on the PGresult don't have to reference the PGconn.
*/
PGNoticeHooks noticeHooks;
+ PGOAuthNoticeHooks oauthNoticeHooks;
PGEvent *events;
int nEvents;
int client_encoding; /* encoding id */
@@ -343,6 +362,17 @@ typedef struct pg_conn_host
* found in password file. */
} pg_conn_host;
+typedef enum pg_oauth_flow_type
+{
+ OAUTH_DEVICE_CODE,
+ OAUTH_CLIENT_CREDENTIALS,
+ OAUTH_AUTH,
+ OAUTH_AUTH_PKCE,
+ OAUTH_REFRESH_TOKEN,
+ OAUTH_AUTH_CODE,
+ OAUTH_NONE
+} pg_oauth_flow_type;
+
/*
* PGconn stores all the state data associated with a single connection
* to a backend.
@@ -403,6 +433,9 @@ struct pg_conn
char *oauth_client_id; /* client identifier */
char *oauth_client_secret; /* client secret */
char *oauth_scope; /* access token scope */
+ char *oauth_bearer_token; /* oauth token */
+ char *oauth_flow_type; /* oauth flow type */
+ char *oauth_refresh_token; /* refresh token */
bool oauth_want_retry; /* should we retry on failure? */
/* Optional file to write trace info to */
@@ -412,6 +445,9 @@ struct pg_conn
/* Callback procedures for notice message processing */
PGNoticeHooks noticeHooks;
+ /* Callback procedures for notifying messages during oauth flows*/
+ PGOAuthNoticeHooks oauthNoticeHooks;
+
/* Event procs registered via PQregisterEventProc */
PGEvent *events; /* expandable array of event data */
int nEvents; /* number of active events */
@@ -677,6 +713,7 @@ extern void pqClearAsyncResult(PGconn *conn);
extern void pqSaveErrorResult(PGconn *conn);
extern PGresult *pqPrepareAsyncResult(PGconn *conn);
extern void pqInternalNotice(const PGNoticeHooks *hooks, const char *fmt,...) pg_attribute_printf(2, 3);
+extern void pqInternalOAuthNotice(const PGOAuthNoticeHooks *hooks, const char *fmt,...);
extern void pqSaveMessageField(PGresult *res, char code,
const char *value);
extern void pqSaveParameterStatus(PGconn *conn, const char *name,
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-11-23 20:05 Jacob Champion <jchampion@timescale.com>
parent: mahendrakar s <mahendrakarforpg@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2022-11-23 20:05 UTC (permalink / raw)
To: mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; +Cc: hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>
On 11/23/22 01:58, mahendrakar s wrote:
> We validated on libpq handling OAuth natively with different flows
> with different OIDC certified providers.
>
> Flows: Device Code, Client Credentials and Refresh Token.
> Providers: Microsoft, Google and Okta.
Great, thank you!
> Also validated with OAuth provider Github.
(How did you get discovery working? I tried this and had to give up
eventually.)
> We propose using OpenID Connect (OIDC) as the protocol, instead of
> OAuth, as it is:
> - Discovery mechanism to bridge the differences and provide metadata.
> - Stricter protocol and certification process to reliably identify
> which providers can be supported.
> - OIDC is designed for authentication, while the main purpose of OAUTH is to
> authorize applications on behalf of the user.
How does this differ from the previous proposal? The OAUTHBEARER SASL
mechanism already relies on OIDC for discovery. (I think that decision
is confusing from an architectural and naming standpoint, but I don't
think they really had an alternative...)
> Github is not OIDC certified, so won’t be supported with this proposal.
> However, it may be supported in the future through the ability for the
> extension to provide custom discovery document content.
Right.
> OpenID configuration has a well-known discovery mechanism
> for the provider configuration URI which is
> defined in OpenID Connect. It allows libpq to fetch
> metadata about provider (i.e endpoints, supported grants, response types, etc).
Sure, but this is already how the original PoC works. The test suite
implements an OIDC provider, for instance. Is there something different
to this that I'm missing?
> In the attached patch (based on V2 patch in the thread and does not
> contain Samay's changes):
> - Provider can configure issuer url and scope through the options hook.)
> - Server passes on an open discovery url and scope to libpq.
> - Libpq handles OAuth flow based on the flow_type sent in the
> connection string [1].
> - Added callbacks to notify a structure to client tools if OAuth flow
> requires user interaction.
> - Pg backend uses hooks to validate bearer token.
Thank you for the sample!
> Note that authentication code flow with PKCE for GUI clients is not
> implemented yet.
>
> Proposed next steps:
> - Broaden discussion to reach agreement on the approach.
High-level thoughts on this particular patch (I assume you're not
looking for low-level implementation comments yet):
0) The original hook proposal upthread, I thought, was about allowing
libpq's flow implementation to be switched out by the application. I
don't see that approach taken here. It's fine if that turned out to be a
bad idea, of course, but this patch doesn't seem to match what we were
talking about.
1) I'm really concerned about the sudden explosion of flows. We went
from one flow (Device Authorization) to six. It's going to be hard
enough to validate that *one* flow is useful and can be securely
deployed by end users; I don't think we're going to be able to maintain
six, especially in combination with my statement that iddawc is not an
appropriate dependency for us.
I'd much rather give applications the ability to use their own OAuth
code, and then maintain within libpq only the flows that are broadly
useful. This ties back to (0) above.
2) Breaking the refresh token into its own pseudoflow is, I think,
passing the buck onto the user for something that's incredibly security
sensitive. The refresh token is powerful; I don't really want it to be
printed anywhere, let alone copy-pasted by the user. Imagine the
phishing opportunities.
If we want to support refresh tokens, I believe we should be developing
a plan to cache and secure them within the client. They should be used
as an accelerator for other flows, not as their own flow.
3) I don't like the departure from the OAUTHBEARER mechanism that's
presented here. For one, since I can't see a sample plugin that makes
use of the "flow type" magic numbers that have been added, I don't
really understand why the extension to the mechanism is necessary.
For two, if we think OAUTHBEARER is insufficient, the people who wrote
it would probably like to hear about it. Claiming support for a spec,
and then implementing an extension without review from the people who
wrote the spec, is not something I'm personally interested in doing.
4) The test suite is still broken, so it's difficult to see these things
in practice for review purposes.
> - Implement libpq changes without iddawc
This in particular will be much easier with a functioning test suite,
and with a smaller number of flows.
> - Prototype GUI flow with pgAdmin
Cool!
Thanks,
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-11-24 03:45 Andrey Chudnovsky <achudnovskij@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
0 siblings, 2 replies; 424+ messages in thread
From: Andrey Chudnovsky @ 2022-11-24 03:45 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: mahendrakar s <mahendrakarforpg@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>
> How does this differ from the previous proposal? The OAUTHBEARER SASL
> mechanism already relies on OIDC for discovery. (I think that decision
> is confusing from an architectural and naming standpoint, but I don't
> think they really had an alternative...)
Mostly terminology questions here. OAUTHBEARER SASL appears to be the
spec about using OAUTH2 tokens for Authentication.
While any OAUTH2 can generally work, we propose to specifically
highlight that only OIDC providers can be supported, as we need the
discovery document.
And we won't be able to support Github under that requirement.
Since the original patch used that too - no change on that, just
confirmation that we need OIDC compliance.
> 0) The original hook proposal upthread, I thought, was about allowing
> libpq's flow implementation to be switched out by the application. I
> don't see that approach taken here. It's fine if that turned out to be a
> bad idea, of course, but this patch doesn't seem to match what we were
> talking about.
We still plan to allow the client to pass the token. Which is a
generic way to implement its own OAUTH flows.
> 1) I'm really concerned about the sudden explosion of flows. We went
> from one flow (Device Authorization) to six. It's going to be hard
> enough to validate that *one* flow is useful and can be securely
> deployed by end users; I don't think we're going to be able to maintain
> six, especially in combination with my statement that iddawc is not an
> appropriate dependency for us.
> I'd much rather give applications the ability to use their own OAuth
> code, and then maintain within libpq only the flows that are broadly
> useful. This ties back to (0) above.
We consider the following set of flows to be minimum required:
- Client Credentials - For Service to Service scenarios.
- Authorization Code with PKCE - For rich clients,including pgAdmin.
- Device code - for psql (and possibly other non-GUI clients).
- Refresh code (separate discussion)
Which is pretty much the list described here:
https://oauth.net/2/grant-types/ and in OAUTH2 specs.
Client Credentials is very simple, so does Refresh Code.
If you prefer to pick one of the richer flows, Authorization code for
GUI scenarios is probably much more widely used.
Plus it's easier to implement too, as interaction goes through a
series of callbacks. No polling required.
> 2) Breaking the refresh token into its own pseudoflow is, I think,
> passing the buck onto the user for something that's incredibly security
> sensitive. The refresh token is powerful; I don't really want it to be
> printed anywhere, let alone copy-pasted by the user. Imagine the
> phishing opportunities.
> If we want to support refresh tokens, I believe we should be developing
> a plan to cache and secure them within the client. They should be used
> as an accelerator for other flows, not as their own flow.
It's considered a separate "grant_type" in the specs / APIs.
https://openid.net/specs/openid-connect-core-1_0.html#RefreshTokens
For the clients, it would be storing the token and using it to authenticate.
On the question of sensitivity, secure credentials stores are
different for each platform, with a lot of cloud offerings for this.
pgAdmin, for example, has its own way to secure credentials to avoid
asking users for passwords every time the app is opened.
I believe we should delegate the refresh token management to the clients.
>3) I don't like the departure from the OAUTHBEARER mechanism that's
> presented here. For one, since I can't see a sample plugin that makes
> use of the "flow type" magic numbers that have been added, I don't
> really understand why the extension to the mechanism is necessary.
I don't think it's much of a departure, but rather a separation of
responsibilities between libpq and upstream clients.
As libpq can be used in different apps, the client would need
different types of flows/grants.
I.e. those need to be provided to libpq at connection initialization
or some other point.
We will change to "grant_type" though and use string to be closer to the spec.
What do you think is the best way for the client to signal which OAUTH
flow should be used?
On Wed, Nov 23, 2022 at 12:05 PM Jacob Champion <jchampion@timescale.com> wrote:
>
> On 11/23/22 01:58, mahendrakar s wrote:
> > We validated on libpq handling OAuth natively with different flows
> > with different OIDC certified providers.
> >
> > Flows: Device Code, Client Credentials and Refresh Token.
> > Providers: Microsoft, Google and Okta.
>
> Great, thank you!
>
> > Also validated with OAuth provider Github.
>
> (How did you get discovery working? I tried this and had to give up
> eventually.)
>
> > We propose using OpenID Connect (OIDC) as the protocol, instead of
> > OAuth, as it is:
> > - Discovery mechanism to bridge the differences and provide metadata.
> > - Stricter protocol and certification process to reliably identify
> > which providers can be supported.
> > - OIDC is designed for authentication, while the main purpose of OAUTH is to
> > authorize applications on behalf of the user.
>
> How does this differ from the previous proposal? The OAUTHBEARER SASL
> mechanism already relies on OIDC for discovery. (I think that decision
> is confusing from an architectural and naming standpoint, but I don't
> think they really had an alternative...)
>
> > Github is not OIDC certified, so won’t be supported with this proposal.
> > However, it may be supported in the future through the ability for the
> > extension to provide custom discovery document content.
>
> Right.
>
> > OpenID configuration has a well-known discovery mechanism
> > for the provider configuration URI which is
> > defined in OpenID Connect. It allows libpq to fetch
> > metadata about provider (i.e endpoints, supported grants, response types, etc).
>
> Sure, but this is already how the original PoC works. The test suite
> implements an OIDC provider, for instance. Is there something different
> to this that I'm missing?
>
> > In the attached patch (based on V2 patch in the thread and does not
> > contain Samay's changes):
> > - Provider can configure issuer url and scope through the options hook.)
> > - Server passes on an open discovery url and scope to libpq.
> > - Libpq handles OAuth flow based on the flow_type sent in the
> > connection string [1].
> > - Added callbacks to notify a structure to client tools if OAuth flow
> > requires user interaction.
> > - Pg backend uses hooks to validate bearer token.
>
> Thank you for the sample!
>
> > Note that authentication code flow with PKCE for GUI clients is not
> > implemented yet.
> >
> > Proposed next steps:
> > - Broaden discussion to reach agreement on the approach.
>
> High-level thoughts on this particular patch (I assume you're not
> looking for low-level implementation comments yet):
>
> 0) The original hook proposal upthread, I thought, was about allowing
> libpq's flow implementation to be switched out by the application. I
> don't see that approach taken here. It's fine if that turned out to be a
> bad idea, of course, but this patch doesn't seem to match what we were
> talking about.
>
> 1) I'm really concerned about the sudden explosion of flows. We went
> from one flow (Device Authorization) to six. It's going to be hard
> enough to validate that *one* flow is useful and can be securely
> deployed by end users; I don't think we're going to be able to maintain
> six, especially in combination with my statement that iddawc is not an
> appropriate dependency for us.
>
> I'd much rather give applications the ability to use their own OAuth
> code, and then maintain within libpq only the flows that are broadly
> useful. This ties back to (0) above.
>
> 2) Breaking the refresh token into its own pseudoflow is, I think,
> passing the buck onto the user for something that's incredibly security
> sensitive. The refresh token is powerful; I don't really want it to be
> printed anywhere, let alone copy-pasted by the user. Imagine the
> phishing opportunities.
>
> If we want to support refresh tokens, I believe we should be developing
> a plan to cache and secure them within the client. They should be used
> as an accelerator for other flows, not as their own flow.
>
> 3) I don't like the departure from the OAUTHBEARER mechanism that's
> presented here. For one, since I can't see a sample plugin that makes
> use of the "flow type" magic numbers that have been added, I don't
> really understand why the extension to the mechanism is necessary.
>
> For two, if we think OAUTHBEARER is insufficient, the people who wrote
> it would probably like to hear about it. Claiming support for a spec,
> and then implementing an extension without review from the people who
> wrote the spec, is not something I'm personally interested in doing.
>
> 4) The test suite is still broken, so it's difficult to see these things
> in practice for review purposes.
>
> > - Implement libpq changes without iddawc
>
> This in particular will be much easier with a functioning test suite,
> and with a smaller number of flows.
>
> > - Prototype GUI flow with pgAdmin
>
> Cool!
>
> Thanks,
> --Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-11-24 08:20 mahendrakar s <mahendrakarforpg@gmail.com>
parent: Andrey Chudnovsky <achudnovskij@gmail.com>
1 sibling, 1 reply; 424+ messages in thread
From: mahendrakar s @ 2022-11-24 08:20 UTC (permalink / raw)
To: Andrey Chudnovsky <achudnovskij@gmail.com>; +Cc: Jacob Champion <jchampion@timescale.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>
Hi Jacob,
I had validated Github by skipping the discovery mechanism and letting
the provider extension pass on the endpoints. This is just for
validation purposes.
If it needs to be supported, then need a way to send the discovery
document from extension.
Thanks,
Mahendrakar.
On Thu, 24 Nov 2022 at 09:16, Andrey Chudnovsky <achudnovskij@gmail.com> wrote:
>
> > How does this differ from the previous proposal? The OAUTHBEARER SASL
> > mechanism already relies on OIDC for discovery. (I think that decision
> > is confusing from an architectural and naming standpoint, but I don't
> > think they really had an alternative...)
> Mostly terminology questions here. OAUTHBEARER SASL appears to be the
> spec about using OAUTH2 tokens for Authentication.
> While any OAUTH2 can generally work, we propose to specifically
> highlight that only OIDC providers can be supported, as we need the
> discovery document.
> And we won't be able to support Github under that requirement.
> Since the original patch used that too - no change on that, just
> confirmation that we need OIDC compliance.
>
> > 0) The original hook proposal upthread, I thought, was about allowing
> > libpq's flow implementation to be switched out by the application. I
> > don't see that approach taken here. It's fine if that turned out to be a
> > bad idea, of course, but this patch doesn't seem to match what we were
> > talking about.
> We still plan to allow the client to pass the token. Which is a
> generic way to implement its own OAUTH flows.
>
> > 1) I'm really concerned about the sudden explosion of flows. We went
> > from one flow (Device Authorization) to six. It's going to be hard
> > enough to validate that *one* flow is useful and can be securely
> > deployed by end users; I don't think we're going to be able to maintain
> > six, especially in combination with my statement that iddawc is not an
> > appropriate dependency for us.
>
> > I'd much rather give applications the ability to use their own OAuth
> > code, and then maintain within libpq only the flows that are broadly
> > useful. This ties back to (0) above.
> We consider the following set of flows to be minimum required:
> - Client Credentials - For Service to Service scenarios.
> - Authorization Code with PKCE - For rich clients,including pgAdmin.
> - Device code - for psql (and possibly other non-GUI clients).
> - Refresh code (separate discussion)
> Which is pretty much the list described here:
> https://oauth.net/2/grant-types/ and in OAUTH2 specs.
> Client Credentials is very simple, so does Refresh Code.
> If you prefer to pick one of the richer flows, Authorization code for
> GUI scenarios is probably much more widely used.
> Plus it's easier to implement too, as interaction goes through a
> series of callbacks. No polling required.
>
> > 2) Breaking the refresh token into its own pseudoflow is, I think,
> > passing the buck onto the user for something that's incredibly security
> > sensitive. The refresh token is powerful; I don't really want it to be
> > printed anywhere, let alone copy-pasted by the user. Imagine the
> > phishing opportunities.
>
> > If we want to support refresh tokens, I believe we should be developing
> > a plan to cache and secure them within the client. They should be used
> > as an accelerator for other flows, not as their own flow.
> It's considered a separate "grant_type" in the specs / APIs.
> https://openid.net/specs/openid-connect-core-1_0.html#RefreshTokens
>
> For the clients, it would be storing the token and using it to authenticate.
> On the question of sensitivity, secure credentials stores are
> different for each platform, with a lot of cloud offerings for this.
> pgAdmin, for example, has its own way to secure credentials to avoid
> asking users for passwords every time the app is opened.
> I believe we should delegate the refresh token management to the clients.
>
> >3) I don't like the departure from the OAUTHBEARER mechanism that's
> > presented here. For one, since I can't see a sample plugin that makes
> > use of the "flow type" magic numbers that have been added, I don't
> > really understand why the extension to the mechanism is necessary.
> I don't think it's much of a departure, but rather a separation of
> responsibilities between libpq and upstream clients.
> As libpq can be used in different apps, the client would need
> different types of flows/grants.
> I.e. those need to be provided to libpq at connection initialization
> or some other point.
> We will change to "grant_type" though and use string to be closer to the spec.
> What do you think is the best way for the client to signal which OAUTH
> flow should be used?
>
> On Wed, Nov 23, 2022 at 12:05 PM Jacob Champion <jchampion@timescale.com> wrote:
> >
> > On 11/23/22 01:58, mahendrakar s wrote:
> > > We validated on libpq handling OAuth natively with different flows
> > > with different OIDC certified providers.
> > >
> > > Flows: Device Code, Client Credentials and Refresh Token.
> > > Providers: Microsoft, Google and Okta.
> >
> > Great, thank you!
> >
> > > Also validated with OAuth provider Github.
> >
> > (How did you get discovery working? I tried this and had to give up
> > eventually.)
> >
> > > We propose using OpenID Connect (OIDC) as the protocol, instead of
> > > OAuth, as it is:
> > > - Discovery mechanism to bridge the differences and provide metadata.
> > > - Stricter protocol and certification process to reliably identify
> > > which providers can be supported.
> > > - OIDC is designed for authentication, while the main purpose of OAUTH is to
> > > authorize applications on behalf of the user.
> >
> > How does this differ from the previous proposal? The OAUTHBEARER SASL
> > mechanism already relies on OIDC for discovery. (I think that decision
> > is confusing from an architectural and naming standpoint, but I don't
> > think they really had an alternative...)
> >
> > > Github is not OIDC certified, so won’t be supported with this proposal.
> > > However, it may be supported in the future through the ability for the
> > > extension to provide custom discovery document content.
> >
> > Right.
> >
> > > OpenID configuration has a well-known discovery mechanism
> > > for the provider configuration URI which is
> > > defined in OpenID Connect. It allows libpq to fetch
> > > metadata about provider (i.e endpoints, supported grants, response types, etc).
> >
> > Sure, but this is already how the original PoC works. The test suite
> > implements an OIDC provider, for instance. Is there something different
> > to this that I'm missing?
> >
> > > In the attached patch (based on V2 patch in the thread and does not
> > > contain Samay's changes):
> > > - Provider can configure issuer url and scope through the options hook.)
> > > - Server passes on an open discovery url and scope to libpq.
> > > - Libpq handles OAuth flow based on the flow_type sent in the
> > > connection string [1].
> > > - Added callbacks to notify a structure to client tools if OAuth flow
> > > requires user interaction.
> > > - Pg backend uses hooks to validate bearer token.
> >
> > Thank you for the sample!
> >
> > > Note that authentication code flow with PKCE for GUI clients is not
> > > implemented yet.
> > >
> > > Proposed next steps:
> > > - Broaden discussion to reach agreement on the approach.
> >
> > High-level thoughts on this particular patch (I assume you're not
> > looking for low-level implementation comments yet):
> >
> > 0) The original hook proposal upthread, I thought, was about allowing
> > libpq's flow implementation to be switched out by the application. I
> > don't see that approach taken here. It's fine if that turned out to be a
> > bad idea, of course, but this patch doesn't seem to match what we were
> > talking about.
> >
> > 1) I'm really concerned about the sudden explosion of flows. We went
> > from one flow (Device Authorization) to six. It's going to be hard
> > enough to validate that *one* flow is useful and can be securely
> > deployed by end users; I don't think we're going to be able to maintain
> > six, especially in combination with my statement that iddawc is not an
> > appropriate dependency for us.
> >
> > I'd much rather give applications the ability to use their own OAuth
> > code, and then maintain within libpq only the flows that are broadly
> > useful. This ties back to (0) above.
> >
> > 2) Breaking the refresh token into its own pseudoflow is, I think,
> > passing the buck onto the user for something that's incredibly security
> > sensitive. The refresh token is powerful; I don't really want it to be
> > printed anywhere, let alone copy-pasted by the user. Imagine the
> > phishing opportunities.
> >
> > If we want to support refresh tokens, I believe we should be developing
> > a plan to cache and secure them within the client. They should be used
> > as an accelerator for other flows, not as their own flow.
> >
> > 3) I don't like the departure from the OAUTHBEARER mechanism that's
> > presented here. For one, since I can't see a sample plugin that makes
> > use of the "flow type" magic numbers that have been added, I don't
> > really understand why the extension to the mechanism is necessary.
> >
> > For two, if we think OAUTHBEARER is insufficient, the people who wrote
> > it would probably like to hear about it. Claiming support for a spec,
> > and then implementing an extension without review from the people who
> > wrote the spec, is not something I'm personally interested in doing.
> >
> > 4) The test suite is still broken, so it's difficult to see these things
> > in practice for review purposes.
> >
> > > - Implement libpq changes without iddawc
> >
> > This in particular will be much easier with a functioning test suite,
> > and with a smaller number of flows.
> >
> > > - Prototype GUI flow with pgAdmin
> >
> > Cool!
> >
> > Thanks,
> > --Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-11-29 21:12 Jacob Champion <jchampion@timescale.com>
parent: Andrey Chudnovsky <achudnovskij@gmail.com>
1 sibling, 0 replies; 424+ messages in thread
From: Jacob Champion @ 2022-11-29 21:12 UTC (permalink / raw)
To: Andrey Chudnovsky <achudnovskij@gmail.com>; +Cc: mahendrakar s <mahendrakarforpg@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>
On 11/23/22 19:45, Andrey Chudnovsky wrote:
> Mostly terminology questions here. OAUTHBEARER SASL appears to be the
> spec about using OAUTH2 tokens for Authentication.
> While any OAUTH2 can generally work, we propose to specifically
> highlight that only OIDC providers can be supported, as we need the
> discovery document.
*If* you're using in-band discovery, yes. But I thought your use case
was explicitly tailored to out-of-band token retrieval:
> The client knows how to get a token for a particular principal
> and doesn't need any additional information other than human readable
> messages.
In that case, isn't OAuth sufficient? There's definitely a need to
document the distinction, but I don't think we have to require OIDC as
long as the client application makes up for the missing information.
(OAUTHBEARER makes the openid-configuration error member optional,
presumably for this reason.)
>> 0) The original hook proposal upthread, I thought, was about allowing
>> libpq's flow implementation to be switched out by the application. I
>> don't see that approach taken here. It's fine if that turned out to be a
>> bad idea, of course, but this patch doesn't seem to match what we were
>> talking about.
> We still plan to allow the client to pass the token. Which is a
> generic way to implement its own OAUTH flows.
Okay. But why push down the implementation into the server?
To illustrate what I mean, here's the architecture of my proposed patchset:
+-------+ +----------+
| | -------------- Empty Token ------------> | |
| libpq | <----- Error Result (w/ Discovery ) ---- | |
| | | |
| +--------+ +--------------+ | |
| | iddawc | <--- [ Flow ] ----> | Issuer/ | | Postgres |
| | | <-- Access Token -- | Authz Server | | |
| +--------+ +--------------+ | +-----------+
| | | | |
| | -------------- Access Token -----------> | > | Validator |
| | <---- Authorization Success/Failure ---- | < | |
| | | +-----------+
+-------+ +----------+
In this implementation, there's only one black box: the validator, which
is responsible for taking an access token from an untrusted client,
verifying that it was issued correctly for the Postgres service, and
either 1) determining whether the bearer is authorized to access the
database, or 2) determining the authenticated ID of the bearer so that
the HBA can decide whether they're authorized. (Or both.)
This approach is limited by the flows that we explicitly enable within
libpq and its OAuth implementation library. You mentioned that you
wanted to support other flows, including clients with out-of-band
knowledge, and I suggested:
> If you wanted to override [iddawc's]
> behavior as a client, you could replace the builtin flow with your
> own, by registering a set of callbacks.
In other words, the hooks would replace iddawc in the above diagram.
In my mind, something like this:
+-------+ +----------+
+------+ | ----------- Empty Token ------------> | Postgres |
| | < | <---------- Error Result ------------ | |
| Hook | | | +-----------+
| | | | | |
+------+ > | ------------ Access Token ----------> | > | Validator |
| | <--- Authorization Success/Failure -- | < | |
| libpq | | +-----------+
+-------+ +----------+
Now there's a second black box -- the client hook -- which takes an
OAUTHBEARER error result (which may or may not have OIDC discovery
information) and returns the access token. How it does this is
unspecified -- it'll probably use some OAuth 2.0 flow, but maybe not.
Maybe it sends the user to a web browser; maybe it uses some of the
magic provider-specific libraries you mentioned upthread. It might have
a refresh token cached so it doesn't have to involve the user at all.
Crucially, though, the two black boxes remain independent of each other.
They have well-defined inputs and outputs (the client hook could be
roughly described as "implement get_auth_token()"). Their correctness
can be independently verified against published OAuth specs and/or
provider documentation. And the client application still makes a single
call to PQconnect*().
Compare this to the architecture proposed by your patch:
Client App
+----------------------+
| +-------+ +----------+
| | libpq | | Postgres |
| PQconnect > | | | +-------+
| +------+ | ------- Flow Type (!) -------> | > | |
| +- < | Hook | < | <------- Error Result -------- | < | |
| [ get +------+ | | | |
| token ] | | | | |
| | | | | | Hooks |
| v | | | | |
| PQconnect > | ----> | ------ Access Token ---------> | > | |
| | | <--- Authz Success/Failure --- | < | |
| +-------+ | +-------+
+----------------------+ +----------+
Rather than decouple things, I think this proposal drives a spike
through the client app, libpq, and the server. Please correct me if I've
misunderstood pieces of the patch, but the following is my view of it:
What used to be a validator hook on the server side now actively
participates in the client-side flow for some reason. (I still don't
understand what the server is supposed to do with that knowledge.
Changing your authz requirements based on the flow the client wants to
use seems like a good way to introduce bugs.)
The client-side hook is now coupled to the application logic: you have
to know to expect an error from the first PQconnect*() call, then check
whatever magic your hook has done for you to be able to set up the
second call to PQconnect*() with the correctly scoped bearer token. So
if you want to switch between the internal libpq OAuth implementation
and your own hook, you have to rewrite your app logic.
On top of all that, the "flow type code" being sent is a custom
extension to OAUTHBEARER that appears to be incompatible with the RFC's
discovery exchange (which is done by sending an empty auth token during
the first round trip).
> We consider the following set of flows to be minimum required:
> - Client Credentials - For Service to Service scenarios.
Okay, that's simple enough that I think it could probably be maintained
inside libpq with minimal cost. At the same time, is it complicated
enough that you need libpq to do it for you?
Maybe once we get the hooks ironed out, it'll be more obvious what the
tradeoff is...
> If you prefer to pick one of the richer flows, Authorization code for
> GUI scenarios is probably much more widely used.
> Plus it's easier to implement too, as interaction goes through a
> series of callbacks. No polling required.
I don't think flows requiring the invocation of web browsers and custom
URL handlers are a clear fit for libpq. For a first draft, at least, I
think that use case should be pushed upward into the client application
via a custom hook.
>> If we want to support refresh tokens, I believe we should be developing
>> a plan to cache and secure them within the client. They should be used
>> as an accelerator for other flows, not as their own flow.
> It's considered a separate "grant_type" in the specs / APIs.
> https://openid.net/specs/openid-connect-core-1_0.html#RefreshTokens
Yes, but that doesn't mean we have to expose it to users via a
connection option. You don't get a refresh token out of the blue; you
get it by going through some other flow, and then you use it in
preference to going through that flow again later.
> For the clients, it would be storing the token and using it to authenticate.
> On the question of sensitivity, secure credentials stores are
> different for each platform, with a lot of cloud offerings for this.
> pgAdmin, for example, has its own way to secure credentials to avoid
> asking users for passwords every time the app is opened.
> I believe we should delegate the refresh token management to the clients.
Delegating to client apps would be fine (and implicitly handled by a
token hook, because the client app would receive the refresh token
directly rather than going through libpq). Delegating to end users, not
so much. Printing a refresh token to stderr as proposed here is, I
think, making things unnecessarily difficult (and/or dangerous) for users.
>> 3) I don't like the departure from the OAUTHBEARER mechanism that's
>> presented here. For one, since I can't see a sample plugin that makes
>> use of the "flow type" magic numbers that have been added, I don't
>> really understand why the extension to the mechanism is necessary.
> I don't think it's much of a departure, but rather a separation of
> responsibilities between libpq and upstream clients.
Given the proposed architectures above, 1) I think this is further
coupling the components, not separating them; and 2) I can't agree that
an incompatible discovery mechanism is "not much of a departure". If
OAUTHBEARER's functionality isn't good enough for some reason, let's
talk about why.
> As libpq can be used in different apps, the client would need
> different types of flows/grants.
> I.e. those need to be provided to libpq at connection initialization
> or some other point.
Why do libpq (or the server!) need to know those things at all, if
they're not going to implement the flow?
> We will change to "grant_type" though and use string to be closer to the spec.
> What do you think is the best way for the client to signal which OAUTH
> flow should be used?
libpq should not need to know the grant type in use if the client is
bypassing its internal implementation entirely.
Thanks,
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-11-29 21:19 Jacob Champion <jchampion@timescale.com>
parent: mahendrakar s <mahendrakarforpg@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2022-11-29 21:19 UTC (permalink / raw)
To: mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; +Cc: hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>
On 11/24/22 00:20, mahendrakar s wrote:
> I had validated Github by skipping the discovery mechanism and letting
> the provider extension pass on the endpoints. This is just for
> validation purposes.
> If it needs to be supported, then need a way to send the discovery
> document from extension.
Yeah. I had originally bounced around the idea that we could send a
data:// URL, but I think that opens up problems.
You're supposed to be able to link the issuer URI with the URI you got
the configuration from, and if they're different, you bail out. If a
server makes up its own OpenID configuration, we'd have to bypass that
safety check, and decide what the risks and mitigations are... Not sure
it's worth it.
Especially if you could just lobby GitHub to, say, provide an OpenID
config. (Maybe there's a security-related reason they don't.)
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-12-06 00:15 Andrey Chudnovsky <achudnovskij@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
0 siblings, 1 reply; 424+ messages in thread
From: Andrey Chudnovsky @ 2022-12-06 00:15 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: mahendrakar s <mahendrakarforpg@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>
Jacob,
Thanks for your feedback.
I think we can focus on the roles and responsibilities of the components first.
Details of the patch can be elaborated. Like "flow type code" is a
mistake on our side, and we will use the term "grant_type" which is
defined by OIDC spec. As well as details of usage of refresh_token.
> Rather than decouple things, I think this proposal drives a spike
> through the client app, libpq, and the server. Please correct me if I've
> misunderstood pieces of the patch, but the following is my view of it:
> What used to be a validator hook on the server side now actively
> participates in the client-side flow for some reason. (I still don't
> understand what the server is supposed to do with that knowledge.
> Changing your authz requirements based on the flow the client wants to
> use seems like a good way to introduce bugs.)
> The client-side hook is now coupled to the application logic: you have
> to know to expect an error from the first PQconnect*() call, then check
> whatever magic your hook has done for you to be able to set up the
> second call to PQconnect*() with the correctly scoped bearer token. So
> if you want to switch between the internal libpq OAuth implementation
> and your own hook, you have to rewrite your app logic.
Basically Yes. We propose an increase of the server side hook responsibility.
From just validating the token, to also return the provider root URL
and required audience. And possibly provide more metadata in the
future.
Which is in our opinion aligned with SASL protocol, where the server
side is responsible for telling the client auth requirements based on
the requested role in the startup packet.
Our understanding is that in the original patch that information came
purely from hba, and we propose extension being able to control that
metadata.
As we see extension as being owned by the identity provider, compared
to HBA which is owned by the server administrator or cloud provider.
This change of the roles is based on the vision of 4 independent actor
types in the ecosystem:
1. Identity Providers (Okta, Google, Microsoft, other OIDC providers).
- Publish open source extensions for PostgreSQL.
- Don't have to own the server deployments, and must ensure their
extensions can work in any environment. This is where we think
additional hook responsibility helps.
2. Server Owners / PAAS providers (On premise admins, Cloud providers,
multi-cloud PAAS providers).
- Install extensions and configure HBA to allow clients to
authenticate with the identity providers of their choice.
3. Client Application Developers (Data Wis, integration tools,
PgAdmin, monitoring tools, e.t.c.)
- Independent from specific Identity providers or server providers.
Write one code for all identity providers.
- Rely on application deployment owners to configure which OIDC
provider to use across client and server setups.
4. Application Deployment Owners (End customers setting up applications)
- The only actor actually aware of which identity provider to use.
Configures the stack based on the Identity and PostgreSQL deployments
they have.
The critical piece of the vision is (3.) above is applications
agnostic of the identity providers. Those applications rely on
properly configured servers and rich driver logic (libpq,
com.postgresql, npgsql) to allow their application to popup auth
windows or do service-to-service authentication with any provider. In
our view that would significantly democratize the deployment of OAUTH
authentication in the community.
In order to allow this separation, we propose:
1. HBA + Extension is the single source of truth of Provider root URL
+ Required Audience for each role. If some backfill for missing OIDC
discovery is needed, the provider-specific extension would be
providing it.
2. Client Application knows which grant_type to use in which scenario.
But can be coded without knowledge of a specific provider. So can't
provide discovery details.
3. Driver (libpq, others) - coordinate the authentication flow based
on client grant_type and identity provider metadata to allow client
applications to use any flow with any provider in a unified way.
Yes, this would require a little more complicated flow between
components than in your original patch. And yes, more complexity comes
with more opportunity to make bugs.
However, I see PG Server and Libpq as the places which can have more
complexity. For the purpose of making work for the community
participants easier and simplify adoption.
Does this make sense to you?
On Tue, Nov 29, 2022 at 1:20 PM Jacob Champion <jchampion@timescale.com> wrote:
>
> On 11/24/22 00:20, mahendrakar s wrote:
> > I had validated Github by skipping the discovery mechanism and letting
> > the provider extension pass on the endpoints. This is just for
> > validation purposes.
> > If it needs to be supported, then need a way to send the discovery
> > document from extension.
>
> Yeah. I had originally bounced around the idea that we could send a
> data:// URL, but I think that opens up problems.
>
> You're supposed to be able to link the issuer URI with the URI you got
> the configuration from, and if they're different, you bail out. If a
> server makes up its own OpenID configuration, we'd have to bypass that
> safety check, and decide what the risks and mitigations are... Not sure
> it's worth it.
>
> Especially if you could just lobby GitHub to, say, provide an OpenID
> config. (Maybe there's a security-related reason they don't.)
>
> --Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-12-07 19:06 Jacob Champion <jchampion@timescale.com>
parent: Andrey Chudnovsky <achudnovskij@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2022-12-07 19:06 UTC (permalink / raw)
To: Andrey Chudnovsky <achudnovskij@gmail.com>; +Cc: mahendrakar s <mahendrakarforpg@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>
On Mon, Dec 5, 2022 at 4:15 PM Andrey Chudnovsky <achudnovskij@gmail.com> wrote:
> I think we can focus on the roles and responsibilities of the components first.
> Details of the patch can be elaborated. Like "flow type code" is a
> mistake on our side, and we will use the term "grant_type" which is
> defined by OIDC spec. As well as details of usage of refresh_token.
(For the record, whether we call it "flow type" or "grant type"
doesn't address my concern.)
> Basically Yes. We propose an increase of the server side hook responsibility.
> From just validating the token, to also return the provider root URL
> and required audience. And possibly provide more metadata in the
> future.
I think it's okay to have the extension and HBA collaborate to provide
discovery information. Your proposal goes further than that, though,
and makes the server aware of the chosen client flow. That appears to
be an architectural violation: why does an OAuth resource server need
to know the client flow at all?
> Which is in our opinion aligned with SASL protocol, where the server
> side is responsible for telling the client auth requirements based on
> the requested role in the startup packet.
You've proposed an alternative SASL mechanism. There's nothing wrong
with that, per se, but I think it should be clear why we've chosen
something nonstandard.
> Our understanding is that in the original patch that information came
> purely from hba, and we propose extension being able to control that
> metadata.
> As we see extension as being owned by the identity provider, compared
> to HBA which is owned by the server administrator or cloud provider.
That seems reasonable, considering how tightly coupled the Issuer and
the token validation process are.
> 2. Server Owners / PAAS providers (On premise admins, Cloud providers,
> multi-cloud PAAS providers).
> - Install extensions and configure HBA to allow clients to
> authenticate with the identity providers of their choice.
(For a future conversation: they need to set up authorization, too,
with custom scopes or some other magic. It's not enough to check who
the token belongs to; even if Postgres is just using the verified
email from OpenID as an authenticator, you have to also know that the
user authorized the token -- and therefore the client -- to access
Postgres on their behalf.)
> 3. Client Application Developers (Data Wis, integration tools,
> PgAdmin, monitoring tools, e.t.c.)
> - Independent from specific Identity providers or server providers.
> Write one code for all identity providers.
Ideally, yes, but that only works if all identity providers implement
the same flows in compatible ways. We're already seeing instances
where that's not the case and we'll necessarily have to deal with that
up front.
> - Rely on application deployment owners to configure which OIDC
> provider to use across client and server setups.
> 4. Application Deployment Owners (End customers setting up applications)
> - The only actor actually aware of which identity provider to use.
> Configures the stack based on the Identity and PostgreSQL deployments
> they have.
(I have doubts that the roles will be as decoupled in practice as you
have described them, but I'd rather defer that for now.)
> The critical piece of the vision is (3.) above is applications
> agnostic of the identity providers. Those applications rely on
> properly configured servers and rich driver logic (libpq,
> com.postgresql, npgsql) to allow their application to popup auth
> windows or do service-to-service authentication with any provider. In
> our view that would significantly democratize the deployment of OAUTH
> authentication in the community.
That seems to be restating the goal of OAuth and OIDC. Can you explain
how the incompatible change allows you to accomplish this better than
standard implementations?
> In order to allow this separation, we propose:
> 1. HBA + Extension is the single source of truth of Provider root URL
> + Required Audience for each role. If some backfill for missing OIDC
> discovery is needed, the provider-specific extension would be
> providing it.
> 2. Client Application knows which grant_type to use in which scenario.
> But can be coded without knowledge of a specific provider. So can't
> provide discovery details.
> 3. Driver (libpq, others) - coordinate the authentication flow based
> on client grant_type and identity provider metadata to allow client
> applications to use any flow with any provider in a unified way.
>
> Yes, this would require a little more complicated flow between
> components than in your original patch.
Why? I claim that standard OAUTHBEARER can handle all of that. What
does your proposed architecture (the third diagram) enable that my
proposed hook (the second diagram) doesn't?
> And yes, more complexity comes
> with more opportunity to make bugs.
> However, I see PG Server and Libpq as the places which can have more
> complexity. For the purpose of making work for the community
> participants easier and simplify adoption.
>
> Does this make sense to you?
Some of it, but it hasn't really addressed the questions from my last mail.
Thanks,
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-12-07 23:22 Andrey Chudnovsky <achudnovskij@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
0 siblings, 1 reply; 424+ messages in thread
From: Andrey Chudnovsky @ 2022-12-07 23:22 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: mahendrakar s <mahendrakarforpg@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>
> I think it's okay to have the extension and HBA collaborate to provide
> discovery information. Your proposal goes further than that, though,
> and makes the server aware of the chosen client flow. That appears to
> be an architectural violation: why does an OAuth resource server need
> to know the client flow at all?
Ok. It may have left there from intermediate iterations. We did
consider making extension drive the flow for specific grant_type, but
decided against that idea. For the same reason you point to.
Is it correct that your main concern about use of grant_type was that
it's propagated to the server? Then yes, we will remove sending it to
the server.
> Ideally, yes, but that only works if all identity providers implement
> the same flows in compatible ways. We're already seeing instances
> where that's not the case and we'll necessarily have to deal with that
> up front.
Yes, based on our analysis OIDC spec is detailed enough, that
providers implementing that one, can be supported with generic code in
libpq / client.
Github specifically won't fit there though. Microsoft Azure AD,
Google, Okta (including Auth0) will.
Theoretically discovery documents can be returned from the extension
(server-side) which is provider specific. Though we didn't plan to
prioritize that.
> That seems to be restating the goal of OAuth and OIDC. Can you explain
> how the incompatible change allows you to accomplish this better than
> standard implementations?
Do you refer to passing grant_type to the server? Which we will get
rid of in the next iteration. Or other incompatible changes as well?
> Why? I claim that standard OAUTHBEARER can handle all of that. What
> does your proposed architecture (the third diagram) enable that my
> proposed hook (the second diagram) doesn't?
The hook proposed on the 2nd diagram effectively delegates all Oauth
flows implementations to the client.
We propose libpq takes care of pulling OpenId discovery and coordination.
Which is effectively Diagram 1 + more flows + server hook providing
root url/audience.
Created the diagrams with all components for 3 flows:
1. Authorization code grant (Clients with Browser access):
+----------------------+ +----------+
| +-------+ |
|
| PQconnect | | |
|
| [auth_code] | | |
+-----------+
| -> | | -------------- Empty Token ------------> | >
| |
| | libpq | <----- Error(w\ Root URL + Audience ) -- | <
| Pre-Auth |
| | | |
| Hook |
| | | |
+-----------+
| | | +--------------+ | |
| | | -------[GET]---------> | OIDC | | Postgres |
| +------+ | <--Provider Metadata-- | Discovery | | |
| +- < | Hook | < | +--------------+ |
|
| | +------+ | |
|
| v | | |
|
| [get auth | | |
|
| code] | | |
|
|<user action>| | |
|
| | | | |
|
| + | | |
|
| PQconnect > | +--------+ +--------------+ |
|
| | | iddawc | <-- [ Auth code ]-> | Issuer/ | | |
| | | | <-- Access Token -- | Authz Server | | |
| | +--------+ +--------------+ | |
| | | |
+-----------+
| | | -------------- Access Token -----------> | >
| Validator |
| | | <---- Authorization Success/Failure ---- | <
| Hook |
| +------+ | |
+-----------+
| +-< | Hook | | |
|
| v +------+ | |
|
|[store +-------+ |
|
| refresh_token] +----------+
+----------------------+
2. Device code grant
+----------------------+ +----------+
| +-------+ |
|
| PQconnect | | |
|
| [auth_code] | | |
+-----------+
| -> | | -------------- Empty Token ------------> | >
| |
| | libpq | <----- Error(w\ Root URL + Audience ) -- | <
| Pre-Auth |
| | | |
| Hook |
| | | |
+-----------+
| | | +--------------+ | |
| | | -------[GET]---------> | OIDC | | Postgres |
| +------+ | <--Provider Metadata-- | Discovery | | |
| +- < | Hook | < | +--------------+ |
|
| | +------+ | |
|
| v | | |
|
| [device | +---------+ +--------------+ |
|
| code] | | iddawc | | Issuer/ | |
|
|<user action>| | | --[ Device code ]-> | Authz Server | |
|
| | |<polling>| --[ Device code ]-> | | |
|
| | | | --[ Device code ]-> | | |
|
| | | | | | | |
| | | | <-- Access Token -- | | | |
| | +---------+ +--------------+ | |
| | | |
+-----------+
| | | -------------- Access Token -----------> | >
| Validator |
| | | <---- Authorization Success/Failure ---- | <
| Hook |
| +------+ | |
+-----------+
| +-< | Hook | | |
|
| v +------+ | |
|
|[store +-------+ |
|
| refresh_token] +----------+
+----------------------+
3. Non-interactive flows (Client Secret / Refresh_Token)
+----------------------+ +----------+
| +-------+ |
|
| PQconnect | | |
|
| [grant_type]| | | |
| -> | | |
+-----------+
| | | -------------- Empty Token ------------> | >
| |
| | libpq | <----- Error(w\ Root URL + Audience ) -- | <
| Pre-Auth |
| | | |
| Hook |
| | | |
+-----------+
| | | +--------------+ | |
| | | -------[GET]---------> | OIDC | | Postgres |
| | | <--Provider Metadata-- | Discovery | | |
| | | +--------------+ |
|
| | | |
|
| | +--------+ +--------------+ |
|
| | | iddawc | <-- [ Secret ]----> | Issuer/ | | |
| | | | <-- Access Token -- | Authz Server | | |
| | +--------+ +--------------+ | |
| | | |
+-----------+
| | | -------------- Access Token -----------> | >
| Validator |
| | | <---- Authorization Success/Failure ---- | <
| Hook |
| | | |
+-----------+
| +-------+ +----------+
+----------------------+
I think what was the most confusing in our latest patch is that
flow_type was passed to the server.
We are not proposing this going forward.
> (For a future conversation: they need to set up authorization, too,
> with custom scopes or some other magic. It's not enough to check who
> the token belongs to; even if Postgres is just using the verified
> email from OpenID as an authenticator, you have to also know that the
> user authorized the token -- and therefore the client -- to access
> Postgres on their behalf.)
My understanding is that metadata in the tokens is provider specific,
so server side hook would be the right place to handle that.
Plus I can envision for some providers it can make sense to make a
remote call to pull some information.
The way we implement Azure AD auth today in PAAS PostgreSQL offering:
- Server administrator uses special extension functions to create
Azure AD enabled PostgreSQL roles.
- PostgreSQL extension maps Roles to unique identity Ids (UID) in the Directory.
- Connection flow: If the token is valid and Role => UID mapping
matches, we authenticate as the Role.
- Then its native PostgreSQL role based access control takes care of privileges.
This is the same for both User- and System-to-system authorization.
Though I assume different providers may treat user- and system-
identities differently. So their extension would handle that.
Thanks!
Andrey.
On Wed, Dec 7, 2022 at 11:06 AM Jacob Champion <jchampion@timescale.com> wrote:
>
> On Mon, Dec 5, 2022 at 4:15 PM Andrey Chudnovsky <achudnovskij@gmail.com> wrote:
> > I think we can focus on the roles and responsibilities of the components first.
> > Details of the patch can be elaborated. Like "flow type code" is a
> > mistake on our side, and we will use the term "grant_type" which is
> > defined by OIDC spec. As well as details of usage of refresh_token.
>
> (For the record, whether we call it "flow type" or "grant type"
> doesn't address my concern.)
>
> > Basically Yes. We propose an increase of the server side hook responsibility.
> > From just validating the token, to also return the provider root URL
> > and required audience. And possibly provide more metadata in the
> > future.
>
> I think it's okay to have the extension and HBA collaborate to provide
> discovery information. Your proposal goes further than that, though,
> and makes the server aware of the chosen client flow. That appears to
> be an architectural violation: why does an OAuth resource server need
> to know the client flow at all?
>
> > Which is in our opinion aligned with SASL protocol, where the server
> > side is responsible for telling the client auth requirements based on
> > the requested role in the startup packet.
>
> You've proposed an alternative SASL mechanism. There's nothing wrong
> with that, per se, but I think it should be clear why we've chosen
> something nonstandard.
>
> > Our understanding is that in the original patch that information came
> > purely from hba, and we propose extension being able to control that
> > metadata.
> > As we see extension as being owned by the identity provider, compared
> > to HBA which is owned by the server administrator or cloud provider.
>
> That seems reasonable, considering how tightly coupled the Issuer and
> the token validation process are.
>
> > 2. Server Owners / PAAS providers (On premise admins, Cloud providers,
> > multi-cloud PAAS providers).
> > - Install extensions and configure HBA to allow clients to
> > authenticate with the identity providers of their choice.
>
> (For a future conversation: they need to set up authorization, too,
> with custom scopes or some other magic. It's not enough to check who
> the token belongs to; even if Postgres is just using the verified
> email from OpenID as an authenticator, you have to also know that the
> user authorized the token -- and therefore the client -- to access
> Postgres on their behalf.)
>
> > 3. Client Application Developers (Data Wis, integration tools,
> > PgAdmin, monitoring tools, e.t.c.)
> > - Independent from specific Identity providers or server providers.
> > Write one code for all identity providers.
>
> Ideally, yes, but that only works if all identity providers implement
> the same flows in compatible ways. We're already seeing instances
> where that's not the case and we'll necessarily have to deal with that
> up front.
>
> > - Rely on application deployment owners to configure which OIDC
> > provider to use across client and server setups.
> > 4. Application Deployment Owners (End customers setting up applications)
> > - The only actor actually aware of which identity provider to use.
> > Configures the stack based on the Identity and PostgreSQL deployments
> > they have.
>
> (I have doubts that the roles will be as decoupled in practice as you
> have described them, but I'd rather defer that for now.)
>
> > The critical piece of the vision is (3.) above is applications
> > agnostic of the identity providers. Those applications rely on
> > properly configured servers and rich driver logic (libpq,
> > com.postgresql, npgsql) to allow their application to popup auth
> > windows or do service-to-service authentication with any provider. In
> > our view that would significantly democratize the deployment of OAUTH
> > authentication in the community.
>
> That seems to be restating the goal of OAuth and OIDC. Can you explain
> how the incompatible change allows you to accomplish this better than
> standard implementations?
>
> > In order to allow this separation, we propose:
> > 1. HBA + Extension is the single source of truth of Provider root URL
> > + Required Audience for each role. If some backfill for missing OIDC
> > discovery is needed, the provider-specific extension would be
> > providing it.
> > 2. Client Application knows which grant_type to use in which scenario.
> > But can be coded without knowledge of a specific provider. So can't
> > provide discovery details.
> > 3. Driver (libpq, others) - coordinate the authentication flow based
> > on client grant_type and identity provider metadata to allow client
> > applications to use any flow with any provider in a unified way.
> >
> > Yes, this would require a little more complicated flow between
> > components than in your original patch.
>
> Why? I claim that standard OAUTHBEARER can handle all of that. What
> does your proposed architecture (the third diagram) enable that my
> proposed hook (the second diagram) doesn't?
>
> > And yes, more complexity comes
> > with more opportunity to make bugs.
> > However, I see PG Server and Libpq as the places which can have more
> > complexity. For the purpose of making work for the community
> > participants easier and simplify adoption.
> >
> > Does this make sense to you?
>
> Some of it, but it hasn't really addressed the questions from my last mail.
>
> Thanks,
> --Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-12-08 04:25 Andrey Chudnovsky <achudnovskij@gmail.com>
parent: Andrey Chudnovsky <achudnovskij@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: Andrey Chudnovsky @ 2022-12-08 04:25 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: mahendrakar s <mahendrakarforpg@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>
That being said, the Diagram 2 would look like this with our proposal:
+----------------------+ +----------+
| +-------+ | Postgres |
| PQconnect ->| | |
|
| | | |
+-----------+
| | | -------------- Empty Token ------------> | >
| |
| | libpq | <----- Error(w\ Root URL + Audience ) -- | <
| Pre-Auth |
| +------+ | |
| Hook |
| +- < | Hook | | |
+-----------+
| | +------+ | | |
| v | | |
|
| [get token]| | |
|
| | | | |
|
| + | | |
+-----------+
| PQconnect > | | -------------- Access Token -----------> | >
| Validator |
| | | <---- Authorization Success/Failure ---- | <
| Hook |
| | | |
+-----------+
| +-------+ |
| +----------------------+
+----------+
With the application taking care of all Token acquisition logic. While
the server-side hook is participating in the pre-authentication reply.
That is definitely a required scenario for the long term and the
easiest to implement in the client core.
And if we can do at least that flow in PG16 it will be a strong
foundation to provide more support for specific grants in libpq going
forward.
Does the diagram above look good to you? We can then start cleaning up
the patch to get that in first.
Thanks!
Andrey.
On Wed, Dec 7, 2022 at 3:22 PM Andrey Chudnovsky <achudnovskij@gmail.com> wrote:
>
> > I think it's okay to have the extension and HBA collaborate to provide
> > discovery information. Your proposal goes further than that, though,
> > and makes the server aware of the chosen client flow. That appears to
> > be an architectural violation: why does an OAuth resource server need
> > to know the client flow at all?
>
> Ok. It may have left there from intermediate iterations. We did
> consider making extension drive the flow for specific grant_type, but
> decided against that idea. For the same reason you point to.
> Is it correct that your main concern about use of grant_type was that
> it's propagated to the server? Then yes, we will remove sending it to
> the server.
>
> > Ideally, yes, but that only works if all identity providers implement
> > the same flows in compatible ways. We're already seeing instances
> > where that's not the case and we'll necessarily have to deal with that
> > up front.
>
> Yes, based on our analysis OIDC spec is detailed enough, that
> providers implementing that one, can be supported with generic code in
> libpq / client.
> Github specifically won't fit there though. Microsoft Azure AD,
> Google, Okta (including Auth0) will.
> Theoretically discovery documents can be returned from the extension
> (server-side) which is provider specific. Though we didn't plan to
> prioritize that.
>
> > That seems to be restating the goal of OAuth and OIDC. Can you explain
> > how the incompatible change allows you to accomplish this better than
> > standard implementations?
>
> Do you refer to passing grant_type to the server? Which we will get
> rid of in the next iteration. Or other incompatible changes as well?
>
> > Why? I claim that standard OAUTHBEARER can handle all of that. What
> > does your proposed architecture (the third diagram) enable that my
> > proposed hook (the second diagram) doesn't?
>
> The hook proposed on the 2nd diagram effectively delegates all Oauth
> flows implementations to the client.
> We propose libpq takes care of pulling OpenId discovery and coordination.
> Which is effectively Diagram 1 + more flows + server hook providing
> root url/audience.
>
> Created the diagrams with all components for 3 flows:
> 1. Authorization code grant (Clients with Browser access):
> +----------------------+ +----------+
> | +-------+ |
> |
> | PQconnect | | |
> |
> | [auth_code] | | |
> +-----------+
> | -> | | -------------- Empty Token ------------> | >
> | |
> | | libpq | <----- Error(w\ Root URL + Audience ) -- | <
> | Pre-Auth |
> | | | |
> | Hook |
> | | | |
> +-----------+
> | | | +--------------+ | |
> | | | -------[GET]---------> | OIDC | | Postgres |
> | +------+ | <--Provider Metadata-- | Discovery | | |
> | +- < | Hook | < | +--------------+ |
> |
> | | +------+ | |
> |
> | v | | |
> |
> | [get auth | | |
> |
> | code] | | |
> |
> |<user action>| | |
> |
> | | | | |
> |
> | + | | |
> |
> | PQconnect > | +--------+ +--------------+ |
> |
> | | | iddawc | <-- [ Auth code ]-> | Issuer/ | | |
> | | | | <-- Access Token -- | Authz Server | | |
> | | +--------+ +--------------+ | |
> | | | |
> +-----------+
> | | | -------------- Access Token -----------> | >
> | Validator |
> | | | <---- Authorization Success/Failure ---- | <
> | Hook |
> | +------+ | |
> +-----------+
> | +-< | Hook | | |
> |
> | v +------+ | |
> |
> |[store +-------+ |
> |
> | refresh_token] +----------+
> +----------------------+
>
> 2. Device code grant
> +----------------------+ +----------+
> | +-------+ |
> |
> | PQconnect | | |
> |
> | [auth_code] | | |
> +-----------+
> | -> | | -------------- Empty Token ------------> | >
> | |
> | | libpq | <----- Error(w\ Root URL + Audience ) -- | <
> | Pre-Auth |
> | | | |
> | Hook |
> | | | |
> +-----------+
> | | | +--------------+ | |
> | | | -------[GET]---------> | OIDC | | Postgres |
> | +------+ | <--Provider Metadata-- | Discovery | | |
> | +- < | Hook | < | +--------------+ |
> |
> | | +------+ | |
> |
> | v | | |
> |
> | [device | +---------+ +--------------+ |
> |
> | code] | | iddawc | | Issuer/ | |
> |
> |<user action>| | | --[ Device code ]-> | Authz Server | |
> |
> | | |<polling>| --[ Device code ]-> | | |
> |
> | | | | --[ Device code ]-> | | |
> |
> | | | | | | | |
> | | | | <-- Access Token -- | | | |
> | | +---------+ +--------------+ | |
> | | | |
> +-----------+
> | | | -------------- Access Token -----------> | >
> | Validator |
> | | | <---- Authorization Success/Failure ---- | <
> | Hook |
> | +------+ | |
> +-----------+
> | +-< | Hook | | |
> |
> | v +------+ | |
> |
> |[store +-------+ |
> |
> | refresh_token] +----------+
> +----------------------+
>
> 3. Non-interactive flows (Client Secret / Refresh_Token)
> +----------------------+ +----------+
> | +-------+ |
> |
> | PQconnect | | |
> |
> | [grant_type]| | | |
> | -> | | |
> +-----------+
> | | | -------------- Empty Token ------------> | >
> | |
> | | libpq | <----- Error(w\ Root URL + Audience ) -- | <
> | Pre-Auth |
> | | | |
> | Hook |
> | | | |
> +-----------+
> | | | +--------------+ | |
> | | | -------[GET]---------> | OIDC | | Postgres |
> | | | <--Provider Metadata-- | Discovery | | |
> | | | +--------------+ |
> |
> | | | |
> |
> | | +--------+ +--------------+ |
> |
> | | | iddawc | <-- [ Secret ]----> | Issuer/ | | |
> | | | | <-- Access Token -- | Authz Server | | |
> | | +--------+ +--------------+ | |
> | | | |
> +-----------+
> | | | -------------- Access Token -----------> | >
> | Validator |
> | | | <---- Authorization Success/Failure ---- | <
> | Hook |
> | | | |
> +-----------+
> | +-------+ +----------+
> +----------------------+
>
> I think what was the most confusing in our latest patch is that
> flow_type was passed to the server.
> We are not proposing this going forward.
>
> > (For a future conversation: they need to set up authorization, too,
> > with custom scopes or some other magic. It's not enough to check who
> > the token belongs to; even if Postgres is just using the verified
> > email from OpenID as an authenticator, you have to also know that the
> > user authorized the token -- and therefore the client -- to access
> > Postgres on their behalf.)
>
> My understanding is that metadata in the tokens is provider specific,
> so server side hook would be the right place to handle that.
> Plus I can envision for some providers it can make sense to make a
> remote call to pull some information.
>
> The way we implement Azure AD auth today in PAAS PostgreSQL offering:
> - Server administrator uses special extension functions to create
> Azure AD enabled PostgreSQL roles.
> - PostgreSQL extension maps Roles to unique identity Ids (UID) in the Directory.
> - Connection flow: If the token is valid and Role => UID mapping
> matches, we authenticate as the Role.
> - Then its native PostgreSQL role based access control takes care of privileges.
>
> This is the same for both User- and System-to-system authorization.
> Though I assume different providers may treat user- and system-
> identities differently. So their extension would handle that.
>
> Thanks!
> Andrey.
>
> On Wed, Dec 7, 2022 at 11:06 AM Jacob Champion <jchampion@timescale.com> wrote:
> >
> > On Mon, Dec 5, 2022 at 4:15 PM Andrey Chudnovsky <achudnovskij@gmail.com> wrote:
> > > I think we can focus on the roles and responsibilities of the components first.
> > > Details of the patch can be elaborated. Like "flow type code" is a
> > > mistake on our side, and we will use the term "grant_type" which is
> > > defined by OIDC spec. As well as details of usage of refresh_token.
> >
> > (For the record, whether we call it "flow type" or "grant type"
> > doesn't address my concern.)
> >
> > > Basically Yes. We propose an increase of the server side hook responsibility.
> > > From just validating the token, to also return the provider root URL
> > > and required audience. And possibly provide more metadata in the
> > > future.
> >
> > I think it's okay to have the extension and HBA collaborate to provide
> > discovery information. Your proposal goes further than that, though,
> > and makes the server aware of the chosen client flow. That appears to
> > be an architectural violation: why does an OAuth resource server need
> > to know the client flow at all?
> >
> > > Which is in our opinion aligned with SASL protocol, where the server
> > > side is responsible for telling the client auth requirements based on
> > > the requested role in the startup packet.
> >
> > You've proposed an alternative SASL mechanism. There's nothing wrong
> > with that, per se, but I think it should be clear why we've chosen
> > something nonstandard.
> >
> > > Our understanding is that in the original patch that information came
> > > purely from hba, and we propose extension being able to control that
> > > metadata.
> > > As we see extension as being owned by the identity provider, compared
> > > to HBA which is owned by the server administrator or cloud provider.
> >
> > That seems reasonable, considering how tightly coupled the Issuer and
> > the token validation process are.
> >
> > > 2. Server Owners / PAAS providers (On premise admins, Cloud providers,
> > > multi-cloud PAAS providers).
> > > - Install extensions and configure HBA to allow clients to
> > > authenticate with the identity providers of their choice.
> >
> > (For a future conversation: they need to set up authorization, too,
> > with custom scopes or some other magic. It's not enough to check who
> > the token belongs to; even if Postgres is just using the verified
> > email from OpenID as an authenticator, you have to also know that the
> > user authorized the token -- and therefore the client -- to access
> > Postgres on their behalf.)
> >
> > > 3. Client Application Developers (Data Wis, integration tools,
> > > PgAdmin, monitoring tools, e.t.c.)
> > > - Independent from specific Identity providers or server providers.
> > > Write one code for all identity providers.
> >
> > Ideally, yes, but that only works if all identity providers implement
> > the same flows in compatible ways. We're already seeing instances
> > where that's not the case and we'll necessarily have to deal with that
> > up front.
> >
> > > - Rely on application deployment owners to configure which OIDC
> > > provider to use across client and server setups.
> > > 4. Application Deployment Owners (End customers setting up applications)
> > > - The only actor actually aware of which identity provider to use.
> > > Configures the stack based on the Identity and PostgreSQL deployments
> > > they have.
> >
> > (I have doubts that the roles will be as decoupled in practice as you
> > have described them, but I'd rather defer that for now.)
> >
> > > The critical piece of the vision is (3.) above is applications
> > > agnostic of the identity providers. Those applications rely on
> > > properly configured servers and rich driver logic (libpq,
> > > com.postgresql, npgsql) to allow their application to popup auth
> > > windows or do service-to-service authentication with any provider. In
> > > our view that would significantly democratize the deployment of OAUTH
> > > authentication in the community.
> >
> > That seems to be restating the goal of OAuth and OIDC. Can you explain
> > how the incompatible change allows you to accomplish this better than
> > standard implementations?
> >
> > > In order to allow this separation, we propose:
> > > 1. HBA + Extension is the single source of truth of Provider root URL
> > > + Required Audience for each role. If some backfill for missing OIDC
> > > discovery is needed, the provider-specific extension would be
> > > providing it.
> > > 2. Client Application knows which grant_type to use in which scenario.
> > > But can be coded without knowledge of a specific provider. So can't
> > > provide discovery details.
> > > 3. Driver (libpq, others) - coordinate the authentication flow based
> > > on client grant_type and identity provider metadata to allow client
> > > applications to use any flow with any provider in a unified way.
> > >
> > > Yes, this would require a little more complicated flow between
> > > components than in your original patch.
> >
> > Why? I claim that standard OAUTHBEARER can handle all of that. What
> > does your proposed architecture (the third diagram) enable that my
> > proposed hook (the second diagram) doesn't?
> >
> > > And yes, more complexity comes
> > > with more opportunity to make bugs.
> > > However, I see PG Server and Libpq as the places which can have more
> > > complexity. For the purpose of making work for the community
> > > participants easier and simplify adoption.
> > >
> > > Does this make sense to you?
> >
> > Some of it, but it hasn't really addressed the questions from my last mail.
> >
> > Thanks,
> > --Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-12-09 00:41 Jacob Champion <jchampion@timescale.com>
parent: Andrey Chudnovsky <achudnovskij@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2022-12-09 00:41 UTC (permalink / raw)
To: Andrey Chudnovsky <achudnovskij@gmail.com>; +Cc: mahendrakar s <mahendrakarforpg@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>
On Wed, Dec 7, 2022 at 3:22 PM Andrey Chudnovsky
<achudnovskij@gmail.com> wrote:
>
>> I think it's okay to have the extension and HBA collaborate to
>> provide discovery information. Your proposal goes further than
>> that, though, and makes the server aware of the chosen client flow.
>> That appears to be an architectural violation: why does an OAuth
>> resource server need to know the client flow at all?
>
> Ok. It may have left there from intermediate iterations. We did
> consider making extension drive the flow for specific grant_type,
> but decided against that idea. For the same reason you point to. Is
> it correct that your main concern about use of grant_type was that
> it's propagated to the server? Then yes, we will remove sending it
> to the server.
Okay. Yes, that was my primary concern.
>> Ideally, yes, but that only works if all identity providers
>> implement the same flows in compatible ways. We're already seeing
>> instances where that's not the case and we'll necessarily have to
>> deal with that up front.
>
> Yes, based on our analysis OIDC spec is detailed enough, that
> providers implementing that one, can be supported with generic code
> in libpq / client. Github specifically won't fit there though.
> Microsoft Azure AD, Google, Okta (including Auth0) will.
> Theoretically discovery documents can be returned from the extension
> (server-side) which is provider specific. Though we didn't plan to
> prioritize that.
As another example, Google's device authorization grant is incompatible
with the spec (which they co-authored). I want to say I had problems
with Azure AD not following that spec either, but I don't remember
exactly what they were. I wouldn't be surprised to find more tiny
departures once we get deeper into implementation.
>> That seems to be restating the goal of OAuth and OIDC. Can you
>> explain how the incompatible change allows you to accomplish this
>> better than standard implementations?
>
> Do you refer to passing grant_type to the server? Which we will get
> rid of in the next iteration. Or other incompatible changes as well?
Just the grant type, yeah.
>> Why? I claim that standard OAUTHBEARER can handle all of that.
>> What does your proposed architecture (the third diagram) enable
>> that my proposed hook (the second diagram) doesn't?
>
> The hook proposed on the 2nd diagram effectively delegates all Oauth
> flows implementations to the client. We propose libpq takes care of
> pulling OpenId discovery and coordination. Which is effectively
> Diagram 1 + more flows + server hook providing root url/audience.
>
> Created the diagrams with all components for 3 flows: [snip]
(I'll skip ahead to your later mail on this.)
>> (For a future conversation: they need to set up authorization,
>> too, with custom scopes or some other magic. It's not enough to
>> check who the token belongs to; even if Postgres is just using the
>> verified email from OpenID as an authenticator, you have to also
>> know that the user authorized the token -- and therefore the client
>> -- to access Postgres on their behalf.)
>
> My understanding is that metadata in the tokens is provider
> specific, so server side hook would be the right place to handle
> that. Plus I can envision for some providers it can make sense to
> make a remote call to pull some information.
The server hook is the right place to check the scopes, yes, but I think
the DBA should be able to specify what those scopes are to begin with.
The provider of the extension shouldn't be expected by the architecture
to hardcode those decisions, even if Azure AD chooses to short-circuit
that choice and provide magic instead.
On 12/7/22 20:25, Andrey Chudnovsky wrote:
> That being said, the Diagram 2 would look like this with our proposal:
> [snip]
>
> With the application taking care of all Token acquisition logic. While
> the server-side hook is participating in the pre-authentication reply.
>
> That is definitely a required scenario for the long term and the
> easiest to implement in the client core.> And if we can do at least that flow in PG16 it will be a strong
> foundation to provide more support for specific grants in libpq going
> forward.
Agreed.
> Does the diagram above look good to you? We can then start cleaning up
> the patch to get that in first.
I maintain that the hook doesn't need to hand back artifacts to the
client for a second PQconnect call. It can just use those artifacts to
obtain the access token and hand that right back to libpq. (I think any
requirement that clients be rewritten to call PQconnect twice will
probably be a sticking point for adoption of an OAuth patch.)
That said, now that your proposal is also compatible with OAUTHBEARER, I
can pony up some code to hopefully prove my point. (I don't know if I'll
be able to do that by the holidays though.)
Thanks!
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-12-13 05:06 Andrey Chudnovsky <achudnovskij@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
0 siblings, 1 reply; 424+ messages in thread
From: Andrey Chudnovsky @ 2022-12-13 05:06 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: mahendrakar s <mahendrakarforpg@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>
> The server hook is the right place to check the scopes, yes, but I think
> the DBA should be able to specify what those scopes are to begin with.
> The provider of the extension shouldn't be expected by the architecture
> to hardcode those decisions, even if Azure AD chooses to short-circuit
> that choice and provide magic instead.
Hardcode is definitely not expected, but customization for identity
provider specific, I think, should be allowed.
I can provide a couple of advanced use cases which happen in the cloud
deployments world, and require per-role management:
- Multi-tenant deployments, when root provider URL would be different
for different roles, based on which tenant they come from.
- Federation to multiple providers. Solutions like Amazon Cognito
which offer a layer of abstraction with several providers
transparently supported.
If your concern is extension not honoring the DBA configured values:
Would a server-side logic to prefer HBA value over extension-provided
resolve this concern?
We are definitely biased towards the cloud deployment scenarios, where
direct access to .hba files is usually not offered at all.
Let's find the middle ground here.
A separate reason for creating this pre-authentication hook is further
extensibility to support more metadata.
Specifically when we add support for OAUTH flows to libpq, server-side
extensions can help bridge the gap between the identity provider
implementation and OAUTH/OIDC specs.
For example, that could allow the Github extension to provide an OIDC
discovery document.
I definitely see identity providers as institutional actors here which
can be given some power through the extension hooks to customize the
behavior within the framework.
> I maintain that the hook doesn't need to hand back artifacts to the
> client for a second PQconnect call. It can just use those artifacts to
> obtain the access token and hand that right back to libpq. (I think any
> requirement that clients be rewritten to call PQconnect twice will
> probably be a sticking point for adoption of an OAuth patch.)
Obtaining a token is an asynchronous process with a human in the loop.
Not sure if expecting a hook function to return a token synchronously
is the best option here.
Can that be an optional return value of the hook in cases when a token
can be obtained synchronously?
On Thu, Dec 8, 2022 at 4:41 PM Jacob Champion <jchampion@timescale.com> wrote:
>
> On Wed, Dec 7, 2022 at 3:22 PM Andrey Chudnovsky
> <achudnovskij@gmail.com> wrote:
> >
> >> I think it's okay to have the extension and HBA collaborate to
> >> provide discovery information. Your proposal goes further than
> >> that, though, and makes the server aware of the chosen client flow.
> >> That appears to be an architectural violation: why does an OAuth
> >> resource server need to know the client flow at all?
> >
> > Ok. It may have left there from intermediate iterations. We did
> > consider making extension drive the flow for specific grant_type,
> > but decided against that idea. For the same reason you point to. Is
> > it correct that your main concern about use of grant_type was that
> > it's propagated to the server? Then yes, we will remove sending it
> > to the server.
>
> Okay. Yes, that was my primary concern.
>
> >> Ideally, yes, but that only works if all identity providers
> >> implement the same flows in compatible ways. We're already seeing
> >> instances where that's not the case and we'll necessarily have to
> >> deal with that up front.
> >
> > Yes, based on our analysis OIDC spec is detailed enough, that
> > providers implementing that one, can be supported with generic code
> > in libpq / client. Github specifically won't fit there though.
> > Microsoft Azure AD, Google, Okta (including Auth0) will.
> > Theoretically discovery documents can be returned from the extension
> > (server-side) which is provider specific. Though we didn't plan to
> > prioritize that.
>
> As another example, Google's device authorization grant is incompatible
> with the spec (which they co-authored). I want to say I had problems
> with Azure AD not following that spec either, but I don't remember
> exactly what they were. I wouldn't be surprised to find more tiny
> departures once we get deeper into implementation.
>
> >> That seems to be restating the goal of OAuth and OIDC. Can you
> >> explain how the incompatible change allows you to accomplish this
> >> better than standard implementations?
> >
> > Do you refer to passing grant_type to the server? Which we will get
> > rid of in the next iteration. Or other incompatible changes as well?
>
> Just the grant type, yeah.
>
> >> Why? I claim that standard OAUTHBEARER can handle all of that.
> >> What does your proposed architecture (the third diagram) enable
> >> that my proposed hook (the second diagram) doesn't?
> >
> > The hook proposed on the 2nd diagram effectively delegates all Oauth
> > flows implementations to the client. We propose libpq takes care of
> > pulling OpenId discovery and coordination. Which is effectively
> > Diagram 1 + more flows + server hook providing root url/audience.
> >
> > Created the diagrams with all components for 3 flows: [snip]
>
> (I'll skip ahead to your later mail on this.)
>
> >> (For a future conversation: they need to set up authorization,
> >> too, with custom scopes or some other magic. It's not enough to
> >> check who the token belongs to; even if Postgres is just using the
> >> verified email from OpenID as an authenticator, you have to also
> >> know that the user authorized the token -- and therefore the client
> >> -- to access Postgres on their behalf.)
> >
> > My understanding is that metadata in the tokens is provider
> > specific, so server side hook would be the right place to handle
> > that. Plus I can envision for some providers it can make sense to
> > make a remote call to pull some information.
>
> The server hook is the right place to check the scopes, yes, but I think
> the DBA should be able to specify what those scopes are to begin with.
> The provider of the extension shouldn't be expected by the architecture
> to hardcode those decisions, even if Azure AD chooses to short-circuit
> that choice and provide magic instead.
>
> On 12/7/22 20:25, Andrey Chudnovsky wrote:
> > That being said, the Diagram 2 would look like this with our proposal:
> > [snip]
> >
> > With the application taking care of all Token acquisition logic. While
> > the server-side hook is participating in the pre-authentication reply.
> >
> > That is definitely a required scenario for the long term and the
> > easiest to implement in the client core.> And if we can do at least that flow in PG16 it will be a strong
> > foundation to provide more support for specific grants in libpq going
> > forward.
>
> Agreed.
> > Does the diagram above look good to you? We can then start cleaning up
> > the patch to get that in first.
>
> I maintain that the hook doesn't need to hand back artifacts to the
> client for a second PQconnect call. It can just use those artifacts to
> obtain the access token and hand that right back to libpq. (I think any
> requirement that clients be rewritten to call PQconnect twice will
> probably be a sticking point for adoption of an OAuth patch.)
>
> That said, now that your proposal is also compatible with OAUTHBEARER, I
> can pony up some code to hopefully prove my point. (I don't know if I'll
> be able to do that by the holidays though.)
>
> Thanks!
> --Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2022-12-16 23:18 Jacob Champion <jchampion@timescale.com>
parent: Andrey Chudnovsky <achudnovskij@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2022-12-16 23:18 UTC (permalink / raw)
To: Andrey Chudnovsky <achudnovskij@gmail.com>; +Cc: mahendrakar s <mahendrakarforpg@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com <smilingsamay@gmail.com>
On Mon, Dec 12, 2022 at 9:06 PM Andrey Chudnovsky
<achudnovskij@gmail.com> wrote:
> If your concern is extension not honoring the DBA configured values:
> Would a server-side logic to prefer HBA value over extension-provided
> resolve this concern?
Yeah. It also seals the role of the extension here as "optional".
> We are definitely biased towards the cloud deployment scenarios, where
> direct access to .hba files is usually not offered at all.
> Let's find the middle ground here.
Sure. I don't want to make this difficult in cloud scenarios --
obviously I'd like for Timescale Cloud to be able to make use of this
too. But if we make this easy for a lone DBA (who doesn't have any
institutional power with the providers) to use correctly and securely,
then it should follow that the providers who _do_ have power and
resources will have an easy time of it as well. The reverse isn't
necessarily true. So I'm definitely planning to focus on the DBA case
first.
> A separate reason for creating this pre-authentication hook is further
> extensibility to support more metadata.
> Specifically when we add support for OAUTH flows to libpq, server-side
> extensions can help bridge the gap between the identity provider
> implementation and OAUTH/OIDC specs.
> For example, that could allow the Github extension to provide an OIDC
> discovery document.
>
> I definitely see identity providers as institutional actors here which
> can be given some power through the extension hooks to customize the
> behavior within the framework.
We'll probably have to make some compromises in this area, but I think
they should be carefully considered exceptions and not a core feature
of the mechanism. The gaps you point out are just fragmentation, and
adding custom extensions to deal with it leads to further
fragmentation instead of providing pressure on providers to just
implement the specs. Worst case, we open up new exciting security
flaws, and then no one can analyze them independently because no one
other than the provider knows how the two sides work together anymore.
Don't get me wrong; it would be naive to proceed as if the OAUTHBEARER
spec were perfect, because it's clearly not. But if we need to make
extensions to it, we can participate in IETF discussions and make our
case publicly for review, rather than enshrining MS/GitHub/Google/etc.
versions of the RFC and enabling that proliferation as a Postgres core
feature.
> Obtaining a token is an asynchronous process with a human in the loop.
> Not sure if expecting a hook function to return a token synchronously
> is the best option here.
> Can that be an optional return value of the hook in cases when a token
> can be obtained synchronously?
I don't think the hook is generally going to be able to return a token
synchronously, and I expect the final design to be async-first. As far
as I know, this will need to be solved for the builtin flows as well
(you don't want a synchronous HTTP call to block your PQconnectPoll
architecture), so the hook should be able to make use of whatever
solution we land on for that.
This is hand-wavy, and I don't expect it to be easy to solve. I just
don't think we have to solve it twice.
Have a good end to the year!
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-01-12 19:08 mahendrakar s <mahendrakarforpg@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
0 siblings, 1 reply; 424+ messages in thread
From: mahendrakar s @ 2023-01-12 19:08 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: Andrey Chudnovsky <achudnovskij@gmail.com>; hlinnaka@iki.fi, Michael Paquier <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com
Hi All,
Changes added to Jacob's patch(v2) as per the discussion in the thread.
The changes allow the customer to send the OAUTH BEARER token through psql
connection string.
Example:
psql -U user@example.com -d 'dbname=postgres oauth_bearer_token=abc'
To configure OAUTH, the pg_hba.conf line look like:
local all all oauth
provider=oauth_provider issuer="https://example.com"; scope="openid email"
We also added hook to libpq to pass on the metadata about the issuer.
Thanks,
Mahendrakar.
On Sat, 17 Dec 2022 at 04:48, Jacob Champion <jchampion@timescale.com>
wrote:
>
> On Mon, Dec 12, 2022 at 9:06 PM Andrey Chudnovsky
> <achudnovskij@gmail.com> wrote:
> > If your concern is extension not honoring the DBA configured values:
> > Would a server-side logic to prefer HBA value over extension-provided
> > resolve this concern?
>
> Yeah. It also seals the role of the extension here as "optional".
>
> > We are definitely biased towards the cloud deployment scenarios, where
> > direct access to .hba files is usually not offered at all.
> > Let's find the middle ground here.
>
> Sure. I don't want to make this difficult in cloud scenarios --
> obviously I'd like for Timescale Cloud to be able to make use of this
> too. But if we make this easy for a lone DBA (who doesn't have any
> institutional power with the providers) to use correctly and securely,
> then it should follow that the providers who _do_ have power and
> resources will have an easy time of it as well. The reverse isn't
> necessarily true. So I'm definitely planning to focus on the DBA case
> first.
>
> > A separate reason for creating this pre-authentication hook is further
> > extensibility to support more metadata.
> > Specifically when we add support for OAUTH flows to libpq, server-side
> > extensions can help bridge the gap between the identity provider
> > implementation and OAUTH/OIDC specs.
> > For example, that could allow the Github extension to provide an OIDC
> > discovery document.
> >
> > I definitely see identity providers as institutional actors here which
> > can be given some power through the extension hooks to customize the
> > behavior within the framework.
>
> We'll probably have to make some compromises in this area, but I think
> they should be carefully considered exceptions and not a core feature
> of the mechanism. The gaps you point out are just fragmentation, and
> adding custom extensions to deal with it leads to further
> fragmentation instead of providing pressure on providers to just
> implement the specs. Worst case, we open up new exciting security
> flaws, and then no one can analyze them independently because no one
> other than the provider knows how the two sides work together anymore.
>
> Don't get me wrong; it would be naive to proceed as if the OAUTHBEARER
> spec were perfect, because it's clearly not. But if we need to make
> extensions to it, we can participate in IETF discussions and make our
> case publicly for review, rather than enshrining MS/GitHub/Google/etc.
> versions of the RFC and enabling that proliferation as a Postgres core
> feature.
>
> > Obtaining a token is an asynchronous process with a human in the loop.
> > Not sure if expecting a hook function to return a token synchronously
> > is the best option here.
> > Can that be an optional return value of the hook in cases when a token
> > can be obtained synchronously?
>
> I don't think the hook is generally going to be able to return a token
> synchronously, and I expect the final design to be async-first. As far
> as I know, this will need to be solved for the builtin flows as well
> (you don't want a synchronous HTTP call to block your PQconnectPoll
> architecture), so the hook should be able to make use of whatever
> solution we land on for that.
>
> This is hand-wavy, and I don't expect it to be easy to solve. I just
> don't think we have to solve it twice.
>
> Have a good end to the year!
> --Jacob
Attachments:
v3-0001-libpq-add-OAUTHBEARER-SASL-mechanism-and-call-back-hooks.patch.gz (application/gzip, 8.5K) download
v3-0002-backend-add-OAUTHBEARER-SASL-mechanishm.patch.gz (application/gzip, 8.6K) download
v3-0004-common-jsonapi-support-FRONTEND-clients.patch.gz (application/gzip, 5.9K) download
v3-0003-simple-oauth_provider-extension.patch.gz (application/gzip, 1.7K) download
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-01-15 20:03 Andrey Chudnovsky <achudnovskij@gmail.com>
parent: mahendrakar s <mahendrakarforpg@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: Andrey Chudnovsky @ 2023-01-15 20:03 UTC (permalink / raw)
To: mahendrakar s <mahendrakarforpg@gmail.com>; +Cc: Jacob Champion <jchampion@timescale.com>; hlinnaka@iki.fi, Michael Paquier <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com
More information on the latest patch.
1. We aligned the implementation with the barebone SASL for OAUTH
described here - https://www.rfc-editor.org/rfc/rfc7628
The flow can be explained in the diagram below:
+----------------------+ +----------+
| +-------+ | Postgres |
| PQconnect ->| | | |
| | | | +-----------+
| | | ---------- Empty Token---------> | > | |
| | libpq | <-- Error(Discovery + Scope ) -- | < | Pre-Auth |
| +------+ | | | Hook |
| +- < | Hook | | | +-----------+
| | +------+ | | |
| v | | | |
| [get token]| | | |
| | | | | |
| + | | | +-----------+
| PQconnect > | | --------- Access Token --------> | > | Validator |
| | | <---------- Auth Result -------- | < | Hook |
| | | | +-----------+
| +-------+ | |
+----------------------+ +----------+
2. Removed Device Code implementation in libpq. Several reasons:
- Reduce scope and focus on the protocol first.
- Device code implementation uses iddawc dependency. Taking this
dependency is a controversial step which requires broader discussion.
- Device code implementation without iddaws would significantly
increase the scope of the patch, as libpq needs to poll the token
endpoint, setup different API calls, e.t.c.
- That flow should canonically only be used for clients which can't
invoke browsers. If it is the only flow to be implemented, it can be
used in the context when it's not expected by the OAUTH protocol.
3. Temporarily removed test suite. We are actively working on aligning
the tests with the latest changes. Will add a patch with tests soon.
We will change the "V3" prefix to make it the next after the previous
iterations.
Thanks!
Andrey.
On Thu, Jan 12, 2023 at 11:08 AM mahendrakar s
<mahendrakarforpg@gmail.com> wrote:
>
> Hi All,
>
> Changes added to Jacob's patch(v2) as per the discussion in the thread.
>
> The changes allow the customer to send the OAUTH BEARER token through psql connection string.
>
> Example:
> psql -U user@example.com -d 'dbname=postgres oauth_bearer_token=abc'
>
> To configure OAUTH, the pg_hba.conf line look like:
> local all all oauth provider=oauth_provider issuer="https://example.com"; scope="openid email"
>
> We also added hook to libpq to pass on the metadata about the issuer.
>
> Thanks,
> Mahendrakar.
>
>
> On Sat, 17 Dec 2022 at 04:48, Jacob Champion <jchampion@timescale.com> wrote:
> >
> > On Mon, Dec 12, 2022 at 9:06 PM Andrey Chudnovsky
> > <achudnovskij@gmail.com> wrote:
> > > If your concern is extension not honoring the DBA configured values:
> > > Would a server-side logic to prefer HBA value over extension-provided
> > > resolve this concern?
> >
> > Yeah. It also seals the role of the extension here as "optional".
> >
> > > We are definitely biased towards the cloud deployment scenarios, where
> > > direct access to .hba files is usually not offered at all.
> > > Let's find the middle ground here.
> >
> > Sure. I don't want to make this difficult in cloud scenarios --
> > obviously I'd like for Timescale Cloud to be able to make use of this
> > too. But if we make this easy for a lone DBA (who doesn't have any
> > institutional power with the providers) to use correctly and securely,
> > then it should follow that the providers who _do_ have power and
> > resources will have an easy time of it as well. The reverse isn't
> > necessarily true. So I'm definitely planning to focus on the DBA case
> > first.
> >
> > > A separate reason for creating this pre-authentication hook is further
> > > extensibility to support more metadata.
> > > Specifically when we add support for OAUTH flows to libpq, server-side
> > > extensions can help bridge the gap between the identity provider
> > > implementation and OAUTH/OIDC specs.
> > > For example, that could allow the Github extension to provide an OIDC
> > > discovery document.
> > >
> > > I definitely see identity providers as institutional actors here which
> > > can be given some power through the extension hooks to customize the
> > > behavior within the framework.
> >
> > We'll probably have to make some compromises in this area, but I think
> > they should be carefully considered exceptions and not a core feature
> > of the mechanism. The gaps you point out are just fragmentation, and
> > adding custom extensions to deal with it leads to further
> > fragmentation instead of providing pressure on providers to just
> > implement the specs. Worst case, we open up new exciting security
> > flaws, and then no one can analyze them independently because no one
> > other than the provider knows how the two sides work together anymore.
> >
> > Don't get me wrong; it would be naive to proceed as if the OAUTHBEARER
> > spec were perfect, because it's clearly not. But if we need to make
> > extensions to it, we can participate in IETF discussions and make our
> > case publicly for review, rather than enshrining MS/GitHub/Google/etc.
> > versions of the RFC and enabling that proliferation as a Postgres core
> > feature.
> >
> > > Obtaining a token is an asynchronous process with a human in the loop.
> > > Not sure if expecting a hook function to return a token synchronously
> > > is the best option here.
> > > Can that be an optional return value of the hook in cases when a token
> > > can be obtained synchronously?
> >
> > I don't think the hook is generally going to be able to return a token
> > synchronously, and I expect the final design to be async-first. As far
> > as I know, this will need to be solved for the builtin flows as well
> > (you don't want a synchronous HTTP call to block your PQconnectPoll
> > architecture), so the hook should be able to make use of whatever
> > solution we land on for that.
> >
> > This is hand-wavy, and I don't expect it to be easy to solve. I just
> > don't think we have to solve it twice.
> >
> > Have a good end to the year!
> > --Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-01-17 22:43 Jacob Champion <jchampion@timescale.com>
parent: Andrey Chudnovsky <achudnovskij@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2023-01-17 22:43 UTC (permalink / raw)
To: Andrey Chudnovsky <achudnovskij@gmail.com>; +Cc: mahendrakar s <mahendrakarforpg@gmail.com>; hlinnaka@iki.fi, Michael Paquier <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com
On Sun, Jan 15, 2023 at 12:03 PM Andrey Chudnovsky
<achudnovskij@gmail.com> wrote:
> 2. Removed Device Code implementation in libpq. Several reasons:
> - Reduce scope and focus on the protocol first.
> - Device code implementation uses iddawc dependency. Taking this
> dependency is a controversial step which requires broader discussion.
> - Device code implementation without iddaws would significantly
> increase the scope of the patch, as libpq needs to poll the token
> endpoint, setup different API calls, e.t.c.
> - That flow should canonically only be used for clients which can't
> invoke browsers. If it is the only flow to be implemented, it can be
> used in the context when it's not expected by the OAUTH protocol.
I'm not understanding the concern in the final point -- providers
generally require you to opt into device authorization, at least as far
as I can tell. So if you decide that it's not appropriate for your use
case... don't enable it. (And I haven't seen any claims that opting into
device authorization weakens the other flows in any way. So if we're
going to implement a flow in libpq, I still think device authorization
is the best choice, since it works on headless machines as well as those
with browsers.)
All of this points at a bigger question to the community: if we choose
not to provide a flow implementation in libpq, is adding OAUTHBEARER
worth the additional maintenance cost?
My personal vote would be "no". I think the hook-only approach proposed
here would ensure that only larger providers would implement it in
practice, and in that case I'd rather spend cycles on generic SASL.
> 3. Temporarily removed test suite. We are actively working on aligning
> the tests with the latest changes. Will add a patch with tests soon.
Okay. Case in point, the following change to the patch appears to be
invalid JSON:
> + appendStringInfo(&buf,
> + "{ "
> + "\"status\": \"invalid_token\", "
> + "\"openid-configuration\": \"%s\","
> + "\"scope\": \"%s\" ",
> + "\"issuer\": \"%s\" ",
> + "}",
Additionally, the "issuer" field added here is not part of the RFC. I've
written my thoughts about unofficial extensions upthread but haven't
received a response, so I'm going to start being more strident: Please,
for the sake of reviewers, call out changes you've made to the spec, and
why they're justified.
The patches seem to be out of order now (and the documentation in the
commit messages has been removed).
Thanks,
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-01-18 01:53 Andrey Chudnovsky <achudnovskij@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
0 siblings, 1 reply; 424+ messages in thread
From: Andrey Chudnovsky @ 2023-01-18 01:53 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: mahendrakar s <mahendrakarforpg@gmail.com>; hlinnaka@iki.fi, Michael Paquier <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com
> All of this points at a bigger question to the community: if we choose
> not to provide a flow implementation in libpq, is adding OAUTHBEARER
> worth the additional maintenance cost?
> My personal vote would be "no". I think the hook-only approach proposed
> here would ensure that only larger providers would implement it in
> practice
Flow implementations in libpq are definitely a long term plan, and I
agree that it would democratise the adoption.
In the previous posts in this conversation I outlined the ones I think
we should support.
However, I don't see why it's strictly necessary to couple those.
As long as the SASL exchange for OAUTHBEARER mechanism is supported by
the protocol, the Client side can evolve at its own pace.
At the same time, the current implementation allows clients to start
building provider-agnostic OAUTH support. By using iddawc or OAUTH
client implementations in the respective platforms.
So I wouldn't refer to "larger providers", but rather "more motivated
clients" here. Which definitely overlaps, but keeps the system open.
> I'm not understanding the concern in the final point -- providers
> generally require you to opt into device authorization, at least as far
> as I can tell. So if you decide that it's not appropriate for your use
> case... don't enable it. (And I haven't seen any claims that opting into
> device authorization weakens the other flows in any way. So if we're
> going to implement a flow in libpq, I still think device authorization
> is the best choice, since it works on headless machines as well as those
> with browsers.)
I agree with the statement that Device code is the best first choice
if we absolutely have to pick one.
Though I don't think we have to.
While device flow can be used for all kinds of user-facing
applications, it's specifically designed for input-constrained
scenarios. As clearly stated in the Abstract here -
https://www.rfc-editor.org/rfc/rfc8628
The authorization code with pkce flow is recommended by the RFSc and
major providers for cases when it's feasible.
The long term goal is to provide both, though I don't see why the
backbone protocol implementation first wouldn't add value.
Another point is user authentication is one side of the whole story
and the other critical one is system-to-system authentication. Where
we have Client Credentials and Certificates.
With the latter it is much harder to get generically implemented, as
provider-specific tokens need to be signed.
Adding the other reasoning, I think libpq support for specific flows
can get in the further iterations, after the protocol support.
> in that case I'd rather spend cycles on generic SASL.
I see 2 approaches to generic SASL:
(a). Generic SASL is a framework used in the protocol, with the
mechanisms implemented on top and exposed to the DBAs as auth types to
configure in hba.
This is the direction we're going here, which is well aligned with the
existing hba-based auth configuration.
(b). Generic SASL exposed to developers on the server- and client-
side to extend on. It seems to be a much longer shot.
The specific points of large ambiguity are libpq distribution model
(which you pointed to) and potential pluggability of insecure
mechanisms.
I do see (a) as a sweet spot with a lot of value for various
participants with much less ambiguity.
> Additionally, the "issuer" field added here is not part of the RFC. I've
> written my thoughts about unofficial extensions upthread but haven't
> received a response, so I'm going to start being more strident: Please,
> for the sake of reviewers, call out changes you've made to the spec, and
> why they're justified.
Thanks for your feedback on this. We had this discussion as well, and
added that as a convenience for the client to identify the provider.
I don't see a reason why an issuer would be absolutely necessary, so
we will get your point that sticking to RFCs is a safer choice.
> The patches seem to be out of order now (and the documentation in the
> commit messages has been removed).
Feedback taken. Work in progress.
On Tue, Jan 17, 2023 at 2:44 PM Jacob Champion <jchampion@timescale.com> wrote:
>
> On Sun, Jan 15, 2023 at 12:03 PM Andrey Chudnovsky
> <achudnovskij@gmail.com> wrote:
> > 2. Removed Device Code implementation in libpq. Several reasons:
> > - Reduce scope and focus on the protocol first.
> > - Device code implementation uses iddawc dependency. Taking this
> > dependency is a controversial step which requires broader discussion.
> > - Device code implementation without iddaws would significantly
> > increase the scope of the patch, as libpq needs to poll the token
> > endpoint, setup different API calls, e.t.c.
> > - That flow should canonically only be used for clients which can't
> > invoke browsers. If it is the only flow to be implemented, it can be
> > used in the context when it's not expected by the OAUTH protocol.
>
> I'm not understanding the concern in the final point -- providers
> generally require you to opt into device authorization, at least as far
> as I can tell. So if you decide that it's not appropriate for your use
> case... don't enable it. (And I haven't seen any claims that opting into
> device authorization weakens the other flows in any way. So if we're
> going to implement a flow in libpq, I still think device authorization
> is the best choice, since it works on headless machines as well as those
> with browsers.)
>
> All of this points at a bigger question to the community: if we choose
> not to provide a flow implementation in libpq, is adding OAUTHBEARER
> worth the additional maintenance cost?
>
> My personal vote would be "no". I think the hook-only approach proposed
> here would ensure that only larger providers would implement it in
> practice, and in that case I'd rather spend cycles on generic SASL.
>
> > 3. Temporarily removed test suite. We are actively working on aligning
> > the tests with the latest changes. Will add a patch with tests soon.
>
> Okay. Case in point, the following change to the patch appears to be
> invalid JSON:
>
> > + appendStringInfo(&buf,
> > + "{ "
> > + "\"status\": \"invalid_token\", "
> > + "\"openid-configuration\": \"%s\","
> > + "\"scope\": \"%s\" ",
> > + "\"issuer\": \"%s\" ",
> > + "}",
>
> Additionally, the "issuer" field added here is not part of the RFC. I've
> written my thoughts about unofficial extensions upthread but haven't
> received a response, so I'm going to start being more strident: Please,
> for the sake of reviewers, call out changes you've made to the spec, and
> why they're justified.
>
> The patches seem to be out of order now (and the documentation in the
> commit messages has been removed).
>
> Thanks,
> --Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-01-25 04:46 mahendrakar s <mahendrakarforpg@gmail.com>
parent: Andrey Chudnovsky <achudnovskij@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: mahendrakar s @ 2023-01-25 04:46 UTC (permalink / raw)
To: Andrey Chudnovsky <achudnovskij@gmail.com>; +Cc: Jacob Champion <jchampion@timescale.com>; hlinnaka@iki.fi, Michael Paquier <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com
Hi All,
The "issuer" field has been removed to align with the RFC
implementation - https://www.rfc-editor.org/rfc/rfc7628.
This patch "v6" is a single patch to support the OAUTH BEARER token
through psql connection string.
Below flow is supported. Added the documentation in the commit messages.
+----------------------+ +----------+
| +-------+ | Postgres |
| PQconnect ->| | | |
| | | | +-----------+
| | | ---------- Empty Token---------> | > | |
| | libpq | <-- Error(Discovery + Scope ) -- | < | Pre-Auth |
| +------+ | | | Hook |
| +- < | Hook | | | +-----------+
| | +------+ | | |
| v | | | |
| [get token]| | | |
| | | | | |
| + | | | +-----------+
| PQconnect > | | --------- Access Token --------> | > | Validator |
| | | <---------- Auth Result -------- | < | Hook |
| | | | +-----------+
| +-------+ | |
+----------------------+ +----------+
Please note that we are working on modifying/adding new tests (from
Jacob's Patch) with the latest changes. Will add a patch with tests
soon.
Thanks,
Mahendrakar.
On Wed, 18 Jan 2023 at 07:24, Andrey Chudnovsky <achudnovskij@gmail.com> wrote:
>
> > All of this points at a bigger question to the community: if we choose
> > not to provide a flow implementation in libpq, is adding OAUTHBEARER
> > worth the additional maintenance cost?
>
> > My personal vote would be "no". I think the hook-only approach proposed
> > here would ensure that only larger providers would implement it in
> > practice
>
> Flow implementations in libpq are definitely a long term plan, and I
> agree that it would democratise the adoption.
> In the previous posts in this conversation I outlined the ones I think
> we should support.
>
> However, I don't see why it's strictly necessary to couple those.
> As long as the SASL exchange for OAUTHBEARER mechanism is supported by
> the protocol, the Client side can evolve at its own pace.
>
> At the same time, the current implementation allows clients to start
> building provider-agnostic OAUTH support. By using iddawc or OAUTH
> client implementations in the respective platforms.
> So I wouldn't refer to "larger providers", but rather "more motivated
> clients" here. Which definitely overlaps, but keeps the system open.
>
> > I'm not understanding the concern in the final point -- providers
> > generally require you to opt into device authorization, at least as far
> > as I can tell. So if you decide that it's not appropriate for your use
> > case... don't enable it. (And I haven't seen any claims that opting into
> > device authorization weakens the other flows in any way. So if we're
> > going to implement a flow in libpq, I still think device authorization
> > is the best choice, since it works on headless machines as well as those
> > with browsers.)
> I agree with the statement that Device code is the best first choice
> if we absolutely have to pick one.
> Though I don't think we have to.
>
> While device flow can be used for all kinds of user-facing
> applications, it's specifically designed for input-constrained
> scenarios. As clearly stated in the Abstract here -
> https://www.rfc-editor.org/rfc/rfc8628
> The authorization code with pkce flow is recommended by the RFSc and
> major providers for cases when it's feasible.
> The long term goal is to provide both, though I don't see why the
> backbone protocol implementation first wouldn't add value.
>
> Another point is user authentication is one side of the whole story
> and the other critical one is system-to-system authentication. Where
> we have Client Credentials and Certificates.
> With the latter it is much harder to get generically implemented, as
> provider-specific tokens need to be signed.
>
> Adding the other reasoning, I think libpq support for specific flows
> can get in the further iterations, after the protocol support.
>
> > in that case I'd rather spend cycles on generic SASL.
> I see 2 approaches to generic SASL:
> (a). Generic SASL is a framework used in the protocol, with the
> mechanisms implemented on top and exposed to the DBAs as auth types to
> configure in hba.
> This is the direction we're going here, which is well aligned with the
> existing hba-based auth configuration.
> (b). Generic SASL exposed to developers on the server- and client-
> side to extend on. It seems to be a much longer shot.
> The specific points of large ambiguity are libpq distribution model
> (which you pointed to) and potential pluggability of insecure
> mechanisms.
>
> I do see (a) as a sweet spot with a lot of value for various
> participants with much less ambiguity.
>
> > Additionally, the "issuer" field added here is not part of the RFC. I've
> > written my thoughts about unofficial extensions upthread but haven't
> > received a response, so I'm going to start being more strident: Please,
> > for the sake of reviewers, call out changes you've made to the spec, and
> > why they're justified.
> Thanks for your feedback on this. We had this discussion as well, and
> added that as a convenience for the client to identify the provider.
> I don't see a reason why an issuer would be absolutely necessary, so
> we will get your point that sticking to RFCs is a safer choice.
>
> > The patches seem to be out of order now (and the documentation in the
> > commit messages has been removed).
> Feedback taken. Work in progress.
>
> On Tue, Jan 17, 2023 at 2:44 PM Jacob Champion <jchampion@timescale.com> wrote:
> >
> > On Sun, Jan 15, 2023 at 12:03 PM Andrey Chudnovsky
> > <achudnovskij@gmail.com> wrote:
> > > 2. Removed Device Code implementation in libpq. Several reasons:
> > > - Reduce scope and focus on the protocol first.
> > > - Device code implementation uses iddawc dependency. Taking this
> > > dependency is a controversial step which requires broader discussion.
> > > - Device code implementation without iddaws would significantly
> > > increase the scope of the patch, as libpq needs to poll the token
> > > endpoint, setup different API calls, e.t.c.
> > > - That flow should canonically only be used for clients which can't
> > > invoke browsers. If it is the only flow to be implemented, it can be
> > > used in the context when it's not expected by the OAUTH protocol.
> >
> > I'm not understanding the concern in the final point -- providers
> > generally require you to opt into device authorization, at least as far
> > as I can tell. So if you decide that it's not appropriate for your use
> > case... don't enable it. (And I haven't seen any claims that opting into
> > device authorization weakens the other flows in any way. So if we're
> > going to implement a flow in libpq, I still think device authorization
> > is the best choice, since it works on headless machines as well as those
> > with browsers.)
> >
> > All of this points at a bigger question to the community: if we choose
> > not to provide a flow implementation in libpq, is adding OAUTHBEARER
> > worth the additional maintenance cost?
> >
> > My personal vote would be "no". I think the hook-only approach proposed
> > here would ensure that only larger providers would implement it in
> > practice, and in that case I'd rather spend cycles on generic SASL.
> >
> > > 3. Temporarily removed test suite. We are actively working on aligning
> > > the tests with the latest changes. Will add a patch with tests soon.
> >
> > Okay. Case in point, the following change to the patch appears to be
> > invalid JSON:
> >
> > > + appendStringInfo(&buf,
> > > + "{ "
> > > + "\"status\": \"invalid_token\", "
> > > + "\"openid-configuration\": \"%s\","
> > > + "\"scope\": \"%s\" ",
> > > + "\"issuer\": \"%s\" ",
> > > + "}",
> >
> > Additionally, the "issuer" field added here is not part of the RFC. I've
> > written my thoughts about unofficial extensions upthread but haven't
> > received a response, so I'm going to start being more strident: Please,
> > for the sake of reviewers, call out changes you've made to the spec, and
> > why they're justified.
> >
> > The patches seem to be out of order now (and the documentation in the
> > commit messages has been removed).
> >
> > Thanks,
> > --Jacob
Attachments:
v6-0004-common-jsonapi-support-FRONTEND-clients.patch.gz (application/x-gzip, 6.4K) download
v6-0002-backend-add-OAUTHBEARER-SASL-mechanishm.patch.gz (application/x-gzip, 9.5K) download
v6-0003-Add-a-very-simple-oauth_provider-extension.patch.gz (application/x-gzip, 2.0K) download
v6-0001-libpq-add-OAUTHBEARER-SASL-mechanism-and-call-back-hooks.patch.gz (application/x-gzip, 9.1K) download
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-02-20 22:35 Stephen Frost <sfrost@snowman.net>
parent: mahendrakar s <mahendrakarforpg@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: Stephen Frost @ 2023-02-20 22:35 UTC (permalink / raw)
To: mahendrakar s <mahendrakarforpg@gmail.com>; +Cc: Andrey Chudnovsky <achudnovskij@gmail.com>; Jacob Champion <jchampion@timescale.com>; hlinnaka@iki.fi, Michael Paquier <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com
Greetings,
* mahendrakar s (mahendrakarforpg@gmail.com) wrote:
> The "issuer" field has been removed to align with the RFC
> implementation - https://www.rfc-editor.org/rfc/rfc7628.
> This patch "v6" is a single patch to support the OAUTH BEARER token
> through psql connection string.
> Below flow is supported. Added the documentation in the commit messages.
>
> +----------------------+ +----------+
> | +-------+ | Postgres |
> | PQconnect ->| | | |
> | | | | +-----------+
> | | | ---------- Empty Token---------> | > | |
> | | libpq | <-- Error(Discovery + Scope ) -- | < | Pre-Auth |
> | +------+ | | | Hook |
> | +- < | Hook | | | +-----------+
> | | +------+ | | |
> | v | | | |
> | [get token]| | | |
> | | | | | |
> | + | | | +-----------+
> | PQconnect > | | --------- Access Token --------> | > | Validator |
> | | | <---------- Auth Result -------- | < | Hook |
> | | | | +-----------+
> | +-------+ | |
> +----------------------+ +----------+
>
> Please note that we are working on modifying/adding new tests (from
> Jacob's Patch) with the latest changes. Will add a patch with tests
> soon.
Having skimmed back through this thread again, I still feel that the
direction that was originally being taken (actually support something in
libpq and the backend, be it with libiddawc or something else or even
our own code, and not just throw hooks in various places) makes a lot
more sense and is a lot closer to how Kerberos and client-side certs and
even LDAP auth work today. That also seems like a much better answer
for our users when it comes to new authentication methods than having
extensions and making libpq developers have to write their own custom
code, not to mention that we'd still need to implement something in psql
to provide such a hook if we are to have psql actually usefully exercise
this, no?
In the Kerberos test suite we have today, we actually bring up a proper
Kerberos server, set things up, and then test end-to-end installing a
keytab for the server, getting a TGT, getting a service ticket, testing
authentication and encryption, etc. Looking around, it seems like the
equivilant would perhaps be to use Glewlwyd and libiddawc or libcurl and
our own code to really be able to test this and show that it works and
that we're doing it correctly, and to let us know if we break something.
Thanks,
Stephen
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-02-21 22:24 Jacob Champion <jchampion@timescale.com>
parent: Stephen Frost <sfrost@snowman.net>
0 siblings, 2 replies; 424+ messages in thread
From: Jacob Champion @ 2023-02-21 22:24 UTC (permalink / raw)
To: Stephen Frost <sfrost@snowman.net>; +Cc: mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; hlinnaka@iki.fi, Michael Paquier <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com
On Mon, Feb 20, 2023 at 2:35 PM Stephen Frost <sfrost@snowman.net> wrote:
> Having skimmed back through this thread again, I still feel that the
> direction that was originally being taken (actually support something in
> libpq and the backend, be it with libiddawc or something else or even
> our own code, and not just throw hooks in various places) makes a lot
> more sense and is a lot closer to how Kerberos and client-side certs and
> even LDAP auth work today.
Cool, that helps focus the effort. Thanks!
> That also seems like a much better answer
> for our users when it comes to new authentication methods than having
> extensions and making libpq developers have to write their own custom
> code, not to mention that we'd still need to implement something in psql
> to provide such a hook if we are to have psql actually usefully exercise
> this, no?
I don't mind letting clients implement their own flows... as long as
it's optional. So even if we did use a hook in the end, I agree that
we've got to exercise it ourselves.
> In the Kerberos test suite we have today, we actually bring up a proper
> Kerberos server, set things up, and then test end-to-end installing a
> keytab for the server, getting a TGT, getting a service ticket, testing
> authentication and encryption, etc. Looking around, it seems like the
> equivilant would perhaps be to use Glewlwyd and libiddawc or libcurl and
> our own code to really be able to test this and show that it works and
> that we're doing it correctly, and to let us know if we break something.
The original patchset includes a test server in Python -- a major
advantage being that you can test the client and server independently
of each other, since the implementation is so asymmetric. Additionally
testing against something like Glewlwyd would be a great way to stack
coverage. (If we *only* test against a packaged server, though, it'll
be harder to test our stuff in the presence of malfunctions and other
corner cases.)
Thanks,
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-02-22 07:00 Andrey Chudnovsky <achudnovskij@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
1 sibling, 0 replies; 424+ messages in thread
From: Andrey Chudnovsky @ 2023-02-22 07:00 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: Stephen Frost <sfrost@snowman.net>; mahendrakar s <mahendrakarforpg@gmail.com>; hlinnaka@iki.fi, Michael Paquier <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com
Thanks for the feedback,
> Having skimmed back through this thread again, I still feel that the
> direction that was originally being taken (actually support something in
> libpq and the backend, be it with libiddawc or something else or even
> our own code, and not just throw hooks in various places) makes a lot
> more sense and is a lot closer to how Kerberos and client-side certs and
> even LDAP auth work today. That also seems like a much better answer
> for our users when it comes to new authentication methods than having
> extensions and making libpq developers have to write their own custom
> code, not to mention that we'd still need to implement something in psql
> to provide such a hook if we are to have psql actually usefully exercise
> this, no?
libpq implementation is the long term plan. However, our intention is
to start with the protocol implementation which allows us to build on
top of.
While device code is the right solution for psql, having that as the
only one can result in incentive to use it in the cases it's not
intended to.
Reasonably good implementation should support all of the following:
(1.) authorization code with pkce (for GUI applications)
(2.) device code (for console user logins)
(3.) client secret
(4.) some support for client certificate flow
(1.) and (4.) require more work to get implemented, though necessary
for encouraging the most secure grant types.
As we didn't have those pieces, we're proposing starting with the
protocol, which can be used by the ecosystem to build token flow
implementations.
Then add the libpq support for individual grant types.
We originally looked at starting with bare bone protocol for PG16 and
adding libpq support in PG17.
That plan won't happen, though still splitting the work into separate
stages would make more sense in my opinion.
Several questions to follow up:
(a.) Would you support committing the protocol first? or you see libpq
implementation for grants as the prerequisite to consider the auth
type?
(b.) As of today, the server side core does not validate that the
token is actually a valid jwt token. Instead relies on the extensions
to do the validation.
Do you think server core should do the basic validation before passing
to extensions to prevent the auth type being used for anything other
than OAUTH flows?
Tests are the plan for the commit-ready implementation.
Thanks!
Andrey.
On Tue, Feb 21, 2023 at 2:24 PM Jacob Champion <jchampion@timescale.com> wrote:
>
> On Mon, Feb 20, 2023 at 2:35 PM Stephen Frost <sfrost@snowman.net> wrote:
> > Having skimmed back through this thread again, I still feel that the
> > direction that was originally being taken (actually support something in
> > libpq and the backend, be it with libiddawc or something else or even
> > our own code, and not just throw hooks in various places) makes a lot
> > more sense and is a lot closer to how Kerberos and client-side certs and
> > even LDAP auth work today.
>
> Cool, that helps focus the effort. Thanks!
>
> > That also seems like a much better answer
> > for our users when it comes to new authentication methods than having
> > extensions and making libpq developers have to write their own custom
> > code, not to mention that we'd still need to implement something in psql
> > to provide such a hook if we are to have psql actually usefully exercise
> > this, no?
>
> I don't mind letting clients implement their own flows... as long as
> it's optional. So even if we did use a hook in the end, I agree that
> we've got to exercise it ourselves.
>
> > In the Kerberos test suite we have today, we actually bring up a proper
> > Kerberos server, set things up, and then test end-to-end installing a
> > keytab for the server, getting a TGT, getting a service ticket, testing
> > authentication and encryption, etc. Looking around, it seems like the
> > equivilant would perhaps be to use Glewlwyd and libiddawc or libcurl and
> > our own code to really be able to test this and show that it works and
> > that we're doing it correctly, and to let us know if we break something.
>
> The original patchset includes a test server in Python -- a major
> advantage being that you can test the client and server independently
> of each other, since the implementation is so asymmetric. Additionally
> testing against something like Glewlwyd would be a great way to stack
> coverage. (If we *only* test against a packaged server, though, it'll
> be harder to test our stuff in the presence of malfunctions and other
> corner cases.)
>
> Thanks,
> --Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-02-23 18:47 Stephen Frost <sfrost@snowman.net>
parent: Jacob Champion <jchampion@timescale.com>
1 sibling, 1 reply; 424+ messages in thread
From: Stephen Frost @ 2023-02-23 18:47 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; hlinnaka@iki.fi, Michael Paquier <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com
Greetings,
* Jacob Champion (jchampion@timescale.com) wrote:
> On Mon, Feb 20, 2023 at 2:35 PM Stephen Frost <sfrost@snowman.net> wrote:
> > Having skimmed back through this thread again, I still feel that the
> > direction that was originally being taken (actually support something in
> > libpq and the backend, be it with libiddawc or something else or even
> > our own code, and not just throw hooks in various places) makes a lot
> > more sense and is a lot closer to how Kerberos and client-side certs and
> > even LDAP auth work today.
>
> Cool, that helps focus the effort. Thanks!
Great, glad to hear that.
> > That also seems like a much better answer
> > for our users when it comes to new authentication methods than having
> > extensions and making libpq developers have to write their own custom
> > code, not to mention that we'd still need to implement something in psql
> > to provide such a hook if we are to have psql actually usefully exercise
> > this, no?
>
> I don't mind letting clients implement their own flows... as long as
> it's optional. So even if we did use a hook in the end, I agree that
> we've got to exercise it ourselves.
This really doesn't feel like a great area to try and do hooks or
similar in, not the least because that approach has been tried and tried
again (PAM, GSSAPI, SASL would all be examples..) and frankly none of
them has turned out great (which is why we can't just tell people "well,
install the pam_oauth2 and watch everything work!") and this strikes me
as trying to do that yet again but worse as it's not even a dedicated
project trying to solve the problem but more like a side project. SCRAM
was good, we've come a long way thanks to that, this feels like it
should be more in line with that rather than trying to invent yet
another new "generic" set of hooks/APIs that will just cause DBAs and
our users headaches trying to make work.
> > In the Kerberos test suite we have today, we actually bring up a proper
> > Kerberos server, set things up, and then test end-to-end installing a
> > keytab for the server, getting a TGT, getting a service ticket, testing
> > authentication and encryption, etc. Looking around, it seems like the
> > equivilant would perhaps be to use Glewlwyd and libiddawc or libcurl and
> > our own code to really be able to test this and show that it works and
> > that we're doing it correctly, and to let us know if we break something.
>
> The original patchset includes a test server in Python -- a major
> advantage being that you can test the client and server independently
> of each other, since the implementation is so asymmetric. Additionally
> testing against something like Glewlwyd would be a great way to stack
> coverage. (If we *only* test against a packaged server, though, it'll
> be harder to test our stuff in the presence of malfunctions and other
> corner cases.)
Oh, that's even better- I agree entirely that having test code that can
be instructed to return specific errors so that we can test that our
code responds properly is great (and is why pgbackrest has things like
a stub'd out libpq, fake s3, GCS, and Azure servers, and more) and would
certainly want to keep that, even if we also build out a test that uses
a real server to provide integration testing with not-our-code too.
Thanks!
Stephen
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-02-23 23:04 Andrey Chudnovsky <achudnovskij@gmail.com>
parent: Stephen Frost <sfrost@snowman.net>
0 siblings, 1 reply; 424+ messages in thread
From: Andrey Chudnovsky @ 2023-02-23 23:04 UTC (permalink / raw)
To: Stephen Frost <sfrost@snowman.net>; +Cc: Jacob Champion <jchampion@timescale.com>; mahendrakar s <mahendrakarforpg@gmail.com>; hlinnaka@iki.fi, Michael Paquier <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com
> This really doesn't feel like a great area to try and do hooks or
> similar in, not the least because that approach has been tried and tried
> again (PAM, GSSAPI, SASL would all be examples..) and frankly none of
> them has turned out great (which is why we can't just tell people "well,
> install the pam_oauth2 and watch everything work!") and this strikes me
> as trying to do that yet again but worse as it's not even a dedicated
> project trying to solve the problem but more like a side project.
In this case it's not intended to be an open-ended hook, but rather an
implementation of a specific rfc (rfc-7628) which defines a
client-server communication for the authentication flow.
The rfc itself does leave a lot of flexibility on specific parts of
the implementation. Which do require hooks:
(1.) Server side hook to validate the token, which is specific to the
OAUTH provider.
(2.) Client side hook to request the client to obtain the token.
On (1.), we would need a hook for the OAUTH provider extension to do
validation. We can though do some basic check that the credential is
indeed a JWT token signed by the requested issuer.
Specifically (2.) is where we can provide a layer in libpq to simplify
the integration. i.e. implement some OAUTH flows.
Though we would need some flexibility for the clients to bring their own token:
For example there are cases where the credential to obtain the token
is stored in a separate secure location and the token is returned from
a separate service or pushed from a more secure environment.
> another new "generic" set of hooks/APIs that will just cause DBAs and
> our users headaches trying to make work.
As I mentioned above, it's an rfc implementation, rather than our invention.
When it comes to DBAs and the users.
Builtin libpq implementations which allows psql and pgadmin to
seamlessly connect should suffice those needs.
While extensibility would allow the ecosystem to be open for OAUTH
providers, SAAS developers, PAAS providers and other institutional
players.
Thanks!
Andrey.
On Thu, Feb 23, 2023 at 10:47 AM Stephen Frost <sfrost@snowman.net> wrote:
>
> Greetings,
>
> * Jacob Champion (jchampion@timescale.com) wrote:
> > On Mon, Feb 20, 2023 at 2:35 PM Stephen Frost <sfrost@snowman.net> wrote:
> > > Having skimmed back through this thread again, I still feel that the
> > > direction that was originally being taken (actually support something in
> > > libpq and the backend, be it with libiddawc or something else or even
> > > our own code, and not just throw hooks in various places) makes a lot
> > > more sense and is a lot closer to how Kerberos and client-side certs and
> > > even LDAP auth work today.
> >
> > Cool, that helps focus the effort. Thanks!
>
> Great, glad to hear that.
>
> > > That also seems like a much better answer
> > > for our users when it comes to new authentication methods than having
> > > extensions and making libpq developers have to write their own custom
> > > code, not to mention that we'd still need to implement something in psql
> > > to provide such a hook if we are to have psql actually usefully exercise
> > > this, no?
> >
> > I don't mind letting clients implement their own flows... as long as
> > it's optional. So even if we did use a hook in the end, I agree that
> > we've got to exercise it ourselves.
>
> This really doesn't feel like a great area to try and do hooks or
> similar in, not the least because that approach has been tried and tried
> again (PAM, GSSAPI, SASL would all be examples..) and frankly none of
> them has turned out great (which is why we can't just tell people "well,
> install the pam_oauth2 and watch everything work!") and this strikes me
> as trying to do that yet again but worse as it's not even a dedicated
> project trying to solve the problem but more like a side project. SCRAM
> was good, we've come a long way thanks to that, this feels like it
> should be more in line with that rather than trying to invent yet
> another new "generic" set of hooks/APIs that will just cause DBAs and
> our users headaches trying to make work.
>
> > > In the Kerberos test suite we have today, we actually bring up a proper
> > > Kerberos server, set things up, and then test end-to-end installing a
> > > keytab for the server, getting a TGT, getting a service ticket, testing
> > > authentication and encryption, etc. Looking around, it seems like the
> > > equivilant would perhaps be to use Glewlwyd and libiddawc or libcurl and
> > > our own code to really be able to test this and show that it works and
> > > that we're doing it correctly, and to let us know if we break something.
> >
> > The original patchset includes a test server in Python -- a major
> > advantage being that you can test the client and server independently
> > of each other, since the implementation is so asymmetric. Additionally
> > testing against something like Glewlwyd would be a great way to stack
> > coverage. (If we *only* test against a packaged server, though, it'll
> > be harder to test our stuff in the presence of malfunctions and other
> > corner cases.)
>
> Oh, that's even better- I agree entirely that having test code that can
> be instructed to return specific errors so that we can test that our
> code responds properly is great (and is why pgbackrest has things like
> a stub'd out libpq, fake s3, GCS, and Azure servers, and more) and would
> certainly want to keep that, even if we also build out a test that uses
> a real server to provide integration testing with not-our-code too.
>
> Thanks!
>
> Stephen
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-02-27 20:31 Stephen Frost <sfrost@snowman.net>
parent: Andrey Chudnovsky <achudnovskij@gmail.com>
0 siblings, 0 replies; 424+ messages in thread
From: Stephen Frost @ 2023-02-27 20:31 UTC (permalink / raw)
To: Andrey Chudnovsky <achudnovskij@gmail.com>; +Cc: Jacob Champion <jchampion@timescale.com>; mahendrakar s <mahendrakarforpg@gmail.com>; hlinnaka@iki.fi, Michael Paquier <michael@paquier.xyz>; pgsql-hackers; smilingsamay@gmail.com
Greetings,
* Andrey Chudnovsky (achudnovskij@gmail.com) wrote:
> > This really doesn't feel like a great area to try and do hooks or
> > similar in, not the least because that approach has been tried and tried
> > again (PAM, GSSAPI, SASL would all be examples..) and frankly none of
> > them has turned out great (which is why we can't just tell people "well,
> > install the pam_oauth2 and watch everything work!") and this strikes me
> > as trying to do that yet again but worse as it's not even a dedicated
> > project trying to solve the problem but more like a side project.
>
> In this case it's not intended to be an open-ended hook, but rather an
> implementation of a specific rfc (rfc-7628) which defines a
> client-server communication for the authentication flow.
> The rfc itself does leave a lot of flexibility on specific parts of
> the implementation. Which do require hooks:
Color me skeptical on an RFC that requires hooks.
> (1.) Server side hook to validate the token, which is specific to the
> OAUTH provider.
> (2.) Client side hook to request the client to obtain the token.
Perhaps I'm missing it... but weren't these handled with what the
original patch that Jacob had was doing?
> On (1.), we would need a hook for the OAUTH provider extension to do
> validation. We can though do some basic check that the credential is
> indeed a JWT token signed by the requested issuer.
>
> Specifically (2.) is where we can provide a layer in libpq to simplify
> the integration. i.e. implement some OAUTH flows.
> Though we would need some flexibility for the clients to bring their own token:
> For example there are cases where the credential to obtain the token
> is stored in a separate secure location and the token is returned from
> a separate service or pushed from a more secure environment.
In those cases... we could, if we wanted, simply implement the code to
actually pull the token, no? We don't *have* to have a hook here for
this, we could just make it work.
> > another new "generic" set of hooks/APIs that will just cause DBAs and
> > our users headaches trying to make work.
> As I mentioned above, it's an rfc implementation, rather than our invention.
While I only took a quick look, I didn't see anything in that RFC that
explicitly says that hooks or a plugin or a library or such is required
to meet the RFC. Sure, there are places which say that the
implementation is specific to a particular server or client but that's
not the same thing.
> When it comes to DBAs and the users.
> Builtin libpq implementations which allows psql and pgadmin to
> seamlessly connect should suffice those needs.
> While extensibility would allow the ecosystem to be open for OAUTH
> providers, SAAS developers, PAAS providers and other institutional
> players.
Each to end up writing their own code to do largely the same thing
without the benefit of the larger community to be able to review and
ensure that it's done properly?
That doesn't sound like a great approach to me.
Thanks,
Stephen
Attachments:
signature.asc (application/pgp-signature, 832B) download
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-04-27 17:35 Jacob Champion <jchampion@timescale.com>
parent: Jacob Champion <jchampion@timescale.com>
1 sibling, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2023-04-27 17:35 UTC (permalink / raw)
To: pgsql-hackers; +Cc: hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>
On Fri, Sep 23, 2022 at 3:39 PM Jacob Champion <jchampion@timescale.com> wrote:
> Here's a newly rebased v5. (They're all zipped now, which I probably
> should have done a while back, sorry.)
To keep this current, v7 is rebased over latest, without the pluggable
authentication patches. This doesn't yet address the architectural
feedback that was discussed previously, so if you're primarily
interested in that, you can safely ignore this version of the
patchset.
The key changes here include
- Meson support, for both the build and the pytest suite
- Cirrus support (and unsurprisingly, Mac and Windows builds fail due
to the Linux-oriented draft code)
- A small tweak to support iddawc down to 0.9.8 (shipped with e.g.
Debian Bullseye)
- Removal of the authn_id test extension in favor of SYSTEM_USER
The meson+pytest support was big enough that I split it into its own
patch. It's not very polished yet, but it mostly works, and when
running tests via Meson it'll now spin up a test server for you. My
virtualenv approach apparently interacts poorly with the multiarch
Cirrus setup (64-bit tests pass, 32-bit tests fail).
Moving forward, the first thing I plan to tackle is asynchronous
operation, so that polling clients can still operate sanely. If I can
find a good solution there, the conversations about possible extension
points should get a lot easier.
Thanks,
--Jacob
Attachments:
since-v5.diff.txt (text/plain, 33.8K) download
1: a94a11d56c < -: ---------- Add support for custom authentication methods
2: 973f622fea < -: ---------- Add sample extension to test custom auth provider hooks
3: b4a0ab5a4e < -: ---------- Add tests for test_auth_provider extension
4: 49715c3c98 < -: ---------- Add support for "map" and custom auth options
5: 2b2e8d3050 ! 1: c3698bbc3d common/jsonapi: support FRONTEND clients
@@ src/common/jsonapi.c: parse_object(JsonLexContext *lex, JsonSemAction *sem)
#endif
@@ src/common/jsonapi.c: json_lex_string(JsonLexContext *lex)
- char *const end = lex->input + lex->input_length;
- int hi_surrogate = -1;
+ return code; \
+ } while (0)
- if (lex->strval != NULL)
- resetStringInfo(lex->strval);
@@ src/common/jsonapi.c: json_lex_string(JsonLexContext *lex)
Assert(lex->input_length > 0);
s = lex->token_start;
@@ src/common/jsonapi.c: json_lex_string(JsonLexContext *lex)
- return JSON_UNICODE_ESCAPE_FORMAT;
- }
+ else
+ FAIL_AT_CHAR_END(JSON_UNICODE_ESCAPE_FORMAT);
}
- if (lex->strval != NULL)
+ if (lex->parse_strval)
@@ src/common/jsonapi.c: json_lex_string(JsonLexContext *lex)
+ appendPQExpBufferChar(lex->strval, (char) ch);
}
else
- return JSON_UNICODE_HIGH_ESCAPE;
+ FAIL_AT_CHAR_END(JSON_UNICODE_HIGH_ESCAPE);
#endif /* FRONTEND */
}
}
@@ src/common/jsonapi.c: json_lex_string(JsonLexContext *lex)
+ else if (lex->parse_strval)
{
if (hi_surrogate != -1)
- return JSON_UNICODE_LOW_SURROGATE;
+ FAIL_AT_CHAR_END(JSON_UNICODE_LOW_SURROGATE);
@@ src/common/jsonapi.c: json_lex_string(JsonLexContext *lex)
case '"':
case '\\':
@@ src/common/jsonapi.c: json_lex_string(JsonLexContext *lex)
+ appendStrValChar(lex->strval, '\t');
break;
default:
- /* Not a valid string escape, so signal error. */
+
@@ src/common/jsonapi.c: json_lex_string(JsonLexContext *lex)
/*
@@ src/common/jsonapi.c: json_lex_string(JsonLexContext *lex)
/*
* s will be incremented at the top of the loop, so set it to just
@@ src/common/jsonapi.c: json_lex_string(JsonLexContext *lex)
- if (hi_surrogate != -1)
return JSON_UNICODE_LOW_SURROGATE;
+ }
+#ifdef FRONTEND
+ if (lex->parse_strval && PQExpBufferBroken(lex->strval))
@@ src/common/jsonapi.c: report_parse_error(JsonParseContext ctx, JsonLexContext *l
-}
-
/*
- * Construct a detail message for a JSON error.
+ * Construct an (already translated) detail message for a JSON error.
*
- * Note that the error message generated by this routine may not be
- * palloc'd, making it unsafe for frontend code as there is no way to
-- * know if this can be safery pfree'd or not.
+- * know if this can be safely pfree'd or not.
+ * The returned allocation is either static or owned by the JsonLexContext and
+ * should not be freed.
*/
@@ src/common/jsonapi.c: report_parse_error(JsonParseContext ctx, JsonLexContext *l
return _("\\u0000 cannot be converted to text.");
case JSON_UNICODE_ESCAPE_FORMAT:
@@ src/common/jsonapi.c: json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
- return _("Unicode low surrogate must follow a high surrogate.");
+ /* note: this case is only reachable in frontend not backend */
+ return _("Unicode escape values cannot be used for code point values above 007F when the encoding is not UTF8.");
+ case JSON_UNICODE_UNTRANSLATABLE:
+- /* note: this case is only reachable in backend not frontend */
++ /*
++ * note: this case is only reachable in backend not frontend.
++ * #ifdef it away so the frontend doesn't try to link against
++ * backend functionality.
++ */
++#ifndef FRONTEND
+ return psprintf(_("Unicode escape value could not be translated to the server's encoding %s."),
+ GetDatabaseEncodingName());
++#else
++ Assert(false);
++ break;
++#endif
+ case JSON_UNICODE_HIGH_SURROGATE:
+ return _("Unicode high surrogate must not follow a high surrogate.");
+ case JSON_UNICODE_LOW_SURROGATE:
+@@ src/common/jsonapi.c: json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
+ break;
}
- /*
@@ src/include/common/jsonapi.h
-#include "lib/stringinfo.h"
-
- typedef enum
+ typedef enum JsonTokenType
{
JSON_TOKEN_INVALID,
-@@ src/include/common/jsonapi.h: typedef enum
+@@ src/include/common/jsonapi.h: typedef enum JsonParseErrorType
JSON_EXPECTED_OBJECT_NEXT,
JSON_EXPECTED_STRING,
JSON_INVALID_TOKEN,
@@ src/include/common/jsonapi.h: typedef enum
JSON_UNICODE_CODE_POINT_ZERO,
JSON_UNICODE_ESCAPE_FORMAT,
JSON_UNICODE_HIGH_ESCAPE,
-@@ src/include/common/jsonapi.h: typedef enum
- JSON_UNICODE_LOW_SURROGATE
+@@ src/include/common/jsonapi.h: typedef enum JsonParseErrorType
+ JSON_SEM_ACTION_FAILED /* error should already be reported */
} JsonParseErrorType;
+/*
@@ src/include/common/jsonapi.h: typedef struct JsonLexContext
+ StrValType *errormsg;
} JsonLexContext;
- typedef void (*json_struct_action) (void *state);
+ typedef JsonParseErrorType (*json_struct_action) (void *state);
@@ src/include/common/jsonapi.h: extern PGDLLIMPORT JsonSemAction nullSemAction;
*/
extern JsonParseErrorType json_count_array_elements(JsonLexContext *lex,
6: c18d7da6cc ! 2: 0cd726fd55 libpq: add OAUTHBEARER SASL mechanism
@@ Commit message
- handle cases where the client has been set up with an issuer and
scope, but the Postgres server wants to use something different
- improve error debuggability during the OAuth handshake
+ - migrate JSON parsing to the new JSON_SEM_ACTION_FAILED API convention
- ...and more.
## configure ##
@@ configure: fi
+ as_fn_error $? "library 'iddawc' is required for OAuth support" "$LINENO" 5
+fi
+
++ # Check for an older spelling of i_get_openid_config
++ for ac_func in i_load_openid_config
++do :
++ ac_fn_c_check_func "$LINENO" "i_load_openid_config" "ac_cv_func_i_load_openid_config"
++if test "x$ac_cv_func_i_load_openid_config" = xyes; then :
++ cat >>confdefs.h <<_ACEOF
++#define HAVE_I_LOAD_OPENID_CONFIG 1
++_ACEOF
++
++fi
++done
++
+fi
+
# for contrib/sepgsql
@@ configure.ac: fi
+if test "$with_oauth" = yes ; then
+ AC_CHECK_LIB(iddawc, i_init_session, [], [AC_MSG_ERROR([library 'iddawc' is required for OAuth support])])
++ # Check for an older spelling of i_get_openid_config
++ AC_CHECK_FUNCS([i_load_openid_config])
+fi
+
# for contrib/sepgsql
@@ configure.ac: elif test "$with_uuid" = ossp ; then
AC_CHECK_HEADERS(crtdefs.h)
fi
+ ## meson.build ##
+@@ meson.build: endif
+
+
+
++###############################################################
++# Library: oauth
++###############################################################
++
++oauthopt = get_option('oauth')
++if not oauthopt.disabled()
++ oauth = dependency('libiddawc', required: oauthopt)
++
++ if oauth.found()
++ cdata.set('USE_OAUTH', 1)
++ # Check for an older spelling of i_get_openid_config
++ if cc.has_function('i_load_openid_config',
++ dependencies: oauth, args: test_c_args)
++ cdata.set('HAVE_I_LOAD_OPENID_CONFIG', 1)
++ endif
++ endif
++else
++ oauth = not_found_dep
++endif
++
++
+ ###############################################################
+ # Library: Tcl (for pltcl)
+ #
+@@ meson.build: libpq_deps += [
+ gssapi,
+ ldap_r,
+ libintl,
++ oauth,
+ ssl,
+ ]
+
+@@ meson.build: if meson.version().version_compare('>=0.57')
+ 'llvm': llvm,
+ 'lz4': lz4,
+ 'nls': libintl,
++ 'oauth': oauth,
+ 'openssl': ssl,
+ 'pam': pam,
+ 'plperl': perl_dep,
+
+ ## meson_options.txt ##
+@@ meson_options.txt: option('lz4', type : 'feature', value: 'auto',
+ option('nls', type: 'feature', value: 'auto',
+ description: 'native language support')
+
++option('oauth', type : 'feature', value: 'auto',
++ description: 'OAuth 2.0 support')
++
+ option('pam', type : 'feature', value: 'auto',
+ description: 'build with PAM support')
+
+
## src/Makefile.global.in ##
@@ src/Makefile.global.in: with_ldap = @with_ldap@
with_libxml = @with_libxml@
@@ src/Makefile.global.in: with_ldap = @with_ldap@
with_uuid = @with_uuid@
with_zlib = @with_zlib@
+ ## src/common/meson.build ##
+@@ src/common/meson.build: common_sources_frontend_static += files(
+ # For the server build of pgcommon, depend on lwlocknames_h, because at least
+ # cryptohash_openssl.c, hmac_openssl.c depend on it. That's arguably a
+ # layering violation, but ...
++#
++# XXX Frontend builds need libpq's pqexpbuffer.h, so adjust the include paths
++# appropriately. This seems completely broken.
+ pgcommon = {}
+ pgcommon_variants = {
+ '_srv': internal_lib_args + {
++ 'include_directories': include_directories('.'),
+ 'sources': common_sources + [lwlocknames_h],
+ 'dependencies': [backend_common_code],
+ },
+ '': default_lib_args + {
++ 'include_directories': include_directories('../interfaces/libpq', '.'),
+ 'sources': common_sources_frontend_static,
+ 'dependencies': [frontend_common_code],
+ },
+ '_shlib': default_lib_args + {
+ 'pic': true,
++ 'include_directories': include_directories('../interfaces/libpq', '.'),
+ 'sources': common_sources_frontend_shlib,
+ 'dependencies': [frontend_common_code],
+ },
+@@ src/common/meson.build: foreach name, opts : pgcommon_variants
+ c_args = opts.get('c_args', []) + common_cflags[cflagname]
+ cflag_libs += static_library('libpgcommon@0@_@1@'.format(name, cflagname),
+ c_pch: pch_c_h,
+- include_directories: include_directories('.'),
+ kwargs: opts + {
+ 'sources': sources,
+ 'c_args': c_args,
+@@ src/common/meson.build: foreach name, opts : pgcommon_variants
+ lib = static_library('libpgcommon@0@'.format(name),
+ link_with: cflag_libs,
+ c_pch: pch_c_h,
+- include_directories: include_directories('.'),
+ kwargs: opts + {
+ 'dependencies': opts['dependencies'] + [ssl],
+ }
+
## src/include/common/oauth-common.h (new) ##
@@
+/*-------------------------------------------------------------------------
@@ src/include/common/oauth-common.h (new)
+#endif /* OAUTH_COMMON_H */
## src/include/pg_config.h.in ##
+@@
+ /* Define to 1 if __builtin_constant_p(x) implies "i"(x) acceptance. */
+ #undef HAVE_I_CONSTRAINT__BUILTIN_CONSTANT_P
+
++/* Define to 1 if you have the `i_load_openid_config' function. */
++#undef HAVE_I_LOAD_OPENID_CONFIG
++
+ /* Define to 1 if you have the `kqueue' function. */
+ #undef HAVE_KQUEUE
+
@@
/* Define to 1 if you have the `crypto' library (-lcrypto). */
#undef HAVE_LIBCRYPTO
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+#include "fe-auth.h"
+#include "mb/pg_wchar.h"
+
++#ifdef HAVE_I_LOAD_OPENID_CONFIG
++/* Older versions of iddawc used 'load' instead of 'get' for some APIs. */
++#define i_get_openid_config i_load_openid_config
++#endif
++
+/* The exported OAuth callback mechanism. */
+static void *oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism);
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+ (ctx)->errmsg = (ctx)->errbuf.data; \
+ } while (0)
+
-+static void
++static JsonParseErrorType
+oauth_json_object_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (oauth_json_has_error(ctx))
-+ return; /* short-circuit */
++ return JSON_SUCCESS; /* short-circuit */
+
+ if (ctx->target_field)
+ {
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+ }
+
+ ++ctx->nested;
++ return JSON_SUCCESS; /* TODO: switch all of these to JSON_SEM_ACTION_FAILED */
+}
+
-+static void
++static JsonParseErrorType
+oauth_json_object_end(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (oauth_json_has_error(ctx))
-+ return; /* short-circuit */
++ return JSON_SUCCESS; /* short-circuit */
+
+ --ctx->nested;
++ return JSON_SUCCESS;
+}
+
-+static void
++static JsonParseErrorType
+oauth_json_object_field_start(void *state, char *name, bool isnull)
+{
+ struct json_ctx *ctx = state;
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+ {
+ /* short-circuit */
+ free(name);
-+ return;
++ return JSON_SUCCESS;
+ }
+
+ if (ctx->nested == 1)
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+ }
+
+ free(name);
++ return JSON_SUCCESS;
+}
+
-+static void
++static JsonParseErrorType
+oauth_json_array_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (oauth_json_has_error(ctx))
-+ return; /* short-circuit */
++ return JSON_SUCCESS; /* short-circuit */
+
+ if (!ctx->nested)
+ {
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
++
++ return JSON_SUCCESS;
+}
+
-+static void
++static JsonParseErrorType
+oauth_json_scalar(void *state, char *token, JsonTokenType type)
+{
+ struct json_ctx *ctx = state;
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+ {
+ /* short-circuit */
+ free(token);
-+ return;
++ return JSON_SUCCESS;
+ }
+
+ if (!ctx->nested)
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+ ctx->target_field = NULL;
+ ctx->target_field_name = NULL;
+
-+ return; /* don't free the token we're using */
++ return JSON_SUCCESS; /* don't free the token we're using */
+ }
+
+ oauth_json_set_error(ctx,
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+ }
+
+ free(token);
++ return JSON_SUCCESS;
+}
+
+static bool
@@ src/interfaces/libpq/fe-auth.c: pg_SASL_continue(PGconn *conn, int payloadlen, b
&done, &success);
## src/interfaces/libpq/fe-auth.h ##
-@@ src/interfaces/libpq/fe-auth.h: extern const pg_fe_sasl_mech pg_scram_mech;
- extern char *pg_fe_scram_build_secret(const char *password,
+@@ src/interfaces/libpq/fe-auth.h: extern char *pg_fe_scram_build_secret(const char *password,
+ int iterations,
const char **errstr);
+/* Mechanisms in fe-auth-oauth.c */
@@ src/interfaces/libpq/fe-auth.h: extern const pg_fe_sasl_mech pg_scram_mech;
## src/interfaces/libpq/fe-connect.c ##
@@ src/interfaces/libpq/fe-connect.c: static const internalPQconninfoOption PQconninfoOptions[] = {
- "Target-Session-Attrs", "", 15, /* sizeof("prefer-standby") = 15 */
- offsetof(struct pg_conn, target_session_attrs)},
+ "Load-Balance-Hosts", "", 8, /* sizeof("disable") = 8 */
+ offsetof(struct pg_conn, load_balance_hosts)},
+ /* OAuth v2 */
+ {"oauth_issuer", NULL, NULL, NULL,
@@ src/interfaces/libpq/fe-connect.c: keep_going: /* We will come back to here
/*
@@ src/interfaces/libpq/fe-connect.c: freePGconn(PGconn *conn)
- free(conn->outBuffer);
free(conn->rowBuf);
free(conn->target_session_attrs);
+ free(conn->load_balance_hosts);
+ free(conn->oauth_issuer);
+ free(conn->oauth_discovery_uri);
+ free(conn->oauth_client_id);
@@ src/interfaces/libpq/fe-connect.c: freePGconn(PGconn *conn)
## src/interfaces/libpq/libpq-int.h ##
@@ src/interfaces/libpq/libpq-int.h: struct pg_conn
- char *ssl_max_protocol_version; /* maximum TLS protocol version */
- char *target_session_attrs; /* desired session properties */
+ char *require_auth; /* name of the expected auth method */
+ char *load_balance_hosts; /* load balance over hosts */
+ /* OAuth v2 */
+ char *oauth_issuer; /* token issuer URL */
@@ src/interfaces/libpq/libpq-int.h: struct pg_conn
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
+
+ ## src/interfaces/libpq/meson.build ##
+@@ src/interfaces/libpq/meson.build: if gssapi.found()
+ )
+ endif
+
++if oauth.found()
++ libpq_sources += files('fe-auth-oauth.c')
++endif
++
+ export_file = custom_target('libpq.exports',
+ kwargs: gen_export_kwargs,
+ )
+
+ ## src/makefiles/meson.build ##
+@@ src/makefiles/meson.build: pgxs_deps = {
+ 'llvm': llvm,
+ 'lz4': lz4,
+ 'nls': libintl,
++ 'oauth': oauth,
+ 'pam': pam,
+ 'perl': perl_dep,
+ 'python': python3_dep,
7: f6a81f50f2 ! 3: 77889eb986 backend: add OAUTHBEARER SASL mechanism
@@ src/backend/libpq/auth.c
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
#include "libpq/scram.h"
+@@
+ */
+ static void auth_failed(Port *port, int status, const char *logdetail);
+ static char *recv_password_packet(Port *port);
+-static void set_authn_id(Port *port, const char *id);
+
+
+ /*----------------------------------------------------------------
+@@ src/backend/libpq/auth.c: static int CheckRADIUSAuth(Port *port);
+ static int PerformRadiusTransaction(const char *server, const char *secret, const char *portstr, const char *identifier, const char *user_name, const char *passwd);
+
+
+-/*
+- * Maximum accepted size of GSS and SSPI authentication tokens.
+- * We also use this as a limit on ordinary password packet lengths.
+- *
+- * Kerberos tickets are usually quite small, but the TGTs issued by Windows
+- * domain controllers include an authorization field known as the Privilege
+- * Attribute Certificate (PAC), which contains the user's Windows permissions
+- * (group memberships etc.). The PAC is copied into all tickets obtained on
+- * the basis of this TGT (even those issued by Unix realms which the Windows
+- * realm trusts), and can be several kB in size. The maximum token size
+- * accepted by Windows systems is determined by the MaxAuthToken Windows
+- * registry setting. Microsoft recommends that it is not set higher than
+- * 65535 bytes, so that seems like a reasonable limit for us as well.
+- */
+-#define PG_MAX_AUTH_TOKEN_LENGTH 65535
+-
+ /*----------------------------------------------------------------
+ * Global authentication functions
+ *----------------------------------------------------------------
@@ src/backend/libpq/auth.c: auth_failed(Port *port, int status, const char *logdetail)
case uaRADIUS:
errstr = gettext_noop("RADIUS authentication failed for user \"%s\"");
@@ src/backend/libpq/auth.c: auth_failed(Port *port, int status, const char *logdet
+ case uaOAuth:
+ errstr = gettext_noop("OAuth bearer authentication failed for user \"%s\"");
+ break;
- case uaCustom:
- {
- CustomAuthProvider *provider = get_provider_by_name(port->hba->custom_provider);
+ default:
+ errstr = gettext_noop("authentication failed for user \"%s\": invalid authentication method");
+ break;
+@@ src/backend/libpq/auth.c: auth_failed(Port *port, int status, const char *logdetail)
+ * lifetime of MyClientConnectionInfo, so it is safe to pass a string that is
+ * managed by an external library.
+ */
+-static void
++void
+ set_authn_id(Port *port, const char *id)
+ {
+ Assert(id);
@@ src/backend/libpq/auth.c: ClientAuthentication(Port *port)
case uaTrust:
status = STATUS_OK;
@@ src/backend/libpq/auth.c: ClientAuthentication(Port *port)
+ case uaOAuth:
+ status = CheckSASLAuth(&pg_be_oauth_mech, port, NULL, NULL);
+ break;
- case uaCustom:
- {
- CustomAuthProvider *provider = get_provider_by_name(port->hba->custom_provider);
+ }
+
+ if ((status == STATUS_OK && port->hba->clientcert == clientCertFull)
## src/backend/libpq/hba.c ##
@@ src/backend/libpq/hba.c: static const char *const UserAuthName[] =
+ "ldap",
"cert",
"radius",
- "custom",
- "peer"
+ "peer",
+ "oauth",
};
-
+ /*
@@ src/backend/libpq/hba.c: parse_hba_line(TokenizedAuthLine *tok_line, int elevel)
#endif
else if (strcmp(token->string, "radius") == 0)
parsedline->auth_method = uaRADIUS;
+ else if (strcmp(token->string, "oauth") == 0)
+ parsedline->auth_method = uaOAuth;
- else if (strcmp(token->string, "custom") == 0)
- parsedline->auth_method = uaCustom;
else
+ {
+ ereport(elevel,
@@ src/backend/libpq/hba.c: parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
+ hbaline->auth_method != uaPeer &&
hbaline->auth_method != uaGSS &&
hbaline->auth_method != uaSSPI &&
- hbaline->auth_method != uaCert &&
-+ hbaline->auth_method != uaOAuth &&
- hbaline->auth_method != uaCustom)
-- INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, cert and custom"));
-+ INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, cert, oauth, and custom"));
+- hbaline->auth_method != uaCert)
+- INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, and cert"));
++ hbaline->auth_method != uaCert &&
++ hbaline->auth_method != uaOAuth)
++ INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, cert, and oauth"));
hbaline->usermap = pstrdup(val);
}
else if (strcmp(name, "clientcert") == 0)
@@ src/backend/libpq/hba.c: parse_hba_auth_opt(char *name, char *val, HbaLine *hbal
+ else
+ hbaline->oauth_skip_usermap = false;
+ }
- else if (strcmp(name, "provider") == 0)
+ else
{
- REQUIRE_AUTH_OPTION(uaCustom, "provider", "custom");
+ ereport(elevel,
+
+ ## src/backend/libpq/meson.build ##
+@@
+ # Copyright (c) 2022-2023, PostgreSQL Global Development Group
+
+ backend_sources += files(
++ 'auth-oauth.c',
+ 'auth-sasl.c',
+ 'auth-scram.c',
+ 'auth.c',
## src/backend/utils/misc/guc_tables.c ##
@@
@@ src/backend/utils/misc/guc_tables.c
#include "libpq/auth.h"
#include "libpq/libpq.h"
+#include "libpq/oauth.h"
+ #include "libpq/scram.h"
+ #include "nodes/queryjumble.h"
#include "optimizer/cost.h"
- #include "optimizer/geqo.h"
- #include "optimizer/optimizer.h"
@@ src/backend/utils/misc/guc_tables.c: struct config_string ConfigureNamesString[] =
- check_backtrace_functions, assign_backtrace_functions, NULL
+ check_io_direct, assign_io_direct, NULL
},
+ {
+ {"oauth_validator_command", PGC_SIGHUP, CONN_AUTH_AUTH,
+ gettext_noop("Command to validate OAuth v2 bearer tokens."),
+ NULL,
-+ GUC_SUPERUSER_ONLY
++ GUC_SUPERUSER_ONLY | GUC_NOT_IN_SAMPLE
+ },
+ &oauth_validator_command,
+ "",
@@ src/backend/utils/misc/guc_tables.c: struct config_string ConfigureNamesString[]
{
{NULL, 0, 0, NULL, NULL}, NULL, NULL, NULL, NULL, NULL
+ ## src/include/libpq/auth.h ##
+@@
+
+ #include "libpq/libpq-be.h"
+
++/*
++ * Maximum accepted size of GSS and SSPI authentication tokens.
++ * We also use this as a limit on ordinary password packet lengths.
++ *
++ * Kerberos tickets are usually quite small, but the TGTs issued by Windows
++ * domain controllers include an authorization field known as the Privilege
++ * Attribute Certificate (PAC), which contains the user's Windows permissions
++ * (group memberships etc.). The PAC is copied into all tickets obtained on
++ * the basis of this TGT (even those issued by Unix realms which the Windows
++ * realm trusts), and can be several kB in size. The maximum token size
++ * accepted by Windows systems is determined by the MaxAuthToken Windows
++ * registry setting. Microsoft recommends that it is not set higher than
++ * 65535 bytes, so that seems like a reasonable limit for us as well.
++ */
++#define PG_MAX_AUTH_TOKEN_LENGTH 65535
++
+ extern PGDLLIMPORT char *pg_krb_server_keyfile;
+ extern PGDLLIMPORT bool pg_krb_caseins_users;
+ extern PGDLLIMPORT bool pg_gss_accept_deleg;
+@@ src/include/libpq/auth.h: extern PGDLLIMPORT char *pg_krb_realm;
+ extern void ClientAuthentication(Port *port);
+ extern void sendAuthRequest(Port *port, AuthRequest areq, const char *extradata,
+ int extralen);
++extern void set_authn_id(Port *port, const char *id);
+
+ /* Hook for plugins to get control in ClientAuthentication() */
+ typedef void (*ClientAuthentication_hook_type) (Port *, int);
+
## src/include/libpq/hba.h ##
@@ src/include/libpq/hba.h: typedef enum UserAuth
+ uaLDAP,
uaCert,
uaRADIUS,
- uaCustom,
- uaPeer
-#define USER_AUTH_LAST uaPeer /* Must be last value of this enum */
+ uaPeer,
@@ src/include/libpq/hba.h: typedef struct HbaLine
+ char *oauth_issuer;
+ char *oauth_scope;
+ bool oauth_skip_usermap;
- char *custom_provider;
- List *custom_auth_options;
} HbaLine;
+
+ typedef struct IdentLine
## src/include/libpq/oauth.h (new) ##
@@
8: e71df89b8f < -: ---------- Add a very simple authn_id extension
9: 73adeb3645 ! 4: 573a2ca3bc Add pytest suite for OAuth
@@ src/test/python/README (new)
+A test suite for exercising both the libpq client and the server backend at the
+protocol level, based on pytest and Construct.
+
++WARNING! This suite takes superuser-level control of the cluster under test,
++writing to the server config, creating and destroying databases, etc. It also
++spins up various ephemeral TCP services. This is not safe for production servers
++and therefore must be explicitly opted into by setting PG_TEST_EXTRA=python in
++the environment.
++
+The test suite currently assumes that the standard PG* environment variables
+point to the database under test and are sufficient to log in a superuser on
+that system. In other words, a bare `psql` needs to Just Work before the test
@@ src/test/python/README (new)
+
+The first run of
+
-+ make installcheck
++ make installcheck PG_TEST_EXTRA=python
+
+will install a local virtual environment and all needed dependencies. During
+development, if libpq changes incompatibly, you can issue
@@ src/test/python/README (new)
+The Makefile is there for convenience, but you don't have to use it. Activate
+the virtualenv to be able to use pytest directly:
+
++ $ export PG_TEST_EXTRA=python
+ $ source venv/bin/activate
+ $ py.test -k oauth
+ ...
@@ src/test/python/client/test_oauth.py (new)
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+ ## src/test/python/conftest.py (new) ##
+@@
++#
++# Copyright 2023 Timescale, Inc.
++# SPDX-License-Identifier: PostgreSQL
++#
++
++import os
++
++import pytest
++
++
++@pytest.fixture(scope="session", autouse=True)
++def _check_PG_TEST_EXTRA(request):
++ """
++ Automatically skips the whole suite if PG_TEST_EXTRA doesn't contain
++ 'python'. pytestmark doesn't seem to work in a top-level conftest.py, so
++ I've made this an autoused fixture instead.
++
++ TODO: there are tests here that are probably safe, but until I do a full
++ analysis on which are and which are not, I've made the entire thing opt-in.
++ """
++ extra_tests = os.getenv("PG_TEST_EXTRA", "").split()
++ if "python" not in extra_tests:
++ pytest.skip("Potentially unsafe test 'python' not enabled in PG_TEST_EXTRA")
+
## src/test/python/pq3.py (new) ##
@@
+#
@@ src/test/python/pq3.py (new)
+
+# Pq3
+
++
+# Adapted from construct.core.EnumIntegerString
+class EnumNamedByte:
+ def __init__(self, val, name):
@@ src/test/python/pytest.ini (new)
## src/test/python/requirements.txt (new) ##
@@
+black
-+cryptography~=3.4.6
++# cryptography 39.x removes a lot of platform support, beware
++cryptography~=38.0.4
+construct~=2.10.61
+isort~=5.6
++# TODO: update to psycopg[c] 3.1
+psycopg2~=2.8.6
-+pytest~=6.1
-+pytest-asyncio~=0.14.0
++pytest~=7.3
++pytest-asyncio~=0.21.0
## src/test/python/server/__init__.py (new) ##
@@ src/test/python/server/conftest.py (new)
+
+import pq3
+
++BLOCKING_TIMEOUT = 2 # the number of seconds to wait for blocking calls
++
+
+@pytest.fixture
+def connect():
@@ src/test/python/server/conftest.py (new)
+ addr = (pq3.pghost(), pq3.pgport())
+
+ try:
-+ sock = socket.create_connection(addr, timeout=2)
++ sock = socket.create_connection(addr, timeout=BLOCKING_TIMEOUT)
+ except ConnectionError as e:
+ pytest.skip(f"unable to connect to {addr}: {e}")
+
@@ src/test/python/server/test_oauth.py (new)
@@
+#
+# Copyright 2021 VMware, Inc.
++# Portions Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
@@ src/test/python/server/test_oauth.py (new)
+
+import pq3
+
++from .conftest import BLOCKING_TIMEOUT
++
+MAX_SASL_MESSAGE_LENGTH = 65535
+
+INVALID_AUTHORIZATION_ERRCODE = b"28000"
@@ src/test/python/server/test_oauth.py (new)
+
+SHARED_MEM_NAME = "oauth-pytest"
+MAX_TOKEN_SIZE = 4096
-+MAX_UINT16 = 2 ** 16 - 1
++MAX_UINT16 = 2**16 - 1
+
+
+def skip_if_no_postgres():
@@ src/test/python/server/test_oauth.py (new)
+ addr = (pq3.pghost(), pq3.pgport())
+
+ try:
-+ with socket.create_connection(addr, timeout=2):
++ with socket.create_connection(addr, timeout=BLOCKING_TIMEOUT):
+ pass
+ except ConnectionError as e:
+ pytest.skip(f"unable to connect to {addr}: {e}")
@@ src/test/python/server/test_oauth.py (new)
+ return connect()
+
+
-+@pytest.fixture(scope="module", autouse=True)
-+def authn_id_extension(oauth_ctx):
-+ """
-+ Performs a `CREATE EXTENSION authn_id` in the test database. This fixture is
-+ autoused, so tests don't need to rely on it.
-+ """
-+ conn = psycopg2.connect(database=oauth_ctx.dbname)
-+ conn.autocommit = True
-+
-+ with contextlib.closing(conn):
-+ c = conn.cursor()
-+ c.execute("CREATE EXTENSION authn_id;")
-+
-+
+@pytest.fixture(scope="session")
+def shared_mem():
+ """
@@ src/test/python/server/test_oauth.py (new)
+ expect_handshake_success(conn)
+
+ # Make sure that the server has not set an authenticated ID.
-+ pq3.send(conn, pq3.types.Query, query=b"SELECT authn_id();")
++ pq3.send(conn, pq3.types.Query, query=b"SELECT system_user;")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
@@ src/test/python/server/test_oauth.py (new)
+ expect_handshake_success(conn)
+
+ # Check the reported authn_id.
-+ pq3.send(conn, pq3.types.Query, query=b"SELECT authn_id();")
++ pq3.send(conn, pq3.types.Query, query=b"SELECT system_user;")
+ resp = receive_until(conn, pq3.types.DataRow)
+
++ expected = authn_id
++ if expected is not None:
++ expected = b"oauth:" + expected
++
+ row = resp.payload
-+ assert row.columns == [authn_id]
++ assert row.columns == [expected]
+
+
+class ExpectedError(object):
@@ src/test/python/server/test_oauth.py (new)
+ expect_handshake_success(conn)
+
+ # Check the user identity.
-+ pq3.send(conn, pq3.types.Query, query=b"SELECT authn_id();")
++ pq3.send(conn, pq3.types.Query, query=b"SELECT system_user;")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
-+ expected = oauth_ctx.user.encode("utf-8")
++ expected = b"oauth:" + oauth_ctx.user.encode("utf-8")
+ assert row.columns == [expected]
+
+
@@ src/test/python/server/validate_bearer.py (new)
+import sys
+from multiprocessing import shared_memory
+
-+MAX_UINT16 = 2 ** 16 - 1
++MAX_UINT16 = 2**16 - 1
+
+
+def remove_shm_from_resource_tracker():
10: ab32128f34 < -: ---------- contrib/oauth: switch to pluggable auth API
-: ---------- > 5: 4490d029b5 squash! Add pytest suite for OAuth
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-05-19 22:01 Jacob Champion <jchampion@timescale.com>
parent: Jacob Champion <jchampion@timescale.com>
0 siblings, 3 replies; 424+ messages in thread
From: Jacob Champion @ 2023-05-19 22:01 UTC (permalink / raw)
To: pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; +Cc: hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On 4/27/23 10:35, Jacob Champion wrote:
> Moving forward, the first thing I plan to tackle is asynchronous
> operation, so that polling clients can still operate sanely. If I can
> find a good solution there, the conversations about possible extension
> points should get a lot easier.
Attached is patchset v8, now with concurrency and 300% more cURL! And
many more questions to answer.
This is a full reimplementation of the client-side OAuth flow. It's an
async-first engine built on top of cURL's multi handles. All pending
operations are multiplexed into a single epoll set (the "altsock"),
which is exposed through PQsocket() for the duration of the OAuth flow.
Clients return to the flow on their next call to PQconnectPoll().
Andrey and Mahendrakar: you'll probably be interested in the
conn->async_auth() callback, conn->altsock, and the pg_fe_run_oauth_flow
entry point. This is intended to be the foundation for alternative flows.
I've kept the blocking iddawc implementation for comparison, but if
you're running the tests against it, be aware that the asynchronous
tests will, predictably, hang. Skip them with `py.test -k 'not
asynchronous'`.
= The Good =
- PQconnectPoll() is no longer indefinitely blocked on a single
connection's OAuth handshake. (iddawc doesn't appear to have any
asynchronous primitives in its API, unless I've missed something crucial.)
- We now have a swappable entry point. Alternative flows could be
implemented by applications without forcing clients to redesign their
polling loops (PQconnect* should just work as expected).
- We have full control over corner cases in our default flow. Debugging
failures is much nicer, with explanations of exactly what has gone wrong
and where, compared to iddawc's "I_ERROR" messages.
- cURL is not a lightweight library by any means, but we're no longer
bundling things like web servers that we're not going to use.
= The Bad =
- Unsurprisingly, there's a lot more code now that we're implementing
the flow ourselves. The client patch has tripled in size, and we'd be on
the hook for implementing and staying current with the RFCs.
- The client implementation is currently epoll-/Linux-specific. I think
kqueue shouldn't be too much trouble for the BSDs, but it's even more
code to maintain.
- Some clients in the wild (psycopg2/psycopg) suppress all notifications
during PQconnectPoll(). To accommodate them, I no longer use the
noticeHooks for communicating the user code, but that means we have to
come up with some other way to let applications override the printing to
stderr. Something like the OpenSSL decryption callback, maybe?
= The Ugly =
- Unless someone is aware of some amazing Winsock magic, I'm pretty sure
the multiplexed-socket approach is dead in the water on Windows. I think
the strategy there probably has to be a background thread plus a fake
"self-pipe" (loopback socket) for polling... which may be controversial?
- We have to figure out how to initialize cURL in a thread-safe manner.
Newer versions of libcurl and OpenSSL improve upon this situation, but I
don't think there's a way to check at compile time whether the
initialization strategy is safe or not (and even at runtime, I think
there may be a chicken-and-egg problem with the API, where it's not safe
to check for thread-safe initialization until after you've safely
initialized).
= Next Steps =
There are so many TODOs in the cURL implementation: it's been a while
since I've done any libcurl programming, it all needs to be hardened,
and I need to comb through the relevant specs again. But I don't want to
gold-plate it if this overall approach is unacceptable. So, questions
for the gallery:
1) Would starting up a background thread (pooled or not) be acceptable
on Windows? Alternatively, does anyone know enough Winsock deep magic to
combine multiple pending events into one (selectable!) socket?
2) If a background thread is acceptable on one platform, does it make
more sense to use one on every platform and just have synchronous code
everywhere? Or should we use a threadless async implementation when we can?
3) Is the current conn->async_auth() entry point sufficient for an
application to implement the Microsoft flows discussed upthread?
4) Would we want to try to require a new enough cURL/OpenSSL to avoid
thread safety problems during initialization, or do we need to introduce
some API equivalent to PQinitOpenSSL?
5) Does this maintenance tradeoff (full control over the client vs. a
large amount of RFC-governed code) seem like it could be okay?
Thanks,
--Jacob
Attachments:
since-v7.diff.txt (text/plain, 39.9K) download
1: c3698bbc3d = 1: 6434d90105 common/jsonapi: support FRONTEND clients
2: 0cd726fd55 < -: ---------- libpq: add OAUTHBEARER SASL mechanism
-: ---------- > 2: 13ddf2b6b3 libpq: add OAUTHBEARER SASL mechanism
3: 77889eb986 = 3: 0b0b0f2b33 backend: add OAUTHBEARER SASL mechanism
4: 573a2ca3bc ! 4: 0e8ddadcbf Add pytest suite for OAuth
@@ Commit message
dependencies will be installed into ./venv for you. See the README for
more details.
+ For iddawc, asynchronous tests still hang, as expected. Bad-interval
+ tests fail because iddawc apparently doesn't care that the interval is
+ bad.
+
## src/test/python/.gitignore (new) ##
@@
+__pycache__/
@@ src/test/python/client/conftest.py (new)
+import threading
+
+import psycopg2
++import psycopg2.extras
+import pytest
+
+import pq3
@@ src/test/python/client/conftest.py (new)
+ def run(self):
+ try:
+ conn = psycopg2.connect(host="127.0.0.1", **self._kwargs)
++ self._pump_async(conn)
+ conn.close()
+ except Exception as e:
+ self.exception = e
@@ src/test/python/client/conftest.py (new)
+ self.exception = None
+ raise e
+
++ def _pump_async(self, conn):
++ """
++ Polls a psycopg2 connection until it's completed. (Synchronous
++ connections will work here too; they'll just immediately return OK.)
++ """
++ psycopg2.extras.wait_select(conn)
++
+
+@pytest.fixture
+def accept(server_socket):
@@ src/test/python/client/conftest.py (new)
+ return sock, client
+
+ yield factory
-+ client.check_completed()
++
++ if client is not None:
++ client.check_completed()
+
+
+@pytest.fixture
@@ src/test/python/client/test_oauth.py (new)
@@
+#
+# Copyright 2021 VMware, Inc.
++# Portions Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
@@ src/test/python/client/test_oauth.py (new)
+import threading
+import time
+import urllib.parse
++from numbers import Number
+
+import psycopg2
+import pytest
@@ src/test/python/client/test_oauth.py (new)
+ finish_handshake(conn)
+
+
++class RawResponse(str):
++ """
++ Returned by registered endpoint callbacks to take full control of the
++ response. Usually, return values are converted to JSON; a RawResponse body
++ will be passed to the client as-is, allowing endpoint implementations to
++ issue invalid JSON.
++ """
++
++ pass
++
++
+class OpenIDProvider(threading.Thread):
+ """
+ A thread that runs a mock OpenID provider server.
@@ src/test/python/client/test_oauth.py (new)
+
+ def run(self):
+ try:
-+ self.server.serve_forever()
++ # XXX socketserver.serve_forever() has a serious architectural
++ # issue: its select loop wakes up every `poll_interval` seconds to
++ # see if the server is shutting down. The default, 500 ms, only lets
++ # us run two tests every second. But the faster we go, the more CPU
++ # we burn unnecessarily...
++ self.server.serve_forever(poll_interval=0.01)
+ except Exception as e:
+ self.exception = e
+
@@ src/test/python/client/test_oauth.py (new)
+ self.endpoint_paths = {}
+ self._endpoints = {}
+
++ # Provide a standard discovery document by default; tests can
++ # override it.
++ self.register_endpoint(
++ None,
++ "GET",
++ "/.well-known/openid-configuration",
++ self._default_discovery_handler,
++ )
++
+ def register_endpoint(self, name, method, path, func):
+ if method not in self._endpoints:
+ self._endpoints[method] = {}
+
+ self._endpoints[method][path] = func
-+ self.endpoint_paths[name] = path
++
++ if name is not None:
++ self.endpoint_paths[name] = path
+
+ def endpoint(self, method, path):
+ if method not in self._endpoints:
@@ src/test/python/client/test_oauth.py (new)
+
+ return self._endpoints[method].get(path)
+
++ def _default_discovery_handler(self, headers, params):
++ doc = {
++ "issuer": self.issuer,
++ "response_types_supported": ["token"],
++ "subject_types_supported": ["public"],
++ "id_token_signing_alg_values_supported": ["RS256"],
++ "grant_types_supported": [
++ "urn:ietf:params:oauth:grant-type:device_code"
++ ],
++ }
++
++ for name, path in self.endpoint_paths.items():
++ doc[name] = self.issuer + path
++
++ return 200, doc
++
+ class _Server(http.server.HTTPServer):
+ def handle_error(self, request, addr):
+ self.shutdown_request(request)
@@ src/test/python/client/test_oauth.py (new)
+ class _Handler(http.server.BaseHTTPRequestHandler):
+ timeout = BLOCKING_TIMEOUT
+
-+ def _discovery_handler(self, headers, params):
-+ oauth = self.server.oauth
-+
-+ doc = {
-+ "issuer": oauth.issuer,
-+ "response_types_supported": ["token"],
-+ "subject_types_supported": ["public"],
-+ "id_token_signing_alg_values_supported": ["RS256"],
-+ }
-+
-+ for name, path in oauth.endpoint_paths.items():
-+ doc[name] = oauth.issuer + path
-+
-+ return 200, doc
-+
+ def _handle(self, *, params=None, handler=None):
+ oauth = self.server.oauth
+ assert self.headers["Host"] == oauth.host
@@ src/test/python/client/test_oauth.py (new)
+ handler is not None
+ ), f"no registered endpoint for {self.command} {self.path}"
+
-+ code, resp = handler(self.headers, params)
++ result = handler(self.headers, params)
++
++ if len(result) == 2:
++ headers = {"Content-Type": "application/json"}
++ code, resp = result
++ else:
++ code, headers, resp = result
+
+ self.send_response(code)
-+ self.send_header("Content-Type", "application/json")
++ for h, v in headers.items():
++ self.send_header(h, v)
+ self.end_headers()
+
-+ resp = json.dumps(resp)
-+ resp = resp.encode("utf-8")
-+ self.wfile.write(resp)
++ if resp is not None:
++ if not isinstance(resp, RawResponse):
++ resp = json.dumps(resp)
++ resp = resp.encode("utf-8")
++ self.wfile.write(resp)
+
+ self.close_connection = True
+
+ def do_GET(self):
-+ if self.path == "/.well-known/openid-configuration":
-+ self._handle(handler=self._discovery_handler)
-+ return
-+
+ self._handle()
+
+ def _request_body(self):
@@ src/test/python/client/test_oauth.py (new)
+@pytest.mark.parametrize("secret", [None, "", "hunter2"])
+@pytest.mark.parametrize("scope", [None, "", "openid email"])
+@pytest.mark.parametrize("retries", [0, 1])
++@pytest.mark.parametrize(
++ "asynchronous",
++ [
++ pytest.param(False, id="synchronous"),
++ pytest.param(True, id="asynchronous"),
++ ],
++)
+def test_oauth_with_explicit_issuer(
-+ capfd, accept, openid_provider, retries, scope, secret
++ capfd, accept, openid_provider, asynchronous, retries, scope, secret
+):
+ client_id = secrets.token_hex()
+
@@ src/test/python/client/test_oauth.py (new)
+ oauth_client_id=client_id,
+ oauth_client_secret=secret,
+ oauth_scope=scope,
++ async_=asynchronous,
+ )
+
+ device_code = secrets.token_hex()
@@ src/test/python/client/test_oauth.py (new)
+ assert expected in stderr
+
+
-+def test_oauth_requires_client_id(accept, openid_provider):
-+ sock, client = accept(
-+ oauth_issuer=openid_provider.issuer,
-+ # Do not set a client ID; this should cause a client error after the
-+ # server asks for OAUTHBEARER and the client tries to contact the
-+ # issuer.
-+ )
-+
++def expect_disconnected_handshake(sock):
++ """
++ Helper for any tests that expect the client to disconnect immediately after
++ being sent the OAUTHBEARER SASL method. Generally speaking, this requires
++ the client to have an oauth_issuer set so that it doesn't try to go through
++ discovery.
++ """
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake.
@@ src/test/python/client/test_oauth.py (new)
+ # The client should disconnect at this point.
+ assert not conn.read()
+
++
++def test_oauth_requires_client_id(accept, openid_provider):
++ sock, client = accept(
++ oauth_issuer=openid_provider.issuer,
++ # Do not set a client ID; this should cause a client error after the
++ # server asks for OAUTHBEARER and the client tries to contact the
++ # issuer.
++ )
++
++ expect_disconnected_handshake(sock)
++
+ expected_error = "no oauth_client_id is set"
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
@@ src/test/python/client/test_oauth.py (new)
+ finish_handshake(conn)
+
+
++def alt_patterns(*patterns):
++ """
++ Just combines multiple alternative regexes into one. It's not very efficient
++ but IMO it's easier to read and maintain.
++ """
++ pat = ""
++
++ for p in patterns:
++ if pat:
++ pat += "|"
++ pat += f"({p})"
++
++ return pat
++
++
+@pytest.mark.parametrize(
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
-+ {
-+ "error": "invalid_client",
-+ "error_description": "client authentication failed",
-+ },
-+ r"client authentication failed \(invalid_client\)",
++ (
++ 400,
++ {
++ "error": "invalid_client",
++ "error_description": "client authentication failed",
++ },
++ ),
++ r"failed to obtain device authorization: client authentication failed \(invalid_client\)",
+ id="authentication failure with description",
+ ),
+ pytest.param(
-+ {"error": "invalid_request"},
-+ r"\(invalid_request\)",
++ (400, {"error": "invalid_request"}),
++ r"failed to obtain device authorization: \(invalid_request\)",
+ id="invalid request without description",
+ ),
+ pytest.param(
-+ {},
-+ r"failed to obtain device authorization",
++ (400, {}),
++ alt_patterns(
++ r'failed to parse token error response: field "error" is missing',
++ r"failed to obtain device authorization: \(iddawc error I_ERROR_PARAM\)",
++ ),
+ id="broken error response",
+ ),
++ pytest.param(
++ (200, RawResponse(r'{ "interval": 3.5.8 }')),
++ alt_patterns(
++ r"failed to parse device authorization: Token .* is invalid",
++ r"failed to obtain device authorization: \(iddawc error I_ERROR\)",
++ ),
++ id="non-numeric interval",
++ ),
++ pytest.param(
++ (200, RawResponse(r'{ "interval": 08 }')),
++ alt_patterns(
++ r"failed to parse device authorization: Token .* is invalid",
++ r"failed to obtain device authorization: \(iddawc error I_ERROR\)",
++ ),
++ id="invalid numeric interval",
++ ),
+ ],
+)
+def test_oauth_device_authorization_failures(
@@ src/test/python/client/test_oauth.py (new)
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
-+ return 400, failure_mode
++ return failure_mode
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
@@ src/test/python/client/test_oauth.py (new)
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
-+ with sock:
-+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
-+ # Initiate a handshake, which should result in the above endpoints
-+ # being called.
-+ startup = pq3.recv1(conn, cls=pq3.Startup)
-+ assert startup.proto == pq3.protocol(3, 0)
++ expect_disconnected_handshake(sock)
+
-+ pq3.send(
-+ conn,
-+ pq3.types.AuthnRequest,
-+ type=pq3.authn.SASL,
-+ body=[b"OAUTHBEARER", b""],
-+ )
++ # Now make sure the client correctly failed.
++ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
++ client.check_completed()
+
-+ # The client should not continue the connection due to the hardcoded
-+ # provider failure; we disconnect here.
++
++Missing = object() # sentinel for test_oauth_device_authorization_bad_json()
++
++
++@pytest.mark.parametrize(
++ "bad_value",
++ [
++ pytest.param({"device_code": 3}, id="object"),
++ pytest.param([1, 2, 3], id="array"),
++ pytest.param("some string", id="string"),
++ pytest.param(4, id="numeric"),
++ pytest.param(False, id="boolean"),
++ pytest.param(None, id="null"),
++ pytest.param(Missing, id="missing"),
++ ],
++)
++@pytest.mark.parametrize(
++ "field_name,ok_type,required",
++ [
++ ("device_code", str, True),
++ ("user_code", str, True),
++ ("verification_uri", str, True),
++ ("interval", int, False),
++ ],
++)
++def test_oauth_device_authorization_bad_json_schema(
++ accept, openid_provider, field_name, ok_type, required, bad_value
++):
++ # To make the test matrix easy, just skip the tests that aren't actually
++ # interesting (field of the correct type, missing optional field).
++ if bad_value is Missing and not required:
++ pytest.skip("not interesting: optional field")
++ elif type(bad_value) == ok_type: # not isinstance(), because bool is an int
++ pytest.skip("not interesting: correct type")
++
++ sock, client = accept(
++ oauth_issuer=openid_provider.issuer,
++ oauth_client_id=secrets.token_hex(),
++ )
++
++ # Set up our provider callbacks.
++ # NOTE that these callbacks will be called on a background thread. Don't do
++ # any unprotected state mutation here.
++
++ def authorization_endpoint(headers, params):
++ # Begin with an acceptable base response...
++ resp = {
++ "device_code": "my-device-code",
++ "user_code": "my-user-code",
++ "interval": 0,
++ "verification_uri": "https://example.com",
++ "expires_in": 5,
++ }
++
++ # ...then tweak it so the client fails.
++ if bad_value is Missing:
++ del resp[field_name]
++ else:
++ resp[field_name] = bad_value
++
++ return 200, resp
++
++ openid_provider.register_endpoint(
++ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
++ )
++
++ def token_endpoint(headers, params):
++ assert False, "token endpoint was invoked unexpectedly"
++
++ openid_provider.register_endpoint(
++ "token_endpoint", "POST", "/token", token_endpoint
++ )
++
++ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
++ if bad_value is Missing:
++ error_pattern = f'field "{field_name}" is missing'
++ elif ok_type == str:
++ error_pattern = f'field "{field_name}" must be a string'
++ elif ok_type == int:
++ error_pattern = f'field "{field_name}" must be a number'
++ else:
++ assert False, "update error_pattern for new failure mode"
++
++ # XXX iddawc doesn't really check for problems in the device authorization
++ # response, leading to this patchwork:
++ if field_name == "verification_uri":
++ error_pattern = alt_patterns(
++ error_pattern,
++ "issuer did not provide a verification URI",
++ )
++ elif field_name == "user_code":
++ error_pattern = alt_patterns(
++ error_pattern,
++ "issuer did not provide a user code",
++ )
++ else:
++ error_pattern = alt_patterns(
++ error_pattern,
++ r"failed to obtain access token: \(iddawc error I_ERROR_PARAM\)",
++ )
++
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
@@ src/test/python/client/test_oauth.py (new)
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
-+ {
-+ "error": "expired_token",
-+ "error_description": "the device code has expired",
-+ },
-+ r"the device code has expired \(expired_token\)",
++ (
++ 400,
++ {
++ "error": "expired_token",
++ "error_description": "the device code has expired",
++ },
++ ),
++ r"failed to obtain access token: the device code has expired \(expired_token\)",
+ id="expired token with description",
+ ),
+ pytest.param(
-+ {"error": "access_denied"},
-+ r"\(access_denied\)",
++ (400, {"error": "access_denied"}),
++ r"failed to obtain access token: \(access_denied\)",
+ id="access denied without description",
+ ),
+ pytest.param(
-+ {},
-+ r"OAuth token retrieval failed",
-+ id="broken error response",
++ (400, {}),
++ alt_patterns(
++ r'failed to parse token error response: field "error" is missing',
++ r"failed to obtain access token: \(iddawc error I_ERROR_PARAM\)",
++ ),
++ id="empty error response",
++ ),
++ pytest.param(
++ (200, {}, {}),
++ alt_patterns(
++ r"failed to parse access token response: no content type was provided",
++ r"failed to obtain access token: \(iddawc error I_ERROR\)",
++ ),
++ id="missing content type",
++ ),
++ pytest.param(
++ (200, {"Content-Type": "text/plain"}, {}),
++ alt_patterns(
++ r"failed to parse access token response: unexpected content type",
++ r"failed to obtain access token: \(iddawc error I_ERROR\)",
++ ),
++ id="wrong content type",
+ ),
+ ],
+)
@@ src/test/python/client/test_oauth.py (new)
+ )
+
+ retry_lock = threading.Lock()
++ final_sent = False
+
+ def token_endpoint(headers, params):
+ with retry_lock:
-+ nonlocal retries
++ nonlocal retries, final_sent
+
+ # If the test wants to force the client to retry, return an
+ # authorization_pending response and decrement the retry count.
@@ src/test/python/client/test_oauth.py (new)
+ retries -= 1
+ return 400, {"error": "authorization_pending"}
+
-+ return 400, failure_mode
++ # We should only return our failure_mode response once; any further
++ # requests indicate that the client isn't correctly bailing out.
++ assert not final_sent, "client continued after token error"
++
++ final_sent = True
++
++ return failure_mode
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
-+ with sock:
-+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
-+ # Initiate a handshake, which should result in the above endpoints
-+ # being called.
-+ startup = pq3.recv1(conn, cls=pq3.Startup)
-+ assert startup.proto == pq3.protocol(3, 0)
++ expect_disconnected_handshake(sock)
+
-+ pq3.send(
-+ conn,
-+ pq3.types.AuthnRequest,
-+ type=pq3.authn.SASL,
-+ body=[b"OAUTHBEARER", b""],
-+ )
++ # Now make sure the client correctly failed.
++ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
++ client.check_completed()
+
-+ # The client should not continue the connection due to the hardcoded
-+ # provider failure; we disconnect here.
++
++@pytest.mark.parametrize(
++ "bad_value",
++ [
++ pytest.param({"device_code": 3}, id="object"),
++ pytest.param([1, 2, 3], id="array"),
++ pytest.param("some string", id="string"),
++ pytest.param(4, id="numeric"),
++ pytest.param(False, id="boolean"),
++ pytest.param(None, id="null"),
++ pytest.param(Missing, id="missing"),
++ ],
++)
++@pytest.mark.parametrize(
++ "field_name,ok_type,required",
++ [
++ ("access_token", str, True),
++ ("token_type", str, True),
++ ],
++)
++def test_oauth_token_bad_json_schema(
++ accept, openid_provider, field_name, ok_type, required, bad_value
++):
++ # To make the test matrix easy, just skip the tests that aren't actually
++ # interesting (field of the correct type, missing optional field).
++ if bad_value is Missing and not required:
++ pytest.skip("not interesting: optional field")
++ elif type(bad_value) == ok_type: # not isinstance(), because bool is an int
++ pytest.skip("not interesting: correct type")
++
++ sock, client = accept(
++ oauth_issuer=openid_provider.issuer,
++ oauth_client_id=secrets.token_hex(),
++ )
++
++ # Set up our provider callbacks.
++ # NOTE that these callbacks will be called on a background thread. Don't do
++ # any unprotected state mutation here.
++
++ def authorization_endpoint(headers, params):
++ resp = {
++ "device_code": "my-device-code",
++ "user_code": "my-user-code",
++ "interval": 0,
++ "verification_uri": "https://example.com",
++ "expires_in": 5,
++ }
++
++ return 200, resp
++
++ openid_provider.register_endpoint(
++ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
++ )
++
++ def token_endpoint(headers, params):
++ # Begin with an acceptable base response...
++ resp = {
++ "access_token": secrets.token_urlsafe(),
++ "token_type": "bearer",
++ }
++
++ # ...then tweak it so the client fails.
++ if bad_value is Missing:
++ del resp[field_name]
++ else:
++ resp[field_name] = bad_value
++
++ return 200, resp
++
++ openid_provider.register_endpoint(
++ "token_endpoint", "POST", "/token", token_endpoint
++ )
++
++ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
++ error_pattern = "failed to parse access token response: "
++ if bad_value is Missing:
++ error_pattern += f'field "{field_name}" is missing'
++ elif ok_type == str:
++ error_pattern += f'field "{field_name}" must be a string'
++ elif ok_type == int:
++ error_pattern += f'field "{field_name}" must be a number'
++ else:
++ assert False, "update error_pattern for new failure mode"
++
++ # XXX iddawc is fairly silent on the topic.
++ error_pattern = alt_patterns(
++ error_pattern,
++ r"failed to obtain access token: \(iddawc error I_ERROR_PARAM\)",
++ )
++
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
@@ src/test/python/client/test_oauth.py (new)
+
+
+@pytest.mark.parametrize(
++ "bad_response,expected_error",
++ [
++ pytest.param(
++ (200, {"Content-Type": "text/plain"}, {}),
++ r'failed to parse OpenID discovery document: unexpected content type "text/plain"',
++ id="not JSON",
++ ),
++ pytest.param(
++ (200, {}, {}),
++ r"failed to parse OpenID discovery document: no content type was provided",
++ id="no Content-Type",
++ ),
++ pytest.param(
++ (204, {}, None),
++ r"failed to fetch OpenID discovery document: unexpected response code 204",
++ id="no content",
++ ),
++ pytest.param(
++ (301, {"Location": "https://localhost/"}, None),
++ r"failed to fetch OpenID discovery document: unexpected response code 301",
++ id="redirection",
++ ),
++ pytest.param(
++ (404, {}),
++ r"failed to fetch OpenID discovery document: unexpected response code 404",
++ id="not found",
++ ),
++ pytest.param(
++ (200, RawResponse("blah\x00blah")),
++ r"failed to parse OpenID discovery document: response contains embedded NULLs",
++ id="NULL bytes in document",
++ ),
++ pytest.param(
++ (200, 123),
++ r"failed to parse OpenID discovery document: top-level element must be an object",
++ id="scalar at top level",
++ ),
++ pytest.param(
++ (200, []),
++ r"failed to parse OpenID discovery document: top-level element must be an object",
++ id="array at top level",
++ ),
++ pytest.param(
++ (200, RawResponse("{")),
++ r"failed to parse OpenID discovery document.* input string ended unexpectedly",
++ id="unclosed object",
++ ),
++ pytest.param(
++ (200, RawResponse(r'{ "hello": ] }')),
++ r"failed to parse OpenID discovery document.* Expected JSON value",
++ id="bad array",
++ ),
++ pytest.param(
++ (200, {"issuer": 123}),
++ r'failed to parse OpenID discovery document: field "issuer" must be a string',
++ id="non-string issuer",
++ ),
++ pytest.param(
++ (200, {"issuer": ["something"]}),
++ r'failed to parse OpenID discovery document: field "issuer" must be a string',
++ id="issuer array",
++ ),
++ pytest.param(
++ (200, {"issuer": {}}),
++ r'failed to parse OpenID discovery document: field "issuer" must be a string',
++ id="issuer object",
++ ),
++ pytest.param(
++ (200, {"grant_types_supported": 123}),
++ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
++ id="scalar grant types field",
++ ),
++ pytest.param(
++ (200, {"grant_types_supported": {}}),
++ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
++ id="object grant types field",
++ ),
++ pytest.param(
++ (200, {"grant_types_supported": [123]}),
++ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
++ id="non-string grant types",
++ ),
++ pytest.param(
++ (200, {"grant_types_supported": ["something", 123]}),
++ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
++ id="non-string grant types later in the list",
++ ),
++ pytest.param(
++ (200, {"grant_types_supported": ["something", {}]}),
++ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
++ id="object grant types later in the list",
++ ),
++ pytest.param(
++ (200, {"grant_types_supported": ["something", ["something"]]}),
++ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
++ id="embedded array grant types later in the list",
++ ),
++ pytest.param(
++ (
++ 200,
++ {
++ "grant_types_supported": ["something"],
++ "token_endpoint": "https://example.com/",
++ "issuer": 123,
++ },
++ ),
++ r'failed to parse OpenID discovery document: field "issuer" must be a string',
++ id="non-string issuer after other valid fields",
++ ),
++ pytest.param(
++ (
++ 200,
++ {
++ "ignored": {"grant_types_supported": 123, "token_endpoint": 123},
++ "issuer": 123,
++ },
++ ),
++ r'failed to parse OpenID discovery document: field "issuer" must be a string',
++ id="non-string issuer after other ignored fields",
++ ),
++ pytest.param(
++ (200, {"token_endpoint": "https://example.com/"}),
++ r'failed to parse OpenID discovery document: field "issuer" is missing',
++ id="missing issuer",
++ ),
++ pytest.param(
++ (200, {"issuer": "https://example.com/"}),
++ r'failed to parse OpenID discovery document: field "token_endpoint" is missing',
++ id="missing token endpoint",
++ ),
++ pytest.param(
++ (
++ 200,
++ {
++ "issuer": "https://example.com",
++ "token_endpoint": "https://example.com/token",
++ "device_authorization_endpoint": "https://example.com/dev",
++ },
++ ),
++ r'cannot run OAuth device authorization: issuer "https://example.com" does not support device code grants',
++ id="missing device code grants",
++ ),
++ pytest.param(
++ (
++ 200,
++ {
++ "issuer": "https://example.com",
++ "token_endpoint": "https://example.com/token",
++ "grant_types_supported": [
++ "urn:ietf:params:oauth:grant-type:device_code"
++ ],
++ },
++ ),
++ r'cannot run OAuth device authorization: issuer "https://example.com" does not provide a device authorization endpoint',
++ id="missing device_authorization_endpoint",
++ ),
++ #
++ # Exercise HTTP-level failures by breaking the protocol. Note that the
++ # error messages here are implementation-dependent.
++ #
++ pytest.param(
++ (1000, {}),
++ r"failed to fetch OpenID discovery document: Unsupported protocol \(.*\)",
++ id="invalid HTTP response code",
++ ),
++ pytest.param(
++ (200, {"Content-Length": -1}, {}),
++ r"failed to fetch OpenID discovery document: Weird server reply \(.*Content-Length.*\)",
++ id="bad HTTP Content-Length",
++ ),
++ ],
++)
++def test_oauth_discovery_provider_failure(
++ accept, openid_provider, bad_response, expected_error
++):
++ sock, client = accept(
++ oauth_issuer=openid_provider.issuer,
++ oauth_client_id=secrets.token_hex(),
++ )
++
++ def failing_discovery_handler(headers, params):
++ return bad_response
++
++ openid_provider.register_endpoint(
++ None,
++ "GET",
++ "/.well-known/openid-configuration",
++ failing_discovery_handler,
++ )
++
++ expect_disconnected_handshake(sock)
++
++ # XXX iddawc doesn't differentiate...
++ expected_error = alt_patterns(
++ expected_error,
++ r"failed to fetch OpenID discovery document \(iddawc error I_ERROR(_PARAM)?\)",
++ )
++
++ with pytest.raises(psycopg2.OperationalError, match=expected_error):
++ client.check_completed()
++
++
++@pytest.mark.parametrize(
+ "sasl_err,resp_type,resp_payload,expected_error",
+ [
+ pytest.param(
@@ src/test/python/client/test_oauth.py (new)
+ "server sent additional OAuth data",
+ id="broken server: SASL success after error",
+ ),
++ pytest.param(
++ {"status": "invalid_request"},
++ pq3.types.AuthnRequest,
++ dict(type=pq3.authn.SASL, body=[b"OAUTHBEARER", b""]),
++ "duplicate SASL authentication request",
++ id="broken server: SASL reinitialization after error",
++ ),
+ ],
+)
+def test_oauth_server_error(accept, sasl_err, resp_type, resp_payload, expected_error):
5: 4490d029b5 ! 5: 65c319a6a3 squash! Add pytest suite for OAuth
@@ .cirrus.yml: task:
sysctl kern.corefile='/tmp/cores/%N.%P.core'
setup_additional_packages_script: |
- #pkg install -y ...
-+ pkg install -y iddawc
++ pkg install -y curl
# NB: Intentionally build without -Dllvm. The freebsd image size is already
# large enough to make VM startup slow, and even without llvm freebsd
@@ .cirrus.yml: task:
--buildtype=debug \
-Dcassert=true -Duuid=bsd -Dtcl_version=tcl86 -Ddtrace=auto \
-DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
-+ -Doauth=enabled \
++ -Doauth=curl \
-Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \
build
EOF
@@ .cirrus.yml: LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
--with-libxslt
--with-llvm
--with-lz4
-+ --with-oauth
++ --with-oauth=curl
--with-pam
--with-perl
--with-python
@@ .cirrus.yml: LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
LINUX_MESON_FEATURES: &LINUX_MESON_FEATURES >-
-Dllvm=enabled
-+ -Doauth=enabled
++ -Doauth=curl
-Duuid=e2fs
@@ .cirrus.yml: task:
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install \
-+ libiddawc-dev \
-+ libiddawc-dev:i386 \
++ libcurl4-openssl-dev \
++ libcurl4-openssl-dev:i386 \
+ python3-venv \
matrix:
@@ .cirrus.yml: task:
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
-+ DEBIAN_FRONTEND=noninteractive apt-get -y install libiddawc-dev
++ DEBIAN_FRONTEND=noninteractive apt-get -y install libcurl4-openssl-dev
###
# Test that code can be built with gcc/clang without warnings
@@ meson.build: foreach test_dir : tests
+ test_group = test_dir['name']
+ test_output = test_result_dir / test_group / kind
+ test_kwargs = {
-+ 'protocol': 'tap',
++ #'protocol': 'tap',
+ 'suite': test_group,
+ 'timeout': 1000,
+ 'depends': test_deps,
@@ meson.build: foreach test_dir : tests
+ pytest = venv_path / 'bin' / 'py.test'
+ test_command = [
+ pytest,
++ # Avoid running these tests against an existing database.
++ '--temp-instance', test_output / 'tmp_check',
++
+ # FIXME pytest-tap's stream feature accidentally suppresses errors that
+ # are critical for debugging:
+ # https://github.com/python-tap/pytest-tap/issues/30
-+ # Fix -- or maybe don't use the meson TAP protocol for now?
-+ '--tap-stream',
-+ # Avoid running these tests against an existing database.
-+ '--temp-instance', test_output / 'tmp_check',
++ # Don't use the meson TAP protocol for now...
++ #'--tap-stream',
+ ]
+
+ foreach pyt : t['tests']
@@ src/test/python/client/test_oauth.py: import pq3
+# The client tests need libpq to have been compiled with OAuth support; skip
+# them otherwise.
+pytestmark = pytest.mark.skipif(
-+ os.getenv("with_oauth") == "no",
++ os.getenv("with_oauth") == "none",
+ reason="OAuth client tests require --with-oauth support",
+)
+
@@ src/test/python/meson.build (new)
+ './test_pq3.py',
+ ],
+ 'env': {
-+ 'with_oauth': oauth.found() ? 'yes' : 'no',
++ 'with_oauth': oauth_library,
+
+ # Point to the default database; the tests will create their own databases
+ # as needed.
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-05-23 11:22 Daniele Varrazzo <daniele.varrazzo@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
2 siblings, 1 reply; 424+ messages in thread
From: Daniele Varrazzo @ 2023-05-23 11:22 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Sat, 20 May 2023 at 00:01, Jacob Champion <jchampion@timescale.com> wrote:
> - Some clients in the wild (psycopg2/psycopg) suppress all notifications
> during PQconnectPoll().
If there is anything we can improve in psycopg please reach out.
-- Daniele
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-05-23 15:56 Jacob Champion <jchampion@timescale.com>
parent: Daniele Varrazzo <daniele.varrazzo@gmail.com>
0 siblings, 0 replies; 424+ messages in thread
From: Jacob Champion @ 2023-05-23 15:56 UTC (permalink / raw)
To: Daniele Varrazzo <daniele.varrazzo@gmail.com>; +Cc: pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Tue, May 23, 2023 at 4:22 AM Daniele Varrazzo
<daniele.varrazzo@gmail.com> wrote:
> On Sat, 20 May 2023 at 00:01, Jacob Champion <jchampion@timescale.com> wrote:
> > - Some clients in the wild (psycopg2/psycopg) suppress all notifications
> > during PQconnectPoll().
>
> If there is anything we can improve in psycopg please reach out.
Will do, thank you! But in this case, I think there's nothing to
improve in psycopg -- in fact, it highlighted the problem with my
initial design, and now I think the notice processor will never be an
appropriate avenue for communication of the user code.
The biggest issue is that there's a chicken-and-egg situation: if
you're using the synchronous PQconnect* API, you can't override the
notice hooks while the handshake is in progress, because you don't
have a connection handle yet. The second problem is that there are a
bunch of parameters coming back from the server (user code,
verification URI, expiration time) that the application may choose to
display or use, and communicating those pieces in a (probably already
translated) flat text string is a pretty hostile API.
So I think we'll probably need to provide a global handler API,
similar to the passphrase hook we currently provide, that can receive
these pieces separately and assemble them however the application
desires. The hard part will be to avoid painting ourselves into a
corner, because this particular information is specific to the device
authorization flow, and if we ever want to add other flows into libpq,
we'll probably not want to add even more hooks.
Thanks,
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-06-29 16:28 Jacob Champion <jchampion@timescale.com>
parent: Jacob Champion <jchampion@timescale.com>
2 siblings, 0 replies; 424+ messages in thread
From: Jacob Champion @ 2023-06-29 16:28 UTC (permalink / raw)
To: pgsql-hackers; +Cc: hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>
On 5/19/23 15:01, Jacob Champion wrote:
> But I don't want to
> gold-plate it if this overall approach is unacceptable. So, questions
> for the gallery:
>
> 1) Would starting up a background thread (pooled or not) be acceptable
> on Windows? Alternatively, does anyone know enough Winsock deep magic to
> combine multiple pending events into one (selectable!) socket?
>
> 2) If a background thread is acceptable on one platform, does it make
> more sense to use one on every platform and just have synchronous code
> everywhere? Or should we use a threadless async implementation when we can?
>
> 3) Is the current conn->async_auth() entry point sufficient for an
> application to implement the Microsoft flows discussed upthread?
>
> 4) Would we want to try to require a new enough cURL/OpenSSL to avoid
> thread safety problems during initialization, or do we need to introduce
> some API equivalent to PQinitOpenSSL?
>
> 5) Does this maintenance tradeoff (full control over the client vs. a
> large amount of RFC-governed code) seem like it could be okay?
There was additional interest at PGCon, so I've registered this in the
commitfest.
Potential reviewers should be aware that the current implementation
requires Linux (or, more specifically, epoll), as the cfbot shows. But
if you have any opinions on the above questions, those will help me
tackle the other platforms. :D
Thanks!
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-07-01 04:28 Thomas Munro <thomas.munro@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
2 siblings, 1 reply; 424+ messages in thread
From: Thomas Munro @ 2023-07-01 04:28 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Sat, May 20, 2023 at 10:01 AM Jacob Champion <jchampion@timescale.com> wrote:
> - The client implementation is currently epoll-/Linux-specific. I think
> kqueue shouldn't be too much trouble for the BSDs, but it's even more
> code to maintain.
I guess you also need a fallback that uses plain old POSIX poll()? I
see you're not just using epoll but also timerfd. Could that be
converted to plain old timeout bookkeeping? That should be enough to
get every other Unix and *possibly* also Windows to work with the same
code path.
> - Unless someone is aware of some amazing Winsock magic, I'm pretty sure
> the multiplexed-socket approach is dead in the water on Windows. I think
> the strategy there probably has to be a background thread plus a fake
> "self-pipe" (loopback socket) for polling... which may be controversial?
I am not a Windows user or hacker, but there are certainly several
ways to multiplex sockets. First there is the WSAEventSelect() +
WaitForMultipleObjects() approach that latch.c uses. It has the
advantage that it allows socket readiness to be multiplexed with
various other things that use Windows "events". But if you don't need
that, ie you *only* need readiness-based wakeup for a bunch of sockets
and no other kinds of fd or object, you can use winsock's plain old
select() or its fairly faithful poll() clone called WSAPoll(). It
looks a bit like that'd be true here if you could kill the timerfd?
It's a shame to write modern code using select(), but you can find
lots of shouting all over the internet about WSAPoll()'s defects, most
famously the cURL guys[1] whose blog is widely cited, so people still
do it. Possibly some good news on that front: by my reading of the
docs, it looks like that problem was fixed in Windows 10 2004[2] which
itself is by now EOL, so all systems should have the fix? I suspect
that means that, finally, you could probably just use the same poll()
code path for Unix (when epoll is not available) *and* Windows these
days, making porting a lot easier. But I've never tried it, so I
don't know what other problems there might be. Another thing people
complain about is the lack of socketpair() or similar in winsock which
means you unfortunately can't easily make anonymous
select/poll-compatible local sockets, but that doesn't seem to be
needed here.
[1] https://daniel.haxx.se/blog/2012/10/10/wsapoll-is-broken/
[2] https://learn.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-wsapoll
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-07-05 21:00 Jacob Champion <jchampion@timescale.com>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2023-07-05 21:00 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Fri, Jun 30, 2023 at 9:29 PM Thomas Munro <thomas.munro@gmail.com> wrote:
>
> On Sat, May 20, 2023 at 10:01 AM Jacob Champion <jchampion@timescale.com> wrote:
> > - The client implementation is currently epoll-/Linux-specific. I think
> > kqueue shouldn't be too much trouble for the BSDs, but it's even more
> > code to maintain.
>
> I guess you also need a fallback that uses plain old POSIX poll()?
The use of the epoll API here is to combine several sockets into one,
not to actually call epoll_wait() itself. kqueue descriptors should
let us do the same, IIUC.
> I see you're not just using epoll but also timerfd. Could that be
> converted to plain old timeout bookkeeping? That should be enough to
> get every other Unix and *possibly* also Windows to work with the same
> code path.
I might be misunderstanding your suggestion, but I think our internal
bookkeeping is orthogonal to that. The use of timerfd here allows us
to forward libcurl's timeout requirements up to the top-level
PQsocket(). As an example, libcurl is free to tell us to call it again
in ten milliseconds, and we have to make sure a nonblocking client
calls us again after that elapses; otherwise they might hang waiting
for data that's not coming.
> > - Unless someone is aware of some amazing Winsock magic, I'm pretty sure
> > the multiplexed-socket approach is dead in the water on Windows. I think
> > the strategy there probably has to be a background thread plus a fake
> > "self-pipe" (loopback socket) for polling... which may be controversial?
>
> I am not a Windows user or hacker, but there are certainly several
> ways to multiplex sockets. First there is the WSAEventSelect() +
> WaitForMultipleObjects() approach that latch.c uses.
I don't think that strategy plays well with select() clients, though
-- it requires a handle array, and we've just got the one socket.
My goal is to maintain compatibility with existing PQconnectPoll()
applications, where the only way we get to communicate with the client
is through the PQsocket() for the connection. Ideally, you shouldn't
have to completely rewrite your application loop just to make use of
OAuth. (I assume a requirement like that would be a major roadblock to
committing this -- and if that's not a correct assumption, then I
guess my job gets a lot easier?)
> It's a shame to write modern code using select(), but you can find
> lots of shouting all over the internet about WSAPoll()'s defects, most
> famously the cURL guys[1] whose blog is widely cited, so people still
> do it.
Right -- that's basically the root of my concern. I can't guarantee
that existing Windows clients out there are all using
WaitForMultipleObjects(). From what I can tell, whatever we hand up
through PQsocket() has to be fully Winsock-/select-compatible.
> Another thing people
> complain about is the lack of socketpair() or similar in winsock which
> means you unfortunately can't easily make anonymous
> select/poll-compatible local sockets, but that doesn't seem to be
> needed here.
For the background-thread implementation, it probably would be. I've
been looking at libevent (BSD-licensed) and its socketpair hack for
Windows...
Thanks!
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-07-05 22:07 Thomas Munro <thomas.munro@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
0 siblings, 1 reply; 424+ messages in thread
From: Thomas Munro @ 2023-07-05 22:07 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Thu, Jul 6, 2023 at 9:00 AM Jacob Champion <jchampion@timescale.com> wrote:
> My goal is to maintain compatibility with existing PQconnectPoll()
> applications, where the only way we get to communicate with the client
> is through the PQsocket() for the connection. Ideally, you shouldn't
> have to completely rewrite your application loop just to make use of
> OAuth. (I assume a requirement like that would be a major roadblock to
> committing this -- and if that's not a correct assumption, then I
> guess my job gets a lot easier?)
Ah, right, I get it.
I guess there are a couple of ways to do it if we give up the goal of
no-code-change-for-the-client:
1. Generalised PQsocket(), that so that a client can call something like:
int PQpollset(const PGConn *conn, struct pollfd fds[], int fds_size,
int *nfds, int *timeout_ms);
That way, libpq could tell you about which events it would like to
wait for on which fds, and when it would like you to call it back due
to timeout, and you can either pass that information directly to
poll() or WSAPoll() or some equivalent interface (we don't care, we
just gave you the info you need), or combine it in obvious ways with
whatever else you want to multiplex with in your client program.
2. Convert those events into new libpq events like 'I want you to
call me back in 100ms', and 'call me back when socket #42 has data',
and let clients handle that by managing their own poll set etc. (This
is something I've speculated about to support more efficient
postgres_fdw shard query multiplexing; gotta figure out how to get
multiple connections' events into one WaitEventSet...)
I guess there is a practical middle ground where client code on
systems that have epoll/kqueue can use OAUTHBEARER without any code
change, and the feature is available on other systems too but you'll
have to change your client code to use one of those interfaces or else
you get an error 'coz we just can't do it. Or, more likely in the
first version, you just can't do it at all... Doesn't seem that bad
to me.
BTW I will happily do the epoll->kqueue port work if necessary.
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-07-06 16:56 Jacob Champion <jchampion@timescale.com>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 2 replies; 424+ messages in thread
From: Jacob Champion @ 2023-07-06 16:56 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Wed, Jul 5, 2023 at 3:07 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> I guess there are a couple of ways to do it if we give up the goal of
> no-code-change-for-the-client:
>
> 1. Generalised PQsocket(), that so that a client can call something like:
>
> int PQpollset(const PGConn *conn, struct pollfd fds[], int fds_size,
> int *nfds, int *timeout_ms);
>
> That way, libpq could tell you about which events it would like to
> wait for on which fds, and when it would like you to call it back due
> to timeout, and you can either pass that information directly to
> poll() or WSAPoll() or some equivalent interface (we don't care, we
> just gave you the info you need), or combine it in obvious ways with
> whatever else you want to multiplex with in your client program.
I absolutely wanted something like this while I was writing the code
(it would have made things much easier), but I'd feel bad adding that
much complexity to the API if the vast majority of connections use
exactly one socket. Are there other use cases in libpq where you think
this expanded API could be useful? Maybe to lift some of the existing
restrictions for PQconnectPoll(), add async DNS resolution, or
something?
Couple complications I can think of at the moment:
1. Clients using persistent pollsets will have to remove old
descriptors, presumably by tracking the delta since the last call,
which might make for a rough transition. Bookkeeping bugs probably
wouldn't show up unless they used OAuth in their test suites. With the
current model, that's more hidden and libpq takes responsibility for
getting it right.
2. In the future, we might need to think carefully around situations
where we want multiple PGConn handles to share descriptors (e.g.
multiplexed backend connections). I avoid tricky questions at the
moment by assigning only one connection per multi pool.
> 2. Convert those events into new libpq events like 'I want you to
> call me back in 100ms', and 'call me back when socket #42 has data',
> and let clients handle that by managing their own poll set etc. (This
> is something I've speculated about to support more efficient
> postgres_fdw shard query multiplexing; gotta figure out how to get
> multiple connections' events into one WaitEventSet...)
Something analogous to libcurl's socket and timeout callbacks [1],
then? Or is there an existing libpq API you were thinking about using?
> I guess there is a practical middle ground where client code on
> systems that have epoll/kqueue can use OAUTHBEARER without any code
> change, and the feature is available on other systems too but you'll
> have to change your client code to use one of those interfaces or else
> you get an error 'coz we just can't do it.
That's a possibility -- if your platform is able to do it nicely,
might as well use it. (In a similar vein, I'd personally vote against
having every platform use a background thread, even if we decided to
implement it for Windows.)
> Or, more likely in the
> first version, you just can't do it at all... Doesn't seem that bad
> to me.
Any initial opinions on whether it's worse or better than a worker thread?
> BTW I will happily do the epoll->kqueue port work if necessary.
And I will happily take you up on that; thanks!
--Jacob
[1] https://curl.se/libcurl/c/CURLMOPT_SOCKETFUNCTION.html
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-07-06 20:47 Thomas Munro <thomas.munro@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
1 sibling, 1 reply; 424+ messages in thread
From: Thomas Munro @ 2023-07-06 20:47 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Fri, Jul 7, 2023 at 4:57 AM Jacob Champion <jchampion@timescale.com> wrote:
> On Wed, Jul 5, 2023 at 3:07 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> > 2. Convert those events into new libpq events like 'I want you to
> > call me back in 100ms', and 'call me back when socket #42 has data',
> > and let clients handle that by managing their own poll set etc. (This
> > is something I've speculated about to support more efficient
> > postgres_fdw shard query multiplexing; gotta figure out how to get
> > multiple connections' events into one WaitEventSet...)
>
> Something analogous to libcurl's socket and timeout callbacks [1],
> then? Or is there an existing libpq API you were thinking about using?
Yeah. Libpq already has an event concept. I did some work on getting
long-lived WaitEventSet objects to be used in various places, some of
which got committed[1], but not yet the parts related to postgres_fdw
(which uses libpq connections to talk to other PostgreSQL servers, and
runs into the limitations of PQsocket()). Horiguchi-san had the good
idea of extending the event system to cover socket changes, but I
haven't actually tried it yet. One day.
> > Or, more likely in the
> > first version, you just can't do it at all... Doesn't seem that bad
> > to me.
>
> Any initial opinions on whether it's worse or better than a worker thread?
My vote is that it's perfectly fine to make a new feature that only
works on some OSes. If/when someone wants to work on getting it going
on Windows/AIX/Solaris (that's the complete set of no-epoll, no-kqueue
OSes we target), they can write the patch.
[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJAC4Oqao%3DqforhNey20J8CiG2R%3DoBPqvfR0vOJrFysGw...
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-07-07 18:48 Jacob Champion <jchampion@timescale.com>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2023-07-07 18:48 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Thu, Jul 6, 2023 at 1:48 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> On Fri, Jul 7, 2023 at 4:57 AM Jacob Champion <jchampion@timescale.com> wrote:
> > Something analogous to libcurl's socket and timeout callbacks [1],
> > then? Or is there an existing libpq API you were thinking about using?
>
> Yeah. Libpq already has an event concept.
Thanks -- I don't know how I never noticed libpq-events.h before.
Per-connection events (or callbacks) might bring up the same
chicken-and-egg situation discussed above, with the notice hook. We'll
be fine as long as PQconnectStart is guaranteed to return before the
PQconnectPoll engine gets to authentication, and it looks like that's
true with today's implementation, which returns pessimistically at
several points instead of just trying to continue the exchange. But I
don't know if that's intended as a guarantee for the future. At the
very least we would have to pin that implementation detail.
> > > Or, more likely in the
> > > first version, you just can't do it at all... Doesn't seem that bad
> > > to me.
> >
> > Any initial opinions on whether it's worse or better than a worker thread?
>
> My vote is that it's perfectly fine to make a new feature that only
> works on some OSes. If/when someone wants to work on getting it going
> on Windows/AIX/Solaris (that's the complete set of no-epoll, no-kqueue
> OSes we target), they can write the patch.
Okay. I'm curious to hear others' thoughts on that, too, if anyone's lurking.
Thanks!
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-07-07 21:16 Andrey Chudnovsky <achudnovskij@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
0 siblings, 1 reply; 424+ messages in thread
From: Andrey Chudnovsky @ 2023-07-07 21:16 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: Thomas Munro <thomas.munro@gmail.com>; pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
Thanks Jacob for making progress on this.
> 3) Is the current conn->async_auth() entry point sufficient for an
> application to implement the Microsoft flows discussed upthread?
Please confirm my understanding of the flow is correct:
1. Client calls PQconnectStart.
- The client doesn't know yet what is the issuer and the scope.
- Parameters are strings, so callback is not provided yet.
2. Client gets PgConn from PQconnectStart return value and updates
conn->async_auth to its own callback.
3. Client polls PQconnectPoll and checks conn->sasl_state until the
value is SASL_ASYNC
4. Client accesses conn->oauth_issuer and conn->oauth_scope and uses
those info to trigger the token flow.
5. Expectations on async_auth:
a. It returns PGRES_POLLING_READING while token acquisition is going on
b. It returns PGRES_POLLING_OK and sets conn->sasl_state->token
when token acquisition succeeds.
6. Is the client supposed to do anything with the altsock parameter?
Is the above accurate understanding?
If yes, it looks workable with a couple of improvements I think would be nice:
1. Currently, oauth_exchange function sets conn->async_auth =
pg_fe_run_oauth_flow and starts Device Code flow automatically when
receiving challenge and metadata from the server.
There probably should be a way for the client to prevent default
Device Code flow from triggering.
2. The current signature and expectations from async_auth function
seems to be tightly coupled with the internal implementation:
- Pieces of information need to be picked and updated in different
places in the PgConn structure.
- Function is expected to return PostgresPollingStatusType which
is used to communicate internal state to the client.
Would it make sense to separate the internal callback used to
communicate with Device Code flow from client facing API?
I.e. introduce a new client facing structure and enum to facilitate
callback and its return value.
-----------
On a separate note:
The backend code currently spawns an external command for token validation.
As we discussed before, an extension hook would be a more efficient
extensibility option.
We see clients make 10k+ connections using OAuth tokens per minute to
our service, and stating external processes would be too much overhead
here.
-----------
> 5) Does this maintenance tradeoff (full control over the client vs. a
> large amount of RFC-governed code) seem like it could be okay?
It's nice for psql to have Device Code flow. Can be made even more
convenient with refresh tokens support.
And for clients on resource constrained devices to be able to
authenticate with Client Credentials (app secret) without bringing
more dependencies.
In most other cases, upstream PostgreSQL drivers written in higher
level languages have libraries / abstractions to implement OAUTH flows
for the platforms they support.
On Fri, Jul 7, 2023 at 11:48 AM Jacob Champion <jchampion@timescale.com> wrote:
>
> On Thu, Jul 6, 2023 at 1:48 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> > On Fri, Jul 7, 2023 at 4:57 AM Jacob Champion <jchampion@timescale.com> wrote:
> > > Something analogous to libcurl's socket and timeout callbacks [1],
> > > then? Or is there an existing libpq API you were thinking about using?
> >
> > Yeah. Libpq already has an event concept.
>
> Thanks -- I don't know how I never noticed libpq-events.h before.
>
> Per-connection events (or callbacks) might bring up the same
> chicken-and-egg situation discussed above, with the notice hook. We'll
> be fine as long as PQconnectStart is guaranteed to return before the
> PQconnectPoll engine gets to authentication, and it looks like that's
> true with today's implementation, which returns pessimistically at
> several points instead of just trying to continue the exchange. But I
> don't know if that's intended as a guarantee for the future. At the
> very least we would have to pin that implementation detail.
>
> > > > Or, more likely in the
> > > > first version, you just can't do it at all... Doesn't seem that bad
> > > > to me.
> > >
> > > Any initial opinions on whether it's worse or better than a worker thread?
> >
> > My vote is that it's perfectly fine to make a new feature that only
> > works on some OSes. If/when someone wants to work on getting it going
> > on Windows/AIX/Solaris (that's the complete set of no-epoll, no-kqueue
> > OSes we target), they can write the patch.
>
> Okay. I'm curious to hear others' thoughts on that, too, if anyone's lurking.
>
> Thanks!
> --Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-07-08 01:00 Thomas Munro <thomas.munro@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
1 sibling, 1 reply; 424+ messages in thread
From: Thomas Munro @ 2023-07-08 01:00 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Fri, Jul 7, 2023 at 4:57 AM Jacob Champion <jchampion@timescale.com> wrote:
> On Wed, Jul 5, 2023 at 3:07 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> > BTW I will happily do the epoll->kqueue port work if necessary.
>
> And I will happily take you up on that; thanks!
Some initial hacking, about 2 coffees' worth:
https://github.com/macdice/postgres/commits/oauth-kqueue
This compiles on FreeBSD and macOS, but I didn't have time to figure
out all your Python testing magic so I don't know if it works yet and
it's still red on CI... one thing I wondered about is the *altsock =
timerfd part which I couldn't do.
The situation on macOS is a little odd: the man page says EVFILT_TIMER
is not implemented. But clearly it is, we can read the source code as
I had to do to find out which unit of time it defaults to[1] (huh,
Apple's github repo for Darwin appears to have been archived recently
-- no more source code updates? that'd be a shame!), and it works
exactly as expected in simple programs. So I would just assume it
works until we see evidence otherwise. (We already use a couple of
other things on macOS more or less by accident because configure finds
them, where they are undocumented or undeclared.)
[1] https://github.com/apple/darwin-xnu/blob/main/bsd/kern/kern_event.c#L1345
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-07-10 23:21 Jacob Champion <jchampion@timescale.com>
parent: Andrey Chudnovsky <achudnovskij@gmail.com>
0 siblings, 0 replies; 424+ messages in thread
From: Jacob Champion @ 2023-07-10 23:21 UTC (permalink / raw)
To: Andrey Chudnovsky <achudnovskij@gmail.com>; +Cc: Thomas Munro <thomas.munro@gmail.com>; pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Fri, Jul 7, 2023 at 2:16 PM Andrey Chudnovsky <achudnovskij@gmail.com> wrote:
> Please confirm my understanding of the flow is correct:
> 1. Client calls PQconnectStart.
> - The client doesn't know yet what is the issuer and the scope.
Right. (Strictly speaking it doesn't even know that OAuth will be used
for the connection, yet, though at some point we'll be able to force
the issue with e.g. `require_auth=oauth`. That's not currently
implemented.)
> - Parameters are strings, so callback is not provided yet.
> 2. Client gets PgConn from PQconnectStart return value and updates
> conn->async_auth to its own callback.
This is where some sort of official authn callback registration (see
above reply to Daniele) would probably come in handy.
> 3. Client polls PQconnectPoll and checks conn->sasl_state until the
> value is SASL_ASYNC
In my head, the client's custom callback would always be invoked
during the call to PQconnectPoll, rather than making the client do
work in between calls. That way, a client can use custom flows even
with a synchronous PQconnectdb().
> 4. Client accesses conn->oauth_issuer and conn->oauth_scope and uses
> those info to trigger the token flow.
Right.
> 5. Expectations on async_auth:
> a. It returns PGRES_POLLING_READING while token acquisition is going on
> b. It returns PGRES_POLLING_OK and sets conn->sasl_state->token
> when token acquisition succeeds.
Yes. Though the token should probably be returned through some
explicit part of the callback, now that you mention it...
> 6. Is the client supposed to do anything with the altsock parameter?
The callback needs to set the altsock up with a select()able
descriptor, which wakes up the client when more work is ready to be
done. Without that, you can't handle multiple connections on a single
thread.
> If yes, it looks workable with a couple of improvements I think would be nice:
> 1. Currently, oauth_exchange function sets conn->async_auth =
> pg_fe_run_oauth_flow and starts Device Code flow automatically when
> receiving challenge and metadata from the server.
> There probably should be a way for the client to prevent default
> Device Code flow from triggering.
Agreed. I'd like the client to be able to override this directly.
> 2. The current signature and expectations from async_auth function
> seems to be tightly coupled with the internal implementation:
> - Pieces of information need to be picked and updated in different
> places in the PgConn structure.
> - Function is expected to return PostgresPollingStatusType which
> is used to communicate internal state to the client.
> Would it make sense to separate the internal callback used to
> communicate with Device Code flow from client facing API?
> I.e. introduce a new client facing structure and enum to facilitate
> callback and its return value.
Yep, exactly right! I just wanted to check that the architecture
*looked* sufficient before pulling it up into an API.
> On a separate note:
> The backend code currently spawns an external command for token validation.
> As we discussed before, an extension hook would be a more efficient
> extensibility option.
> We see clients make 10k+ connections using OAuth tokens per minute to
> our service, and stating external processes would be too much overhead
> here.
+1. I'm curious, though -- what language do you expect to use to write
a production validator hook? Surely not low-level C...?
> > 5) Does this maintenance tradeoff (full control over the client vs. a
> > large amount of RFC-governed code) seem like it could be okay?
>
> It's nice for psql to have Device Code flow. Can be made even more
> convenient with refresh tokens support.
> And for clients on resource constrained devices to be able to
> authenticate with Client Credentials (app secret) without bringing
> more dependencies.
>
> In most other cases, upstream PostgreSQL drivers written in higher
> level languages have libraries / abstractions to implement OAUTH flows
> for the platforms they support.
Yeah, I'm really interested in seeing which existing high-level flows
can be mixed in through a driver. Trying not to get too far ahead of
myself :D
Thanks for the review!
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-07-10 23:50 Jacob Champion <jchampion@timescale.com>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2023-07-10 23:50 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Fri, Jul 7, 2023 at 6:01 PM Thomas Munro <thomas.munro@gmail.com> wrote:
>
> On Fri, Jul 7, 2023 at 4:57 AM Jacob Champion <jchampion@timescale.com> wrote:
> > On Wed, Jul 5, 2023 at 3:07 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> > > BTW I will happily do the epoll->kqueue port work if necessary.
> >
> > And I will happily take you up on that; thanks!
>
> Some initial hacking, about 2 coffees' worth:
> https://github.com/macdice/postgres/commits/oauth-kqueue
>
> This compiles on FreeBSD and macOS, but I didn't have time to figure
> out all your Python testing magic so I don't know if it works yet and
> it's still red on CI...
This is awesome, thank you!
I need to look into the CI more, but it looks like the client tests
are passing, which is a good sign. (I don't understand why the
server-side tests are failing on FreeBSD, but they shouldn't be using
the libpq code at all, so I think your kqueue implementation is in the
clear. Cirrus doesn't have the logs from the server-side test failures
anywhere -- probably a bug in my Meson patch.)
> one thing I wondered about is the *altsock =
> timerfd part which I couldn't do.
I did that because I'm not entirely sure that libcurl is guaranteed to
have cleared out all its sockets from the mux, and I didn't want to
invite spurious wakeups. I should probably verify whether or not
that's possible. If so, we could just make that code resilient to
early wakeup, so that it matters less, or set up a second kqueue that
only holds the timer if that turns out to be unacceptable?
> The situation on macOS is a little odd: the man page says EVFILT_TIMER
> is not implemented. But clearly it is, we can read the source code as
> I had to do to find out which unit of time it defaults to[1] (huh,
> Apple's github repo for Darwin appears to have been archived recently
> -- no more source code updates? that'd be a shame!), and it works
> exactly as expected in simple programs. So I would just assume it
> works until we see evidence otherwise. (We already use a couple of
> other things on macOS more or less by accident because configure finds
> them, where they are undocumented or undeclared.)
Huh. Something to keep an eye on... might be a problem with older versions?
Thanks!
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-07-11 17:50 Jacob Champion <jchampion@timescale.com>
parent: Jacob Champion <jchampion@timescale.com>
0 siblings, 2 replies; 424+ messages in thread
From: Jacob Champion @ 2023-07-11 17:50 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Mon, Jul 10, 2023 at 4:50 PM Jacob Champion <jchampion@timescale.com> wrote:
> I don't understand why the
> server-side tests are failing on FreeBSD, but they shouldn't be using
> the libpq code at all, so I think your kqueue implementation is in the
> clear.
Oh, whoops, it's just the missed CLOEXEC flag in the final patch. (If
the write side of the pipe gets copied around, it hangs open and the
validator never sees the "end" of the token.) I'll switch the logic
around to set the flag on the write side instead of unsetting it on
the read side.
I have a WIP patch that passes tests on FreeBSD, which I'll clean up
and post Sometime Soon. macOS builds now but still fails before it
runs the test; looks like it's having trouble finding OpenSSL during
`pip install` of the test modules...
Thanks!
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-07-12 01:37 Thomas Munro <thomas.munro@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
1 sibling, 1 reply; 424+ messages in thread
From: Thomas Munro @ 2023-07-12 01:37 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Wed, Jul 12, 2023 at 5:50 AM Jacob Champion <jchampion@timescale.com> wrote:
> Oh, whoops, it's just the missed CLOEXEC flag in the final patch. (If
> the write side of the pipe gets copied around, it hangs open and the
> validator never sees the "end" of the token.) I'll switch the logic
> around to set the flag on the write side instead of unsetting it on
> the read side.
Oops, sorry about that. Glad to hear it's all working!
(FTR my parenthetical note about macOS/XNU sources on Github was a
false alarm: the "apple" account has stopped publishing a redundant
copy of that, but "apple-oss-distributions" is the account I should
have been looking at and it is live. I guess it migrated at some
point, or something. Phew.)
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-07-13 04:51 Andrey Chudnovsky <achudnovskij@gmail.com>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 0 replies; 424+ messages in thread
From: Andrey Chudnovsky @ 2023-07-13 04:51 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: Jacob Champion <jchampion@timescale.com>; pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
> > - Parameters are strings, so callback is not provided yet.
> > 2. Client gets PgConn from PQconnectStart return value and updates
> > conn->async_auth to its own callback.
>
> This is where some sort of official authn callback registration (see
> above reply to Daniele) would probably come in handy.
+1
> > 3. Client polls PQconnectPoll and checks conn->sasl_state until the
> > value is SASL_ASYNC
>
> In my head, the client's custom callback would always be invoked
> during the call to PQconnectPoll, rather than making the client do
> work in between calls. That way, a client can use custom flows even
> with a synchronous PQconnectdb().
The way I see this API working is the asynchronous client needs at least 2
PQConnectPoll calls:
1. To be notified of what the authentication requirements are and get
parameters.
2. When it acquires the token, the callback is used to inform libpq of the
token and return PGRES_POLLING_OK.
For the synchronous client, the callback implementation would need to be
aware of the fact that synchronous implementation invokes callback
frequently and be implemented accordingly.
Bottom lime, I don't see much problem with the current proposal. Just the
way of callback to know that OAUTH token is requested and get parameters
relies on PQconnectPoll being invoked after corresponding parameters of
conn object are populated.
> > > 5. Expectations on async_auth:
> > > a. It returns PGRES_POLLING_READING while token acquisition is
going on
> > > b. It returns PGRES_POLLING_OK and sets conn->sasl_state->token
> > > when token acquisition succeeds.
> >
> > Yes. Though the token should probably be returned through some
> > explicit part of the callback, now that you mention it...
>
> > 6. Is the client supposed to do anything with the altsock parameter?
>
> The callback needs to set the altsock up with a select()able
> descriptor, which wakes up the client when more work is ready to be
> done. Without that, you can't handle multiple connections on a single
> thread.
Ok, thanks for clarification.
> > On a separate note:
> > The backend code currently spawns an external command for token
validation.
> > As we discussed before, an extension hook would be a more efficient
> > extensibility option.
> > We see clients make 10k+ connections using OAuth tokens per minute to
> > our service, and stating external processes would be too much overhead
> > here.
>
> +1. I'm curious, though -- what language do you expect to use to write
> a production validator hook? Surely not low-level C...?
For the server side code, it would likely be identity providers publishing
extensions to validate their tokens.
Those can do that in C too. Or extensions now can be implemented in Rust
using pgrx. Which is developer friendly enough in my opinion.
> Yeah, I'm really interested in seeing which existing high-level flows
> can be mixed in through a driver. Trying not to get too far ahead of
> myself :D
I can think of the following as the most common:
1. Authorization code with PKCE. This is by far the most common for the
user login flows. Requires to spin up a browser and listen to redirect
URL/Port. Most high level platforms have libraries to do both.
2. Client Certificates. This requires an identity provider specific library
to construct and sign the token. The providers publish SDKs to do that for
most common app development platforms.
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-07-17 23:55 Jacob Champion <jchampion@timescale.com>
parent: Jacob Champion <jchampion@timescale.com>
1 sibling, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2023-07-17 23:55 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Tue, Jul 11, 2023 at 10:50 AM Jacob Champion
<jchampion@timescale.com> wrote:
> I have a WIP patch that passes tests on FreeBSD, which I'll clean up
> and post Sometime Soon. macOS builds now but still fails before it
> runs the test; looks like it's having trouble finding OpenSSL during
> `pip install` of the test modules...
Hi Thomas,
v9 folds in your kqueue implementation (thanks again!) and I have a
quick question to check my understanding:
> + case CURL_POLL_REMOVE:
> + /*
> + * We don't know which of these is currently registered, perhaps
> + * both, so we try to remove both. This means we need to tolerate
> + * ENOENT below.
> + */
> + EV_SET(&ev[nev], socket, EVFILT_READ, EV_DELETE, 0, 0, 0);
> + nev++;
> + EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_DELETE, 0, 0, 0);
> + nev++;
> + break;
We're not setting EV_RECEIPT for these -- is that because none of the
filters we're using are EV_CLEAR, and so it doesn't matter if we
accidentally pull pending events off the queue during the kevent() call?
v9 also improves the Cirrus debugging experience and fixes more issues
on macOS, so the tests should be green there now. The final patch in the
series works around what I think is a build bug in psycopg2 2.9 [1] for
the BSDs+meson.
Thanks,
--Jacob
[1] https://github.com/psycopg/psycopg2/issues/1599
Attachments:
since-v8.diff.txt (text/plain, 18.1K) download
1: 6434d90105 = 1: 9c6a340119 common/jsonapi: support FRONTEND clients
2: 13ddf2b6b3 ! 2: 8072d0416e libpq: add OAUTHBEARER SASL mechanism
@@ Commit message
development headers. Pass `curl` or `iddawc` to --with-oauth/-Doauth
during configuration.
+ Thomas Munro wrote the kqueue() implementation for oauth-curl; thanks!
+
Several TODOs:
- don't retry forever if the server won't accept our token
- perform several sanity checks on the OAuth issuer's responses
@@ meson.build: if meson.version().version_compare('>=0.57')
'plperl': perl_dep,
## meson_options.txt ##
-@@ meson_options.txt: option('lz4', type : 'feature', value: 'auto',
+@@ meson_options.txt: option('lz4', type: 'feature', value: 'auto',
option('nls', type: 'feature', value: 'auto',
- description: 'native language support')
+ description: 'Native language support')
+option('oauth', type : 'combo', choices : ['auto', 'none', 'curl', 'iddawc'],
+ value: 'auto',
+ description: 'use LIB for OAuth 2.0 support (curl, iddawc)')
+
- option('pam', type : 'feature', value: 'auto',
- description: 'build with PAM support')
+ option('pam', type: 'feature', value: 'auto',
+ description: 'PAM support')
## src/Makefile.global.in ##
@@ src/Makefile.global.in: with_ldap = @with_ldap@
## src/common/meson.build ##
@@ src/common/meson.build: common_sources_frontend_static += files(
- # For the server build of pgcommon, depend on lwlocknames_h, because at least
- # cryptohash_openssl.c, hmac_openssl.c depend on it. That's arguably a
+ # least cryptohash_openssl.c, hmac_openssl.c depend on it.
+ # controldata_utils.c depends on wait_event_types_h. That's arguably a
# layering violation, but ...
+#
+# XXX Frontend builds need libpq's pqexpbuffer.h, so adjust the include paths
@@ src/common/meson.build: common_sources_frontend_static += files(
pgcommon_variants = {
'_srv': internal_lib_args + {
+ 'include_directories': include_directories('.'),
- 'sources': common_sources + [lwlocknames_h],
+ 'sources': common_sources + [lwlocknames_h] + [wait_event_types_h],
'dependencies': [backend_common_code],
},
'': default_lib_args + {
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+
+#include <curl/curl.h>
+#include <math.h>
++#ifdef HAVE_SYS_EPOLL_H
+#include <sys/epoll.h>
+#include <sys/timerfd.h>
++#endif
++#ifdef HAVE_SYS_EVENT_H
++#include <sys/event.h>
++#endif
+#include <unistd.h>
+
+#include "common/jsonapi.h"
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+#include "libpq-int.h"
+#include "mb/pg_wchar.h"
+
++#ifdef HAVE_SYS_EVENT_H
++/* macOS doesn't define the time unit macros, but uses milliseconds by default. */
++#ifndef NOTE_MSECONDS
++#define NOTE_MSECONDS 0
++#endif
++#endif
++
+/*
+ * Parsed JSON Representations
+ *
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+{
+ OAuthStep step; /* where are we in the flow? */
+
++#ifdef HAVE_SYS_EPOLL_H
+ int timerfd; /* a timerfd for signaling async timeouts */
++#endif
+ pgsocket mux; /* the multiplexer socket containing all descriptors
+ tracked by cURL, plus the timerfd */
+ CURLM *curlm; /* top-level multi handle for cURL operations */
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+
+ if (actx->mux != PGINVALID_SOCKET)
+ close(actx->mux);
++#ifdef HAVE_SYS_EPOLL_H
+ if (actx->timerfd >= 0)
+ close(actx->timerfd);
++#endif
+
+ free(actx);
+}
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ * Sets up the actx->mux, which is the altsock that PQconnectPoll clients will
+ * select() on instead of the Postgres socket during OAuth negotiation.
+ *
-+ * This is just an epoll set abstracting multiple other descriptors. A timerfd
-+ * is always part of the set; it's just disabled when we're not using it.
++ * This is just an epoll set or kqueue abstracting multiple other descriptors.
++ * A timerfd is always part of the set when using epoll; it's just disabled
++ * when we're not using it.
+ */
+static bool
+setup_multiplexer(struct async_ctx *actx)
+{
++#ifdef HAVE_SYS_EPOLL_H
+ struct epoll_event ev = {.events = EPOLLIN};
+
+ actx->mux = epoll_create1(EPOLL_CLOEXEC);
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ }
+
+ return true;
++#endif
++#ifdef HAVE_SYS_EVENT_H
++ actx->mux = kqueue();
++ if (actx->mux < 0)
++ {
++ actx_error(actx, "failed to create kqueue: %m");
++ return false;
++ }
++
++ return true;
++#endif
++
++ actx_error(actx, "here's a nickel kid, get yourself a better computer");
++ return false;
+}
+
+/*
-+ * Adds and removes sockets from the multiplexer epoll set, as directed by the
++ * Adds and removes sockets from the multiplexer set, as directed by the
+ * cURL multi handle.
+ */
+static int
+register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
+ void *socketp)
+{
++#ifdef HAVE_SYS_EPOLL_H
+ struct async_ctx *actx = ctx;
+ struct epoll_event ev = {0};
+ int res;
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+
+ return -1;
+ }
++#endif
++#ifdef HAVE_SYS_EVENT_H
++ struct async_ctx *actx = ctx;
++ struct kevent ev[2] = {{0}};
++ struct kevent ev_out[2];
++ struct timespec timeout = {0};
++ int nev = 0;
++ int res;
++
++ switch (what)
++ {
++ case CURL_POLL_IN:
++ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD, 0, 0, 0);
++ nev++;
++ break;
++
++ case CURL_POLL_OUT:
++ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD, 0, 0, 0);
++ nev++;
++ break;
++
++ case CURL_POLL_INOUT:
++ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD, 0, 0, 0);
++ nev++;
++ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD, 0, 0, 0);
++ nev++;
++ break;
++
++ case CURL_POLL_REMOVE:
++ /*
++ * We don't know which of these is currently registered, perhaps
++ * both, so we try to remove both. This means we need to tolerate
++ * ENOENT below.
++ */
++ EV_SET(&ev[nev], socket, EVFILT_READ, EV_DELETE, 0, 0, 0);
++ nev++;
++ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_DELETE, 0, 0, 0);
++ nev++;
++ break;
++
++ default:
++ actx_error(actx, "unknown cURL socket operation (%d)", what);
++ return -1;
++ }
++
++ res = kevent(actx->mux, ev, nev, ev_out, lengthof(ev_out), &timeout);
++ if (res < 0)
++ {
++ actx_error(actx, "could not modify kqueue: %m");
++ return -1;
++ }
++
++ /*
++ * We can't use the simple errno version of kevent, because we need to skip
++ * over ENOENT while still allowing a second change to be processed. So we
++ * need a longer-form error checking loop.
++ */
++ for (int i = 0; i < res; ++i)
++ {
++ if (ev_out[i].flags & EV_ERROR && ev_out[i].data != ENOENT)
++ {
++ errno = ev_out[i].data;
++ switch (what)
++ {
++ case CURL_POLL_REMOVE:
++ actx_error(actx, "could not delete from kqueue: %m");
++ break;
++ default:
++ actx_error(actx, "could not add to kqueue: %m");
++ }
++ return -1;
++ }
++ }
++#endif
+
+ return 0;
+}
+
+/*
-+ * Adds or removes timeouts from the multiplexer epoll set, as directed by the
-+ * cURL multi handle. Rather than continually adding and removing the timerfd,
-+ * we keep it in the epoll set at all times and just disarm it when it's not
++ * Adds or removes timeouts from the multiplexer set, as directed by the
++ * cURL multi handle. Rather than continually adding and removing the timer,
++ * we keep it in the set at all times and just disarm it when it's not
+ * needed.
+ */
+static int
+register_timer(CURLM *curlm, long timeout, void *ctx)
+{
++#if HAVE_SYS_EPOLL_H
+ struct async_ctx *actx = ctx;
+ struct itimerspec spec = {0};
+
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ actx_error(actx, "setting timerfd to %ld: %m", timeout);
+ return -1;
+ }
++#endif
++#ifdef HAVE_SYS_EVENT_H
++ struct async_ctx *actx = ctx;
++ struct kevent ev;
++
++ EV_SET(&ev, 1, EVFILT_TIMER, timeout < 0 ? EV_DELETE : EV_ADD,
++ NOTE_MSECONDS, timeout, 0);
++ if (kevent(actx->mux, &ev, 1, NULL, 0, NULL) < 0 && errno != ENOENT)
++ {
++ actx_error(actx, "setting kqueue timer to %ld: %m", timeout);
++ return -1;
++ }
++#endif
+
+ return 0;
+}
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ }
+
+ actx->mux = PGINVALID_SOCKET;
++#ifdef HAVE_SYS_EPOLL_H
+ actx->timerfd = -1;
++#endif
+
+ state->async_ctx = actx;
+
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ case OAUTH_STEP_TOKEN_REQUEST:
+ {
+ const struct token_error *err;
++#ifdef HAVE_SYS_EPOLL_H
+ struct itimerspec spec = {0};
++#endif
++#ifdef HAVE_SYS_EVENT_H
++ struct kevent ev = {0};
++#endif
+
+ if (!finish_token_request(actx, &tok))
+ goto error_return;
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ * Wait for the required interval before issuing the next request.
+ */
+ Assert(actx->authz.interval > 0);
++#ifdef HAVE_SYS_EPOLL_H
+ spec.it_value.tv_sec = actx->authz.interval;
+
+ if (timerfd_settime(actx->timerfd, 0 /* no flags */, &spec, NULL) < 0)
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ }
+
+ *altsock = actx->timerfd;
++#endif
++#ifdef HAVE_SYS_EVENT_H
++ // XXX: I guess this wants to be hidden in a routine
++ EV_SET(&ev, 1, EVFILT_TIMER, EV_ADD, NOTE_MSECONDS,
++ actx->authz.interval * 1000, 0);
++ if (kevent(actx->mux, &ev, 1, NULL, 0, NULL) < 0)
++ {
++ actx_error(actx, "failed to set kqueue timer: %m");
++ goto error_return;
++ }
++ // XXX: why did we change the altsock in the epoll version?
++#endif
+ actx->step = OAUTH_STEP_WAIT_INTERVAL;
+ break;
+ }
3: 0b0b0f2b33 ! 3: 07be9375aa backend: add OAUTHBEARER SASL mechanism
@@ Commit message
correctly is delegated entirely to the oauth_validator_command).
Several TODOs:
- - port to platforms other than "modern Linux"
+ - port to platforms other than "modern Linux/BSD"
- overhaul the communication with oauth_validator_command, which is
currently a bad hack on OpenPipeStream()
- implement more sanity checks on the OAUTHBEARER message format and
@@ src/backend/libpq/auth-oauth.c (new)
+static bool validate(Port *port, const char *auth, const char **logdetail);
+static bool run_validator_command(Port *port, const char *token);
+static bool check_exit(FILE **fh, const char *command);
-+static bool unset_cloexec(int fd);
++static bool set_cloexec(int fd);
+static bool username_ok_for_shell(const char *username);
+
+#define KVSEP 0x01
@@ src/backend/libpq/auth-oauth.c (new)
+ * MUST read all data off of the pipe before writing anything).
+ * TODO: port to Windows using _pipe().
+ */
-+ rc = pipe2(pipefd, O_CLOEXEC);
++ rc = pipe(pipefd);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
@@ src/backend/libpq/auth-oauth.c (new)
+ rfd = pipefd[0];
+ wfd = pipefd[1];
+
-+ /* Allow the read pipe be passed to the child. */
-+ if (!unset_cloexec(rfd))
++ if (!set_cloexec(wfd))
+ {
+ /* error message was already logged */
+ goto cleanup;
@@ src/backend/libpq/auth-oauth.c (new)
+ }
+
+ /* Execute the command. */
-+ fh = OpenPipeStream(command.data, "re");
-+ /* TODO: handle failures */
++ fh = OpenPipeStream(command.data, "r");
++ if (!fh)
++ {
++ ereport(COMMERROR,
++ (errcode_for_file_access(),
++ errmsg("opening pipe to OAuth validator: %m")));
++ goto cleanup;
++ }
+
+ /* We don't need the read end of the pipe anymore. */
+ close(rfd);
@@ src/backend/libpq/auth-oauth.c (new)
+}
+
+static bool
-+unset_cloexec(int fd)
++set_cloexec(int fd)
+{
+ int flags;
+ int rc;
@@ src/backend/libpq/auth-oauth.c (new)
+ return false;
+ }
+
-+ rc = fcntl(fd, F_SETFD, flags & ~FD_CLOEXEC);
++ rc = fcntl(fd, F_SETFD, flags | FD_CLOEXEC);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
-+ errmsg("could not unset FD_CLOEXEC for child pipe: %m")));
++ errmsg("could not set FD_CLOEXEC for child pipe: %m")));
+ return false;
+ }
+
@@ src/include/libpq/auth.h
+
extern PGDLLIMPORT char *pg_krb_server_keyfile;
extern PGDLLIMPORT bool pg_krb_caseins_users;
- extern PGDLLIMPORT bool pg_gss_accept_deleg;
-@@ src/include/libpq/auth.h: extern PGDLLIMPORT char *pg_krb_realm;
+ extern PGDLLIMPORT bool pg_gss_accept_delegation;
+@@ src/include/libpq/auth.h: extern PGDLLIMPORT bool pg_gss_accept_delegation;
extern void ClientAuthentication(Port *port);
extern void sendAuthRequest(Port *port, AuthRequest areq, const char *extradata,
int extralen);
4: 0e8ddadcbf ! 4: 71cedc6ff5 Add pytest suite for OAuth
@@ src/test/python/requirements.txt (new)
+construct~=2.10.61
+isort~=5.6
+# TODO: update to psycopg[c] 3.1
-+psycopg2~=2.8.6
++psycopg2~=2.9.6
+pytest~=7.3
+pytest-asyncio~=0.21.0
5: 65c319a6a3 ! 5: 9b02e14829 squash! Add pytest suite for OAuth
@@ Commit message
TODOs:
- The --tap-stream option to pytest-tap is slightly broken during test
failures (it suppresses error information), which impedes debugging.
- - Unsurprisingly, Windows and Mac builds fail on the Linux-specific
- backend changes.
+ - Unsurprisingly, Windows builds fail on the Linux-/BSD-specific backend
+ changes. 32-bit builds on Ubuntu fail during testing as well.
- pyca/cryptography is pinned at an old version. Since we use it for
testing and not security, this isn't a critical problem yet, but it's
not ideal. Newer versions require a Rust compiler to build, and while
@@ meson.build: foreach test_dir : tests
+ test_command = [
+ pytest,
+ # Avoid running these tests against an existing database.
-+ '--temp-instance', test_output / 'tmp_check',
++ '--temp-instance', test_output / 'data',
+
+ # FIXME pytest-tap's stream feature accidentally suppresses errors that
+ # are critical for debugging:
@@ src/test/python/meson.build (new)
@@
+# Copyright (c) 2023, PostgreSQL Global Development Group
+
++pytest_env = {
++ 'with_oauth': oauth_library,
++
++ # Point to the default database; the tests will create their own databases as
++ # needed.
++ 'PGDATABASE': 'postgres',
++
++ # Avoid the need for a Rust compiler on platforms without prebuilt wheels for
++ # pyca/cryptography.
++ 'CRYPTOGRAPHY_DONT_BUILD_RUST': '1',
++}
++
++# Some modules (psycopg2) need OpenSSL at compile time; for platforms where we
++# might have multiple implementations installed (macOS+brew), try to use the
++# same one that libpq is using.
++if ssl.found()
++ pytest_incdir = ssl.get_variable(pkgconfig: 'includedir', default_value: '')
++ if pytest_incdir != ''
++ pytest_env += { 'CPPFLAGS': '-I@0@'.format(pytest_incdir) }
++ endif
++
++ pytest_libdir = ssl.get_variable(pkgconfig: 'libdir', default_value: '')
++ if pytest_libdir != ''
++ pytest_env += { 'LDFLAGS': '-L@0@'.format(pytest_libdir) }
++ endif
++endif
++
+tests += {
+ 'name': 'python',
+ 'sd': meson.current_source_dir(),
@@ src/test/python/meson.build (new)
+ './test_internals.py',
+ './test_pq3.py',
+ ],
-+ 'env': {
-+ 'with_oauth': oauth_library,
-+
-+ # Point to the default database; the tests will create their own databases
-+ # as needed.
-+ 'PGDATABASE': 'postgres',
-+
-+ # Avoid the need for a Rust compiler on platforms without prebuilt wheels
-+ # for pyca/cryptography.
-+ 'CRYPTOGRAPHY_DONT_BUILD_RUST': '1',
-+ },
++ 'env': pytest_env,
+ },
+}
@@ src/test/python/server/conftest.py: import pq3
+ cleanup_prior_instance(datadir)
+ subprocess.run(["pg_ctl", "-D", datadir, "init"], check=True)
+
++ # The CI looks for *.log files to upload, so the file name here isn't
++ # completely arbitrary.
++ log = os.path.join(datadir, "postmaster.log")
+ port = unused_tcp_port_factory()
-+ log = os.path.join(datadir, "logfile")
+
+ subprocess.run(
+ [
@@ src/test/python/server/conftest.py: import pq3
+ "-l",
+ log,
+ "-o",
-+ f"-c port={port} -c listen_addresses=localhost",
++ " ".join(
++ [
++ f"-c port={port}",
++ "-c listen_addresses=localhost",
++ "-c log_connections=on",
++ ]
++ ),
+ "start",
+ ],
+ check=True,
-: ---------- > 6: 7d179f7e53 XXX work around psycopg2 build failures
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-07-18 23:03 Thomas Munro <thomas.munro@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
0 siblings, 1 reply; 424+ messages in thread
From: Thomas Munro @ 2023-07-18 23:03 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Tue, Jul 18, 2023 at 11:55 AM Jacob Champion <jchampion@timescale.com> wrote:
> We're not setting EV_RECEIPT for these -- is that because none of the
> filters we're using are EV_CLEAR, and so it doesn't matter if we
> accidentally pull pending events off the queue during the kevent() call?
+1 for EV_RECEIPT ("just tell me about errors, don't drain any
events"). I had a vague memory that it caused portability problems.
Just checked... it was OpenBSD I was thinking of, but they finally
added that flag in 6.2 (2017). Our older-than-that BF OpenBSD animal
recently retired so that should be fine. (Yes, without EV_CLEAR it's
"level triggered" not "edge triggered" in epoll terminology, so the
way I had it was not broken, but the way you're suggesting would be
nicer.) Note that you'll have to skip data == 0 (no error) too.
+ #ifdef HAVE_SYS_EVENT_H
+ /* macOS doesn't define the time unit macros, but uses milliseconds
by default. */
+ #ifndef NOTE_MSECONDS
+ #define NOTE_MSECONDS 0
+ #endif
+ #endif
While comparing the cousin OSs' man pages just now, I noticed that
it's not only macOS that lacks NOTE_MSECONDS, it's also OpenBSD and
NetBSD < 10. Maybe just delete that cruft ^^^ and use literal 0 in
fflags directly. FreeBSD, and recently also NetBSD, decided to get
fancy with high resolution timers, but 0 gets the traditional unit of
milliseconds on all platforms (I just wrote it like that because I
started from FreeBSD and didn't know the history/portability story).
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-07-26 16:43 Jacob Champion <jchampion@timescale.com>
parent: Thomas Munro <thomas.munro@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2023-07-26 16:43 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Tue, Jul 18, 2023 at 4:04 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> On Tue, Jul 18, 2023 at 11:55 AM Jacob Champion <jchampion@timescale.com> wrote:
> +1 for EV_RECEIPT ("just tell me about errors, don't drain any
> events").
Sounds good.
> While comparing the cousin OSs' man pages just now, I noticed that
> it's not only macOS that lacks NOTE_MSECONDS, it's also OpenBSD and
> NetBSD < 10. Maybe just delete that cruft ^^^ and use literal 0 in
> fflags directly.
So I don't lose track of it, here's a v10 with those two changes.
Thanks!
--Jacob
Attachments:
since-v9.diff.txt (text/plain, 3.5K) download
1: 9c6a340119 = 1: 0278c7ba90 common/jsonapi: support FRONTEND clients
2: 8072d0416e ! 2: bb3ce4b6a9 libpq: add OAUTHBEARER SASL mechanism
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+#include "libpq-int.h"
+#include "mb/pg_wchar.h"
+
-+#ifdef HAVE_SYS_EVENT_H
-+/* macOS doesn't define the time unit macros, but uses milliseconds by default. */
-+#ifndef NOTE_MSECONDS
-+#define NOTE_MSECONDS 0
-+#endif
-+#endif
-+
+/*
+ * Parsed JSON Representations
+ *
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ switch (what)
+ {
+ case CURL_POLL_IN:
-+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD, 0, 0, 0);
++ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_OUT:
-+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD, 0, 0, 0);
++ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_INOUT:
-+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD, 0, 0, 0);
++ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
-+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD, 0, 0, 0);
++ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ * both, so we try to remove both. This means we need to tolerate
+ * ENOENT below.
+ */
-+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_DELETE, 0, 0, 0);
++ EV_SET(&ev[nev], socket, EVFILT_READ, EV_DELETE | EV_RECEIPT, 0, 0, 0);
+ nev++;
-+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_DELETE, 0, 0, 0);
++ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_DELETE | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ */
+ for (int i = 0; i < res; ++i)
+ {
-+ if (ev_out[i].flags & EV_ERROR && ev_out[i].data != ENOENT)
++ /*
++ * EV_RECEIPT should guarantee one EV_ERROR result for every change,
++ * whether successful or not. Failed entries contain a non-zero errno in
++ * the `data` field.
++ */
++ Assert(ev_out[i].flags & EV_ERROR);
++
++ errno = ev_out[i].data;
++ if (errno && errno != ENOENT)
+ {
-+ errno = ev_out[i].data;
+ switch (what)
+ {
+ case CURL_POLL_REMOVE:
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ struct kevent ev;
+
+ EV_SET(&ev, 1, EVFILT_TIMER, timeout < 0 ? EV_DELETE : EV_ADD,
-+ NOTE_MSECONDS, timeout, 0);
++ 0, timeout, 0);
+ if (kevent(actx->mux, &ev, 1, NULL, 0, NULL) < 0 && errno != ENOENT)
+ {
+ actx_error(actx, "setting kqueue timer to %ld: %m", timeout);
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ // XXX: I guess this wants to be hidden in a routine
-+ EV_SET(&ev, 1, EVFILT_TIMER, EV_ADD, NOTE_MSECONDS,
++ EV_SET(&ev, 1, EVFILT_TIMER, EV_ADD, 0,
+ actx->authz.interval * 1000, 0);
+ if (kevent(actx->mux, &ev, 1, NULL, 0, NULL) < 0)
+ {
3: 07be9375aa = 3: 20b7522228 backend: add OAUTHBEARER SASL mechanism
4: 71cedc6ff5 = 4: f3cec068f9 Add pytest suite for OAuth
5: 9b02e14829 = 5: da1933ac1d squash! Add pytest suite for OAuth
6: 7d179f7e53 = 6: 8f36b5c124 XXX work around psycopg2 build failures
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-08-30 22:57 Jacob Champion <jchampion@timescale.com>
parent: Jacob Champion <jchampion@timescale.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2023-08-30 22:57 UTC (permalink / raw)
To: Thomas Munro <thomas.munro@gmail.com>; +Cc: pgsql-hackers; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
v11 is a quick rebase over the recent Cirrus changes, and I've dropped
0006 now that psycopg2 can build against BSD/Meson setups (thanks Daniele!).
--Jacob
Attachments:
since-v10.diff.txt (text/plain, 3.3K) download
1: 0278c7ba90 = 1: 36409a76ce common/jsonapi: support FRONTEND clients
2: bb3ce4b6a9 = 2: 1356b729db libpq: add OAUTHBEARER SASL mechanism
3: 20b7522228 ! 3: 863a49f863 backend: add OAUTHBEARER SASL mechanism
@@ src/backend/utils/misc/guc_tables.c
#include "nodes/queryjumble.h"
#include "optimizer/cost.h"
@@ src/backend/utils/misc/guc_tables.c: struct config_string ConfigureNamesString[] =
- check_io_direct, assign_io_direct, NULL
+ check_debug_io_direct, assign_debug_io_direct, NULL
},
+ {
4: f3cec068f9 = 4: 348554e5f4 Add pytest suite for OAuth
5: da1933ac1d ! 5: 16d3984a45 squash! Add pytest suite for OAuth
@@ Commit message
- The with_oauth test skip logic should probably be integrated into the
Makefile side as well...
- ## .cirrus.yml ##
-@@ .cirrus.yml: env:
+ ## .cirrus.tasks.yml ##
+@@ .cirrus.tasks.yml: env:
MTEST_ARGS: --print-errorlogs --no-rebuild -C build
PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests
TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf
@@ .cirrus.yml: env:
# What files to preserve in case tests fail
-@@ .cirrus.yml: task:
+@@ .cirrus.tasks.yml: task:
chown root:postgres /tmp/cores
sysctl kern.corefile='/tmp/cores/%N.%P.core'
setup_additional_packages_script: |
@@ .cirrus.yml: task:
# NB: Intentionally build without -Dllvm. The freebsd image size is already
# large enough to make VM startup slow, and even without llvm freebsd
-@@ .cirrus.yml: task:
+@@ .cirrus.tasks.yml: task:
--buildtype=debug \
-Dcassert=true -Duuid=bsd -Dtcl_version=tcl86 -Ddtrace=auto \
-DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
@@ .cirrus.yml: task:
-Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \
build
EOF
-@@ .cirrus.yml: LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
+@@ .cirrus.tasks.yml: LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
--with-libxslt
--with-llvm
--with-lz4
@@ .cirrus.yml: LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
--with-pam
--with-perl
--with-python
-@@ .cirrus.yml: LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
+@@ .cirrus.tasks.yml: LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
LINUX_MESON_FEATURES: &LINUX_MESON_FEATURES >-
-Dllvm=enabled
@@ .cirrus.yml: LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
-Duuid=e2fs
-@@ .cirrus.yml: task:
+@@ .cirrus.tasks.yml: task:
EOF
setup_additional_packages_script: |
@@ .cirrus.yml: task:
matrix:
- name: Linux - Debian Bullseye - Autoconf
-@@ .cirrus.yml: task:
+@@ .cirrus.tasks.yml: task:
folder: $CCACHE_DIR
setup_additional_packages_script: |
@@ src/test/python/requirements.txt
construct~=2.10.61
isort~=5.6
# TODO: update to psycopg[c] 3.1
+-psycopg2~=2.9.6
++psycopg2~=2.9.7
+ pytest~=7.3
+ pytest-asyncio~=0.21.0
## src/test/python/server/conftest.py ##
@@
6: 8f36b5c124 < -: ---------- XXX work around psycopg2 build failures
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-09-06 22:11 Jacob Champion <jchampion@timescale.com>
parent: Jacob Champion <jchampion@timescale.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2023-09-06 22:11 UTC (permalink / raw)
To: mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; +Cc: pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
v12 implements a first draft of a client hook, so applications can
replace either the device prompt or the entire OAuth flow. (Andrey and
Mahendrakar: hopefully this is close to what you need.) It also cleans
up some of the JSON tech debt.
Since (IMO) we don't want to introduce new hooks every time we make
improvements to the internal flows, the new hook is designed to
retrieve multiple pieces of data from the application. Clients either
declare their ability to get that data, or delegate the job to the
next link in the chain, which by default is a no-op. That lets us add
new data types to the end, and older clients will ignore them until
they're taught otherwise. (I'm trying hard not to over-engineer this,
but it seems like the concept of "give me some piece of data to
continue authenticating" could pretty easily subsume things like the
PQsslKeyPassHook if we wanted.)
The PQAUTHDATA_OAUTH_BEARER_TOKEN case is the one that replaces the
flow entirely, as discussed upthread. Your application gets the
discovery URI and the requested scope for the connection. It can then
either delegate back to libpq (e.g. if the issuer isn't one it can
help with), immediately return a token (e.g. if one is already cached
for the current user), or install a nonblocking callback to implement
a custom async flow. When the connection is closed (or fails), the
hook provides a cleanup function to free any resources it may have
allocated.
Thanks,
--Jacob
Attachments:
since-v11.diff.txt (text/plain, 39.3K) download
1: 36409a76ce = 1: 0ff053cf31 common/jsonapi: support FRONTEND clients
2: 1356b729db ! 2: 26a341e1de libpq: add OAUTHBEARER SASL mechanism
@@ Commit message
Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG
The OAuth issuer must support device authorization. No other OAuth flows
- are currently implemented.
+ are currently implemented (but clients may provide their own flows; see
+ below).
The client implementation requires either libcurl or libiddawc and their
development headers. Pass `curl` or `iddawc` to --with-oauth/-Doauth
@@ Commit message
Thomas Munro wrote the kqueue() implementation for oauth-curl; thanks!
+ = PQauthDataHook =
+
+ Clients may override two pieces of OAuth handling using the new
+ PQsetAuthDataHook():
+
+ - PQAUTHDATA_PROMPT_OAUTH_DEVICE: replaces the default user prompt to
+ standard error when using the builtin device authorization flow
+
+ - PQAUTHDATA_OAUTH_BEARER_TOKEN: replaces the entire OAuth flow with a
+ custom asynchronous implementation
+
+ In general, a hook implementation should examine the incoming `type` to
+ decide whether or not to handle a specific piece of authdata; if not, it
+ should delegate to the previous hook in the chain (retrievable via
+ PQgetAuthDataHook()). Otherwise, it should return an integer > 0 and
+ follow the authdata-specific instructions. Returning an integer < 0
+ signals an error condition and abandons the connection attempt.
+
+ == PQAUTHDATA_PROMPT_OAUTH_DEVICE ==
+
+ The hook should display the device prompt (URL + code) using whatever
+ method it prefers.
+
+ == PQAUTHDATA_OAUTH_BEARER_TOKEN ==
+
+ The hook should either directly return a Bearer token for the current
+ user/issuer/scope combination, if one is available without blocking, or
+ else set up an asynchronous callback to retrieve one. See the
+ documentation for PQoauthBearerRequest.
+
Several TODOs:
- don't retry forever if the server won't accept our token
- perform several sanity checks on the OAuth issuer's responses
- handle cases where the client has been set up with an issuer and
scope, but the Postgres server wants to use something different
- improve error debuggability during the OAuth handshake
- - migrate JSON parsing to the new JSON_SEM_ACTION_FAILED API convention
- fix libcurl initialization thread-safety
- harden the libcurl flow implementation
- - figure out how to report the user code and URL without the notice
- processor
+ - figure out pgsocket/int difference on Windows
- ...and more.
## configure ##
@@ src/interfaces/libpq/Makefile: endif
SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi32 -lssl -lsocket -lnsl -lresolv -lintl -lm $(PTHREAD_LIBS), $(LIBS)) $(LDAP_LIBS_FE)
endif
+ ## src/interfaces/libpq/exports.txt ##
+@@ src/interfaces/libpq/exports.txt: PQclosePrepared 188
+ PQclosePortal 189
+ PQsendClosePrepared 190
+ PQsendClosePortal 191
++PQsetAuthDataHook 192
++PQgetAuthDataHook 193
++PQdefaultAuthDataHook 194
+
## src/interfaces/libpq/fe-auth-oauth-curl.c (new) ##
@@
+/*-------------------------------------------------------------------------
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+#include <unistd.h>
+
+#include "common/jsonapi.h"
++#include "fe-auth.h"
+#include "fe-auth-oauth.h"
+#include "libpq-int.h"
+#include "mb/pg_wchar.h"
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ */
+ struct provider provider;
+ struct device_authz authz;
++
++ bool user_prompted; /* have we already sent the authz prompt? */
+};
+
+/*
-+ * Exported function to free the async_ctx, which is stored directly on the
-+ * PGconn. This is called during pqDropConnection() so that we don't leak
-+ * resources even if PQconnectPoll() never calls us back.
++ * Frees the async_ctx, which is stored directly on the PGconn. This is called
++ * during pqDropConnection() so that we don't leak resources even if
++ * PQconnectPoll() never calls us back.
+ *
+ * TODO: we should probably call this at the end of a successful authentication,
+ * too, to proactively free up resources.
+ */
-+void
-+pg_fe_free_oauth_async_ctx(PGconn *conn, void *ctx)
++static void
++free_curl_async_ctx(PGconn *conn, void *ctx)
+{
+ struct async_ctx *actx = ctx;
+
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+#endif
+
+ state->async_ctx = actx;
++ state->free_async_ctx = free_curl_async_ctx;
+
+ initPQExpBuffer(&actx->work_data);
+ initPQExpBuffer(&actx->errbuf);
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ if (!finish_token_request(actx, &tok))
+ goto error_return;
+
-+ /*
-+ * Now that we know the token endpoint isn't broken, give the user
-+ * the login instructions.
-+ *
-+ * TODO: allow client applications to override this handling.
-+ */
-+ fprintf(stderr, "Visit %s and enter the code: %s",
-+ actx->authz.verification_uri, actx->authz.user_code);
++ if (!actx->user_prompted)
++ {
++ int res;
++ PQpromptOAuthDevice prompt = {
++ .verification_uri = actx->authz.verification_uri,
++ .user_code = actx->authz.user_code,
++ /* TODO: optional fields */
++ };
++
++ /*
++ * Now that we know the token endpoint isn't broken, give the
++ * user the login instructions.
++ */
++ res = PQauthDataHook(PQAUTHDATA_PROMPT_OAUTH_DEVICE, conn,
++ &prompt);
++
++ if (!res)
++ {
++ fprintf(stderr, "Visit %s and enter the code: %s",
++ prompt.verification_uri, prompt.user_code);
++ }
++ else if (res < 0)
++ {
++ actx_error(actx, "device prompt failed");
++ goto error_return;
++ }
++
++ actx->user_prompted = true;
++ }
+
+ if (tok.access_token)
+ {
@@ src/interfaces/libpq/fe-auth-oauth-iddawc.c (new)
+
+ if (!user_prompted)
+ {
++ int res;
++ PQpromptOAuthDevice prompt = {
++ .verification_uri = verification_uri,
++ .user_code = user_code,
++ /* TODO: optional fields */
++ };
++
+ /*
+ * Now that we know the token endpoint isn't broken, give the user
+ * the login instructions.
+ */
-+ pqInternalNotice(&conn->noticeHooks,
-+ "Visit %s and enter the code: %s",
-+ verification_uri, user_code);
++ res = PQauthDataHook(PQAUTHDATA_PROMPT_OAUTH_DEVICE, conn,
++ &prompt);
++
++ if (!res)
++ {
++ fprintf(stderr, "Visit %s and enter the code: %s",
++ prompt.verification_uri, prompt.user_code);
++ }
++ else if (res < 0)
++ {
++ appendPQExpBufferStr(&conn->errorMessage,
++ libpq_gettext("device prompt failed\n"));
++ goto cleanup;
++ }
+
+ user_prompted = true;
+ }
@@ src/interfaces/libpq/fe-auth-oauth-iddawc.c (new)
+ /* TODO: actually make this asynchronous */
+ state->token = run_iddawc_auth_flow(conn, conn->oauth_discovery_uri);
+ return state->token ? PGRES_POLLING_OK : PGRES_POLLING_FAILED;
-+}
-+
-+void
-+pg_fe_free_oauth_async_ctx(PGconn *conn, void *ctx)
-+{
-+ /* We currently have no async_ctx, so this should not be called. */
-+ Assert(false);
+}
## src/interfaces/libpq/fe-auth-oauth.c (new) ##
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+ Assert(sasl_mechanism != NULL);
+ Assert(!strcmp(sasl_mechanism, OAUTHBEARER_NAME));
+
-+ state = malloc(sizeof(*state));
++ state = calloc(1, sizeof(*state));
+ if (!state)
+ return NULL;
+
+ state->state = FE_OAUTH_INIT;
+ state->conn = conn;
-+ state->token = NULL;
-+ state->async_ctx = NULL;
+
+ return state;
+}
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+{
+ struct json_ctx *ctx = state;
+
-+ if (oauth_json_has_error(ctx))
-+ return JSON_SUCCESS; /* short-circuit */
-+
+ if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+ }
+
+ ++ctx->nested;
-+ return JSON_SUCCESS; /* TODO: switch all of these to JSON_SEM_ACTION_FAILED */
++ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static JsonParseErrorType
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+{
+ struct json_ctx *ctx = state;
+
-+ if (oauth_json_has_error(ctx))
-+ return JSON_SUCCESS; /* short-circuit */
-+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+{
+ struct json_ctx *ctx = state;
+
-+ if (oauth_json_has_error(ctx))
-+ {
-+ /* short-circuit */
-+ free(name);
-+ return JSON_SUCCESS;
-+ }
-+
+ if (ctx->nested == 1)
+ {
+ if (!strcmp(name, ERROR_STATUS_FIELD))
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+{
+ struct json_ctx *ctx = state;
+
-+ if (oauth_json_has_error(ctx))
-+ return JSON_SUCCESS; /* short-circuit */
-+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+ ctx->target_field_name);
+ }
+
-+ return JSON_SUCCESS;
++ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static JsonParseErrorType
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+{
+ struct json_ctx *ctx = state;
+
-+ if (oauth_json_has_error(ctx))
-+ {
-+ /* short-circuit */
-+ free(token);
-+ return JSON_SUCCESS;
-+ }
-+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+ }
+
+ free(token);
-+ return JSON_SUCCESS;
++ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static bool
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+
+ err = pg_parse_json(&lex, &sem);
+
-+ if (err != JSON_SUCCESS)
-+ {
-+ errmsg = json_errdetail(err, &lex);
-+ }
-+ else if (PQExpBufferDataBroken(ctx.errbuf))
-+ {
-+ errmsg = libpq_gettext("out of memory");
-+ }
-+ else if (ctx.errmsg)
++ if (err == JSON_SEM_ACTION_FAILED)
+ {
-+ errmsg = ctx.errmsg;
++ if (PQExpBufferDataBroken(ctx.errbuf))
++ errmsg = libpq_gettext("out of memory");
++ else if (ctx.errmsg)
++ errmsg = ctx.errmsg;
++ else
++ {
++ /*
++ * Developer error: one of the action callbacks didn't call
++ * oauth_json_set_error() before erroring out.
++ */
++ Assert(oauth_json_has_error(&ctx));
++ errmsg = "<unexpected empty error>";
++ }
+ }
++ else if (err != JSON_SUCCESS)
++ errmsg = json_errdetail(err, &lex);
+
+ if (errmsg)
+ appendPQExpBuffer(&conn->errorMessage,
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+ return true;
+}
+
++static void
++free_request(PGconn *conn, void *vreq)
++{
++ PQoauthBearerRequest *request = vreq;
++
++ if (request->cleanup)
++ request->cleanup(conn, request);
++
++ free(request);
++}
++
++static PostgresPollingStatusType
++run_user_oauth_flow(PGconn *conn, pgsocket *altsock)
++{
++ fe_oauth_state *state = conn->sasl_state;
++ PQoauthBearerRequest *request = state->async_ctx;
++ PostgresPollingStatusType status;
++
++ if (!request->async)
++ {
++ libpq_append_conn_error(conn, "user-defined OAuth flow provided neither a token nor an async callback");
++ return PGRES_POLLING_FAILED;
++ }
++
++ status = request->async(conn, request, altsock);
++ if (status == PGRES_POLLING_FAILED)
++ {
++ libpq_append_conn_error(conn, "user-defined OAuth flow failed");
++ return status;
++ }
++ else if (status == PGRES_POLLING_OK)
++ {
++ /*
++ * We already have a token, so copy it into the state. (We can't
++ * hold onto the original string, since it may not be safe for us to
++ * free() it.)
++ */
++ PQExpBufferData token;
++
++ if (!request->token)
++ {
++ libpq_append_conn_error(conn, "user-defined OAuth flow did not provide a token");
++ return PGRES_POLLING_FAILED;
++ }
++
++ initPQExpBuffer(&token);
++ appendPQExpBuffer(&token, "Bearer %s", request->token);
++
++ if (PQExpBufferDataBroken(token))
++ {
++ libpq_append_conn_error(conn, "out of memory");
++ return PGRES_POLLING_FAILED;
++ }
++
++ state->token = token.data;
++ return PGRES_POLLING_OK;
++ }
++
++ /* TODO: what if no altsock was set? */
++ return status;
++}
++
++static bool
++setup_token_request(PGconn *conn, fe_oauth_state *state)
++{
++ int res;
++ PQoauthBearerRequest request = {
++ .openid_configuration = conn->oauth_discovery_uri,
++ .scope = conn->oauth_scope,
++ };
++
++ Assert(request.openid_configuration);
++
++ /* The client may have overridden the OAuth flow. */
++ res = PQauthDataHook(PQAUTHDATA_OAUTH_BEARER_TOKEN, conn, &request);
++ if (res > 0)
++ {
++ PQoauthBearerRequest *request_copy;
++
++ if (request.token)
++ {
++ /*
++ * We already have a token, so copy it into the state. (We can't
++ * hold onto the original string, since it may not be safe for us to
++ * free() it.)
++ */
++ PQExpBufferData token;
++
++ initPQExpBuffer(&token);
++ appendPQExpBuffer(&token, "Bearer %s", request.token);
++
++ if (PQExpBufferDataBroken(token))
++ {
++ libpq_append_conn_error(conn, "out of memory");
++ goto fail;
++ }
++
++ state->token = token.data;
++
++ /* short-circuit */
++ if (request.cleanup)
++ request.cleanup(conn, &request);
++ return true;
++ }
++
++ request_copy = malloc(sizeof(*request_copy));
++ if (!request_copy)
++ {
++ libpq_append_conn_error(conn, "out of memory");
++ goto fail;
++ }
++
++ memcpy(request_copy, &request, sizeof(request));
++
++ conn->async_auth = run_user_oauth_flow;
++ state->async_ctx = request_copy;
++ state->free_async_ctx = free_request;
++ }
++ else if (res < 0)
++ {
++ libpq_append_conn_error(conn, "user-defined OAuth flow failed");
++ goto fail;
++ }
++ else
++ {
++ /* Use our built-in OAuth flow. */
++ conn->async_auth = pg_fe_run_oauth_flow;
++ }
++
++ return true;
++
++fail:
++ if (request.cleanup)
++ request.cleanup(conn, &request);
++ return false;
++}
++
+static bool
+derive_discovery_uri(PGconn *conn)
+{
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+ }
+
+ /*
-+ * At this point we have to hand the connection over to our
-+ * OAuth implementation. This involves a number of HTTP
-+ * connections and timed waits, so we escape the synchronous
-+ * auth processing and tell PQconnectPoll to transfer control to
-+ * our async implementation.
++ * Decide whether we're using a user-provided OAuth flow, or the
++ * one we have built in.
+ */
-+ conn->async_auth = pg_fe_run_oauth_flow;
-+ state->state = FE_OAUTH_REQUESTING_TOKEN;
-+ return SASL_ASYNC;
-+ }
++ if (!setup_token_request(conn, state))
++ return SASL_FAILED;
+
-+ /*
-+ * If we don't have a discovery URI to be able to request a token,
-+ * we ask the server for one explicitly with an empty token. This
-+ * doesn't require any asynchronous work.
-+ */
-+ state->token = strdup("");
-+ if (!state->token)
++ if (state->token)
++ {
++ /*
++ * A really smart user implementation may have already given
++ * us the token (e.g. if there was an unexpired copy already
++ * cached). In that case, we can just fall through.
++ */
++ }
++ else
++ {
++ /*
++ * Otherwise, we have to hand the connection over to our
++ * OAuth implementation. This involves a number of HTTP
++ * connections and timed waits, so we escape the synchronous
++ * auth processing and tell PQconnectPoll to transfer
++ * control to our async implementation.
++ */
++ Assert(conn->async_auth); /* should have been set already */
++ state->state = FE_OAUTH_REQUESTING_TOKEN;
++ return SASL_ASYNC;
++ }
++ }
++ else
+ {
-+ libpq_append_conn_error(conn, "out of memory");
-+ return SASL_FAILED;
++ /*
++ * If we don't have a discovery URI to be able to request a
++ * token, we ask the server for one explicitly with an empty
++ * token. This doesn't require any asynchronous work.
++ */
++ state->token = strdup("");
++ if (!state->token)
++ {
++ libpq_append_conn_error(conn, "out of memory");
++ return SASL_FAILED;
++ }
+ }
+
+ /* fall through */
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+
+ free(state->token);
+ if (state->async_ctx)
-+ pg_fe_free_oauth_async_ctx(state->conn, state->async_ctx);
++ state->free_async_ctx(state->conn, state->async_ctx);
+
+ free(state);
+}
@@ src/interfaces/libpq/fe-auth-oauth.h (new)
+
+ PGconn *conn;
+ char *token;
++
+ void *async_ctx;
++ void (*free_async_ctx) (PGconn *conn, void *ctx);
+} fe_oauth_state;
+
+extern PostgresPollingStatusType pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock);
-+extern void pg_fe_free_oauth_async_ctx(PGconn *conn, void *ctx);
+
+#endif /* FE_AUTH_OAUTH_H */
@@ src/interfaces/libpq/fe-auth.c: pg_fe_sendauth(AuthRequest areq, int payloadlen,
{
/* Use this message if pg_SASL_continue didn't supply one */
if (conn->errorMessage.len == oldmsglen)
+@@ src/interfaces/libpq/fe-auth.c: PQencryptPasswordConn(PGconn *conn, const char *passwd, const char *user,
+
+ return crypt_pwd;
+ }
++
++PQauthDataHook_type PQauthDataHook = PQdefaultAuthDataHook;
++
++PQauthDataHook_type
++PQgetAuthDataHook(void)
++{
++ return PQauthDataHook;
++}
++
++void
++PQsetAuthDataHook(PQauthDataHook_type hook)
++{
++ PQauthDataHook = hook ? hook : PQdefaultAuthDataHook;
++}
++
++int
++PQdefaultAuthDataHook(PGAuthData type, PGconn *conn, void *data)
++{
++ return 0; /* handle nothing */
++}
## src/interfaces/libpq/fe-auth.h ##
@@
+ #include "libpq-int.h"
++extern PQauthDataHook_type PQauthDataHook;
++
++
/* Prototypes for functions in fe-auth.c */
-extern int pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn);
+extern int pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn,
@@ src/interfaces/libpq/fe-connect.c: keep_going: /* We will come back to here
+ case CONNECTION_AUTHENTICATING:
+ {
+ PostgresPollingStatusType status;
-+ pgsocket altsock;
++ pgsocket altsock = PGINVALID_SOCKET;
+
+ if (!conn->async_auth)
+ {
@@ src/interfaces/libpq/fe-misc.c: pqSocketCheck(PGconn *conn, int forRead, int for
if (result < 0)
## src/interfaces/libpq/libpq-fe.h ##
+@@ src/interfaces/libpq/libpq-fe.h: extern "C"
+ #define LIBPQ_HAS_TRACE_FLAGS 1
+ /* Indicates that PQsslAttribute(NULL, "library") is useful */
+ #define LIBPQ_HAS_SSL_LIBRARY_DETECTION 1
++/* Indicates presence of the PQAUTHDATA_PROMPT_OAUTH_DEVICE authdata hook */
++#define LIBPQ_HAS_PROMPT_OAUTH_DEVICE 1
+
+ /*
+ * Option flags for PQcopyResult
@@ src/interfaces/libpq/libpq-fe.h: typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
@@ src/interfaces/libpq/libpq-fe.h: typedef enum
} ConnStatusType;
typedef enum
+@@ src/interfaces/libpq/libpq-fe.h: typedef enum
+ PQ_PIPELINE_ABORTED
+ } PGpipelineStatus;
+
++typedef enum
++{
++ PQAUTHDATA_PROMPT_OAUTH_DEVICE, /* user must visit a device-authorization URL */
++ PQAUTHDATA_OAUTH_BEARER_TOKEN, /* server requests an OAuth Bearer token */
++} PGAuthData;
++
+ /* PGconn encapsulates a connection to the backend.
+ * The contents of this struct are not supposed to be known to applications.
+ */
+@@ src/interfaces/libpq/libpq-fe.h: extern int PQenv2encoding(void);
+
+ /* === in fe-auth.c === */
+
++typedef struct _PQpromptOAuthDevice
++{
++ const char *verification_uri; /* verification URI to visit */
++ const char *user_code; /* user code to enter */
++} PQpromptOAuthDevice;
++
++typedef struct _PQoauthBearerRequest
++{
++ /* Hook inputs (constant across all calls) */
++ const char * const openid_configuration; /* OIDC discovery URI */
++ const char * const scope; /* required scope(s), or NULL */
++
++ /* Hook outputs */
++
++ /*
++ * Callback implementing a custom asynchronous OAuth flow.
++ *
++ * The callback may return
++ * - PGRES_POLLING_READING/WRITING, to indicate that a file descriptor has
++ * been stored in *altsock and libpq should wait until it is readable or
++ * writable before calling back;
++ * - PGRES_POLLING_OK, to indicate that the flow is complete and
++ * request->token has been set; or
++ * - PGRES_POLLING_FAILED, to indicate that token retrieval has failed.
++ *
++ * This callback is optional. If the token can be obtained without blocking
++ * during the original call to the PQAUTHDATA_OAUTH_BEARER_TOKEN hook, it
++ * may be returned directly, but one of request->async or request->token
++ * must be set by the hook.
++ */
++ PostgresPollingStatusType (*async) (PGconn *conn,
++ struct _PQoauthBearerRequest *request,
++ int *altsock);
++
++ /*
++ * Callback to clean up custom allocations. A hook implementation may use
++ * this to free request->token and any resources in request->user.
++ *
++ * This is technically optional, but highly recommended, because there is no
++ * other indication as to when it is safe to free the token.
++ */
++ void (*cleanup) (PGconn *conn, struct _PQoauthBearerRequest *request);
++
++ /*
++ * The hook should set this to the Bearer token contents for the connection,
++ * once the flow is completed. The token contents must remain available to
++ * libpq until the hook's cleanup callback is called.
++ */
++ char *token;
++
++ /*
++ * Hook-defined data. libpq will not modify this pointer across calls to the
++ * async callback, so it can be used to keep track of application-specific
++ * state. Resources allocated here should be freed by the cleanup callback.
++ */
++ void *user;
++} PQoauthBearerRequest;
++
+ extern char *PQencryptPassword(const char *passwd, const char *user);
+ extern char *PQencryptPasswordConn(PGconn *conn, const char *passwd, const char *user, const char *algorithm);
+
++typedef int (*PQauthDataHook_type) (PGAuthData type, PGconn *conn, void *data);
++extern void PQsetAuthDataHook(PQauthDataHook_type hook);
++extern PQauthDataHook_type PQgetAuthDataHook(void);
++extern int PQdefaultAuthDataHook(PGAuthData type, PGconn *conn, void *data);
++
+ /* === in encnames.c === */
+
+ extern int pg_char_to_encoding(const char *name);
## src/interfaces/libpq/libpq-int.h ##
@@ src/interfaces/libpq/libpq-int.h: typedef struct pg_conn_host
3: 863a49f863 = 3: d02bc9a466 backend: add OAUTHBEARER SASL mechanism
4: 348554e5f4 ! 4: fded01d22b Add pytest suite for OAuth
@@ src/test/python/client/test_oauth.py (new)
+#
+
+import base64
++import collections
++import ctypes
+import http.server
+import json
++import logging
++import os
++import platform
+import secrets
+import sys
+import threading
+import time
++import traceback
++import types
+import urllib.parse
+from numbers import Number
+
@@ src/test/python/client/test_oauth.py (new)
+
+from .conftest import BLOCKING_TIMEOUT
+
++if platform.system() == "Darwin":
++ libpq = ctypes.cdll.LoadLibrary("libpq.5.dylib")
++else:
++ libpq = ctypes.cdll.LoadLibrary("libpq.so.5")
++
+
+def finish_handshake(conn):
+ """
@@ src/test/python/client/test_oauth.py (new)
+ thread.stop()
+
+
++#
++# PQAuthDataHook implementation, matching libpq.h
++#
++
++
++PQAUTHDATA_PROMPT_OAUTH_DEVICE = 0
++PQAUTHDATA_OAUTH_BEARER_TOKEN = 1
++
++PGRES_POLLING_FAILED = 0
++PGRES_POLLING_READING = 1
++PGRES_POLLING_WRITING = 2
++PGRES_POLLING_OK = 3
++
++
++class PQPromptOAuthDevice(ctypes.Structure):
++ _fields_ = [
++ ("verification_uri", ctypes.c_char_p),
++ ("user_code", ctypes.c_char_p),
++ ]
++
++
++class PQOAuthBearerRequest(ctypes.Structure):
++ pass
++
++
++PQOAuthBearerRequest._fields_ = [
++ ("openid_configuration", ctypes.c_char_p),
++ ("scope", ctypes.c_char_p),
++ (
++ "async_",
++ ctypes.CFUNCTYPE(
++ ctypes.c_int,
++ ctypes.c_void_p,
++ ctypes.POINTER(PQOAuthBearerRequest),
++ ctypes.POINTER(ctypes.c_int),
++ ),
++ ),
++ (
++ "cleanup",
++ ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(PQOAuthBearerRequest)),
++ ),
++ ("token", ctypes.c_char_p),
++ ("user", ctypes.c_void_p),
++]
++
++
++@pytest.fixture
++def auth_data_cb():
++ """
++ Tracks calls to the libpq authdata hook. The yielded object contains a calls
++ member that records the data sent to the hook. If a test needs to perform
++ custom actions during a call, it can set the yielded object's impl callback;
++ beware that the callback takes place on a different thread.
++
++ This is done differently from the other callback implementations on purpose.
++ For the others, we can declare test-specific callbacks and have them perform
++ direct assertions on the data they receive. But that won't work for a C
++ callback, because there's no way for us to bubble up the assertion through
++ libpq. Instead, this mock-style approach is taken, where we just record the
++ calls and let the test examine them later.
++ """
++
++ class _Call:
++ pass
++
++ class _cb(object):
++ def __init__(self):
++ self.calls = []
++
++ cb = _cb()
++ cb.impl = None
++
++ # The callback will occur on a different thread, so protect the cb object.
++ cb_lock = threading.Lock()
++
++ @ctypes.CFUNCTYPE(ctypes.c_int, ctypes.c_byte, ctypes.c_void_p, ctypes.c_void_p)
++ def auth_data_cb(typ, pgconn, data):
++ handle_by_default = 0 # does an implementation have to be provided?
++
++ if typ == PQAUTHDATA_PROMPT_OAUTH_DEVICE:
++ cls = PQPromptOAuthDevice
++ handle_by_default = 1
++ elif typ == PQAUTHDATA_OAUTH_BEARER_TOKEN:
++ cls = PQOAuthBearerRequest
++ else:
++ return 0
++
++ call = _Call()
++ call.type = typ
++
++ # The lifetime of the underlying data being pointed to doesn't
++ # necessarily match the lifetime of the Python object, so we can't
++ # reference a Structure's fields after returning. Explicitly copy the
++ # contents over, field by field.
++ data = ctypes.cast(data, ctypes.POINTER(cls))
++ for name, _ in cls._fields_:
++ setattr(call, name, getattr(data.contents, name))
++
++ with cb_lock:
++ cb.calls.append(call)
++
++ if cb.impl:
++ # Pass control back to the test.
++ try:
++ return cb.impl(typ, pgconn, data.contents)
++ except Exception:
++ # This can't escape into the C stack, but we can fail the flow
++ # and hope the traceback gives us enough detail.
++ logging.error(
++ "Exception during authdata hook callback:\n"
++ + traceback.format_exc()
++ )
++ return -1
++
++ return handle_by_default
++
++ libpq.PQsetAuthDataHook(auth_data_cb)
++ try:
++ yield cb
++ finally:
++ # The callback is about to go out of scope, so make sure libpq is
++ # disconnected from it. (We wouldn't want to accidentally influence
++ # later tests anyway.)
++ libpq.PQsetAuthDataHook(None)
++
++
+@pytest.mark.parametrize("secret", [None, "", "hunter2"])
+@pytest.mark.parametrize("scope", [None, "", "openid email"])
+@pytest.mark.parametrize("retries", [0, 1])
@@ src/test/python/client/test_oauth.py (new)
+ ],
+)
+def test_oauth_with_explicit_issuer(
-+ capfd, accept, openid_provider, asynchronous, retries, scope, secret
++ accept, openid_provider, asynchronous, retries, scope, secret, auth_data_cb
+):
+ client_id = secrets.token_hex()
+
@@ src/test/python/client/test_oauth.py (new)
+ finish_handshake(conn)
+
+ if retries:
-+ # Finally, make sure that the client prompted the user with the expected
-+ # authorization URL and user code.
-+ expected = f"Visit {verification_url} and enter the code: {user_code}"
-+ _, stderr = capfd.readouterr()
-+ assert expected in stderr
++ # Finally, make sure that the client prompted the user once with the
++ # expected authorization URL and user code.
++ assert len(auth_data_cb.calls) == 2
++
++ # First call should have been for a custom flow, which we ignored.
++ assert auth_data_cb.calls[0].type == PQAUTHDATA_OAUTH_BEARER_TOKEN
++
++ # Second call is for our user prompt.
++ call = auth_data_cb.calls[1]
++ assert call.type == PQAUTHDATA_PROMPT_OAUTH_DEVICE
++ assert call.verification_uri.decode() == verification_url
++ assert call.user_code.decode() == user_code
+
+
+def expect_disconnected_handshake(sock):
@@ src/test/python/client/test_oauth.py (new)
+ finish_handshake(conn)
+
+
++@pytest.fixture
++def self_pipe():
++ """
++ Yields a pipe fd pair.
++ """
++
++ class _Pipe:
++ pass
++
++ p = _Pipe()
++ p.readfd, p.writefd = os.pipe()
++
++ try:
++ yield p
++ finally:
++ os.close(p.readfd)
++ os.close(p.writefd)
++
++
++@pytest.mark.parametrize("scope", [None, "", "openid email"])
++@pytest.mark.parametrize(
++ "retries",
++ [
++ -1, # no async callback
++ 0, # async callback immediately returns token
++ 1, # async callback waits on altsock once
++ 2, # async callback waits on altsock twice
++ ],
++)
++@pytest.mark.parametrize(
++ "asynchronous",
++ [
++ pytest.param(False, id="synchronous"),
++ pytest.param(True, id="asynchronous"),
++ ],
++)
++def test_user_defined_flow(
++ accept, auth_data_cb, self_pipe, scope, retries, asynchronous
++):
++ issuer = "http://localhost"
++ discovery_uri = issuer + "/.well-known/openid-configuration"
++ access_token = secrets.token_urlsafe()
++
++ sock, _ = accept(
++ oauth_issuer=issuer,
++ oauth_client_id="some-id",
++ oauth_scope=scope,
++ async_=asynchronous,
++ )
++
++ # Track callbacks.
++ attempts = 0
++ wakeup_called = False
++ cleanup_calls = 0
++ lock = threading.Lock()
++
++ def wakeup():
++ """Writes a byte to the wakeup pipe."""
++ nonlocal wakeup_called
++ with lock:
++ wakeup_called = True
++ os.write(self_pipe.writefd, b"\0")
++
++ def get_token(pgconn, request, p_altsock):
++ """
++ Async token callback. While attempts < retries, libpq will be instructed
++ to wait on the self_pipe. When attempts == retries, the token will be
++ set.
++
++ Note that assertions and exceptions raised here are allowed but not very
++ helpful, since they can't bubble through the libpq stack to be collected
++ by the test suite. Try not to rely too heavily on them.
++ """
++ # Make sure libpq passed our user data through.
++ assert request.user == 42
++
++ with lock:
++ nonlocal attempts, wakeup_called
++
++ if attempts:
++ # If we've already started the timer, we shouldn't get a
++ # call back before it trips.
++ assert wakeup_called, "authdata hook was called before the timer"
++
++ # Drain the wakeup byte.
++ os.read(self_pipe.readfd, 1)
++
++ if attempts < retries:
++ attempts += 1
++
++ # Wake up the client in a little bit of time.
++ wakeup_called = False
++ threading.Timer(0.1, wakeup).start()
++
++ # Tell libpq to wait on the other end of the wakeup pipe.
++ p_altsock[0] = self_pipe.readfd
++ return PGRES_POLLING_READING
++
++ # Done!
++ request.token = access_token.encode()
++ return PGRES_POLLING_OK
++
++ @ctypes.CFUNCTYPE(
++ ctypes.c_int,
++ ctypes.c_void_p,
++ ctypes.POINTER(PQOAuthBearerRequest),
++ ctypes.POINTER(ctypes.c_int),
++ )
++ def get_token_wrapper(pgconn, p_request, p_altsock):
++ """
++ Translation layer between C and Python for the async callback.
++ Assertions and exceptions will be swallowed at the boundary, so make
++ sure they don't escape here.
++ """
++ try:
++ return get_token(pgconn, p_request.contents, p_altsock)
++ except Exception:
++ logging.error("Exception during async callback:\n" + traceback.format_exc())
++ return PGRES_POLLING_FAILED
++
++ @ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(PQOAuthBearerRequest))
++ def cleanup(pgconn, p_request):
++ """
++ Should be called exactly once per connection.
++ """
++ nonlocal cleanup_calls
++ with lock:
++ cleanup_calls += 1
++
++ def bearer_hook(typ, pgconn, request):
++ """
++ Implementation of the PQAuthDataHook, which either sets up an async
++ callback or returns the token directly, depending on the value of
++ retries.
++
++ As above, try not to rely too much on assertions/exceptions here.
++ """
++ assert typ == PQAUTHDATA_OAUTH_BEARER_TOKEN
++ request.cleanup = cleanup
++
++ if retries < 0:
++ # Special case: return a token immediately without a callback.
++ request.token = access_token.encode()
++ return 1
++
++ # Tell libpq to call us back.
++ request.async_ = get_token_wrapper
++ request.user = ctypes.c_void_p(42) # will be checked in the callback
++ return 1
++
++ auth_data_cb.impl = bearer_hook
++
++ # Now drive the server side.
++ with sock:
++ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
++ # Initiate a handshake, which should result in our custom callback
++ # being invoked to fetch the token.
++ initial = start_oauth_handshake(conn)
++
++ # Validate and accept the token.
++ auth = get_auth_value(initial)
++ assert auth == f"Bearer {access_token}".encode("ascii")
++
++ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
++ finish_handshake(conn)
++
++ # Check the data provided to the hook.
++ assert len(auth_data_cb.calls) == 1
++
++ call = auth_data_cb.calls[0]
++ assert call.type == PQAUTHDATA_OAUTH_BEARER_TOKEN
++ assert call.openid_configuration.decode() == discovery_uri
++ assert call.scope == (None if scope is None else scope.encode())
++
++ # Make sure we cleaned up after ourselves.
++ assert cleanup_calls == 1
++
++
+def alt_patterns(*patterns):
+ """
+ Just combines multiple alternative regexes into one. It's not very efficient
5: 16d3984a45 ! 5: 38a9691801 squash! Add pytest suite for OAuth
@@ src/test/python/README: To make quick smoke tests possible, slow tests have been
+ $ py.test --temp-instance=./tmp_check
## src/test/python/client/test_oauth.py ##
-@@
- import base64
- import http.server
- import json
-+import os
- import secrets
- import sys
- import threading
@@ src/test/python/client/test_oauth.py: import pq3
from .conftest import BLOCKING_TIMEOUT
@@ src/test/python/client/test_oauth.py: import pq3
+ reason="OAuth client tests require --with-oauth support",
+)
+
-
- def finish_handshake(conn):
- """
+ if platform.system() == "Darwin":
+ libpq = ctypes.cdll.LoadLibrary("libpq.5.dylib")
+ else:
## src/test/python/conftest.py ##
@@ src/test/python/conftest.py: import os
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-11-03 12:28 Shlok Kyal <shlok.kyal.oss@gmail.com>
parent: Jacob Champion <jchampion@timescale.com>
0 siblings, 1 reply; 424+ messages in thread
From: Shlok Kyal @ 2023-11-03 12:28 UTC (permalink / raw)
To: Jacob Champion <jchampion@timescale.com>; +Cc: mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
Hi,
On Fri, 3 Nov 2023 at 17:14, Jacob Champion <jchampion@timescale.com> wrote:
>
> v12 implements a first draft of a client hook, so applications can
> replace either the device prompt or the entire OAuth flow. (Andrey and
> Mahendrakar: hopefully this is close to what you need.) It also cleans
> up some of the JSON tech debt.
I went through CFbot and found that build is failing, links:
https://cirrus-ci.com/task/6061898244816896
https://cirrus-ci.com/task/6624848198238208
https://cirrus-ci.com/task/5217473314684928
https://cirrus-ci.com/task/6343373221527552
Just want to make sure you are aware of these failures.
Thanks,
Shlok Kumar Kyal
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-11-03 23:48 Jacob Champion <champion.p@gmail.com>
parent: Shlok Kyal <shlok.kyal.oss@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2023-11-03 23:48 UTC (permalink / raw)
To: Shlok Kyal <shlok.kyal.oss@gmail.com>; +Cc: Jacob Champion <jchampion@timescale.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Fri, Nov 3, 2023 at 5:28 AM Shlok Kyal <shlok.kyal.oss@gmail.com> wrote:
> Just want to make sure you are aware of these failures.
Thanks for the nudge! Looks like I need to reconcile with the changes
to JsonLexContext in 1c99cde2. I should be able to get to that next
week; in the meantime I'll mark it Waiting on Author.
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-11-08 19:00 Jacob Champion <champion.p@gmail.com>
parent: Jacob Champion <champion.p@gmail.com>
0 siblings, 2 replies; 424+ messages in thread
From: Jacob Champion @ 2023-11-08 19:00 UTC (permalink / raw)
To: Shlok Kyal <shlok.kyal.oss@gmail.com>; +Cc: mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Fri, Nov 3, 2023 at 4:48 PM Jacob Champion <champion.p@gmail.com> wrote:
> Thanks for the nudge! Looks like I need to reconcile with the changes
> to JsonLexContext in 1c99cde2. I should be able to get to that next
> week; in the meantime I'll mark it Waiting on Author.
v13 rebases over latest. The JsonLexContext changes have simplified
0001 quite a bit, and there's probably a bit more minimization that
could be done.
Unfortunately the configure/Makefile build of libpq now seems to be
pulling in an `exit()` dependency in a way that Meson does not. (Or
maybe Meson isn't checking?) I still need to investigate that
difference and fix it, so I recommend Meson if you're looking to
test-drive a build.
Thanks,
--Jacob
Attachments:
since-v12.diff.txt (text/plain, 15.2K) download
1: e80c124c3d ! 1: 1e17a1059f common/jsonapi: support FRONTEND clients
@@ Commit message
memory owned by the JsonLexContext, so clients don't need to worry about
freeing it.
- For convenience, the backend now has destroyJsonLexContext() to mirror
- other create/destroy APIs. The frontend has init/term versions of the
- API to handle stack-allocated JsonLexContexts.
-
We can now partially revert b44669b2ca, now that json_errdetail() works
correctly.
- ## src/backend/utils/adt/jsonfuncs.c ##
-@@ src/backend/utils/adt/jsonfuncs.c: json_object_keys(PG_FUNCTION_ARGS)
- pg_parse_json_or_ereport(lex, sem);
- /* keys are now in state->result */
-
-- pfree(lex->strval->data);
-- pfree(lex->strval);
-- pfree(lex);
-+ destroyJsonLexContext(lex);
- pfree(sem);
-
- MemoryContextSwitchTo(oldcontext);
-
## src/bin/pg_verifybackup/parse_manifest.c ##
-@@ src/bin/pg_verifybackup/parse_manifest.c: void
- json_parse_manifest(JsonManifestParseContext *context, char *buffer,
- size_t size)
- {
-- JsonLexContext *lex;
-+ JsonLexContext lex = {0};
- JsonParseErrorType json_error;
- JsonSemAction sem;
- JsonManifestParseState parse;
-@@ src/bin/pg_verifybackup/parse_manifest.c: json_parse_manifest(JsonManifestParseContext *context, char *buffer,
- parse.state = JM_EXPECT_TOPLEVEL_START;
- parse.saw_version_field = false;
-
-- /* Create a JSON lexing context. */
-- lex = makeJsonLexContextCstringLen(buffer, size, PG_UTF8, true);
-+ /* Initialize a JSON lexing context. */
-+ initJsonLexContextCstringLen(&lex, buffer, size, PG_UTF8, true);
-
- /* Set up semantic actions. */
- sem.semstate = &parse;
@@ src/bin/pg_verifybackup/parse_manifest.c: json_parse_manifest(JsonManifestParseContext *context, char *buffer,
- sem.scalar = json_manifest_scalar;
-
/* Run the actual JSON parser. */
-- json_error = pg_parse_json(lex, &sem);
-+ json_error = pg_parse_json(&lex, &sem);
+ json_error = pg_parse_json(lex, &sem);
if (json_error != JSON_SUCCESS)
- json_manifest_parse_failure(context, "parsing failed");
-+ json_manifest_parse_failure(context, json_errdetail(json_error, &lex));
++ json_manifest_parse_failure(context, json_errdetail(json_error, lex));
if (parse.state != JM_EXPECT_EOF)
json_manifest_parse_failure(context, "manifest ended unexpectedly");
- /* Verify the manifest checksum. */
- verify_manifest_checksum(&parse, buffer, size);
-+
-+ /* Clean up. */
-+ termJsonLexContext(&lex);
- }
-
- /*
## src/bin/pg_verifybackup/t/005_bad_manifest.pl ##
@@ src/bin/pg_verifybackup/t/005_bad_manifest.pl: use Test::More;
@@ src/common/jsonapi.c
/*
* The context of the parser is maintained by the recursive descent
* mechanism, but is passed explicitly to the error reporting routine
-@@ src/common/jsonapi.c: IsValidJsonNumber(const char *str, int len)
- return (!numeric_error) && (total_len == dummy_lex.input_length);
- }
-
-+#ifndef FRONTEND
-+
- /*
- * makeJsonLexContextCstringLen
- *
-- * lex constructor, with or without StringInfo object for de-escaped lexemes.
-+ * lex constructor, with or without a string object for de-escaped lexemes.
- *
- * Without is better as it makes the processing faster, so only make one
- * if really required.
-@@ src/common/jsonapi.c: makeJsonLexContextCstringLen(char *json, int len, int encoding, bool need_escape
- {
- JsonLexContext *lex = palloc0(sizeof(JsonLexContext));
-
-+ initJsonLexContextCstringLen(lex, json, len, encoding, need_escapes);
-+
-+ return lex;
-+}
-+
-+void
-+destroyJsonLexContext(JsonLexContext *lex)
-+{
-+ termJsonLexContext(lex);
-+ pfree(lex);
-+}
-+
-+#endif /* !FRONTEND */
-+
-+void
-+initJsonLexContextCstringLen(JsonLexContext *lex, char *json, int len, int encoding, bool need_escapes)
-+{
- lex->input = lex->token_terminator = lex->line_start = json;
- lex->line_number = 1;
- lex->input_length = len;
+@@ src/common/jsonapi.c: makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
lex->input_encoding = encoding;
-- if (need_escapes)
+ if (need_escapes)
+ {
- lex->strval = makeStringInfo();
-- return lex;
-+ lex->parse_strval = need_escapes;
-+ if (lex->parse_strval)
-+ {
+ /*
+ * This call can fail in FRONTEND code. We defer error handling to time
-+ * of use (json_lex_string()) since there's no way to signal failure
-+ * here, and we might not need to parse any strings anyway.
++ * of use (json_lex_string()) since we might not need to parse any
++ * strings anyway.
+ */
+ lex->strval = createStrVal();
-+ }
+ lex->flags |= JSONLEX_FREE_STRVAL;
++ lex->parse_strval = true;
+ }
+ lex->errormsg = NULL;
-+}
-+
-+void
-+termJsonLexContext(JsonLexContext *lex)
-+{
+
+ return lex;
+ }
+@@ src/common/jsonapi.c: makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
+ void
+ freeJsonLexContext(JsonLexContext *lex)
+ {
+ static const JsonLexContext empty = {0};
+
-+ if (lex->strval)
-+ {
+ if (lex->flags & JSONLEX_FREE_STRVAL)
+ {
+#ifdef FRONTEND
+ destroyPQExpBuffer(lex->strval);
+#else
-+ pfree(lex->strval->data);
-+ pfree(lex->strval);
+ pfree(lex->strval->data);
+ pfree(lex->strval);
+#endif
+ }
-+
+ if (lex->errormsg)
+ {
+#ifdef FRONTEND
@@ src/common/jsonapi.c: makeJsonLexContextCstringLen(char *json, int len, int enco
+ pfree(lex->errormsg->data);
+ pfree(lex->errormsg);
+#endif
-+ }
-+
-+ *lex = empty;
+ }
+ if (lex->flags & JSONLEX_FREE_STRUCT)
+ pfree(lex);
++ else
++ *lex = empty;
}
/*
@@ src/common/jsonapi.c: json_errdetail(JsonParseErrorType error, JsonLexContext *l
+ return lex->errormsg->data;
+}
+ ## src/common/meson.build ##
+@@ src/common/meson.build: common_sources_frontend_static += files(
+ # least cryptohash_openssl.c, hmac_openssl.c depend on it.
+ # controldata_utils.c depends on wait_event_types_h. That's arguably a
+ # layering violation, but ...
++#
++# XXX Frontend builds need libpq's pqexpbuffer.h, so adjust the include paths
++# appropriately. This seems completely broken.
+ pgcommon = {}
+ pgcommon_variants = {
+ '_srv': internal_lib_args + {
++ 'include_directories': include_directories('.'),
+ 'sources': common_sources + [lwlocknames_h] + [wait_event_types_h],
+ 'dependencies': [backend_common_code],
+ },
+ '': default_lib_args + {
++ 'include_directories': include_directories('../interfaces/libpq', '.'),
+ 'sources': common_sources_frontend_static,
+ 'dependencies': [frontend_common_code],
+ # Files in libpgcommon.a should use/export the "xxx_private" versions
+@@ src/common/meson.build: pgcommon_variants = {
+ },
+ '_shlib': default_lib_args + {
+ 'pic': true,
++ 'include_directories': include_directories('../interfaces/libpq', '.'),
+ 'sources': common_sources_frontend_shlib,
+ 'dependencies': [frontend_common_code],
+ },
+@@ src/common/meson.build: foreach name, opts : pgcommon_variants
+ c_args = opts.get('c_args', []) + common_cflags[cflagname]
+ cflag_libs += static_library('libpgcommon@0@_@1@'.format(name, cflagname),
+ c_pch: pch_c_h,
+- include_directories: include_directories('.'),
+ kwargs: opts + {
+ 'sources': sources,
+ 'c_args': c_args,
+@@ src/common/meson.build: foreach name, opts : pgcommon_variants
+ lib = static_library('libpgcommon@0@'.format(name),
+ link_with: cflag_libs,
+ c_pch: pch_c_h,
+- include_directories: include_directories('.'),
+ kwargs: opts + {
+ 'dependencies': opts['dependencies'] + [ssl],
+ }
+
## src/include/common/jsonapi.h ##
@@
#ifndef JSONAPI_H
@@ src/include/common/jsonapi.h: typedef enum JsonParseErrorType
JSON_UNICODE_ESCAPE_FORMAT,
JSON_UNICODE_HIGH_ESCAPE,
@@ src/include/common/jsonapi.h: typedef enum JsonParseErrorType
- JSON_SEM_ACTION_FAILED /* error should already be reported */
+ JSON_SEM_ACTION_FAILED, /* error should already be reported */
} JsonParseErrorType;
+/*
@@ src/include/common/jsonapi.h: typedef enum JsonParseErrorType
/*
* All the fields in this structure should be treated as read-only.
@@ src/include/common/jsonapi.h: typedef struct JsonLexContext
- int lex_level;
+ bits32 flags;
int line_number; /* line number, starting from 1 */
char *line_start; /* where that line starts within input */
- StringInfo strval;
@@ src/include/common/jsonapi.h: typedef struct JsonLexContext
} JsonLexContext;
typedef JsonParseErrorType (*json_struct_action) (void *state);
-@@ src/include/common/jsonapi.h: extern PGDLLIMPORT JsonSemAction nullSemAction;
- */
- extern JsonParseErrorType json_count_array_elements(JsonLexContext *lex,
- int *elements);
-+#ifndef FRONTEND
-
- /*
-- * constructor for JsonLexContext, with or without strval element.
-+ * allocating constructor for JsonLexContext, with or without strval element.
- * If supplied, the strval element will contain a de-escaped version of
- * the lexeme. However, doing this imposes a performance penalty, so
- * it should be avoided if the de-escaped lexeme is not required.
-@@ src/include/common/jsonapi.h: extern JsonLexContext *makeJsonLexContextCstringLen(char *json,
- int encoding,
- bool need_escapes);
-
-+/*
-+ * Counterpart to makeJsonLexContextCstringLen(): clears and deallocates lex.
-+ * The context pointer should not be used after this call.
-+ */
-+extern void destroyJsonLexContext(JsonLexContext *lex);
-+
-+#endif /* !FRONTEND */
-+
-+/*
-+ * stack constructor for JsonLexContext, with or without strval element.
-+ * If supplied, the strval element will contain a de-escaped version of
-+ * the lexeme. However, doing this imposes a performance penalty, so
-+ * it should be avoided if the de-escaped lexeme is not required.
-+ */
-+extern void initJsonLexContextCstringLen(JsonLexContext *lex,
-+ char *json,
-+ int len,
-+ int encoding,
-+ bool need_escapes);
-+
-+/*
-+ * Counterpart to initJsonLexContextCstringLen(): clears the contents of lex,
-+ * but does not deallocate lex itself.
-+ */
-+extern void termJsonLexContext(JsonLexContext *lex);
-+
- /* lex one token */
- extern JsonParseErrorType json_lex(JsonLexContext *lex);
-
2: 02eea9ffe0 ! 2: e941ba5807 libpq: add OAUTHBEARER SASL mechanism
@@ src/Makefile.global.in: with_ldap = @with_ldap@
with_uuid = @with_uuid@
with_zlib = @with_zlib@
- ## src/common/meson.build ##
-@@ src/common/meson.build: common_sources_frontend_static += files(
- # least cryptohash_openssl.c, hmac_openssl.c depend on it.
- # controldata_utils.c depends on wait_event_types_h. That's arguably a
- # layering violation, but ...
-+#
-+# XXX Frontend builds need libpq's pqexpbuffer.h, so adjust the include paths
-+# appropriately. This seems completely broken.
- pgcommon = {}
- pgcommon_variants = {
- '_srv': internal_lib_args + {
-+ 'include_directories': include_directories('.'),
- 'sources': common_sources + [lwlocknames_h] + [wait_event_types_h],
- 'dependencies': [backend_common_code],
- },
- '': default_lib_args + {
-+ 'include_directories': include_directories('../interfaces/libpq', '.'),
- 'sources': common_sources_frontend_static,
- 'dependencies': [frontend_common_code],
- },
- '_shlib': default_lib_args + {
- 'pic': true,
-+ 'include_directories': include_directories('../interfaces/libpq', '.'),
- 'sources': common_sources_frontend_shlib,
- 'dependencies': [frontend_common_code],
- },
-@@ src/common/meson.build: foreach name, opts : pgcommon_variants
- c_args = opts.get('c_args', []) + common_cflags[cflagname]
- cflag_libs += static_library('libpgcommon@0@_@1@'.format(name, cflagname),
- c_pch: pch_c_h,
-- include_directories: include_directories('.'),
- kwargs: opts + {
- 'sources': sources,
- 'c_args': c_args,
-@@ src/common/meson.build: foreach name, opts : pgcommon_variants
- lib = static_library('libpgcommon@0@'.format(name),
- link_with: cflag_libs,
- c_pch: pch_c_h,
-- include_directories: include_directories('.'),
- kwargs: opts + {
- 'dependencies': opts['dependencies'] + [ssl],
- }
-
## src/include/common/oauth-common.h (new) ##
@@
+/*-------------------------------------------------------------------------
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ goto cleanup;
+ }
+
-+ initJsonLexContextCstringLen(&lex, resp->data, resp->len, PG_UTF8, true);
++ makeJsonLexContextCstringLen(&lex, resp->data, resp->len, PG_UTF8, true);
+
+ ctx.errbuf = &actx->errbuf;
+ ctx.fields = fields;
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ success = true;
+
+cleanup:
-+ termJsonLexContext(&lex);
++ freeJsonLexContext(&lex);
+ return success;
+}
+
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+ return false;
+ }
+
-+ initJsonLexContextCstringLen(&lex, msg, msglen, PG_UTF8, true);
++ makeJsonLexContextCstringLen(&lex, msg, msglen, PG_UTF8, true);
+
+ initPQExpBuffer(&ctx.errbuf);
+ sem.semstate = &ctx;
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+
+ /* Don't need the error buffer or the JSON lexer anymore. */
+ termPQExpBuffer(&ctx.errbuf);
-+ termJsonLexContext(&lex);
++ freeJsonLexContext(&lex);
+
+ if (errmsg)
+ return false;
3: b3a731b695 ! 3: 37eaa5ceb6 backend: add OAUTHBEARER SASL mechanism
@@ src/include/libpq/auth.h: extern PGDLLIMPORT bool pg_gss_accept_delegation;
## src/include/libpq/hba.h ##
@@ src/include/libpq/hba.h: typedef enum UserAuth
- uaLDAP,
uaCert,
uaRADIUS,
-- uaPeer
+ uaPeer,
-#define USER_AUTH_LAST uaPeer /* Must be last value of this enum */
-+ uaPeer,
-+ uaOAuth
++ uaOAuth,
+#define USER_AUTH_LAST uaOAuth /* Must be last value of this enum */
} UserAuth;
4: b2a570114e = 4: 7177943f0c Add pytest suite for OAuth
5: 48cd916bfe = 5: bea695c317 squash! Add pytest suite for OAuth
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-11-10 01:42 Andrey Chudnovsky <achudnovskij@gmail.com>
parent: Jacob Champion <champion.p@gmail.com>
1 sibling, 1 reply; 424+ messages in thread
From: Andrey Chudnovsky @ 2023-11-10 01:42 UTC (permalink / raw)
To: Jacob Champion <champion.p@gmail.com>; +Cc: Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
Hi Jacob,
Wanted to follow up on one of the topics discussed here in the past:
Do you plan to support adding an extension hook to validate the token?
It would allow a more efficient integration, then spinning a separate
process.
Thanks!
Andrey.
On Wed, Nov 8, 2023 at 11:00 AM Jacob Champion <champion.p@gmail.com> wrote:
> On Fri, Nov 3, 2023 at 4:48 PM Jacob Champion <champion.p@gmail.com>
> wrote:
> > Thanks for the nudge! Looks like I need to reconcile with the changes
> > to JsonLexContext in 1c99cde2. I should be able to get to that next
> > week; in the meantime I'll mark it Waiting on Author.
>
> v13 rebases over latest. The JsonLexContext changes have simplified
> 0001 quite a bit, and there's probably a bit more minimization that
> could be done.
>
> Unfortunately the configure/Makefile build of libpq now seems to be
> pulling in an `exit()` dependency in a way that Meson does not. (Or
> maybe Meson isn't checking?) I still need to investigate that
> difference and fix it, so I recommend Meson if you're looking to
> test-drive a build.
>
> Thanks,
> --Jacob
>
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-11-15 20:20 Jacob Champion <champion.p@gmail.com>
parent: Andrey Chudnovsky <achudnovskij@gmail.com>
0 siblings, 0 replies; 424+ messages in thread
From: Jacob Champion @ 2023-11-15 20:20 UTC (permalink / raw)
To: Andrey Chudnovsky <achudnovskij@gmail.com>; +Cc: Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Thu, Nov 9, 2023 at 5:43 PM Andrey Chudnovsky <achudnovskij@gmail.com> wrote:
> Do you plan to support adding an extension hook to validate the token?
>
> It would allow a more efficient integration, then spinning a separate process.
I think an API in the style of archive modules might probably be a
good way to go, yeah.
It's probably not very high on the list of priorities, though, since
the inputs and outputs are going to "look" the same whether you're
inside or outside of the server process. The client side is going to
need the bulk of the work/testing/validation. Speaking of which -- how
is the current PQauthDataHook design doing when paired with MS AAD
(er, Entra now I guess)? I haven't had an Azure test bed available for
a while.
Thanks,
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2023-12-05 09:43 Daniel Gustafsson <daniel@yesql.se>
parent: Jacob Champion <champion.p@gmail.com>
1 sibling, 1 reply; 424+ messages in thread
From: Daniel Gustafsson @ 2023-12-05 09:43 UTC (permalink / raw)
To: Jacob Champion <champion.p@gmail.com>; +Cc: Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
> On 8 Nov 2023, at 20:00, Jacob Champion <champion.p@gmail.com> wrote:
> Unfortunately the configure/Makefile build of libpq now seems to be
> pulling in an `exit()` dependency in a way that Meson does not.
I believe this comes from the libcurl and specifically the ntlm_wb support
which is often enabled in system and package manager provided installations.
There isn't really a fix here apart from requiring a libcurl not compiled with
ntlm_wb support, or adding an exception to the exit() check in the Makefile.
Bringing this up with other curl developers to see if it could be fixed we
instead decided to deprecate the whole module as it's quirky and not used much.
This won't help with existing installations but at least it will be deprecated
and removed by the time v17 ships, so gating on a version shipped after its
removal will avoid it.
https://github.com/curl/curl/commit/04540f69cfd4bf16e80e7c190b645f1baf505a84
> (Or maybe Meson isn't checking?) I still need to investigate that
> difference and fix it, so I recommend Meson if you're looking to
> test-drive a build.
There is no corresponding check in the Meson build, which seems like a TODO.
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-01-09 18:48 Jacob Champion <champion.p@gmail.com>
parent: Daniel Gustafsson <daniel@yesql.se>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2024-01-09 18:48 UTC (permalink / raw)
To: daniel@yesql.se; +Cc: Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; pgsql-hackers; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Tue, Dec 5, 2023 at 1:44 AM Daniel Gustafsson <daniel@yesql.se> wrote:
>
> > On 8 Nov 2023, at 20:00, Jacob Champion <champion.p@gmail.com> wrote:
>
> > Unfortunately the configure/Makefile build of libpq now seems to be
> > pulling in an `exit()` dependency in a way that Meson does not.
>
> I believe this comes from the libcurl and specifically the ntlm_wb support
> which is often enabled in system and package manager provided installations.
> There isn't really a fix here apart from requiring a libcurl not compiled with
> ntlm_wb support, or adding an exception to the exit() check in the Makefile.
>
> Bringing this up with other curl developers to see if it could be fixed we
> instead decided to deprecate the whole module as it's quirky and not used much.
> This won't help with existing installations but at least it will be deprecated
> and removed by the time v17 ships, so gating on a version shipped after its
> removal will avoid it.
>
> https://github.com/curl/curl/commit/04540f69cfd4bf16e80e7c190b645f1baf505a84
Ooh, thank you for looking into that and fixing it!
> > (Or maybe Meson isn't checking?) I still need to investigate that
> > difference and fix it, so I recommend Meson if you're looking to
> > test-drive a build.
>
> There is no corresponding check in the Meson build, which seems like a TODO.
Okay, I'll look into that too when I get time.
Thanks,
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-02-21 01:00 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Jacob Champion <champion.p@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2024-02-21 01:00 UTC (permalink / raw)
To: pgsql-hackers; +Cc: daniel@yesql.se, Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
Hi all,
v14 rebases over latest and fixes a warning when assertions are
disabled. 0006 is temporary and hacks past a couple of issues I have
not yet root caused -- one of which makes me wonder if 0001 needs to
be considered alongside the recent pg_combinebackup and incremental
JSON work...?
--Jacob
Attachments:
since-v13.diff.txt (text/plain, 6.3K) download
1: e7f87668ab ! 1: b6e8358f44 common/jsonapi: support FRONTEND clients
@@ Commit message
We can now partially revert b44669b2ca, now that json_errdetail() works
correctly.
- ## src/bin/pg_verifybackup/parse_manifest.c ##
-@@ src/bin/pg_verifybackup/parse_manifest.c: json_parse_manifest(JsonManifestParseContext *context, char *buffer,
- /* Run the actual JSON parser. */
- json_error = pg_parse_json(lex, &sem);
- if (json_error != JSON_SUCCESS)
-- json_manifest_parse_failure(context, "parsing failed");
-+ json_manifest_parse_failure(context, json_errdetail(json_error, lex));
- if (parse.state != JM_EXPECT_EOF)
- json_manifest_parse_failure(context, "manifest ended unexpectedly");
-
-
## src/bin/pg_verifybackup/t/005_bad_manifest.pl ##
@@ src/bin/pg_verifybackup/t/005_bad_manifest.pl: use Test::More;
my $tempdir = PostgreSQL::Test::Utils::tempdir;
@@ src/common/Makefile: override CPPFLAGS += -DVAL_LDFLAGS_EX="\"$(LDFLAGS_EX)\""
+override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common -I$(libpq_srcdir) $(CPPFLAGS)
LIBS += $(PTHREAD_LIBS)
- # If you add objects here, see also src/tools/msvc/Mkvcbuild.pm
+ OBJS_COMMON = \
## src/common/jsonapi.c ##
@@
@@ src/common/meson.build: foreach name, opts : pgcommon_variants
'dependencies': opts['dependencies'] + [ssl],
}
+ ## src/common/parse_manifest.c ##
+@@ src/common/parse_manifest.c: json_parse_manifest(JsonManifestParseContext *context, char *buffer,
+ /* Run the actual JSON parser. */
+ json_error = pg_parse_json(lex, &sem);
+ if (json_error != JSON_SUCCESS)
+- json_manifest_parse_failure(context, "parsing failed");
++ json_manifest_parse_failure(context, json_errdetail(json_error, lex));
+ if (parse.state != JM_EXPECT_EOF)
+ json_manifest_parse_failure(context, "manifest ended unexpectedly");
+
+
## src/include/common/jsonapi.h ##
@@
#ifndef JSONAPI_H
2: 0ab79a168f ! 2: 5fa08a8033 libpq: add OAUTHBEARER SASL mechanism
@@ Commit message
- fix libcurl initialization thread-safety
- harden the libcurl flow implementation
- figure out pgsocket/int difference on Windows
+ - fix intermittent failure in the cleanup callback tests (race
+ condition?)
- ...and more.
## configure ##
@@ src/interfaces/libpq/Makefile: endif
endif
## src/interfaces/libpq/exports.txt ##
-@@ src/interfaces/libpq/exports.txt: PQclosePrepared 188
- PQclosePortal 189
- PQsendClosePrepared 190
+@@ src/interfaces/libpq/exports.txt: PQsendClosePrepared 190
PQsendClosePortal 191
-+PQsetAuthDataHook 192
-+PQgetAuthDataHook 193
-+PQdefaultAuthDataHook 194
+ PQchangePassword 192
+ PQsendPipelineSync 193
++PQsetAuthDataHook 194
++PQgetAuthDataHook 195
++PQdefaultAuthDataHook 196
## src/interfaces/libpq/fe-auth-oauth-curl.c (new) ##
@@
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ */
+ cnt = sscanf(interval_str, "%f", &parsed);
+
-+ Assert(cnt == 1); /* otherwise the lexer screwed up */
++ if (cnt != 1)
++ {
++ /*
++ * Either the lexer screwed up or our assumption above isn't true, and
++ * either way a developer needs to take a look.
++ */
++ Assert(cnt == 1);
++ return 1; /* don't fall through in release builds */
++ }
++
+ parsed = ceilf(parsed);
+
+ if (parsed < 1)
@@ src/interfaces/libpq/fe-auth.c: pg_fe_sendauth(AuthRequest areq, int payloadlen,
{
/* Use this message if pg_SASL_continue didn't supply one */
if (conn->errorMessage.len == oldmsglen)
-@@ src/interfaces/libpq/fe-auth.c: PQencryptPasswordConn(PGconn *conn, const char *passwd, const char *user,
-
- return crypt_pwd;
+@@ src/interfaces/libpq/fe-auth.c: PQchangePassword(PGconn *conn, const char *user, const char *passwd)
+ }
+ }
}
+
+PQauthDataHook_type PQauthDataHook = PQdefaultAuthDataHook;
@@ src/interfaces/libpq/fe-connect.c: keep_going: /* We will come back to here
case CONNECTION_AUTH_OK:
{
/*
-@@ src/interfaces/libpq/fe-connect.c: makeEmptyPGconn(void)
+@@ src/interfaces/libpq/fe-connect.c: pqMakeEmptyPGconn(void)
conn->verbosity = PQERRORS_DEFAULT;
conn->show_context = PQSHOW_CONTEXT_ERRORS;
conn->sock = PGINVALID_SOCKET;
@@ src/interfaces/libpq/libpq-fe.h: extern int PQenv2encoding(void);
+
extern char *PQencryptPassword(const char *passwd, const char *user);
extern char *PQencryptPasswordConn(PGconn *conn, const char *passwd, const char *user, const char *algorithm);
+ extern PGresult *PQchangePassword(PGconn *conn, const char *user, const char *passwd);
+typedef int (*PQauthDataHook_type) (PGAuthData type, PGconn *conn, void *data);
+extern void PQsetAuthDataHook(PQauthDataHook_type hook);
3: fb0cc3f87e ! 3: 13cf3f80b8 backend: add OAUTHBEARER SASL mechanism
@@ src/backend/libpq/hba.c: parse_hba_auth_opt(char *name, char *val, HbaLine *hbal
## src/backend/libpq/meson.build ##
@@
- # Copyright (c) 2022-2023, PostgreSQL Global Development Group
+ # Copyright (c) 2022-2024, PostgreSQL Global Development Group
backend_sources += files(
+ 'auth-oauth.c',
4: 153347752c = 4: 83a55ba4eb Add pytest suite for OAuth
5: 8b85e542a7 ! 5: 49a3b2dfd1 squash! Add pytest suite for OAuth
@@ .cirrus.tasks.yml: task:
# NB: Intentionally build without -Dllvm. The freebsd image size is already
# large enough to make VM startup slow, and even without llvm freebsd
@@ .cirrus.tasks.yml: task:
- --buildtype=debug \
- -Dcassert=true -Duuid=bsd -Dtcl_version=tcl86 -Ddtrace=auto \
+ -Dcassert=true -Dinjection_points=true \
+ -Duuid=bsd -Dtcl_version=tcl86 -Ddtrace=auto \
-DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
+ -Doauth=curl \
-Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \
-: ---------- > 6: a68494323f XXX temporary patches to build and test
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-02-22 14:08 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2024-02-22 14:08 UTC (permalink / raw)
To: pgsql-hackers; +Cc: daniel@yesql.se, Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Tue, Feb 20, 2024 at 5:00 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> v14 rebases over latest and fixes a warning when assertions are
> disabled.
v15 is a housekeeping update that adds typedefs.list entries and runs
pgindent. It also includes a temporary patch from Daniel to get the
cfbot a bit farther (see above discussion on libcurl/exit).
--Jacob
Attachments:
since-v14.diff.txt (text/plain, 80.0K) download
v15-0004-Add-pytest-suite-for-OAuth.patch.gz (application/x-gzip, 34.6K) download
v15-0003-backend-add-OAUTHBEARER-SASL-mechanism.patch.gz (application/x-gzip, 12.6K) download
v15-0002-libpq-add-OAUTHBEARER-SASL-mechanism.patch.gz (application/x-gzip, 34.0K) download
v15-0001-common-jsonapi-support-FRONTEND-clients.patch.gz (application/x-gzip, 6.1K) download
v15-0005-squash-Add-pytest-suite-for-OAuth.patch.gz (application/x-gzip, 7.9K) download
v15-0007-REVERT-temporarily-skip-the-exit-check.patch.gz (application/x-gzip, 637B) download
v15-0006-XXX-temporary-patches-to-build-and-test.patch.gz (application/x-gzip, 895B) download
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-02-24 01:01 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 3 replies; 424+ messages in thread
From: Jacob Champion @ 2024-02-24 01:01 UTC (permalink / raw)
To: pgsql-hackers; +Cc: daniel@yesql.se, Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Thu, Feb 22, 2024 at 6:08 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> v15 is a housekeeping update that adds typedefs.list entries and runs
> pgindent.
v16 is more transformational!
Daniel contributed 0004, which completely replaces the
validator_command architecture with a C module API. This solves a
bunch of problems as discussed upthread and vastly simplifies the test
framework for the server side. 0004 also adds a set of Perl tests,
which will begin to subsume some of the Python server-side tests as I
get around to porting them. (@Daniel: 0005 is my diff against your
original patch, for review.)
0008 has been modified to quickfix the pgcommon linkage on the
Makefile side; my previous attempt at this only fixed Meson. The
patchset is now carrying a lot of squash-cruft, and I plan to flatten
it in the next version.
Thanks,
--Jacob
Attachments:
since-v15.diff.txt (text/plain, 30.7K) download
1: 92cf9bdcb3 = 1: dc523009f2 common/jsonapi: support FRONTEND clients
2: 0a58e64ade = 2: af969e6cea libpq: add OAUTHBEARER SASL mechanism
3: c56bc808b6 = 3: 8906c9d445 backend: add OAUTHBEARER SASL mechanism
-: ---------- > 4: e2566ab594 Introduce OAuth validator libraries
-: ---------- > 5: 26781a7f15 squash! Introduce OAuth validator libraries
4: 35ca8abdad ! 6: 295de92a5a Add pytest suite for OAuth
@@ src/test/python/server/conftest.py (new)
+
+ yield conn_factory
+ ## src/test/python/server/oauthtest.c (new) ##
+@@
++/*-------------------------------------------------------------------------
++ *
++ * oauthtest.c
++ * Test module for serverside OAuth token validation callbacks
++ *
++ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
++ * Portions Copyright (c) 1994, Regents of the University of California
++ *
++ * src/test/python/server/oauthtest.c
++ *
++ *-------------------------------------------------------------------------
++ */
++
++#include "postgres.h"
++
++#include "fmgr.h"
++#include "libpq/oauth.h"
++#include "utils/guc.h"
++#include "utils/memutils.h"
++
++PG_MODULE_MAGIC;
++
++static void test_startup(ValidatorModuleState *state);
++static void test_shutdown(ValidatorModuleState *state);
++static ValidatorModuleResult * test_validate(ValidatorModuleState *state,
++ const char *token,
++ const char *role);
++
++static const OAuthValidatorCallbacks callbacks = {
++ .startup_cb = test_startup,
++ .shutdown_cb = test_shutdown,
++ .validate_cb = test_validate,
++};
++
++static char *expected_bearer = "";
++static bool set_authn_id = false;
++static char *authn_id = "";
++static bool reflect_role = false;
++
++void
++_PG_init(void)
++{
++ DefineCustomStringVariable("oauthtest.expected_bearer",
++ "Expected Bearer token for future connections",
++ NULL,
++ &expected_bearer,
++ "",
++ PGC_SIGHUP,
++ 0,
++ NULL, NULL, NULL);
++
++ DefineCustomBoolVariable("oauthtest.set_authn_id",
++ "Whether to set an authenticated identity",
++ NULL,
++ &set_authn_id,
++ false,
++ PGC_SIGHUP,
++ 0,
++ NULL, NULL, NULL);
++ DefineCustomStringVariable("oauthtest.authn_id",
++ "Authenticated identity to use for future connections",
++ NULL,
++ &authn_id,
++ "",
++ PGC_SIGHUP,
++ 0,
++ NULL, NULL, NULL);
++
++ DefineCustomBoolVariable("oauthtest.reflect_role",
++ "Ignore the bearer token; use the requested role as the authn_id",
++ NULL,
++ &reflect_role,
++ false,
++ PGC_SIGHUP,
++ 0,
++ NULL, NULL, NULL);
++
++ MarkGUCPrefixReserved("oauthtest");
++}
++
++const OAuthValidatorCallbacks *
++_PG_oauth_validator_module_init(void)
++{
++ return &callbacks;
++}
++
++static void
++test_startup(ValidatorModuleState *state)
++{
++}
++
++static void
++test_shutdown(ValidatorModuleState *state)
++{
++}
++
++static ValidatorModuleResult *
++test_validate(ValidatorModuleState *state, const char *token, const char *role)
++{
++ ValidatorModuleResult *res;
++
++ res = palloc0(sizeof(ValidatorModuleResult)); /* TODO: palloc context? */
++
++ if (reflect_role)
++ {
++ res->authenticated = true;
++ res->authn_id = pstrdup(role); /* TODO: constify? */
++ }
++ else
++ {
++ if (*expected_bearer && !strcmp(token, expected_bearer))
++ res->authenticated = true;
++ if (set_authn_id)
++ res->authn_id = authn_id;
++ }
++
++ return res;
++}
+
## src/test/python/server/test_oauth.py (new) ##
@@
+#
@@ src/test/python/server/test_oauth.py (new)
+FEATURE_NOT_SUPPORTED_ERRCODE = b"0A000"
+
+SHARED_MEM_NAME = "oauth-pytest"
-+MAX_TOKEN_SIZE = 4096
+MAX_UINT16 = 2**16 - 1
+
+
@@ src/test/python/server/test_oauth.py (new)
+ dbname = "oauth_test_" + id
+
+ user = "oauth_user_" + id
++ punct_user = "oauth_\"'? ;&!_user_" + id # username w/ punctuation
+ map_user = "oauth_map_user_" + id
+ authz_user = "oauth_authz_user_" + id
+
@@ src/test/python/server/test_oauth.py (new)
+
+ # Create our roles and database.
+ user = sql.Identifier(ctx.user)
++ punct_user = sql.Identifier(ctx.punct_user)
+ map_user = sql.Identifier(ctx.map_user)
+ authz_user = sql.Identifier(ctx.authz_user)
+ dbname = sql.Identifier(ctx.dbname)
+
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(user))
++ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(punct_user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(map_user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(authz_user))
+ c.execute(sql.SQL("CREATE DATABASE {};").format(dbname))
+
-+ # Make this test script the server's oauth_validator.
-+ path = pathlib.Path(__file__).parent / "validate_bearer.py"
-+ path = str(path.absolute())
-+
-+ cmd = f"{shlex.quote(path)} {SHARED_MEM_NAME} <&%f"
-+ c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (cmd,))
-+
+ # Replace pg_hba and pg_ident.
+ c.execute("SHOW hba_file;")
+ hba = c.fetchone()[0]
@@ src/test/python/server/test_oauth.py (new)
+ # Put things back the way they were.
+ c.execute("SELECT pg_reload_conf();")
+
-+ c.execute("ALTER SYSTEM RESET oauth_validator_command;")
+ c.execute(sql.SQL("DROP DATABASE {};").format(dbname))
+ c.execute(sql.SQL("DROP ROLE {};").format(authz_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(map_user))
++ c.execute(sql.SQL("DROP ROLE {};").format(punct_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(user))
+
+
@@ src/test/python/server/test_oauth.py (new)
+ return connect()
+
+
-+@pytest.fixture(scope="session")
-+def shared_mem():
-+ """
-+ Yields a shared memory segment that can be used for communication between
-+ the bearer_token fixture and ./validate_bearer.py.
-+ """
-+ size = MAX_TOKEN_SIZE + 2 # two byte length prefix
-+ mem = shared_memory.SharedMemory(SHARED_MEM_NAME, create=True, size=size)
-+
-+ try:
-+ with contextlib.closing(mem):
-+ yield mem
-+ finally:
-+ mem.unlink()
-+
-+
-+@pytest.fixture()
-+def bearer_token(shared_mem):
++def bearer_token(*, size=16):
+ """
-+ Returns a factory function that, when called, will store a Bearer token in
-+ shared_mem. If token is None (the default), a new token will be generated
-+ using secrets.token_urlsafe() and returned; otherwise the passed token will
-+ be used as-is.
-+
-+ When token is None, the generated token size in bytes may be specified as an
-+ argument; if unset, a small 16-byte token will be generated. The token size
-+ may not exceed MAX_TOKEN_SIZE in any case.
-+
-+ The return value is the token, converted to a bytes object.
-+
-+ As a special case for testing failure modes, accept_any may be set to True.
-+ This signals to the validator command that any bearer token should be
-+ accepted. The returned token in this case may be used or discarded as needed
-+ by the test.
++ Generates a Bearer token using secrets.token_urlsafe(). The generated token
++ size in bytes may be specified; if unset, a small 16-byte token will be
++ generated.
+ """
+
-+ def set_token(token=None, *, size=16, accept_any=False):
-+ if token is not None:
-+ size = len(token)
++ if size % 4:
++ raise ValueError(f"requested token size {size} is not a multiple of 4")
+
-+ if size > MAX_TOKEN_SIZE:
-+ raise ValueError(f"token size {size} exceeds maximum size {MAX_TOKEN_SIZE}")
++ token = secrets.token_urlsafe(size // 4 * 3)
++ assert len(token) == size
+
-+ if token is None:
-+ if size % 4:
-+ raise ValueError(f"requested token size {size} is not a multiple of 4")
-+
-+ token = secrets.token_urlsafe(size // 4 * 3)
-+ assert len(token) == size
-+
-+ try:
-+ token = token.encode("ascii")
-+ except AttributeError:
-+ pass # already encoded
-+
-+ if accept_any:
-+ # Two-byte magic value.
-+ shared_mem.buf[:2] = struct.pack("H", MAX_UINT16)
-+ else:
-+ # Two-byte length prefix, then the token data.
-+ shared_mem.buf[:2] = struct.pack("H", len(token))
-+ shared_mem.buf[2 : size + 2] = token
-+
-+ return token
-+
-+ return set_token
++ return token
+
+
+def begin_oauth_handshake(conn, oauth_ctx, *, user=None):
@@ src/test/python/server/test_oauth.py (new)
+ )
+
+
++@pytest.fixture()
++def setup_validator():
++ """
++ A per-test fixture that sets up the test validator with expected behavior.
++ The setting will be reverted during teardown.
++ """
++ conn = psycopg2.connect("")
++ conn.autocommit = True
++
++ with contextlib.closing(conn):
++ c = conn.cursor()
++ prev = dict()
++
++ def setter(**gucs):
++ for guc, val in gucs.items():
++ # Save the previous value.
++ c.execute(sql.SQL("SHOW oauthtest.{};").format(sql.Identifier(guc)))
++ prev[guc] = c.fetchone()[0]
++
++ c.execute(
++ sql.SQL("ALTER SYSTEM SET oauthtest.{} TO %s;").format(
++ sql.Identifier(guc)
++ ),
++ (val,),
++ )
++ c.execute("SELECT pg_reload_conf();")
++
++ yield setter
++
++ # Restore the previous values.
++ for guc, val in prev.items():
++ c.execute(
++ sql.SQL("ALTER SYSTEM SET oauthtest.{} TO %s;").format(
++ sql.Identifier(guc)
++ ),
++ (val,),
++ )
++ c.execute("SELECT pg_reload_conf();")
++
++
+@pytest.mark.parametrize("token_len", [16, 1024, 4096])
+@pytest.mark.parametrize(
+ "auth_prefix",
@@ src/test/python/server/test_oauth.py (new)
+ b"Bearer ",
+ ],
+)
-+def test_oauth(conn, oauth_ctx, bearer_token, auth_prefix, token_len):
-+ begin_oauth_handshake(conn, oauth_ctx)
-+
++def test_oauth(setup_validator, connect, oauth_ctx, auth_prefix, token_len):
+ # Generate our bearer token with the desired length.
+ token = bearer_token(size=token_len)
-+ auth = auth_prefix + token
++ setup_validator(expected_bearer=token)
++
++ conn = connect()
++ begin_oauth_handshake(conn, oauth_ctx)
+
++ auth = auth_prefix + token.encode("ascii")
+ send_initial_response(conn, auth=auth)
+ expect_handshake_success(conn)
+
@@ src/test/python/server/test_oauth.py (new)
+ "x-._~+/x",
+ ],
+)
-+def test_oauth_bearer_corner_cases(conn, oauth_ctx, bearer_token, token_value):
++def test_oauth_bearer_corner_cases(setup_validator, connect, oauth_ctx, token_value):
++ setup_validator(expected_bearer=token_value)
++
++ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+
-+ send_initial_response(conn, bearer=bearer_token(token_value))
++ send_initial_response(conn, bearer=token_value.encode("ascii"))
+
+ expect_handshake_success(conn)
+
@@ src/test/python/server/test_oauth.py (new)
+ ),
+ ],
+)
-+def test_oauth_authn_id(conn, oauth_ctx, bearer_token, user, authn_id, should_succeed):
-+ token = None
-+
++def test_oauth_authn_id(
++ setup_validator, connect, oauth_ctx, user, authn_id, should_succeed
++):
++ token = bearer_token()
+ authn_id = authn_id(oauth_ctx)
-+ if authn_id is not None:
-+ authn_id = authn_id.encode("ascii")
+
-+ # As a hack to get the validator to reflect arbitrary output from this
-+ # test, encode the desired output as a base64 token. The validator will
-+ # key on the leading "output=" to differentiate this from the random
-+ # tokens generated by secrets.token_urlsafe().
-+ output = b"output=" + authn_id + b"\n"
-+ token = base64.urlsafe_b64encode(output)
++ # Set up the validator appropriately.
++ gucs = dict(expected_bearer=token)
++ if authn_id is not None:
++ gucs["set_authn_id"] = True
++ gucs["authn_id"] = authn_id
++ setup_validator(**gucs)
+
-+ token = bearer_token(token)
++ conn = connect()
+ username = user(oauth_ctx)
-+
+ begin_oauth_handshake(conn, oauth_ctx, user=username)
-+ send_initial_response(conn, bearer=token)
++ send_initial_response(conn, bearer=token.encode("ascii"))
+
+ if not should_succeed:
+ expect_handshake_failure(conn, oauth_ctx)
@@ src/test/python/server/test_oauth.py (new)
+
+ expected = authn_id
+ if expected is not None:
-+ expected = b"oauth:" + expected
++ expected = b"oauth:" + expected.encode("ascii")
+
+ row = resp.payload
+ assert row.columns == [expected]
@@ src/test/python/server/test_oauth.py (new)
+ assert expected in detail
+
+
-+def test_oauth_rejected_bearer(conn, oauth_ctx, bearer_token):
-+ # Generate a new bearer token, which we will proceed not to use.
-+ _ = bearer_token()
-+
++def test_oauth_rejected_bearer(conn, oauth_ctx):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send a bearer token that doesn't match what the validator expects. It
@@ src/test/python/server/test_oauth.py (new)
+ b"Bearer ",
+ b"Bearer a===b",
+ b"Bearer hello!",
++ b"Bearer trailingspace ",
++ b"Bearer trailingtab\t",
+ b"Bearer me@example.com",
++ b"Beare abcd",
+ b'OAuth realm="Example"',
+ b"",
+ ],
+)
-+def test_oauth_invalid_bearer(conn, oauth_ctx, bearer_token, bad_bearer):
++def test_oauth_invalid_bearer(setup_validator, connect, oauth_ctx, bad_bearer):
+ # Tell the validator to accept any token. This ensures that the invalid
+ # bearer tokens are rejected before the validation step.
-+ _ = bearer_token(accept_any=True)
++ setup_validator(reflect_role=True)
+
++ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+ send_initial_response(conn, auth=bad_bearer)
+
@@ src/test/python/server/test_oauth.py (new)
+ err.match(resp)
+
+
-+def test_oauth_empty_initial_response(conn, oauth_ctx, bearer_token):
++def test_oauth_empty_initial_response(setup_validator, connect, oauth_ctx):
++ token = bearer_token()
++ setup_validator(expected_bearer=token)
++
++ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send an initial response without data.
@@ src/test/python/server/test_oauth.py (new)
+ assert not pkt.payload.body
+
+ # Now send the initial data.
-+ data = b"n,,\x01auth=Bearer " + bearer_token() + b"\x01\x01"
++ data = b"n,,\x01auth=Bearer " + token.encode("ascii") + b"\x01\x01"
+ pq3.send(conn, pq3.types.PasswordMessage, data)
+
+ # Server should now complete the handshake.
+ expect_handshake_success(conn)
+
+
-+@pytest.fixture()
-+def set_validator():
-+ """
-+ A per-test fixture that allows a test to override the setting of
-+ oauth_validator_command for the cluster. The setting will be reverted during
-+ teardown.
-+
-+ Passing None will perform an ALTER SYSTEM RESET.
-+ """
-+ conn = psycopg2.connect("")
-+ conn.autocommit = True
-+
-+ with contextlib.closing(conn):
-+ c = conn.cursor()
-+
-+ # Save the previous value.
-+ c.execute("SHOW oauth_validator_command;")
-+ prev_cmd = c.fetchone()[0]
-+
-+ def setter(cmd):
-+ c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (cmd,))
-+ c.execute("SELECT pg_reload_conf();")
-+
-+ yield setter
-+
-+ # Restore the previous value.
-+ c.execute("ALTER SYSTEM SET oauth_validator_command TO %s;", (prev_cmd,))
-+ c.execute("SELECT pg_reload_conf();")
-+
-+
-+def test_oauth_no_validator(oauth_ctx, set_validator, connect, bearer_token):
++# TODO: see if there's a way to test this easily after the API switch
++def xtest_oauth_no_validator(setup_validator, oauth_ctx, connect):
+ # Clear out our validator command, then establish a new connection.
+ set_validator("")
+ conn = connect()
@@ src/test/python/server/test_oauth.py (new)
+ expect_handshake_failure(conn, oauth_ctx)
+
+
-+def test_oauth_validator_role(oauth_ctx, set_validator, connect):
-+ # Switch the validator implementation. This validator will reflect the
-+ # PGUSER as the authenticated identity.
-+ path = pathlib.Path(__file__).parent / "validate_reflect.py"
-+ path = str(path.absolute())
++@pytest.mark.parametrize(
++ "user",
++ [
++ pytest.param(
++ lambda ctx: ctx.user,
++ id="basic username",
++ ),
++ pytest.param(
++ lambda ctx: ctx.punct_user,
++ id="'unsafe' characters are passed through correctly",
++ ),
++ ],
++)
++def test_oauth_validator_role(setup_validator, oauth_ctx, connect, user):
++ username = user(oauth_ctx)
+
-+ set_validator(f"{shlex.quote(path)} '%r' <&%f")
++ # Tell the validator to reflect the PGUSER as the authenticated identity.
++ setup_validator(reflect_role=True)
+ conn = connect()
+
-+ # Log in. Note that the reflection validator ignores the bearer token.
-+ begin_oauth_handshake(conn, oauth_ctx, user=oauth_ctx.user)
++ # Log in. Note that reflection ignores the bearer token.
++ begin_oauth_handshake(conn, oauth_ctx, user=username)
+ send_initial_response(conn, bearer=b"dontcare")
+ expect_handshake_success(conn)
+
@@ src/test/python/server/test_oauth.py (new)
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
-+ expected = b"oauth:" + oauth_ctx.user.encode("utf-8")
++ expected = b"oauth:" + username.encode("utf-8")
+ assert row.columns == [expected]
-+
-+
-+def test_oauth_role_with_shell_unsafe_characters(oauth_ctx, set_validator, connect):
-+ """
-+ XXX This test pins undesirable behavior. We should be able to handle any
-+ valid Postgres role name.
-+ """
-+ # Switch the validator implementation. This validator will reflect the
-+ # PGUSER as the authenticated identity.
-+ path = pathlib.Path(__file__).parent / "validate_reflect.py"
-+ path = str(path.absolute())
-+
-+ set_validator(f"{shlex.quote(path)} '%r' <&%f")
-+ conn = connect()
-+
-+ unsafe_username = "hello'there"
-+ begin_oauth_handshake(conn, oauth_ctx, user=unsafe_username)
-+
-+ # The server should reject the handshake.
-+ send_initial_response(conn, bearer=b"dontcare")
-+ expect_handshake_failure(conn, oauth_ctx)
## src/test/python/server/test_server.py (new) ##
@@
@@ src/test/python/server/test_server.py (new)
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.ReadyForQuery
- ## src/test/python/server/validate_bearer.py (new) ##
-@@
-+#! /usr/bin/env python3
-+#
-+# Copyright 2021 VMware, Inc.
-+# SPDX-License-Identifier: PostgreSQL
-+#
-+# DO NOT USE THIS OAUTH VALIDATOR IN PRODUCTION. It doesn't actually validate
-+# anything, and it logs the bearer token data, which is sensitive.
-+#
-+# This executable is used as an oauth_validator_command in concert with
-+# test_oauth.py. Memory is shared and communicated from that test module's
-+# bearer_token() fixture.
-+#
-+# This script must run under the Postgres server environment; keep the
-+# dependency list fairly standard.
-+
-+import base64
-+import binascii
-+import contextlib
-+import struct
-+import sys
-+from multiprocessing import shared_memory
-+
-+MAX_UINT16 = 2**16 - 1
-+
-+
-+def remove_shm_from_resource_tracker():
-+ """
-+ Monkey-patch multiprocessing.resource_tracker so SharedMemory won't be
-+ tracked. Pulled from this thread, where there are more details:
-+
-+ https://bugs.python.org/issue38119
-+
-+ TL;DR: all clients of shared memory segments automatically destroy them on
-+ process exit, which makes shared memory segments much less useful. This
-+ monkeypatch removes that behavior so that we can defer to the test to manage
-+ the segment lifetime.
-+
-+ Ideally a future Python patch will pull in this fix and then the entire
-+ function can go away.
-+ """
-+ from multiprocessing import resource_tracker
-+
-+ def fix_register(name, rtype):
-+ if rtype == "shared_memory":
-+ return
-+ return resource_tracker._resource_tracker.register(self, name, rtype)
-+
-+ resource_tracker.register = fix_register
-+
-+ def fix_unregister(name, rtype):
-+ if rtype == "shared_memory":
-+ return
-+ return resource_tracker._resource_tracker.unregister(self, name, rtype)
-+
-+ resource_tracker.unregister = fix_unregister
-+
-+ if "shared_memory" in resource_tracker._CLEANUP_FUNCS:
-+ del resource_tracker._CLEANUP_FUNCS["shared_memory"]
-+
-+
-+def main(args):
-+ remove_shm_from_resource_tracker() # XXX remove some day
-+
-+ # Get the expected token from the currently running test.
-+ shared_mem_name = args[0]
-+
-+ mem = shared_memory.SharedMemory(shared_mem_name)
-+ with contextlib.closing(mem):
-+ # First two bytes are the token length.
-+ size = struct.unpack("H", mem.buf[:2])[0]
-+
-+ if size == MAX_UINT16:
-+ # Special case: the test wants us to accept any token.
-+ sys.stderr.write("accepting token without validation\n")
-+ return
-+
-+ # The remainder of the buffer contains the expected token.
-+ assert size <= (mem.size - 2)
-+ expected_token = mem.buf[2 : size + 2].tobytes()
-+
-+ mem.buf[:] = b"\0" * mem.size # scribble over the token
-+
-+ token = sys.stdin.buffer.read()
-+ if token != expected_token:
-+ sys.exit(f"failed to match Bearer token ({token!r} != {expected_token!r})")
-+
-+ # See if the test wants us to print anything. If so, it will have encoded
-+ # the desired output in the token with an "output=" prefix.
-+ try:
-+ # altchars="-_" corresponds to the urlsafe alphabet.
-+ data = base64.b64decode(token, altchars="-_", validate=True)
-+
-+ if data.startswith(b"output="):
-+ sys.stdout.buffer.write(data[7:])
-+
-+ except binascii.Error:
-+ pass
-+
-+
-+if __name__ == "__main__":
-+ main(sys.argv[1:])
-
- ## src/test/python/server/validate_reflect.py (new) ##
-@@
-+#! /usr/bin/env python3
-+#
-+# Copyright 2021 VMware, Inc.
-+# SPDX-License-Identifier: PostgreSQL
-+#
-+# DO NOT USE THIS OAUTH VALIDATOR IN PRODUCTION. It ignores the bearer token
-+# entirely and automatically logs the user in.
-+#
-+# This executable is used as an oauth_validator_command in concert with
-+# test_oauth.py. It expects the user's desired role name as an argument; the
-+# actual token will be discarded and the user will be logged in with the role
-+# name as the authenticated identity.
-+#
-+# This script must run under the Postgres server environment; keep the
-+# dependency list fairly standard.
-+
-+import sys
-+
-+
-+def main(args):
-+ # We have to read the entire token as our first action to unblock the
-+ # server, but we won't actually use it.
-+ _ = sys.stdin.buffer.read()
-+
-+ if len(args) != 1:
-+ sys.exit("usage: ./validate_reflect.py ROLE")
-+
-+ # Log the user in as the provided role.
-+ role = args[0]
-+ print(role)
-+
-+
-+if __name__ == "__main__":
-+ main(sys.argv[1:])
-
## src/test/python/test_internals.py (new) ##
@@
+#
5: fb4cac4e99 ! 7: 7d21be13c0 squash! Add pytest suite for OAuth
@@ src/test/python/meson.build (new)
@@
+# Copyright (c) 2023, PostgreSQL Global Development Group
+
++subdir('server')
++
+pytest_env = {
+ 'with_oauth': oauth_library,
+
@@ src/test/python/server/conftest.py: import pq3
+ f"-c port={port}",
+ "-c listen_addresses=localhost",
+ "-c log_connections=on",
++ "-c shared_preload_libraries=oauthtest",
++ "-c oauth_validator_library=oauthtest",
+ ]
+ ),
+ "start",
@@ src/test/python/server/conftest.py: import pq3
# Have ExitStack close our socket.
stack.enter_context(sock)
+ ## src/test/python/server/meson.build (new) ##
+@@
++# Copyright (c) 2024, PostgreSQL Global Development Group
++
++if not oauth.found()
++ subdir_done()
++endif
++
++oauthtest_sources = files(
++ 'oauthtest.c',
++)
++
++if host_system == 'windows'
++ oauthtest_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
++ '--NAME', 'oauthtest',
++ '--FILEDESC', 'passthrough module to validate OAuth tests',
++ ])
++endif
++
++oauthtest = shared_module('oauthtest',
++ oauthtest_sources,
++ kwargs: pg_test_mod_args,
++)
++test_install_libs += oauthtest
+
## src/test/python/server/test_oauth.py ##
-@@ src/test/python/server/test_oauth.py: MAX_TOKEN_SIZE = 4096
+@@ src/test/python/server/test_oauth.py: SHARED_MEM_NAME = "oauth-pytest"
MAX_UINT16 = 2**16 - 1
@@ src/test/python/server/test_oauth.py: def oauth_ctx():
conn.autocommit = True
with contextlib.closing(conn):
-@@ src/test/python/server/test_oauth.py: def test_oauth_empty_initial_response(conn, oauth_ctx, bearer_token):
+@@ src/test/python/server/test_oauth.py: def receive_until(conn, type):
@pytest.fixture()
--def set_validator():
-+def set_validator(postgres_instance):
+-def setup_validator():
++def setup_validator(postgres_instance):
"""
- A per-test fixture that allows a test to override the setting of
- oauth_validator_command for the cluster. The setting will be reverted during
-@@ src/test/python/server/test_oauth.py: def set_validator():
-
- Passing None will perform an ALTER SYSTEM RESET.
+ A per-test fixture that sets up the test validator with expected behavior.
+ The setting will be reverted during teardown.
"""
- conn = psycopg2.connect("")
+ host, port = postgres_instance
6: 2008e60b3c ! 8: 26dcd5f828 XXX temporary patches to build and test
@@ Commit message
0001; has something changed?
- construct 2.10.70 has some incompatibilities with the current tests
+ ## src/bin/pg_combinebackup/Makefile ##
+@@ src/bin/pg_combinebackup/Makefile: include $(top_builddir)/src/Makefile.global
+
+ override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
+ LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils
++# TODO: fix this properly
++LDFLAGS_INTERNAL += -lpgcommon $(libpq_pgport)
+
+ OBJS = \
+ $(WIN32RES) \
+@@ src/bin/pg_combinebackup/Makefile: OBJS = \
+
+ all: pg_combinebackup
+
+-pg_combinebackup: $(OBJS) | submake-libpgport submake-libpgfeutils
+- $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(libpq_pgport) $(LIBS) -o $@$(X)
++pg_combinebackup: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
++ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
+ install: all installdirs
+ $(INSTALL_PROGRAM) pg_combinebackup$(X) '$(DESTDIR)$(bindir)/pg_combinebackup$(X)'
+
## src/bin/pg_combinebackup/meson.build ##
@@ src/bin/pg_combinebackup/meson.build: endif
@@ src/bin/pg_combinebackup/meson.build: endif
)
bin_targets += pg_combinebackup
+ ## src/bin/pg_verifybackup/Makefile ##
+@@ src/bin/pg_verifybackup/Makefile: top_builddir = ../../..
+ include $(top_builddir)/src/Makefile.global
+
+ # We need libpq only because fe_utils does.
+-LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
++LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils -lpgcommon $(libpq_pgport)
+
+ OBJS = \
+ $(WIN32RES) \
+
## src/test/python/requirements.txt ##
@@
black
7: 64611d33ef = 9: 0ff8e3786a REVERT: temporarily skip the exit check
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-02-27 19:33 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
2 siblings, 0 replies; 424+ messages in thread
From: Jacob Champion @ 2024-02-27 19:33 UTC (permalink / raw)
To: pgsql-hackers; +Cc: daniel@yesql.se, Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Tue, Feb 27, 2024 at 11:20 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> This is done in v17, which is also now based on the two patches pulled
> out by Daniel in [1].
It looks like my patchset has been eaten by a malware scanner:
550 Message content failed content scanning
(Sanesecurity.Foxhole.Mail_gz.UNOFFICIAL)
Was there a recent change to the lists? Is anyone able to see what the
actual error was so I don't do it again?
Thanks,
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-02-28 14:05 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
2 siblings, 2 replies; 424+ messages in thread
From: Jacob Champion @ 2024-02-28 14:05 UTC (permalink / raw)
To: pgsql-hackers
[Trying again, with all patches unzipped and the CC list temporarily
removed to avoid flooding people's inboxes. Original message follows.]
On Fri, Feb 23, 2024 at 5:01 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> The
> patchset is now carrying a lot of squash-cruft, and I plan to flatten
> it in the next version.
This is done in v17, which is also now based on the two patches pulled
out by Daniel in [1]. Besides the squashes, which make up most of the
range-diff, I've fixed a call to strncasecmp() which is not available
on Windows.
Daniel and I discussed trying a Python version of the test server,
since the standard library there should give us more goodies to work
with. A proof of concept is in 0009. I think the big question I have
for it is, how would we communicate what we want the server to do for
the test? (We could perhaps switch on magic values of the client ID?)
In the end I'd like to be testing close to 100% of the failure modes,
and that's likely to mean a lot of back-and-forth if the server
implementation isn't in the Perl process.
--Jacob
[1] https://postgr.es/m/flat/F51F8777-FAF5-49F2-BC5E-8F9EB423ECE0%40yesql.se
Attachments: download all (10 files, .tar.gz)
since-v16.diff.txt (text/plain, 54.6K) download
1: dc523009f2 = 1: 00976d4f75 common/jsonapi: support FRONTEND clients
-: ---------- > 2: d8b567dd55 Refactor SASL exchange to return tri-state status
-: ---------- > 3: 83d78f598c Explicitly require password for SCRAM exchange
2: af969e6cea ! 4: 00c8073807 libpq: add OAUTHBEARER SASL mechanism
@@ src/interfaces/libpq/fe-auth-oauth.h (new)
+#endif /* FE_AUTH_OAUTH_H */
## src/interfaces/libpq/fe-auth-sasl.h ##
-@@
-
- #include "libpq-fe.h"
-
-+/* See pg_fe_sasl_mech.exchange(). */
-+typedef enum
-+{
-+ SASL_COMPLETE,
-+ SASL_FAILED,
-+ SASL_CONTINUE,
+@@ src/interfaces/libpq/fe-auth-sasl.h: typedef enum
+ SASL_COMPLETE = 0,
+ SASL_FAILED,
+ SASL_CONTINUE,
+ SASL_ASYNC,
-+} SASLStatus;
-+
+ } SASLStatus;
+
/*
- * Frontend SASL mechanism callbacks.
- *
@@ src/interfaces/libpq/fe-auth-sasl.h: typedef struct pg_fe_sasl_mech
- * server response once at the start of the authentication exchange to
- * generate an initial response.
- *
-+ * Returns a SASLStatus:
-+ *
-+ * SASL_CONTINUE: The output buffer is filled with a client response. An
-+ * additional server challenge is expected.
-+ *
-+ * SASL_ASYNC: Some asynchronous processing external to the connection
-+ * needs to be done before a response can be generated. The
-+ * mechanism is responsible for setting up conn->async_auth
-+ * appropriately before returning.
-+ *
-+ * SASL_COMPLETE: The SASL exchange has completed successfully.
-+ *
-+ * SASL_FAILED: The exchange has failed and the connection should be
-+ * dropped.
-+ *
- * Input parameters:
*
* state: The opaque mechanism state returned by init()
*
@@ src/interfaces/libpq/fe-auth-sasl.h: typedef struct pg_fe_sasl_mech
* the server expects the client to send a message to start
@@ src/interfaces/libpq/fe-auth-sasl.h: typedef struct pg_fe_sasl_mech
*
- * output: A malloc'd buffer containing the client's response to
- * the server (can be empty), or NULL if the exchange should
-- * be aborted. (*success should be set to false in the
-+ * be aborted. (The callback should return SASL_FAILED in the
- * latter case.)
- *
- * outputlen: The length (0 or higher) of the client response buffer,
- * ignored if output is NULL.
-- *
-- * done: Set to true if the SASL exchange should not continue,
-- * because the exchange is either complete or failed
-- *
-- * success: Set to true if the SASL exchange completed successfully.
-- * Ignored if *done is false.
+ * SASL_CONTINUE: The output buffer is filled with a client response.
+ * Additional server challenge is expected
++ * SASL_ASYNC: Some asynchronous processing external to the
++ * connection needs to be done before a response can be
++ * generated. The mechanism is responsible for setting up
++ * conn->async_auth appropriately before returning.
+ * SASL_COMPLETE: The SASL exchange has completed successfully.
+- * SASL_FAILED: The exchance has failed and the connection should be
++ * SASL_FAILED: The exchange has failed and the connection should be
+ * dropped.
*--------
*/
-- void (*exchange) (void *state, char *input, int inputlen,
-- char **output, int *outputlen,
-- bool *done, bool *success);
+- SASLStatus (*exchange) (void *state, char *input, int inputlen,
+ SASLStatus (*exchange) (void *state, bool final,
+ char *input, int inputlen,
-+ char **output, int *outputlen);
+ char **output, int *outputlen);
/*--------
- * channel_bound()
## src/interfaces/libpq/fe-auth-scram.c ##
@@
/* The exported SCRAM callback mechanism. */
static void *scram_init(PGconn *conn, const char *password,
const char *sasl_mechanism);
--static void scram_exchange(void *opaq, char *input, int inputlen,
-- char **output, int *outputlen,
-- bool *done, bool *success);
+-static SASLStatus scram_exchange(void *opaq, char *input, int inputlen,
+static SASLStatus scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
-+ char **output, int *outputlen);
+ char **output, int *outputlen);
static bool scram_channel_bound(void *opaq);
static void scram_free(void *opaq);
-
@@ src/interfaces/libpq/fe-auth-scram.c: scram_free(void *opaq)
- /*
* Exchange a SCRAM message with backend.
*/
--static void
+ static SASLStatus
-scram_exchange(void *opaq, char *input, int inputlen,
-- char **output, int *outputlen,
-- bool *done, bool *success)
-+static SASLStatus
+scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
-+ char **output, int *outputlen)
+ char **output, int *outputlen)
{
fe_scram_state *state = (fe_scram_state *) opaq;
- PGconn *conn = state->conn;
- const char *errstr = NULL;
-
-- *done = false;
-- *success = false;
- *output = NULL;
- *outputlen = 0;
-
-@@ src/interfaces/libpq/fe-auth-scram.c: scram_exchange(void *opaq, char *input, int inputlen,
- if (inputlen == 0)
- {
- libpq_append_conn_error(conn, "malformed SCRAM message (empty message)");
-- goto error;
-+ return SASL_FAILED;
- }
- if (inputlen != strlen(input))
- {
- libpq_append_conn_error(conn, "malformed SCRAM message (length mismatch)");
-- goto error;
-+ return SASL_FAILED;
- }
- }
-
-@@ src/interfaces/libpq/fe-auth-scram.c: scram_exchange(void *opaq, char *input, int inputlen,
- /* Begin the SCRAM handshake, by sending client nonce */
- *output = build_client_first_message(state);
- if (*output == NULL)
-- goto error;
-+ return SASL_FAILED;
-
- *outputlen = strlen(*output);
-- *done = false;
- state->state = FE_SCRAM_NONCE_SENT;
-- break;
-+ return SASL_CONTINUE;
-
- case FE_SCRAM_NONCE_SENT:
- /* Receive salt and server nonce, send response. */
- if (!read_server_first_message(state, input))
-- goto error;
-+ return SASL_FAILED;
-
- *output = build_client_final_message(state);
- if (*output == NULL)
-- goto error;
-+ return SASL_FAILED;
-
- *outputlen = strlen(*output);
-- *done = false;
- state->state = FE_SCRAM_PROOF_SENT;
-- break;
-+ return SASL_CONTINUE;
-
- case FE_SCRAM_PROOF_SENT:
-- /* Receive server signature */
-- if (!read_server_final_message(state, input))
-- goto error;
--
-- /*
-- * Verify server signature, to make sure we're talking to the
-- * genuine server.
-- */
-- if (!verify_server_signature(state, success, &errstr))
-- {
-- libpq_append_conn_error(conn, "could not verify server signature: %s", errstr);
-- goto error;
-- }
--
-- if (!*success)
- {
-- libpq_append_conn_error(conn, "incorrect server signature");
-+ bool match;
-+
-+ /* Receive server signature */
-+ if (!read_server_final_message(state, input))
-+ return SASL_FAILED;
-+
-+ /*
-+ * Verify server signature, to make sure we're talking to the
-+ * genuine server.
-+ */
-+ if (!verify_server_signature(state, &match, &errstr))
-+ {
-+ libpq_append_conn_error(conn, "could not verify server signature: %s", errstr);
-+ return SASL_FAILED;
-+ }
-+
-+ if (!match)
-+ libpq_append_conn_error(conn, "incorrect server signature");
-+
-+ state->state = FE_SCRAM_FINISHED;
-+ state->conn->client_finished_auth = true;
-+ return match ? SASL_COMPLETE : SASL_FAILED;
- }
-- *done = true;
-- state->state = FE_SCRAM_FINISHED;
-- state->conn->client_finished_auth = true;
-- break;
-
- default:
- /* shouldn't happen */
- libpq_append_conn_error(conn, "invalid SCRAM exchange state");
-- goto error;
-+ break;
- }
-- return;
-
--error:
-- *done = true;
-- *success = false;
-+ return SASL_FAILED;
- }
-
- /*
## src/interfaces/libpq/fe-auth.c ##
@@
@@ src/interfaces/libpq/fe-auth.c: pg_SSPI_startup(PGconn *conn, int use_negotiate,
{
char *initialresponse = NULL;
int initialresponselen;
-- bool done;
-- bool success;
const char *selected_mechanism;
PQExpBufferData mechanism_buf;
- char *password;
+ char *password = NULL;
-+ SASLStatus status;
+ SASLStatus status;
initPQExpBuffer(&mechanism_buf);
-
@@ src/interfaces/libpq/fe-auth.c: pg_SASL_init(PGconn *conn, int payloadlen)
goto error;
}
@@ src/interfaces/libpq/fe-auth.c: pg_SASL_init(PGconn *conn, int payloadlen)
libpq_append_conn_error(conn, "duplicate SASL authentication request");
goto error;
@@ src/interfaces/libpq/fe-auth.c: pg_SASL_init(PGconn *conn, int payloadlen)
- /*
- * Parse the list of SASL authentication mechanisms in the
- * AuthenticationSASL message, and select the best mechanism that we
-- * support. SCRAM-SHA-256-PLUS and SCRAM-SHA-256 are the only ones
-- * supported at the moment, listed by order of decreasing importance.
-+ * support. Mechanisms are listed by order of decreasing importance.
- */
- selected_mechanism = NULL;
- for (;;)
-@@ src/interfaces/libpq/fe-auth.c: pg_SASL_init(PGconn *conn, int payloadlen)
- {
- selected_mechanism = SCRAM_SHA_256_PLUS_NAME;
- conn->sasl = &pg_scram_mech;
-+ conn->password_needed = true;
- }
- #else
- /*
-@@ src/interfaces/libpq/fe-auth.c: pg_SASL_init(PGconn *conn, int payloadlen)
- {
- selected_mechanism = SCRAM_SHA_256_NAME;
conn->sasl = &pg_scram_mech;
-+ conn->password_needed = true;
+ conn->password_needed = true;
}
+#ifdef USE_OAUTH
+ else if (strcmp(mechanism_buf.data, OAUTHBEARER_NAME) == 0 &&
@@ src/interfaces/libpq/fe-auth.c: pg_SASL_init(PGconn *conn, int payloadlen)
if (!selected_mechanism)
@@ src/interfaces/libpq/fe-auth.c: pg_SASL_init(PGconn *conn, int payloadlen)
- /*
- * First, select the password to use for the exchange, complaining if
-- * there isn't one. Currently, all supported SASL mechanisms require a
-- * password, so we can just go ahead here without further distinction.
-+ * there isn't one and the SASL mechanism needs it.
- */
-- conn->password_needed = true;
-- password = conn->connhost[conn->whichhost].password;
-- if (password == NULL)
-- password = conn->pgpass;
-- if (password == NULL || password[0] == '\0')
-+ if (conn->password_needed)
- {
-- appendPQExpBufferStr(&conn->errorMessage,
-- PQnoPasswordSupplied);
-- goto error;
-+ password = conn->connhost[conn->whichhost].password;
-+ if (password == NULL)
-+ password = conn->pgpass;
-+ if (password == NULL || password[0] == '\0')
-+ {
-+ appendPQExpBufferStr(&conn->errorMessage,
-+ PQnoPasswordSupplied);
-+ goto error;
-+ }
- }
-
Assert(conn->sasl);
- /*
@@ src/interfaces/libpq/fe-auth.c: pg_SASL_init(PGconn *conn, int payloadlen)
+ }
/* Get the mechanism-specific Initial Client Response, if any */
-- conn->sasl->exchange(conn->sasl_state,
-- NULL, -1,
-- &initialresponse, &initialresponselen,
-- &done, &success);
+- status = conn->sasl->exchange(conn->sasl_state,
+ status = conn->sasl->exchange(conn->sasl_state, false,
-+ NULL, -1,
-+ &initialresponse, &initialresponselen);
+ NULL, -1,
+ &initialresponse, &initialresponselen);
-- if (done && !success)
-+ if (status == SASL_FAILED)
+ if (status == SASL_FAILED)
goto error;
+ if (status == SASL_ASYNC)
@@ src/interfaces/libpq/fe-auth.c: oom_error:
{
char *output;
int outputlen;
-- bool done;
-- bool success;
- int res;
- char *challenge;
-+ SASLStatus status;
-
- /* Read the SASL challenge from the AuthenticationSASLContinue message. */
- challenge = malloc(payloadlen + 1);
@@ src/interfaces/libpq/fe-auth.c: pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
/* For safety and convenience, ensure the buffer is NULL-terminated. */
challenge[payloadlen] = '\0';
-- conn->sasl->exchange(conn->sasl_state,
-- challenge, payloadlen,
-- &output, &outputlen,
-- &done, &success);
+- status = conn->sasl->exchange(conn->sasl_state,
+ status = conn->sasl->exchange(conn->sasl_state, final,
-+ challenge, payloadlen,
-+ &output, &outputlen);
+ challenge, payloadlen,
+ &output, &outputlen);
free(challenge); /* don't need the input anymore */
-- if (final && !done)
+ if (status == SASL_ASYNC)
+ {
+ /*
@@ src/interfaces/libpq/fe-auth.c: pg_SASL_continue(PGconn *conn, int payloadlen, b
+ return STATUS_OK;
+ }
+
-+ if (final && !(status == SASL_FAILED || status == SASL_COMPLETE))
+ if (final && !(status == SASL_FAILED || status == SASL_COMPLETE))
{
if (outputlen != 0)
- free(output);
-@@ src/interfaces/libpq/fe-auth.c: pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
- * If the exchange is not completed yet, we need to make sure that the
- * SASL mechanism has generated a message to send back.
- */
-- if (output == NULL && !done)
-+ if (output == NULL && status == SASL_CONTINUE)
- {
- libpq_append_conn_error(conn, "no client response found after SASL exchange success");
- return STATUS_ERROR;
-@@ src/interfaces/libpq/fe-auth.c: pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
- return STATUS_ERROR;
- }
-
-- if (done && !success)
-+ if (status == SASL_FAILED)
- return STATUS_ERROR;
-
- return STATUS_OK;
@@ src/interfaces/libpq/fe-auth.c: check_expected_areq(AuthRequest areq, PGconn *conn)
* it. We are responsible for reading any remaining extra data, specific
* to the authentication method. 'payloadlen' is the remaining length in
@@ src/tools/pgindent/typedefs.list: PQArgBlock
PQsslKeyPassHook_OpenSSL_type
PREDICATELOCK
PREDICATELOCKTAG
-@@ src/tools/pgindent/typedefs.list: RuleLock
- RuleStmt
- RunningTransactions
- RunningTransactionsData
-+SASLStatus
- SC_HANDLE
- SECURITY_ATTRIBUTES
- SECURITY_STATUS
@@ src/tools/pgindent/typedefs.list: explain_get_index_name_hook_type
f_smgr
fasthash_state
3: 8906c9d445 ! 5: 29d7e3cbed backend: add OAUTHBEARER SASL mechanism
@@ src/backend/libpq/auth-oauth.c (new)
+ *
+ * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
+ */
-+ if (strncasecmp(auth, BEARER_SCHEME, strlen(BEARER_SCHEME)))
++ if (pg_strncasecmp(auth, BEARER_SCHEME, strlen(BEARER_SCHEME)))
+ return false;
+
+ /* Pull the bearer token out of the auth value. */
4: e2566ab594 ! 6: 0661817808 Introduce OAuth validator libraries
@@ Commit message
for loading in extensions for validating bearer tokens. A lot of
code is left to be written.
+ Co-authored-by: Jacob Champion <jacob.champion@enterprisedb.com>
+
## src/backend/libpq/auth-oauth.c ##
@@
* See the following RFC for more details:
@@ src/backend/libpq/auth-oauth.c: generate_error_response(struct oauth_ctx *ctx, c
*outputlen = buf.len;
}
+-static bool
+-validate(Port *port, const char *auth, const char **logdetail)
+/*-----
++ * Validates the provided Authorization header and returns the token from
++ * within it. NULL is returned on validation failure.
++ *
+ * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
+ * 2.1:
+ *
@@ src/backend/libpq/auth-oauth.c: generate_error_response(struct oauth_ctx *ctx, c
+ *
+ * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
+ */
- static bool
--validate(Port *port, const char *auth, const char **logdetail)
++static const char *
+validate_token_format(const char *header)
{
- static const char *const b64_set =
@@ src/backend/libpq/auth-oauth.c: generate_error_response(struct oauth_ctx *ctx, c
- const char *token;
- size_t span;
- int ret;
--
-- /* TODO: handle logdetail when the test framework can check it */
+ /* If the token is empty or simply too short to be correct */
+ if (!header || strlen(header) <= 7)
+ {
+ ereport(COMMERROR,
+ (errmsg("malformed OAuth bearer token 1")));
-+ return false;
++ return NULL;
+ }
+- /* TODO: handle logdetail when the test framework can check it */
+-
- /*-----
- * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
- * 2.1:
@@ src/backend/libpq/auth-oauth.c: generate_error_response(struct oauth_ctx *ctx, c
- *
- * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
- */
-- if (strncasecmp(auth, BEARER_SCHEME, strlen(BEARER_SCHEME)))
-+ if (strncasecmp(header, BEARER_SCHEME, strlen(BEARER_SCHEME)))
- return false;
+- if (pg_strncasecmp(auth, BEARER_SCHEME, strlen(BEARER_SCHEME)))
+- return false;
++ if (pg_strncasecmp(header, BEARER_SCHEME, strlen(BEARER_SCHEME)))
++ return NULL;
/* Pull the bearer token out of the auth value. */
- token = auth + strlen(BEARER_SCHEME);
@@ src/backend/libpq/auth-oauth.c: generate_error_response(struct oauth_ctx *ctx, c
- errmsg("malformed OAUTHBEARER message"),
+ errmsg("malformed OAuth bearer token 2"),
errdetail("Bearer token is empty.")));
- return false;
+- return false;
++ return NULL;
}
-@@ src/backend/libpq/auth-oauth.c: validate(Port *port, const char *auth, const char **logdetail)
+
+ /*
* Make sure the token contains only allowed characters. Tokens may end
* with any number of '=' characters.
*/
@@ src/backend/libpq/auth-oauth.c: validate(Port *port, const char *auth, const cha
- errmsg("malformed OAUTHBEARER message"),
+ errmsg("malformed OAuth bearer token 3"),
errdetail("Bearer token is not in the correct format.")));
- return false;
+- return false;
++ return NULL;
}
- /* Have the validator check the token. */
- if (!run_validator_command(port, token))
-+ return true;
++ return token;
+}
+
+static bool
+validate(Port *port, const char *auth)
+{
-+ int map_status;
-+ ValidatorModuleResult *ret;
++ int map_status;
++ ValidatorModuleResult *ret;
++ const char *token;
+
+ /* Ensure that we have a correct token to validate */
-+ if (!validate_token_format(auth))
-+ return false;
-+
++ if (!(token = validate_token_format(auth)))
+ return false;
+
+ /* Call the validation function from the validator module */
+ ret = ValidatorCallbacks->validate_cb(validator_module_state,
-+ auth + strlen(BEARER_SCHEME),
-+ port->user_name);
++ token, port->user_name);
+
+ if (!ret->authenticated)
- return false;
-
++ return false;
++
++ if (ret->authn_id)
++ set_authn_id(port, ret->authn_id);
++
if (port->hba->oauth_skip_usermap)
+ {
+ /*
@@ src/backend/libpq/auth-oauth.c: validate(Port *port, const char *auth, const char **logdetail)
}
@@ src/backend/libpq/auth-oauth.c: validate(Port *port, const char *auth, const cha
/* TODO: use logdetail; reduce message duplication */
ereport(LOG,
@@ src/backend/libpq/auth-oauth.c: validate(Port *port, const char *auth, const char **logdetail)
- return false;
}
-+ /* Set the authenticated identity from the validator module */
-+ set_authn_id(port, ret->authn_id);
-+
/* Finally, check the user map. */
- ret = check_usermap(port->hba->usermap, port->user_name,
-+ map_status = check_usermap(port->hba->usermap, port->user_name,
- MyClientConnectionInfo.authn_id, false);
+- MyClientConnectionInfo.authn_id, false);
- return (ret == STATUS_OK);
++ map_status = check_usermap(port->hba->usermap, port->user_name,
++ MyClientConnectionInfo.authn_id, false);
+ return (map_status == STATUS_OK);
}
@@ src/backend/libpq/auth-oauth.c: validate(Port *port, const char *auth, const cha
+load_validator_library(void)
{
- int rc;
--
++ OAuthValidatorModuleInit validator_init;
+
- rc = ClosePipeStream(*fh);
- *fh = NULL;
-+ OAuthValidatorModuleInit validator_init;
-
+-
- if (rc == -1)
- {
- /* pclose() itself failed. */
@@ src/backend/libpq/auth-oauth.c: validate(Port *port, const char *auth, const cha
- else if (rc != 0)
- {
- char *reason = wait_result_to_str(rc);
+-
+- ereport(COMMERROR,
+- (errmsg("failed to execute command \"%s\": %s",
+- command, reason)));
+-
+- pfree(reason);
+- }
+ if (OAuthValidatorLibrary[0] == '\0')
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("oauth_validator_library is not set")));
-- ereport(COMMERROR,
-- (errmsg("failed to execute command \"%s\": %s",
-- command, reason)));
+- return (rc == 0);
+-}
+ validator_init = (OAuthValidatorModuleInit)
+ load_external_function(OAuthValidatorLibrary,
+ "_PG_oauth_validator_module_init", false, NULL);
-- pfree(reason);
-- }
--
-- return (rc == 0);
--}
-+ if (validator_init == NULL)
-+ ereport(ERROR,
-+ (errmsg("%s module \"%s\" have to define the symbol %s",
-+ "OAuth validator", OAuthValidatorLibrary, "_PG_oauth_validator_module_init")));
-
-static bool
-set_cloexec(int fd)
-{
- int flags;
- int rc;
-+ ValidatorCallbacks = (*validator_init) ();
++ if (validator_init == NULL)
++ ereport(ERROR,
++ (errmsg("%s module \"%s\" have to define the symbol %s",
++ "OAuth validator", OAuthValidatorLibrary, "_PG_oauth_validator_module_init")));
- flags = fcntl(fd, F_GETFD);
- if (flags == -1)
@@ src/backend/libpq/auth-oauth.c: validate(Port *port, const char *auth, const cha
- errmsg("could not get fd flags for child pipe: %m")));
- return false;
- }
-+ validator_module_state = (ValidatorModuleState *) palloc0(sizeof(ValidatorModuleState));
-+ if (ValidatorCallbacks->startup_cb != NULL)
-+ ValidatorCallbacks->startup_cb(validator_module_state);
++ ValidatorCallbacks = (*validator_init) ();
- rc = fcntl(fd, F_SETFD, flags | FD_CLOEXEC);
- if (rc < 0)
@@ src/backend/libpq/auth-oauth.c: validate(Port *port, const char *auth, const cha
- errmsg("could not set FD_CLOEXEC for child pipe: %m")));
- return false;
- }
--
++ validator_module_state = (ValidatorModuleState *) palloc0(sizeof(ValidatorModuleState));
++ if (ValidatorCallbacks->startup_cb != NULL)
++ ValidatorCallbacks->startup_cb(validator_module_state);
+
- return true;
+ before_shmem_exit(shutdown_validator_library, 0);
}
@@ src/include/libpq/oauth.h
/* Implementation */
extern const pg_be_sasl_mech pg_be_oauth_mech;
+ ## src/test/modules/meson.build ##
+@@ src/test/modules/meson.build: subdir('gin')
+ subdir('injection_points')
+ subdir('ldap_password_func')
+ subdir('libpq_pipeline')
++subdir('oauth_validator')
+ subdir('plsample')
+ subdir('spgist_name_ops')
+ subdir('ssl_passphrase_callback')
+
## src/test/modules/oauth_validator/.gitignore (new) ##
@@
+# Generated subdirectories
@@ src/test/modules/oauth_validator/expected/validator.out (new)
+(1 row)
+
+ ## src/test/modules/oauth_validator/meson.build (new) ##
+@@
++# Copyright (c) 2024, PostgreSQL Global Development Group
++
++validator_sources = files(
++ 'validator.c',
++)
++
++if host_system == 'windows'
++ validator_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
++ '--NAME', 'validator',
++ '--FILEDESC', 'validator - test OAuth validator module',])
++endif
++
++validator = shared_module('validator',
++ validator_sources,
++ kwargs: pg_test_mod_args,
++)
++test_install_libs += validator
++
++tests += {
++ 'name': 'oauth_validator',
++ 'sd': meson.current_source_dir(),
++ 'bd': meson.current_build_dir(),
++ 'regress': {
++ 'sql': [
++ 'validator',
++ ],
++ },
++ 'tap': {
++ 'tests': [
++ 't/001_server.pl',
++ ],
++ },
++}
+
## src/test/modules/oauth_validator/sql/validator.sql (new) ##
@@
+SELECT 1;
@@ src/test/modules/oauth_validator/t/001_server.pl (new)
+$node->append_conf('postgresql.conf', "oauth_validator_library = 'validator'\n");
+$node->start;
+
-+reset_pg_hba($node, 'all', 'all', 'oauth issuer="127.0.0.1" scope="openid postgres"');
++reset_pg_hba($node, 'all', 'all', 'oauth issuer="127.0.0.1:18080" scope="openid postgres"');
+
-+my $webserver = PostgreSQL::Test::OAuthServer->new(80);
++my $webserver = PostgreSQL::Test::OAuthServer->new(18080);
+
+my $port = $webserver->port();
+
-+is($port, 80, "Port is 80");
++is($port, 18080, "Port is 18080");
+
+$webserver->setup();
+$webserver->run();
+
-+$node->connect_ok("dbname=postgres oauth_client_id=f02c6361-0635", "connect");
++$node->connect_ok("dbname=postgres oauth_client_id=f02c6361-0635", "connect",
++ expected_stderr => qr@Visit https://example\.com/ and enter the code: postgresuser@);
+
+$node->stop;
+
@@ src/test/modules/oauth_validator/validator.c (new)
+
+static void validator_startup(ValidatorModuleState *state);
+static void validator_shutdown(ValidatorModuleState *state);
-+static ValidatorModuleResult *validate_token(ValidatorModuleState *state,
-+ const char *token,
-+ const char *role);
++static ValidatorModuleResult * validate_token(ValidatorModuleState *state,
++ const char *token,
++ const char *role);
+
+static const OAuthValidatorCallbacks validator_callbacks = {
+ .startup_cb = validator_startup,
@@ src/test/modules/oauth_validator/validator.c (new)
+ return res;
+}
+ ## src/test/perl/PostgreSQL/Test/Cluster.pm ##
+@@ src/test/perl/PostgreSQL/Test/Cluster.pm: instead of the default.
+
+ If this regular expression is set, matches it with the output generated.
+
++=item expected_stderr => B<value>
++
++If this regular expression is set, matches it against the standard error
++stream; otherwise the stderr must be empty.
++
+ =item log_like => [ qr/required message/ ]
+
+ =item log_unlike => [ qr/prohibited message/ ]
+@@ src/test/perl/PostgreSQL/Test/Cluster.pm: sub connect_ok
+ like($stdout, $params{expected_stdout}, "$test_name: stdout matches");
+ }
+
+- is($stderr, "", "$test_name: no stderr");
++ if (defined($params{expected_stderr}))
++ {
++ like($stderr, $params{expected_stderr}, "$test_name: stderr matches");
++ }
++ else
++ {
++ is($stderr, "", "$test_name: no stderr");
++ }
+
+ $self->log_check($test_name, $log_location, %params);
+ }
+
## src/test/perl/PostgreSQL/Test/OAuthServer.pm (new) ##
@@
+#!/usr/bin/perl
@@ src/test/perl/PostgreSQL/Test/OAuthServer.pm (new)
+ print $fh "\r\n";
+ print $fh <<EOR;
+ {
-+ "issuer": "http://localhost:80",
-+ "token_endpoint": "http://localhost:80/token",
-+ "device_authorization_endpoint": "http://localhost:80/authorize",
++ "issuer": "http://localhost:$self->{'port'}",
++ "token_endpoint": "http://localhost:$self->{'port'}/token",
++ "device_authorization_endpoint": "http://localhost:$self->{'port'}/authorize",
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
@@ src/test/perl/PostgreSQL/Test/OAuthServer.pm (new)
+}
+
+1;
+
+ ## src/tools/pgindent/typedefs.list ##
+@@ src/tools/pgindent/typedefs.list: NumericSortSupport
+ NumericSumAccum
+ NumericVar
+ OAuthStep
++OAuthValidatorCallbacks
+ OM_uint32
+ OP
+ OSAPerGroupState
+@@ src/tools/pgindent/typedefs.list: VacuumRelation
+ VacuumStmt
+ ValidIOData
+ ValidateIndexState
++ValidatorModuleState
+ ValuesScan
+ ValuesScanState
+ Var
5: 26781a7f15 < -: ---------- squash! Introduce OAuth validator libraries
6: 295de92a5a ! 7: 45755e8461 Add pytest suite for OAuth
@@ Metadata
## Commit message ##
Add pytest suite for OAuth
- Requires Python 3; on the first run of `make installcheck` the
- dependencies will be installed into ./venv for you. See the README for
- more details.
+ Requires Python 3. On the first run of `make installcheck` or `meson
+ test` the dependencies will be installed into a local virtualenv for
+ you. See the README for more details.
+
+ Cirrus has been updated to build OAuth support on Debian and FreeBSD.
+
+ The suite contains a --temp-instance option, analogous to pg_regress's
+ option of the same name, which allows an ephemeral server to be spun up
+ during a test run.
For iddawc, asynchronous tests still hang, as expected. Bad-interval
tests fail because iddawc apparently doesn't care that the interval is
bad.
+ TODOs:
+ - The --tap-stream option to pytest-tap is slightly broken during test
+ failures (it suppresses error information), which impedes debugging.
+ - Unsurprisingly, Windows builds fail on the Linux-/BSD-specific backend
+ changes. 32-bit builds on Ubuntu fail during testing as well.
+ - pyca/cryptography is pinned at an old version. Since we use it for
+ testing and not security, this isn't a critical problem yet, but it's
+ not ideal. Newer versions require a Rust compiler to build, and while
+ many platforms have precompiled wheels, some (FreeBSD) do not. Even
+ with the Rust pieces bypassed, compilation on FreeBSD takes a while.
+ - The with_oauth test skip logic should probably be integrated into the
+ Makefile side as well...
+
+ ## .cirrus.tasks.yml ##
+@@ .cirrus.tasks.yml: env:
+ MTEST_ARGS: --print-errorlogs --no-rebuild -C build
+ PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests
+ TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf
+- PG_TEST_EXTRA: kerberos ldap ssl load_balance
++ PG_TEST_EXTRA: kerberos ldap ssl load_balance python
+
+
+ # What files to preserve in case tests fail
+@@ .cirrus.tasks.yml: task:
+ chown root:postgres /tmp/cores
+ sysctl kern.corefile='/tmp/cores/%N.%P.core'
+ setup_additional_packages_script: |
+- #pkg install -y ...
++ pkg install -y curl
+
+ # NB: Intentionally build without -Dllvm. The freebsd image size is already
+ # large enough to make VM startup slow, and even without llvm freebsd
+@@ .cirrus.tasks.yml: task:
+ -Dcassert=true -Dinjection_points=true \
+ -Duuid=bsd -Dtcl_version=tcl86 -Ddtrace=auto \
+ -DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
++ -Doauth=curl \
+ -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \
+ build
+ EOF
+@@ .cirrus.tasks.yml: LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
+ --with-libxslt
+ --with-llvm
+ --with-lz4
++ --with-oauth=curl
+ --with-pam
+ --with-perl
+ --with-python
+@@ .cirrus.tasks.yml: LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
+
+ LINUX_MESON_FEATURES: &LINUX_MESON_FEATURES >-
+ -Dllvm=enabled
++ -Doauth=curl
+ -Duuid=e2fs
+
+
+@@ .cirrus.tasks.yml: task:
+ EOF
+
+ setup_additional_packages_script: |
+- #apt-get update
+- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
++ apt-get update
++ DEBIAN_FRONTEND=noninteractive apt-get -y install \
++ libcurl4-openssl-dev \
++ libcurl4-openssl-dev:i386 \
++ python3-venv \
+
+ matrix:
+ - name: Linux - Debian Bullseye - Autoconf
+@@ .cirrus.tasks.yml: task:
+ folder: $CCACHE_DIR
+
+ setup_additional_packages_script: |
+- #apt-get update
+- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
++ apt-get update
++ DEBIAN_FRONTEND=noninteractive apt-get -y install libcurl4-openssl-dev
+
+ ###
+ # Test that code can be built with gcc/clang without warnings
+
+ ## meson.build ##
+@@ meson.build: else
+ endif
+
+ testwrap = files('src/tools/testwrap')
++make_venv = files('src/tools/make_venv')
++
++checked_working_venv = false
+
+ foreach test_dir : tests
+ testwrap_base = [
+@@ meson.build: foreach test_dir : tests
+ )
+ endforeach
+ install_suites += test_group
++ elif kind == 'pytest'
++ venv_name = test_dir['name'] + '_venv'
++ venv_path = meson.build_root() / venv_name
++
++ # The Python tests require a working venv module. This is part of the
++ # standard library, but some platforms disable it until a separate package
++ # is installed. Those same platforms don't provide an easy way to check
++ # whether the venv command will work until the first time you try it, so
++ # we decide whether or not to enable these tests on the fly.
++ if not checked_working_venv
++ cmd = run_command(python, '-m', 'venv', venv_path, check: false)
++
++ have_working_venv = (cmd.returncode() == 0)
++ if not have_working_venv
++ warning('A working Python venv module is required to run Python tests.')
++ endif
++
++ checked_working_venv = true
++ endif
++
++ if not have_working_venv
++ continue
++ endif
++
++ # Make sure the temporary installation is in PATH (necessary both for
++ # --temp-instance and for any pip modules compiling against libpq, like
++ # psycopg2).
++ env = test_env
++ env.prepend('PATH', temp_install_bindir, test_dir['bd'])
++
++ foreach name, value : t.get('env', {})
++ env.set(name, value)
++ endforeach
++
++ reqs = files(t['requirements'])
++ test('install_' + venv_name,
++ python,
++ args: [ make_venv, '--requirements', reqs, venv_path ],
++ env: env,
++ priority: setup_tests_priority - 1, # must run after tmp_install
++ is_parallel: false,
++ suite: ['setup'],
++ timeout: 60, # 30s is too short for the cryptography package compile
++ )
++
++ test_group = test_dir['name']
++ test_output = test_result_dir / test_group / kind
++ test_kwargs = {
++ #'protocol': 'tap',
++ 'suite': test_group,
++ 'timeout': 1000,
++ 'depends': test_deps,
++ 'env': env,
++ }
++
++ pytest = venv_path / 'bin' / 'py.test'
++ test_command = [
++ pytest,
++ # Avoid running these tests against an existing database.
++ '--temp-instance', test_output / 'data',
++
++ # FIXME pytest-tap's stream feature accidentally suppresses errors that
++ # are critical for debugging:
++ # https://github.com/python-tap/pytest-tap/issues/30
++ # Don't use the meson TAP protocol for now...
++ #'--tap-stream',
++ ]
++
++ foreach pyt : t['tests']
++ # Similarly to TAP, strip ./ and .py to make the names prettier
++ pyt_p = pyt
++ if pyt_p.startswith('./')
++ pyt_p = pyt_p.split('./')[1]
++ endif
++ if pyt_p.endswith('.py')
++ pyt_p = fs.stem(pyt_p)
++ endif
++
++ test(test_group / pyt_p,
++ python,
++ kwargs: test_kwargs,
++ args: testwrap_base + [
++ '--testgroup', test_group,
++ '--testname', pyt_p,
++ '--', test_command,
++ test_dir['sd'] / pyt,
++ ],
++ )
++ endforeach
++ install_suites += test_group
+ else
+ error('unknown kind @0@ of test in @1@'.format(kind, test_dir['sd']))
+ endif
+
+ ## src/test/meson.build ##
+@@ src/test/meson.build: subdir('authentication')
+ subdir('recovery')
+ subdir('subscription')
+ subdir('modules')
++subdir('python')
+
+ if ssl.found()
+ subdir('ssl')
+
## src/test/python/.gitignore (new) ##
@@
+__pycache__/
@@ src/test/python/README (new)
+
+ export PGDATABASE=postgres
+
-+but you can adjust as needed for your setup.
++but you can adjust as needed for your setup. See also 'Advanced Usage' below.
+
+## Requirements
+
@@ src/test/python/README (new)
+can skip them by saying e.g.
+
+ $ py.test -m 'not slow'
++
++If you'd rather not test against an existing server, you can have the suite spin
++up a temporary one using whatever pg_ctl it finds in PATH:
++
++ $ py.test --temp-instance=./tmp_check
## src/test/python/client/__init__.py (new) ##
@@ src/test/python/client/test_oauth.py (new)
+
+from .conftest import BLOCKING_TIMEOUT
+
++# The client tests need libpq to have been compiled with OAuth support; skip
++# them otherwise.
++pytestmark = pytest.mark.skipif(
++ os.getenv("with_oauth") == "none",
++ reason="OAuth client tests require --with-oauth support",
++)
++
+if platform.system() == "Darwin":
+ libpq = ctypes.cdll.LoadLibrary("libpq.5.dylib")
+else:
@@ src/test/python/conftest.py (new)
+import pytest
+
+
++def pytest_addoption(parser):
++ """
++ Adds custom command line options to py.test. We add one to signal temporary
++ Postgres instance creation for the server tests.
++
++ Per pytest documentation, this must live in the top level test directory.
++ """
++ parser.addoption(
++ "--temp-instance",
++ metavar="DIR",
++ help="create a temporary Postgres instance in DIR",
++ )
++
++
+@pytest.fixture(scope="session", autouse=True)
+def _check_PG_TEST_EXTRA(request):
+ """
@@ src/test/python/conftest.py (new)
+ if "python" not in extra_tests:
+ pytest.skip("Potentially unsafe test 'python' not enabled in PG_TEST_EXTRA")
+ ## src/test/python/meson.build (new) ##
+@@
++# Copyright (c) 2023, PostgreSQL Global Development Group
++
++subdir('server')
++
++pytest_env = {
++ 'with_oauth': oauth_library,
++
++ # Point to the default database; the tests will create their own databases as
++ # needed.
++ 'PGDATABASE': 'postgres',
++
++ # Avoid the need for a Rust compiler on platforms without prebuilt wheels for
++ # pyca/cryptography.
++ 'CRYPTOGRAPHY_DONT_BUILD_RUST': '1',
++}
++
++# Some modules (psycopg2) need OpenSSL at compile time; for platforms where we
++# might have multiple implementations installed (macOS+brew), try to use the
++# same one that libpq is using.
++if ssl.found()
++ pytest_incdir = ssl.get_variable(pkgconfig: 'includedir', default_value: '')
++ if pytest_incdir != ''
++ pytest_env += { 'CPPFLAGS': '-I@0@'.format(pytest_incdir) }
++ endif
++
++ pytest_libdir = ssl.get_variable(pkgconfig: 'libdir', default_value: '')
++ if pytest_libdir != ''
++ pytest_env += { 'LDFLAGS': '-L@0@'.format(pytest_libdir) }
++ endif
++endif
++
++tests += {
++ 'name': 'python',
++ 'sd': meson.current_source_dir(),
++ 'bd': meson.current_build_dir(),
++ 'pytest': {
++ 'requirements': meson.current_source_dir() / 'requirements.txt',
++ 'tests': [
++ './client',
++ './server',
++ './test_internals.py',
++ './test_pq3.py',
++ ],
++ 'env': pytest_env,
++ },
++}
+
## src/test/python/pq3.py (new) ##
@@
+#
@@ src/test/python/pytest.ini (new)
## src/test/python/requirements.txt (new) ##
@@
+black
-+# cryptography 39.x removes a lot of platform support, beware
-+cryptography~=38.0.4
++# cryptography 35.x and later add many platform/toolchain restrictions, beware
++cryptography~=3.4.8
+construct~=2.10.61
+isort~=5.6
+# TODO: update to psycopg[c] 3.1
-+psycopg2~=2.9.6
++psycopg2~=2.9.7
+pytest~=7.3
+pytest-asyncio~=0.21.0
@@ src/test/python/server/__init__.py (new)
## src/test/python/server/conftest.py (new) ##
@@
+#
-+# Copyright 2021 VMware, Inc.
++# Portions Copyright 2021 VMware, Inc.
++# Portions Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
++import collections
+import contextlib
++import os
++import shutil
+import socket
++import subprocess
+import sys
+
+import pytest
@@ src/test/python/server/conftest.py (new)
+BLOCKING_TIMEOUT = 2 # the number of seconds to wait for blocking calls
+
+
++def cleanup_prior_instance(datadir):
++ """
++ Clean up an existing data directory, but make sure it actually looks like a
++ data directory first. (Empty folders will remain untouched, since initdb can
++ populate them.)
++ """
++ required_entries = set(["base", "PG_VERSION", "postgresql.conf"])
++ empty = True
++
++ try:
++ with os.scandir(datadir) as entries:
++ for e in entries:
++ empty = False
++ required_entries.discard(e.name)
++
++ except FileNotFoundError:
++ return # nothing to clean up
++
++ if empty:
++ return # initdb can handle an empty datadir
++
++ if required_entries:
++ pytest.fail(
++ f"--temp-instance directory \"{datadir}\" is not empty and doesn't look like a data directory (missing {', '.join(required_entries)})"
++ )
++
++ # Okay, seems safe enough now.
++ shutil.rmtree(datadir)
++
++
++@pytest.fixture(scope="session")
++def postgres_instance(pytestconfig, unused_tcp_port_factory):
++ """
++ If --temp-instance has been passed to pytest, this fixture runs a temporary
++ Postgres instance on an available port. Otherwise, the fixture will attempt
++ to contact a running Postgres server on (PGHOST, PGPORT); dependent tests
++ will be skipped if the connection fails.
++
++ Yields a (host, port) tuple for connecting to the server.
++ """
++ PGInstance = collections.namedtuple("PGInstance", ["addr", "temporary"])
++
++ datadir = pytestconfig.getoption("temp_instance")
++ if datadir:
++ # We were told to create a temporary instance. Use pg_ctl to set it up
++ # on an unused port.
++ cleanup_prior_instance(datadir)
++ subprocess.run(["pg_ctl", "-D", datadir, "init"], check=True)
++
++ # The CI looks for *.log files to upload, so the file name here isn't
++ # completely arbitrary.
++ log = os.path.join(datadir, "postmaster.log")
++ port = unused_tcp_port_factory()
++
++ subprocess.run(
++ [
++ "pg_ctl",
++ "-D",
++ datadir,
++ "-l",
++ log,
++ "-o",
++ " ".join(
++ [
++ f"-c port={port}",
++ "-c listen_addresses=localhost",
++ "-c log_connections=on",
++ "-c shared_preload_libraries=oauthtest",
++ "-c oauth_validator_library=oauthtest",
++ ]
++ ),
++ "start",
++ ],
++ check=True,
++ )
++
++ yield ("localhost", port)
++
++ subprocess.run(["pg_ctl", "-D", datadir, "stop"], check=True)
++
++ else:
++ # Try to contact an already running server; skip the suite if we can't
++ # find one.
++ addr = (pq3.pghost(), pq3.pgport())
++
++ try:
++ with socket.create_connection(addr, timeout=BLOCKING_TIMEOUT):
++ pass
++ except ConnectionError as e:
++ pytest.skip(f"unable to connect to Postgres server at {addr}: {e}")
++
++ yield addr
++
++
+@pytest.fixture
-+def connect():
++def connect(postgres_instance):
+ """
+ A factory fixture that, when called, returns a socket connected to a
-+ Postgres server, wrapped in a pq3 connection. The calling test will be
-+ skipped automatically if a server is not running at PGHOST:PGPORT, so it's
-+ best to connect as soon as possible after the test case begins, to avoid
-+ doing unnecessary work.
++ Postgres server, wrapped in a pq3 connection. Dependent tests will be
++ skipped if no server is available.
+ """
++ addr = postgres_instance
++
+ # Set up an ExitStack to handle safe cleanup of all of the moving pieces.
+ with contextlib.ExitStack() as stack:
+
+ def conn_factory():
-+ addr = (pq3.pghost(), pq3.pgport())
-+
-+ try:
-+ sock = socket.create_connection(addr, timeout=BLOCKING_TIMEOUT)
-+ except ConnectionError as e:
-+ pytest.skip(f"unable to connect to {addr}: {e}")
++ sock = socket.create_connection(addr, timeout=BLOCKING_TIMEOUT)
+
+ # Have ExitStack close our socket.
+ stack.enter_context(sock)
@@ src/test/python/server/conftest.py (new)
+
+ yield conn_factory
+ ## src/test/python/server/meson.build (new) ##
+@@
++# Copyright (c) 2024, PostgreSQL Global Development Group
++
++if not oauth.found()
++ subdir_done()
++endif
++
++oauthtest_sources = files(
++ 'oauthtest.c',
++)
++
++if host_system == 'windows'
++ oauthtest_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
++ '--NAME', 'oauthtest',
++ '--FILEDESC', 'passthrough module to validate OAuth tests',
++ ])
++endif
++
++oauthtest = shared_module('oauthtest',
++ oauthtest_sources,
++ kwargs: pg_test_mod_args,
++)
++test_install_libs += oauthtest
+
## src/test/python/server/oauthtest.c (new) ##
@@
+/*-------------------------------------------------------------------------
@@ src/test/python/server/test_oauth.py (new)
+MAX_UINT16 = 2**16 - 1
+
+
-+def skip_if_no_postgres():
-+ """
-+ Used by the oauth_ctx fixture to skip this test module if no Postgres server
-+ is running.
-+
-+ This logic is nearly duplicated with the conn fixture. Ideally oauth_ctx
-+ would depend on that, but a module-scope fixture can't depend on a
-+ test-scope fixture, and we haven't reached the rule of three yet.
-+ """
-+ addr = (pq3.pghost(), pq3.pgport())
-+
-+ try:
-+ with socket.create_connection(addr, timeout=BLOCKING_TIMEOUT):
-+ pass
-+ except ConnectionError as e:
-+ pytest.skip(f"unable to connect to {addr}: {e}")
-+
-+
+@contextlib.contextmanager
+def prepend_file(path, lines):
+ """
@@ src/test/python/server/test_oauth.py (new)
+
+
+@pytest.fixture(scope="module")
-+def oauth_ctx():
++def oauth_ctx(postgres_instance):
+ """
+ Creates a database and user that use the oauth auth method. The context
+ object contains the dbname and user attributes as strings to be used during
@@ src/test/python/server/test_oauth.py (new)
+ server running on a local machine, and that the PGUSER has rights to create
+ databases and roles.
+ """
-+ skip_if_no_postgres() # don't bother running these tests without a server
-+
+ id = secrets.token_hex(4)
+
+ class Context:
@@ src/test/python/server/test_oauth.py (new)
+ )
+ ident_lines = (r"oauth /^(.*)@example\.com$ \1",)
+
-+ conn = psycopg2.connect("")
++ host, port = postgres_instance
++ conn = psycopg2.connect(host=host, port=port)
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
@@ src/test/python/server/test_oauth.py (new)
+
+
+@pytest.fixture()
-+def setup_validator():
++def setup_validator(postgres_instance):
+ """
+ A per-test fixture that sets up the test validator with expected behavior.
+ The setting will be reverted during teardown.
+ """
-+ conn = psycopg2.connect("")
++ host, port = postgres_instance
++ conn = psycopg2.connect(host=host, port=port)
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
@@ src/test/python/tls.py (new)
+ "length" / Int16ub,
+ "fragment" / FixedSized(this.length, GreedyBytes),
+)
+
+ ## src/tools/make_venv (new) ##
+@@
++#!/usr/bin/env python3
++
++import argparse
++import subprocess
++import os
++import sys
++
++parser = argparse.ArgumentParser()
++
++parser.add_argument('--requirements', help='path to pip requirements file', type=str)
++parser.add_argument('--privatedir', help='private directory for target', type=str)
++parser.add_argument('venv_path', help='desired venv location')
++
++args = parser.parse_args()
++
++# Decide whether or not to capture stdout into a log file. We only do this if
++# we've been given our own private directory.
++#
++# FIXME Unfortunately this interferes with debugging on Cirrus, because the
++# private directory isn't uploaded in the sanity check's artifacts. When we
++# don't capture the log file, it gets spammed to stdout during build... Is there
++# a way to push this into the meson-log somehow? For now, the capture
++# implementation is commented out.
++logfile = None
++
++if args.privatedir:
++ if not os.path.isdir(args.privatedir):
++ os.mkdir(args.privatedir)
++
++ # FIXME see above comment
++ # logpath = os.path.join(args.privatedir, 'stdout.txt')
++ # logfile = open(logpath, 'w')
++
++def run(*args):
++ kwargs = dict(check=True)
++ if logfile:
++ kwargs.update(stdout=logfile)
++
++ subprocess.run(args, **kwargs)
++
++# Create the virtualenv first.
++run(sys.executable, '-m', 'venv', args.venv_path)
++
++# Update pip next. This helps avoid old pip bugs; the version inside system
++# Pythons tends to be pretty out of date.
++pip = os.path.join(args.venv_path, 'bin', 'pip')
++run(pip, 'install', '-U', 'pip')
++
++# Finally, install the test's requirements. We need pytest and pytest-tap, no
++# matter what the test needs.
++run(pip, 'install', 'pytest', 'pytest-tap')
++if args.requirements:
++ run(pip, 'install', '-r', args.requirements)
7: 7d21be13c0 < -: ---------- squash! Add pytest suite for OAuth
8: 26dcd5f828 ! 8: 0f9f884856 XXX temporary patches to build and test
@@ Commit message
- the new pg_combinebackup utility uses JSON in the frontend without
0001; has something changed?
- construct 2.10.70 has some incompatibilities with the current tests
+ - temporarily skip the exit check (from Daniel Gustafsson); this needs
+ to be turned into an exception for curl rather than a plain exit call
## src/bin/pg_combinebackup/Makefile ##
@@ src/bin/pg_combinebackup/Makefile: include $(top_builddir)/src/Makefile.global
@@ src/bin/pg_verifybackup/Makefile: top_builddir = ../../..
OBJS = \
$(WIN32RES) \
+ ## src/interfaces/libpq/Makefile ##
+@@ src/interfaces/libpq/Makefile: libpq-refs-stamp: $(shlib)
+ ifneq ($(enable_coverage), yes)
+ ifeq (,$(filter aix solaris,$(PORTNAME)))
+ @if nm -A -u $< 2>/dev/null | grep -v -e __cxa_atexit -e __tsan_func_exit | grep exit; then \
+- echo 'libpq must not be calling any function which invokes exit'; exit 1; \
++ echo 'libpq must not be calling any function which invokes exit'; \
+ fi
+ endif
+ endif
+
## src/test/python/requirements.txt ##
@@
black
9: 0ff8e3786a < -: ---------- REVERT: temporarily skip the exit check
-: ---------- > 9: de8f81bd7d WIP: Python OAuth provider implementation
v17-0003-Explicitly-require-password-for-SCRAM-exchange.patch (application/octet-stream, 2.9K) download
From 83d78f598c65bd942e770eefbec0084d040414c3 Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <dgustafsson@postgresql.org>
Date: Fri, 23 Feb 2024 11:19:55 +0100
Subject: [PATCH v17 3/9] Explicitly require password for SCRAM exchange
This refactors the SASL init flow to set password_needed on the two
SCRAM exchanges currently supported. The code already required this
but was set up in such a way that all SASL exchanges required using
a password, a restriction which may not hold for all exchanges (the
example at hand being the proposed OAuthbearer exchange).
This was extracted from a larger patchset to introduce OAuthBearer
authentication and authorization.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://postgr.es/m/d1b467a78e0e36ed85a09adf979d04cf124a9d4b.camel@vmware.com
---
src/interfaces/libpq/fe-auth.c | 26 ++++++++++++++------------
1 file changed, 14 insertions(+), 12 deletions(-)
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index 71dd096605..9f57976b4f 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -446,8 +446,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
/*
* Parse the list of SASL authentication mechanisms in the
* AuthenticationSASL message, and select the best mechanism that we
- * support. SCRAM-SHA-256-PLUS and SCRAM-SHA-256 are the only ones
- * supported at the moment, listed by order of decreasing importance.
+ * support. Mechanisms are listed by order of decreasing importance.
*/
selected_mechanism = NULL;
for (;;)
@@ -487,6 +486,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
selected_mechanism = SCRAM_SHA_256_PLUS_NAME;
conn->sasl = &pg_scram_mech;
+ conn->password_needed = true;
}
#else
/*
@@ -522,6 +522,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
selected_mechanism = SCRAM_SHA_256_NAME;
conn->sasl = &pg_scram_mech;
+ conn->password_needed = true;
}
}
@@ -545,18 +546,19 @@ pg_SASL_init(PGconn *conn, int payloadlen)
/*
* First, select the password to use for the exchange, complaining if
- * there isn't one. Currently, all supported SASL mechanisms require a
- * password, so we can just go ahead here without further distinction.
+ * there isn't one and the selected SASL mechanism needs it.
*/
- conn->password_needed = true;
- password = conn->connhost[conn->whichhost].password;
- if (password == NULL)
- password = conn->pgpass;
- if (password == NULL || password[0] == '\0')
+ if (conn->password_needed)
{
- appendPQExpBufferStr(&conn->errorMessage,
- PQnoPasswordSupplied);
- goto error;
+ password = conn->connhost[conn->whichhost].password;
+ if (password == NULL)
+ password = conn->pgpass;
+ if (password == NULL || password[0] == '\0')
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ PQnoPasswordSupplied);
+ goto error;
+ }
}
Assert(conn->sasl);
--
2.34.1
v17-0004-libpq-add-OAUTHBEARER-SASL-mechanism.patch (application/octet-stream, 118.7K) download
From 00c807380737e91e3f85ebe0301c54faad2565b3 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 13 Apr 2021 10:27:27 -0700
Subject: [PATCH v17 4/9] libpq: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) and OAuth 2.0 Device Authorization
Grants (RFC 8628) on the client side. When speaking to a OAuth-enabled
server, it looks a bit like this:
$ psql 'host=example.org oauth_client_id=f02c6361-0635-...'
Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG
The OAuth issuer must support device authorization. No other OAuth flows
are currently implemented (but clients may provide their own flows; see
below).
The client implementation requires either libcurl or libiddawc and their
development headers. Pass `curl` or `iddawc` to --with-oauth/-Doauth
during configuration.
Thomas Munro wrote the kqueue() implementation for oauth-curl; thanks!
= PQauthDataHook =
Clients may override two pieces of OAuth handling using the new
PQsetAuthDataHook():
- PQAUTHDATA_PROMPT_OAUTH_DEVICE: replaces the default user prompt to
standard error when using the builtin device authorization flow
- PQAUTHDATA_OAUTH_BEARER_TOKEN: replaces the entire OAuth flow with a
custom asynchronous implementation
In general, a hook implementation should examine the incoming `type` to
decide whether or not to handle a specific piece of authdata; if not, it
should delegate to the previous hook in the chain (retrievable via
PQgetAuthDataHook()). Otherwise, it should return an integer > 0 and
follow the authdata-specific instructions. Returning an integer < 0
signals an error condition and abandons the connection attempt.
== PQAUTHDATA_PROMPT_OAUTH_DEVICE ==
The hook should display the device prompt (URL + code) using whatever
method it prefers.
== PQAUTHDATA_OAUTH_BEARER_TOKEN ==
The hook should either directly return a Bearer token for the current
user/issuer/scope combination, if one is available without blocking, or
else set up an asynchronous callback to retrieve one. See the
documentation for PQoauthBearerRequest.
Several TODOs:
- don't retry forever if the server won't accept our token
- perform several sanity checks on the OAuth issuer's responses
- handle cases where the client has been set up with an issuer and
scope, but the Postgres server wants to use something different
- improve error debuggability during the OAuth handshake
- fix libcurl initialization thread-safety
- harden the libcurl flow implementation
- figure out pgsocket/int difference on Windows
- fix intermittent failure in the cleanup callback tests (race
condition?)
- ...and more.
---
configure | 186 ++
configure.ac | 37 +
meson.build | 45 +
meson_options.txt | 4 +
src/Makefile.global.in | 1 +
src/include/common/oauth-common.h | 19 +
src/include/pg_config.h.in | 18 +
src/interfaces/libpq/Makefile | 12 +-
src/interfaces/libpq/exports.txt | 3 +
src/interfaces/libpq/fe-auth-oauth-curl.c | 1981 +++++++++++++++++++
src/interfaces/libpq/fe-auth-oauth-iddawc.c | 319 +++
src/interfaces/libpq/fe-auth-oauth.c | 659 ++++++
src/interfaces/libpq/fe-auth-oauth.h | 42 +
src/interfaces/libpq/fe-auth-sasl.h | 12 +-
src/interfaces/libpq/fe-auth-scram.c | 6 +-
src/interfaces/libpq/fe-auth.c | 107 +-
src/interfaces/libpq/fe-auth.h | 9 +-
src/interfaces/libpq/fe-connect.c | 85 +-
src/interfaces/libpq/fe-misc.c | 7 +-
src/interfaces/libpq/libpq-fe.h | 77 +-
src/interfaces/libpq/libpq-int.h | 14 +
src/interfaces/libpq/meson.build | 9 +
src/makefiles/meson.build | 1 +
src/tools/pgindent/typedefs.list | 10 +
24 files changed, 3634 insertions(+), 29 deletions(-)
create mode 100644 src/include/common/oauth-common.h
create mode 100644 src/interfaces/libpq/fe-auth-oauth-curl.c
create mode 100644 src/interfaces/libpq/fe-auth-oauth-iddawc.c
create mode 100644 src/interfaces/libpq/fe-auth-oauth.c
create mode 100644 src/interfaces/libpq/fe-auth-oauth.h
diff --git a/configure b/configure
index 6b87e5c9a8..1527d5d599 100755
--- a/configure
+++ b/configure
@@ -712,6 +712,7 @@ with_uuid
with_readline
with_systemd
with_selinux
+with_oauth
with_ldap
with_krb_srvnam
krb_srvtab
@@ -859,6 +860,7 @@ with_krb_srvnam
with_pam
with_bsd_auth
with_ldap
+with_oauth
with_bonjour
with_selinux
with_systemd
@@ -1570,6 +1572,7 @@ Optional Packages:
--with-pam build with PAM support
--with-bsd-auth build with BSD Authentication support
--with-ldap build with LDAP support
+ --with-oauth=LIB use LIB for OAuth 2.0 support (curl, iddawc)
--with-bonjour build with Bonjour support
--with-selinux build with SELinux support
--with-systemd build with systemd support
@@ -8794,6 +8797,59 @@ $as_echo "$with_ldap" >&6; }
+#
+# OAuth 2.0
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with OAuth support" >&5
+$as_echo_n "checking whether to build with OAuth support... " >&6; }
+
+
+
+# Check whether --with-oauth was given.
+if test "${with_oauth+set}" = set; then :
+ withval=$with_oauth;
+ case $withval in
+ yes)
+ as_fn_error $? "argument required for --with-oauth option" "$LINENO" 5
+ ;;
+ no)
+ as_fn_error $? "argument required for --with-oauth option" "$LINENO" 5
+ ;;
+ *)
+
+ ;;
+ esac
+
+fi
+
+
+if test x"$with_oauth" = x"" ; then
+ with_oauth=no
+fi
+
+if test x"$with_oauth" = x"curl"; then
+
+$as_echo "#define USE_OAUTH 1" >>confdefs.h
+
+
+$as_echo "#define USE_OAUTH_CURL 1" >>confdefs.h
+
+elif test x"$with_oauth" = x"iddawc"; then
+
+$as_echo "#define USE_OAUTH 1" >>confdefs.h
+
+
+$as_echo "#define USE_OAUTH_IDDAWC 1" >>confdefs.h
+
+elif test x"$with_oauth" != x"no"; then
+ as_fn_error $? "--with-oauth must specify curl or iddawc" "$LINENO" 5
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_oauth" >&5
+$as_echo "$with_oauth" >&6; }
+
+
+
#
# Bonjour
#
@@ -13347,6 +13403,116 @@ fi
+if test "$with_oauth" = curl ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for curl_multi_init in -lcurl" >&5
+$as_echo_n "checking for curl_multi_init in -lcurl... " >&6; }
+if ${ac_cv_lib_curl_curl_multi_init+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lcurl $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char curl_multi_init ();
+int
+main ()
+{
+return curl_multi_init ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_curl_curl_multi_init=yes
+else
+ ac_cv_lib_curl_curl_multi_init=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_curl_curl_multi_init" >&5
+$as_echo "$ac_cv_lib_curl_curl_multi_init" >&6; }
+if test "x$ac_cv_lib_curl_curl_multi_init" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBCURL 1
+_ACEOF
+
+ LIBS="-lcurl $LIBS"
+
+else
+ as_fn_error $? "library 'curl' is required for --with-oauth=curl" "$LINENO" 5
+fi
+
+elif test "$with_oauth" = iddawc ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for i_init_session in -liddawc" >&5
+$as_echo_n "checking for i_init_session in -liddawc... " >&6; }
+if ${ac_cv_lib_iddawc_i_init_session+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-liddawc $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char i_init_session ();
+int
+main ()
+{
+return i_init_session ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_iddawc_i_init_session=yes
+else
+ ac_cv_lib_iddawc_i_init_session=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_iddawc_i_init_session" >&5
+$as_echo "$ac_cv_lib_iddawc_i_init_session" >&6; }
+if test "x$ac_cv_lib_iddawc_i_init_session" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBIDDAWC 1
+_ACEOF
+
+ LIBS="-liddawc $LIBS"
+
+else
+ as_fn_error $? "library 'iddawc' is required for --with-oauth=iddawc" "$LINENO" 5
+fi
+
+ # Check for an older spelling of i_get_openid_config
+ for ac_func in i_load_openid_config
+do :
+ ac_fn_c_check_func "$LINENO" "i_load_openid_config" "ac_cv_func_i_load_openid_config"
+if test "x$ac_cv_func_i_load_openid_config" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_I_LOAD_OPENID_CONFIG 1
+_ACEOF
+
+fi
+done
+
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for security_compute_create_name in -lselinux" >&5
@@ -14372,6 +14538,26 @@ fi
done
+fi
+
+if test "$with_oauth" = curl; then
+ ac_fn_c_check_header_mongrel "$LINENO" "curl/curl.h" "ac_cv_header_curl_curl_h" "$ac_includes_default"
+if test "x$ac_cv_header_curl_curl_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <curl/curl.h> is required for OAuth" "$LINENO" 5
+fi
+
+
+elif test "$with_oauth" = iddawc; then
+ ac_fn_c_check_header_mongrel "$LINENO" "iddawc.h" "ac_cv_header_iddawc_h" "$ac_includes_default"
+if test "x$ac_cv_header_iddawc_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <iddawc.h> is required for OAuth" "$LINENO" 5
+fi
+
+
fi
if test "$PORTNAME" = "win32" ; then
diff --git a/configure.ac b/configure.ac
index 6e64ece11d..b7982221a7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -949,6 +949,29 @@ AC_MSG_RESULT([$with_ldap])
AC_SUBST(with_ldap)
+#
+# OAuth 2.0
+#
+AC_MSG_CHECKING([whether to build with OAuth support])
+PGAC_ARG_REQ(with, oauth, [LIB], [use LIB for OAuth 2.0 support (curl, iddawc)])
+if test x"$with_oauth" = x"" ; then
+ with_oauth=no
+fi
+
+if test x"$with_oauth" = x"curl"; then
+ AC_DEFINE([USE_OAUTH], 1, [Define to 1 to build with OAuth 2.0 support. (--with-oauth)])
+ AC_DEFINE([USE_OAUTH_CURL], 1, [Define to 1 to use libcurl for OAuth support.])
+elif test x"$with_oauth" = x"iddawc"; then
+ AC_DEFINE([USE_OAUTH], 1, [Define to 1 to build with OAuth 2.0 support. (--with-oauth)])
+ AC_DEFINE([USE_OAUTH_IDDAWC], 1, [Define to 1 to use libiddawc for OAuth support.])
+elif test x"$with_oauth" != x"no"; then
+ AC_MSG_ERROR([--with-oauth must specify curl or iddawc])
+fi
+
+AC_MSG_RESULT([$with_oauth])
+AC_SUBST(with_oauth)
+
+
#
# Bonjour
#
@@ -1446,6 +1469,14 @@ fi
AC_SUBST(LDAP_LIBS_FE)
AC_SUBST(LDAP_LIBS_BE)
+if test "$with_oauth" = curl ; then
+ AC_CHECK_LIB(curl, curl_multi_init, [], [AC_MSG_ERROR([library 'curl' is required for --with-oauth=curl])])
+elif test "$with_oauth" = iddawc ; then
+ AC_CHECK_LIB(iddawc, i_init_session, [], [AC_MSG_ERROR([library 'iddawc' is required for --with-oauth=iddawc])])
+ # Check for an older spelling of i_get_openid_config
+ AC_CHECK_FUNCS([i_load_openid_config])
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
AC_CHECK_LIB(selinux, security_compute_create_name, [],
@@ -1637,6 +1668,12 @@ elif test "$with_uuid" = ossp ; then
[AC_MSG_ERROR([header file <ossp/uuid.h> or <uuid.h> is required for OSSP UUID])])])
fi
+if test "$with_oauth" = curl; then
+ AC_CHECK_HEADER(curl/curl.h, [], [AC_MSG_ERROR([header file <curl/curl.h> is required for OAuth])])
+elif test "$with_oauth" = iddawc; then
+ AC_CHECK_HEADER(iddawc.h, [], [AC_MSG_ERROR([header file <iddawc.h> is required for OAuth])])
+fi
+
if test "$PORTNAME" = "win32" ; then
AC_CHECK_HEADERS(crtdefs.h)
fi
diff --git a/meson.build b/meson.build
index 8ed51b6aae..8c83eac03d 100644
--- a/meson.build
+++ b/meson.build
@@ -849,6 +849,49 @@ endif
+###############################################################
+# Library: oauth
+###############################################################
+
+oauth = not_found_dep
+oauth_library = 'none'
+oauthopt = get_option('oauth')
+
+if oauthopt == 'auto' and auto_features.disabled()
+ oauthopt = 'none'
+endif
+
+if oauthopt in ['auto', 'curl']
+ oauth = dependency('libcurl', required: (oauthopt == 'curl'))
+
+ if oauth.found()
+ oauth_library = 'curl'
+ cdata.set('USE_OAUTH', 1)
+ cdata.set('USE_OAUTH_CURL', 1)
+ endif
+endif
+
+if not oauth.found() and oauthopt in ['auto', 'iddawc']
+ oauth = dependency('libiddawc', required: (oauthopt == 'iddawc'))
+
+ if oauth.found()
+ oauth_library = 'iddawc'
+ cdata.set('USE_OAUTH', 1)
+ cdata.set('USE_OAUTH_IDDAWC', 1)
+
+ # Check for an older spelling of i_get_openid_config
+ if cc.has_function('i_load_openid_config',
+ dependencies: oauth, args: test_c_args)
+ cdata.set('HAVE_I_LOAD_OPENID_CONFIG', 1)
+ endif
+ endif
+endif
+
+if oauthopt == 'auto' and auto_features.enabled() and not oauth.found()
+ error('no OAuth implementation library found')
+endif
+
+
###############################################################
# Library: Tcl (for pltcl)
#
@@ -2838,6 +2881,7 @@ libpq_deps += [
gssapi,
ldap_r,
libintl,
+ oauth,
ssl,
]
@@ -3439,6 +3483,7 @@ if meson.version().version_compare('>=0.57')
'llvm': llvm,
'lz4': lz4,
'nls': libintl,
+ 'oauth': oauth,
'openssl': ssl,
'pam': pam,
'plperl': perl_dep,
diff --git a/meson_options.txt b/meson_options.txt
index 249ecc5ffd..f54f7fd717 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -121,6 +121,10 @@ option('lz4', type: 'feature', value: 'auto',
option('nls', type: 'feature', value: 'auto',
description: 'Native language support')
+option('oauth', type : 'combo', choices : ['auto', 'none', 'curl', 'iddawc'],
+ value: 'auto',
+ description: 'use LIB for OAuth 2.0 support (curl, iddawc)')
+
option('pam', type: 'feature', value: 'auto',
description: 'PAM support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 8b3f8c24e0..79b3647834 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -193,6 +193,7 @@ with_ldap = @with_ldap@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
+with_oauth = @with_oauth@
with_system_tzdata = @with_system_tzdata@
with_uuid = @with_uuid@
with_zlib = @with_zlib@
diff --git a/src/include/common/oauth-common.h b/src/include/common/oauth-common.h
new file mode 100644
index 0000000000..5ff3488bfb
--- /dev/null
+++ b/src/include/common/oauth-common.h
@@ -0,0 +1,19 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth-common.h
+ * Declarations for helper functions used for OAuth/OIDC authentication
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/common/oauth-common.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef OAUTH_COMMON_H
+#define OAUTH_COMMON_H
+
+/* Name of SASL mechanism per IANA */
+#define OAUTHBEARER_NAME "OAUTHBEARER"
+
+#endif /* OAUTH_COMMON_H */
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 07e73567dc..f470c77669 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -231,6 +231,9 @@
/* Define to 1 if __builtin_constant_p(x) implies "i"(x) acceptance. */
#undef HAVE_I_CONSTRAINT__BUILTIN_CONSTANT_P
+/* Define to 1 if you have the `i_load_openid_config' function. */
+#undef HAVE_I_LOAD_OPENID_CONFIG
+
/* Define to 1 if you have the `kqueue' function. */
#undef HAVE_KQUEUE
@@ -243,6 +246,12 @@
/* Define to 1 if you have the `crypto' library (-lcrypto). */
#undef HAVE_LIBCRYPTO
+/* Define to 1 if you have the `curl' library (-lcurl). */
+#undef HAVE_LIBCURL
+
+/* Define to 1 if you have the `iddawc' library (-liddawc). */
+#undef HAVE_LIBIDDAWC
+
/* Define to 1 if you have the `ldap' library (-lldap). */
#undef HAVE_LIBLDAP
@@ -711,6 +720,15 @@
/* Define to select named POSIX semaphores. */
#undef USE_NAMED_POSIX_SEMAPHORES
+/* Define to 1 to build with OAuth 2.0 support. (--with-oauth) */
+#undef USE_OAUTH
+
+/* Define to 1 to use libcurl for OAuth support. */
+#undef USE_OAUTH_CURL
+
+/* Define to 1 to use libiddawc for OAuth support. */
+#undef USE_OAUTH_IDDAWC
+
/* Define to 1 to build with OpenSSL support. (--with-ssl=openssl) */
#undef USE_OPENSSL
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 083ca6f4cc..5dab88f095 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -61,6 +61,16 @@ OBJS += \
fe-secure-gssapi.o
endif
+ifneq ($(with_oauth),no)
+OBJS += fe-auth-oauth.o
+
+ifeq ($(with_oauth),iddawc)
+OBJS += fe-auth-oauth-iddawc.o
+else
+OBJS += fe-auth-oauth-curl.o
+endif
+endif
+
ifeq ($(PORTNAME), cygwin)
override shlib = cyg$(NAME)$(DLSUFFIX)
endif
@@ -79,7 +89,7 @@ endif
# that are built correctly for use in a shlib.
SHLIB_LINK_INTERNAL = -lpgcommon_shlib -lpgport_shlib
ifneq ($(PORTNAME), win32)
-SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
+SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lcurl -liddawc -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
else
SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi32 -lssl -lsocket -lnsl -lresolv -lintl -lm $(PTHREAD_LIBS), $(LIBS)) $(LDAP_LIBS_FE)
endif
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 088592deb1..0f8f5e3125 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -193,3 +193,6 @@ PQsendClosePrepared 190
PQsendClosePortal 191
PQchangePassword 192
PQsendPipelineSync 193
+PQsetAuthDataHook 194
+PQgetAuthDataHook 195
+PQdefaultAuthDataHook 196
diff --git a/src/interfaces/libpq/fe-auth-oauth-curl.c b/src/interfaces/libpq/fe-auth-oauth-curl.c
new file mode 100644
index 0000000000..3e20ba5818
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth-curl.c
@@ -0,0 +1,1981 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth-curl.c
+ * The libcurl implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth-curl.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <curl/curl.h>
+#include <math.h>
+#ifdef HAVE_SYS_EPOLL_H
+#include <sys/epoll.h>
+#include <sys/timerfd.h>
+#endif
+#ifdef HAVE_SYS_EVENT_H
+#include <sys/event.h>
+#endif
+#include <unistd.h>
+
+#include "common/jsonapi.h"
+#include "fe-auth.h"
+#include "fe-auth-oauth.h"
+#include "libpq-int.h"
+#include "mb/pg_wchar.h"
+
+/*
+ * Parsed JSON Representations
+ *
+ * As a general rule, we parse and cache only the fields we're currently using.
+ * When adding new fields, ensure the corresponding free_*() function is updated
+ * too.
+ */
+
+/*
+ * The OpenID Provider configuration (alternatively named "authorization server
+ * metadata") jointly described by OpenID Connect Discovery 1.0 and RFC 8414:
+ *
+ * https://openid.net/specs/openid-connect-discovery-1_0.html
+ * https://www.rfc-editor.org/rfc/rfc8414#section-3.2
+ */
+struct provider
+{
+ char *issuer;
+ char *token_endpoint;
+ char *device_authorization_endpoint;
+ struct curl_slist *grant_types_supported;
+};
+
+static void
+free_provider(struct provider *provider)
+{
+ free(provider->issuer);
+ free(provider->token_endpoint);
+ free(provider->device_authorization_endpoint);
+ curl_slist_free_all(provider->grant_types_supported);
+}
+
+/*
+ * The Device Authorization response, described by RFC 8628:
+ *
+ * https://www.rfc-editor.org/rfc/rfc8628#section-3.2
+ */
+struct device_authz
+{
+ char *device_code;
+ char *user_code;
+ char *verification_uri;
+ char *interval_str;
+
+ /* Fields below are parsed from the corresponding string above. */
+ int interval;
+};
+
+static void
+free_device_authz(struct device_authz *authz)
+{
+ free(authz->device_code);
+ free(authz->user_code);
+ free(authz->verification_uri);
+ free(authz->interval_str);
+}
+
+/*
+ * The Token Endpoint error response, as described by RFC 6749:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-5.2
+ *
+ * Note that this response type can also be returned from the Device
+ * Authorization Endpoint.
+ */
+struct token_error
+{
+ char *error;
+ char *error_description;
+};
+
+static void
+free_token_error(struct token_error *err)
+{
+ free(err->error);
+ free(err->error_description);
+}
+
+/*
+ * The Access Token response, as described by RFC 6749:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-4.1.4
+ *
+ * During the Device Authorization flow, several temporary errors are expected
+ * as part of normal operation. To make it easy to handle these in the happy
+ * path, this contains an embedded token_error that is filled in if needed.
+ */
+struct token
+{
+ /* for successful responses */
+ char *access_token;
+ char *token_type;
+
+ /* for error responses */
+ struct token_error err;
+};
+
+static void
+free_token(struct token *tok)
+{
+ free(tok->access_token);
+ free(tok->token_type);
+ free_token_error(&tok->err);
+}
+
+/*
+ * Asynchronous State
+ */
+
+/* States for the overall async machine. */
+typedef enum
+{
+ OAUTH_STEP_INIT,
+ OAUTH_STEP_DISCOVERY,
+ OAUTH_STEP_DEVICE_AUTHORIZATION,
+ OAUTH_STEP_TOKEN_REQUEST,
+ OAUTH_STEP_WAIT_INTERVAL,
+} OAuthStep;
+
+/*
+ * The async_ctx holds onto state that needs to persist across multiple calls to
+ * pg_fe_run_oauth_flow(). Almost everything interacts with this in some way.
+ */
+struct async_ctx
+{
+ OAuthStep step; /* where are we in the flow? */
+
+#ifdef HAVE_SYS_EPOLL_H
+ int timerfd; /* a timerfd for signaling async timeouts */
+#endif
+ pgsocket mux; /* the multiplexer socket containing all
+ * descriptors tracked by cURL, plus the
+ * timerfd */
+ CURLM *curlm; /* top-level multi handle for cURL operations */
+ CURL *curl; /* the (single) easy handle for serial
+ * requests */
+
+ struct curl_slist *headers; /* common headers for all requests */
+ PQExpBufferData work_data; /* scratch buffer for general use (remember to
+ * clear out prior contents first!) */
+
+ /*------
+ * Since a single logical operation may stretch across multiple calls to
+ * our entry point, errors have three parts:
+ *
+ * - errctx: an optional static string, describing the global operation
+ * currently in progress. It'll be translated for you.
+ *
+ * - errbuf: contains the actual error message. Generally speaking, use
+ * actx_error[_str] to manipulate this. This must be filled
+ * with something useful on an error.
+ *
+ * - curl_err: an optional static error buffer used by cURL to put
+ * detailed information about failures. Unfortunately
+ * untranslatable.
+ *
+ * These pieces will be combined into a single error message looking
+ * something like the following, with errctx and/or curl_err omitted when
+ * absent:
+ *
+ * connection to server ... failed: errctx: errbuf (curl_err)
+ */
+ const char *errctx; /* not freed; must point to static allocation */
+ PQExpBufferData errbuf;
+ char curl_err[CURL_ERROR_SIZE];
+
+ /*
+ * These documents need to survive over multiple calls, and are therefore
+ * cached directly in the async_ctx.
+ */
+ struct provider provider;
+ struct device_authz authz;
+
+ bool user_prompted; /* have we already sent the authz prompt? */
+};
+
+/*
+ * Frees the async_ctx, which is stored directly on the PGconn. This is called
+ * during pqDropConnection() so that we don't leak resources even if
+ * PQconnectPoll() never calls us back.
+ *
+ * TODO: we should probably call this at the end of a successful authentication,
+ * too, to proactively free up resources.
+ */
+static void
+free_curl_async_ctx(PGconn *conn, void *ctx)
+{
+ struct async_ctx *actx = ctx;
+
+ Assert(actx); /* oauth_free() shouldn't call us otherwise */
+
+ /*
+ * TODO: in general, none of the error cases below should ever happen if
+ * we have no bugs above. But if we do hit them, surfacing those errors
+ * somehow might be the only way to have a chance to debug them. What's
+ * the best way to do that? Assertions? Spraying messages on stderr?
+ * Bubbling an error code to the top? Appending to the connection's error
+ * message only helps if the bug caused a connection failure; otherwise
+ * it'll be buried...
+ */
+
+ if (actx->curlm && actx->curl)
+ {
+ CURLMcode err = curl_multi_remove_handle(actx->curlm, actx->curl);
+
+ if (err)
+ libpq_append_conn_error(conn,
+ "cURL easy handle removal failed: %s",
+ curl_multi_strerror(err));
+ }
+
+ if (actx->curl)
+ {
+ /*
+ * curl_multi_cleanup() doesn't free any associated easy handles; we
+ * need to do that separately. We only ever have one easy handle per
+ * multi handle.
+ */
+ curl_easy_cleanup(actx->curl);
+ }
+
+ if (actx->curlm)
+ {
+ CURLMcode err = curl_multi_cleanup(actx->curlm);
+
+ if (err)
+ libpq_append_conn_error(conn,
+ "cURL multi handle cleanup failed: %s",
+ curl_multi_strerror(err));
+ }
+
+ free_provider(&actx->provider);
+ free_device_authz(&actx->authz);
+
+ curl_slist_free_all(actx->headers);
+ termPQExpBuffer(&actx->work_data);
+ termPQExpBuffer(&actx->errbuf);
+
+ if (actx->mux != PGINVALID_SOCKET)
+ close(actx->mux);
+#ifdef HAVE_SYS_EPOLL_H
+ if (actx->timerfd >= 0)
+ close(actx->timerfd);
+#endif
+
+ free(actx);
+}
+
+/*
+ * Macros for manipulating actx->errbuf. actx_error() translates and formats a
+ * string for you; actx_error_str() appends a string directly without
+ * translation.
+ */
+
+#define actx_error(ACTX, FMT, ...) \
+ appendPQExpBuffer(&(ACTX)->errbuf, libpq_gettext(FMT), ##__VA_ARGS__)
+
+#define actx_error_str(ACTX, S) \
+ appendPQExpBufferStr(&(ACTX)->errbuf, S)
+
+/*
+ * Macros for getting and setting state for the connection's two cURL handles,
+ * so you don't have to write out the error handling every time. They assume
+ * that they're embedded in a function returning bool, however.
+ */
+
+#define CHECK_MSETOPT(ACTX, OPT, VAL) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLMcode _setopterr = curl_multi_setopt(_actx->curlm, OPT, VAL); \
+ if (_setopterr) { \
+ actx_error(_actx, "failed to set %s on OAuth connection: %s",\
+ #OPT, curl_multi_strerror(_setopterr)); \
+ return false; \
+ } \
+ } while (0)
+
+#define CHECK_SETOPT(ACTX, OPT, VAL) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLcode _setopterr = curl_easy_setopt(_actx->curl, OPT, VAL); \
+ if (_setopterr) { \
+ actx_error(_actx, "failed to set %s on OAuth connection: %s",\
+ #OPT, curl_easy_strerror(_setopterr)); \
+ return false; \
+ } \
+ } while (0)
+
+#define CHECK_GETINFO(ACTX, INFO, OUT) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLcode _getinfoerr = curl_easy_getinfo(_actx->curl, INFO, OUT); \
+ if (_getinfoerr) { \
+ actx_error(_actx, "failed to get %s from OAuth response: %s",\
+ #INFO, curl_easy_strerror(_getinfoerr)); \
+ return false; \
+ } \
+ } while (0)
+
+/*
+ * General JSON Parsing for OAuth Responses
+ */
+
+/*
+ * Represents a single name/value pair in a JSON object. This is the primary
+ * interface to parse_oauth_json().
+ *
+ * All fields are stored internally as strings or lists of strings, so clients
+ * have to explicitly parse other scalar types (though they will have gone
+ * through basic lexical validation). Storing nested objects is not currently
+ * supported, nor is parsing arrays of anything other than strings.
+ */
+struct json_field
+{
+ const char *name; /* name (key) of the member */
+
+ JsonTokenType type; /* currently supports JSON_TOKEN_STRING,
+ * JSON_TOKEN_NUMBER, and
+ * JSON_TOKEN_ARRAY_START */
+ union
+ {
+ char **scalar; /* for all scalar types */
+ struct curl_slist **array; /* for type == JSON_TOKEN_ARRAY_START */
+ };
+
+ bool required; /* REQUIRED field, or just OPTIONAL? */
+};
+
+/* Documentation macros for json_field.required. */
+#define REQUIRED true
+#define OPTIONAL false
+
+/* Parse state for parse_oauth_json(). */
+struct oauth_parse
+{
+ PQExpBuffer errbuf; /* detail message for JSON_SEM_ACTION_FAILED */
+ int nested; /* nesting level (zero is the top) */
+
+ const struct json_field *fields; /* field definition array */
+ const struct json_field *active; /* points inside the fields array */
+};
+
+#define oauth_parse_set_error(ctx, fmt, ...) \
+ appendPQExpBuffer((ctx)->errbuf, libpq_gettext(fmt), ##__VA_ARGS__)
+
+static void
+report_type_mismatch(struct oauth_parse *ctx)
+{
+ char *msgfmt;
+
+ Assert(ctx->active);
+
+ /*
+ * At the moment, the only fields we're interested in are strings,
+ * numbers, and arrays of strings.
+ */
+ switch (ctx->active->type)
+ {
+ case JSON_TOKEN_STRING:
+ msgfmt = "field \"%s\" must be a string";
+ break;
+
+ case JSON_TOKEN_NUMBER:
+ msgfmt = "field \"%s\" must be a number";
+ break;
+
+ case JSON_TOKEN_ARRAY_START:
+ msgfmt = "field \"%s\" must be an array of strings";
+ break;
+
+ default:
+ Assert(false);
+ msgfmt = "field \"%s\" has unexpected type";
+ }
+
+ oauth_parse_set_error(ctx, msgfmt, ctx->active->name);
+}
+
+static JsonParseErrorType
+oauth_json_object_start(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (ctx->active)
+ {
+ /*
+ * Currently, none of the fields we're interested in can be or contain
+ * objects, so we can reject this case outright.
+ */
+ report_type_mismatch(ctx);
+ return JSON_SEM_ACTION_FAILED;
+ }
+
+ ++ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_field_start(void *state, char *name, bool isnull)
+{
+ struct oauth_parse *ctx = state;
+
+ /* We care only about the top-level fields. */
+ if (ctx->nested == 1)
+ {
+ const struct json_field *field = ctx->fields;
+
+ /*
+ * We should never start parsing a new field while a previous one is
+ * still active.
+ *
+ * TODO: this code relies on assertions too much. We need to exit
+ * sanely on internal logic errors, to avoid turning bugs into
+ * vulnerabilities.
+ */
+ Assert(!ctx->active);
+
+ while (field->name)
+ {
+ if (!strcmp(name, field->name))
+ {
+ ctx->active = field;
+ break;
+ }
+
+ ++field;
+ }
+ }
+
+ free(name);
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_end(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_start(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (!ctx->nested)
+ {
+ oauth_parse_set_error(ctx, "top-level element must be an object");
+ return JSON_SEM_ACTION_FAILED;
+ }
+
+ if (ctx->active)
+ {
+ if (ctx->active->type != JSON_TOKEN_ARRAY_START
+ /* The arrays we care about must not have arrays as values. */
+ || ctx->nested > 1)
+ {
+ report_type_mismatch(ctx);
+ return JSON_SEM_ACTION_FAILED;
+ }
+ }
+
+ ++ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_end(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (ctx->active)
+ {
+ /*
+ * This assumes that no target arrays can contain other arrays, which
+ * we check in the array_start callback.
+ */
+ Assert(ctx->nested == 2);
+ Assert(ctx->active->type == JSON_TOKEN_ARRAY_START);
+
+ ctx->active = NULL;
+ }
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_scalar(void *state, char *token, JsonTokenType type)
+{
+ struct oauth_parse *ctx = state;
+ JsonParseErrorType result = JSON_SUCCESS;
+
+ if (!ctx->nested)
+ {
+ oauth_parse_set_error(ctx, "top-level element must be an object");
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ if (ctx->active)
+ {
+ JsonTokenType expected;
+
+ /*
+ * Make sure this matches what the active field expects. Arrays must
+ * contain only strings with the current implementation.
+ */
+ if (ctx->active->type == JSON_TOKEN_ARRAY_START)
+ expected = JSON_TOKEN_STRING;
+ else
+ expected = ctx->active->type;
+
+ if (type != expected)
+ {
+ report_type_mismatch(ctx);
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ /*
+ * FIXME if the JSON field is duplicated, we'll leak the prior value.
+ * Error out in that case instead.
+ */
+ if (ctx->active->type != JSON_TOKEN_ARRAY_START)
+ {
+ Assert(ctx->nested == 1);
+
+ *ctx->active->scalar = token;
+ ctx->active = NULL;
+
+ return JSON_SUCCESS; /* don't free the token */
+ }
+ else /* ctx->target_array */
+ {
+ struct curl_slist *temp;
+
+ Assert(ctx->nested == 2);
+
+ temp = curl_slist_append(*ctx->active->array, token);
+ if (!temp)
+ {
+ oauth_parse_set_error(ctx, "out of memory");
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ *ctx->active->array = temp;
+
+ /*
+ * Note that curl_slist_append() makes a copy of the token, so we
+ * can free it below.
+ */
+ }
+ }
+ else
+ {
+ /* otherwise we just ignore it */
+ }
+
+cleanup:
+ free(token);
+ return result;
+}
+
+/*
+ * A helper function for general JSON parsing. fields is the array of field
+ * definitions with their backing pointers. The response will be parsed from
+ * actx->curl and actx->work_data (as set up by start_request()), and any
+ * parsing errors will be placed into actx->errbuf.
+ */
+static bool
+parse_oauth_json(struct async_ctx *actx, const struct json_field *fields)
+{
+ PQExpBuffer resp = &actx->work_data;
+ char *content_type;
+ JsonLexContext lex = {0};
+ JsonSemAction sem = {0};
+ JsonParseErrorType err;
+ struct oauth_parse ctx = {0};
+ bool success = false;
+
+ /* Make sure the server thinks it's given us JSON. */
+ CHECK_GETINFO(actx, CURLINFO_CONTENT_TYPE, &content_type);
+
+ if (!content_type)
+ {
+ actx_error(actx, "no content type was provided");
+ goto cleanup;
+ }
+ else if (strcasecmp(content_type, "application/json") != 0)
+ {
+ actx_error(actx, "unexpected content type \"%s\"", content_type);
+ goto cleanup;
+ }
+
+ if (strlen(resp->data) != resp->len)
+ {
+ actx_error(actx, "response contains embedded NULLs");
+ goto cleanup;
+ }
+
+ makeJsonLexContextCstringLen(&lex, resp->data, resp->len, PG_UTF8, true);
+
+ ctx.errbuf = &actx->errbuf;
+ ctx.fields = fields;
+ sem.semstate = &ctx;
+
+ sem.object_start = oauth_json_object_start;
+ sem.object_field_start = oauth_json_object_field_start;
+ sem.object_end = oauth_json_object_end;
+ sem.array_start = oauth_json_array_start;
+ sem.array_end = oauth_json_array_end;
+ sem.scalar = oauth_json_scalar;
+
+ err = pg_parse_json(&lex, &sem);
+
+ if (err != JSON_SUCCESS)
+ {
+ /*
+ * For JSON_SEM_ACTION_FAILED, we've already written the error
+ * message. Other errors come directly from pg_parse_json(), already
+ * translated.
+ */
+ if (err != JSON_SEM_ACTION_FAILED)
+ actx_error_str(actx, json_errdetail(err, &lex));
+
+ goto cleanup;
+ }
+
+ /* Check all required fields. */
+ while (fields->name)
+ {
+ if (fields->required && !*fields->scalar && !*fields->array)
+ {
+ actx_error(actx, "field \"%s\" is missing", fields->name);
+ goto cleanup;
+ }
+
+ fields++;
+ }
+
+ success = true;
+
+cleanup:
+ freeJsonLexContext(&lex);
+ return success;
+}
+
+/*
+ * JSON Parser Definitions
+ */
+
+static bool
+parse_provider(struct async_ctx *actx, struct provider *provider)
+{
+ struct json_field fields[] = {
+ {"issuer", JSON_TOKEN_STRING, {&provider->issuer}, REQUIRED},
+ {"token_endpoint", JSON_TOKEN_STRING, {&provider->token_endpoint}, REQUIRED},
+
+ /*----
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - jwks_uri
+ * - response_types_supported
+ * - subject_types_supported
+ * - id_token_signing_alg_values_supported
+ */
+
+ {"device_authorization_endpoint", JSON_TOKEN_STRING, {&provider->device_authorization_endpoint}, OPTIONAL},
+ {"grant_types_supported", JSON_TOKEN_ARRAY_START, {.array = &provider->grant_types_supported}, OPTIONAL},
+
+ {0},
+ };
+
+ return parse_oauth_json(actx, fields);
+}
+
+/*
+ * Parses the "interval" JSON number, corresponding to the number of seconds to
+ * wait between token endpoint requests.
+ *
+ * RFC 8628 is pretty silent on sanity checks for the interval. As a matter of
+ * practicality, round any fractional intervals up to the next second, and clamp
+ * the result at a minimum of one. (Zero-second intervals would result in an
+ * expensive network polling loop.)
+ *
+ * TODO: maybe clamp the upper bound too, based on the libpq timeout and/or the
+ * code expiration time?
+ */
+static int
+parse_interval(const char *interval_str)
+{
+ float parsed;
+ int cnt;
+
+ /*
+ * The JSON lexer has already validated the number, which is stricter than
+ * the %f format, so we should be good to use sscanf().
+ */
+ cnt = sscanf(interval_str, "%f", &parsed);
+
+ if (cnt != 1)
+ {
+ /*
+ * Either the lexer screwed up or our assumption above isn't true, and
+ * either way a developer needs to take a look.
+ */
+ Assert(cnt == 1);
+ return 1; /* don't fall through in release builds */
+ }
+
+ parsed = ceilf(parsed);
+
+ if (parsed < 1)
+ return 1; /* TODO this slows down the tests
+ * considerably... */
+ else if (INT_MAX <= parsed)
+ return INT_MAX;
+
+ return parsed;
+}
+
+static bool
+parse_device_authz(struct async_ctx *actx, struct device_authz *authz)
+{
+ struct json_field fields[] = {
+ {"device_code", JSON_TOKEN_STRING, {&authz->device_code}, REQUIRED},
+ {"user_code", JSON_TOKEN_STRING, {&authz->user_code}, REQUIRED},
+ {"verification_uri", JSON_TOKEN_STRING, {&authz->verification_uri}, REQUIRED},
+
+ /*
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - expires_in
+ */
+
+ {"interval", JSON_TOKEN_NUMBER, {&authz->interval_str}, OPTIONAL},
+
+ {0},
+ };
+
+ if (!parse_oauth_json(actx, fields))
+ return false;
+
+ /*
+ * Parse our numeric fields. Lexing has already completed by this time, so
+ * we at least know they're valid JSON numbers.
+ */
+ if (authz->interval_str)
+ authz->interval = parse_interval(authz->interval_str);
+ else
+ {
+ /* TODO: handle default interval of 5 seconds */
+ }
+
+ return true;
+}
+
+static bool
+parse_token_error(struct async_ctx *actx, struct token_error *err)
+{
+ bool result;
+ struct json_field fields[] = {
+ {"error", JSON_TOKEN_STRING, {&err->error}, REQUIRED},
+
+ {"error_description", JSON_TOKEN_STRING, {&err->error_description}, OPTIONAL},
+
+ {0},
+ };
+
+ result = parse_oauth_json(actx, fields);
+
+ /*
+ * Since token errors are parsed during other active error paths, only
+ * override the errctx if parsing explicitly fails.
+ */
+ if (!result)
+ actx->errctx = "failed to parse token error response";
+
+ return result;
+}
+
+static bool
+parse_access_token(struct async_ctx *actx, struct token *tok)
+{
+ struct json_field fields[] = {
+ {"access_token", JSON_TOKEN_STRING, {&tok->access_token}, REQUIRED},
+ {"token_type", JSON_TOKEN_STRING, {&tok->token_type}, REQUIRED},
+
+ /*
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - scope (only required if different than requested -- TODO check)
+ */
+
+ {0},
+ };
+
+ return parse_oauth_json(actx, fields);
+}
+
+/*
+ * cURL Multi Setup/Callbacks
+ */
+
+/*
+ * Sets up the actx->mux, which is the altsock that PQconnectPoll clients will
+ * select() on instead of the Postgres socket during OAuth negotiation.
+ *
+ * This is just an epoll set or kqueue abstracting multiple other descriptors.
+ * A timerfd is always part of the set when using epoll; it's just disabled
+ * when we're not using it.
+ */
+static bool
+setup_multiplexer(struct async_ctx *actx)
+{
+#ifdef HAVE_SYS_EPOLL_H
+ struct epoll_event ev = {.events = EPOLLIN};
+
+ actx->mux = epoll_create1(EPOLL_CLOEXEC);
+ if (actx->mux < 0)
+ {
+ actx_error(actx, "failed to create epoll set: %m");
+ return false;
+ }
+
+ actx->timerfd = timerfd_create(CLOCK_MONOTONIC, TFD_CLOEXEC);
+ if (actx->timerfd < 0)
+ {
+ actx_error(actx, "failed to create timerfd: %m");
+ return false;
+ }
+
+ if (epoll_ctl(actx->mux, EPOLL_CTL_ADD, actx->timerfd, &ev) < 0)
+ {
+ actx_error(actx, "failed to add timerfd to epoll set: %m");
+ return false;
+ }
+
+ return true;
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ actx->mux = kqueue();
+ if (actx->mux < 0)
+ {
+ actx_error(actx, "failed to create kqueue: %m");
+ return false;
+ }
+
+ return true;
+#endif
+
+ actx_error(actx, "here's a nickel kid, get yourself a better computer");
+ return false;
+}
+
+/*
+ * Adds and removes sockets from the multiplexer set, as directed by the
+ * cURL multi handle.
+ */
+static int
+register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
+ void *socketp)
+{
+#ifdef HAVE_SYS_EPOLL_H
+ struct async_ctx *actx = ctx;
+ struct epoll_event ev = {0};
+ int res;
+ int op = EPOLL_CTL_ADD;
+
+ switch (what)
+ {
+ case CURL_POLL_IN:
+ ev.events = EPOLLIN;
+ break;
+
+ case CURL_POLL_OUT:
+ ev.events = EPOLLOUT;
+ break;
+
+ case CURL_POLL_INOUT:
+ ev.events = EPOLLIN | EPOLLOUT;
+ break;
+
+ case CURL_POLL_REMOVE:
+ op = EPOLL_CTL_DEL;
+ break;
+
+ default:
+ actx_error(actx, "unknown cURL socket operation (%d)", what);
+ return -1;
+ }
+
+ res = epoll_ctl(actx->mux, op, socket, &ev);
+ if (res < 0 && errno == EEXIST)
+ {
+ /* We already had this socket in the pollset. */
+ op = EPOLL_CTL_MOD;
+ res = epoll_ctl(actx->mux, op, socket, &ev);
+ }
+
+ if (res < 0)
+ {
+ switch (op)
+ {
+ case EPOLL_CTL_ADD:
+ actx_error(actx, "could not add to epoll set: %m");
+ break;
+
+ case EPOLL_CTL_DEL:
+ actx_error(actx, "could not delete from epoll set: %m");
+ break;
+
+ default:
+ actx_error(actx, "could not update epoll set: %m");
+ }
+
+ return -1;
+ }
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct async_ctx *actx = ctx;
+ struct kevent ev[2] = {{0}};
+ struct kevent ev_out[2];
+ struct timespec timeout = {0};
+ int nev = 0;
+ int res;
+
+ switch (what)
+ {
+ case CURL_POLL_IN:
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_OUT:
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_INOUT:
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_REMOVE:
+
+ /*
+ * We don't know which of these is currently registered, perhaps
+ * both, so we try to remove both. This means we need to tolerate
+ * ENOENT below.
+ */
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_DELETE | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_DELETE | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ default:
+ actx_error(actx, "unknown cURL socket operation (%d)", what);
+ return -1;
+ }
+
+ res = kevent(actx->mux, ev, nev, ev_out, lengthof(ev_out), &timeout);
+ if (res < 0)
+ {
+ actx_error(actx, "could not modify kqueue: %m");
+ return -1;
+ }
+
+ /*
+ * We can't use the simple errno version of kevent, because we need to
+ * skip over ENOENT while still allowing a second change to be processed.
+ * So we need a longer-form error checking loop.
+ */
+ for (int i = 0; i < res; ++i)
+ {
+ /*
+ * EV_RECEIPT should guarantee one EV_ERROR result for every change,
+ * whether successful or not. Failed entries contain a non-zero errno
+ * in the `data` field.
+ */
+ Assert(ev_out[i].flags & EV_ERROR);
+
+ errno = ev_out[i].data;
+ if (errno && errno != ENOENT)
+ {
+ switch (what)
+ {
+ case CURL_POLL_REMOVE:
+ actx_error(actx, "could not delete from kqueue: %m");
+ break;
+ default:
+ actx_error(actx, "could not add to kqueue: %m");
+ }
+ return -1;
+ }
+ }
+#endif
+
+ return 0;
+}
+
+/*
+ * Adds or removes timeouts from the multiplexer set, as directed by the
+ * cURL multi handle. Rather than continually adding and removing the timer,
+ * we keep it in the set at all times and just disarm it when it's not
+ * needed.
+ */
+static int
+register_timer(CURLM *curlm, long timeout, void *ctx)
+{
+#if HAVE_SYS_EPOLL_H
+ struct async_ctx *actx = ctx;
+ struct itimerspec spec = {0};
+
+ if (timeout < 0)
+ {
+ /* the zero itimerspec will disarm the timer below */
+ }
+ else if (timeout == 0)
+ {
+ /*
+ * A zero timeout means cURL wants us to call back immediately. That's
+ * not technically an option for timerfd, but we can make the timeout
+ * ridiculously short.
+ *
+ * TODO: maybe just signal drive_request() to immediately call back in
+ * this case?
+ */
+ spec.it_value.tv_nsec = 1;
+ }
+ else
+ {
+ spec.it_value.tv_sec = timeout / 1000;
+ spec.it_value.tv_nsec = (timeout % 1000) * 1000000;
+ }
+
+ if (timerfd_settime(actx->timerfd, 0 /* no flags */ , &spec, NULL) < 0)
+ {
+ actx_error(actx, "setting timerfd to %ld: %m", timeout);
+ return -1;
+ }
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct async_ctx *actx = ctx;
+ struct kevent ev;
+
+ EV_SET(&ev, 1, EVFILT_TIMER, timeout < 0 ? EV_DELETE : EV_ADD,
+ 0, timeout, 0);
+ if (kevent(actx->mux, &ev, 1, NULL, 0, NULL) < 0 && errno != ENOENT)
+ {
+ actx_error(actx, "setting kqueue timer to %ld: %m", timeout);
+ return -1;
+ }
+#endif
+
+ return 0;
+}
+
+/*
+ * Initializes the two cURL handles in the async_ctx. The multi handle,
+ * actx->curlm, is what drives the asynchronous engine and tells us what to do
+ * next. The easy handle, actx->curl, encapsulates the state for a single
+ * request/response. It's added to the multi handle as needed, during
+ * start_request().
+ */
+static bool
+setup_curl_handles(struct async_ctx *actx)
+{
+ /*
+ * Create our multi handle. This encapsulates the entire conversation with
+ * cURL for this connection.
+ */
+ actx->curlm = curl_multi_init();
+ if (!actx->curlm)
+ {
+ /* We don't get a lot of feedback on the failure reason. */
+ actx_error(actx, "failed to create cURL multi handle");
+ return false;
+ }
+
+ /*
+ * The multi handle tells us what to wait on using two callbacks. These
+ * will manipulate actx->mux as needed.
+ */
+ CHECK_MSETOPT(actx, CURLMOPT_SOCKETFUNCTION, register_socket);
+ CHECK_MSETOPT(actx, CURLMOPT_SOCKETDATA, actx);
+ CHECK_MSETOPT(actx, CURLMOPT_TIMERFUNCTION, register_timer);
+ CHECK_MSETOPT(actx, CURLMOPT_TIMERDATA, actx);
+
+ /*
+ * Set up an easy handle. All of our requests are made serially, so we
+ * only ever need to keep track of one.
+ */
+ actx->curl = curl_easy_init();
+ if (!actx->curl)
+ {
+ actx_error(actx, "failed to create cURL handle");
+ return false;
+ }
+
+ /*
+ * Multi-threaded applications must set CURLOPT_NOSIGNAL. This requires us
+ * to handle the possibility of SIGPIPE ourselves.
+ *
+ * TODO: This disables DNS resolution timeouts unless libcurl has been
+ * compiled against alternative resolution support. We should check that.
+ *
+ * TODO: handle SIGPIPE via pq_block_sigpipe(), or via a
+ * CURLOPT_SOCKOPTFUNCTION maybe...
+ */
+ CHECK_SETOPT(actx, CURLOPT_NOSIGNAL, 1L);
+
+ /* TODO investigate using conn->Pfdebug and CURLOPT_DEBUGFUNCTION here */
+ CHECK_SETOPT(actx, CURLOPT_VERBOSE, 1L);
+ CHECK_SETOPT(actx, CURLOPT_ERRORBUFFER, actx->curl_err);
+
+ /* TODO */
+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, stderr);
+
+ /*
+ * Only HTTP[S] is allowed. TODO: disallow HTTP without user opt-in
+ */
+ CHECK_SETOPT(actx, CURLOPT_PROTOCOLS, CURLPROTO_HTTP | CURLPROTO_HTTPS);
+
+ /*
+ * Suppress the Accept header to make our request as minimal as possible.
+ * (Ideally we would set it to "application/json" instead, but OpenID is
+ * pretty strict when it comes to provider behavior, so we have to check
+ * what comes back anyway.)
+ */
+ actx->headers = curl_slist_append(actx->headers, "Accept:"); /* TODO: check result */
+ CHECK_SETOPT(actx, CURLOPT_HTTPHEADER, actx->headers);
+
+ return true;
+}
+
+/*
+ * Generic HTTP Request Handlers
+ */
+
+/*
+ * Response callback from cURL; appends the response body into actx->work_data.
+ * See start_request().
+ */
+static size_t
+append_data(char *buf, size_t size, size_t nmemb, void *userdata)
+{
+ PQExpBuffer resp = userdata;
+ size_t len = size * nmemb;
+
+ /* TODO: cap the maximum size */
+ appendBinaryPQExpBuffer(resp, buf, len);
+ /* TODO: check for broken buffer */
+
+ return len;
+}
+
+/*
+ * Begins an HTTP request on the multi handle. The caller should have set up all
+ * request-specific options on actx->curl first. The server's response body will
+ * be accumulated in actx->work_data (which will be reset, so don't store
+ * anything important there across this call).
+ *
+ * Once a request is queued, it can be driven to completion via drive_request().
+ */
+static bool
+start_request(struct async_ctx *actx)
+{
+ CURLMcode err;
+ int running;
+
+ resetPQExpBuffer(&actx->work_data);
+ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data);
+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, &actx->work_data);
+
+ err = curl_multi_add_handle(actx->curlm, actx->curl);
+ if (err)
+ {
+ actx_error(actx, "failed to queue HTTP request: %s",
+ curl_multi_strerror(err));
+ return false;
+ }
+
+ err = curl_multi_socket_action(actx->curlm, CURL_SOCKET_TIMEOUT, 0, &running);
+ if (err)
+ {
+ actx_error(actx, "asynchronous HTTP request failed: %s",
+ curl_multi_strerror(err));
+ return false;
+ }
+
+ /*
+ * Sanity check.
+ *
+ * TODO: even though this is nominally an asynchronous process, there are
+ * apparently operations that can synchronously fail by this point, such
+ * as connections to closed local ports. Maybe we need to let this case
+ * fall through to drive_request instead, or else perform a
+ * curl_multi_info_read immediately.
+ */
+ if (running != 1)
+ {
+ actx_error(actx, "failed to queue HTTP request");
+ return false;
+ }
+
+ return true;
+}
+
+/*
+ * Drives the multi handle towards completion. The caller should have already
+ * set up an asynchronous request via start_request().
+ */
+static PostgresPollingStatusType
+drive_request(struct async_ctx *actx)
+{
+ CURLMcode err;
+ int running;
+ CURLMsg *msg;
+ int msgs_left;
+ bool done;
+
+ err = curl_multi_socket_all(actx->curlm, &running);
+ if (err)
+ {
+ actx_error(actx, "asynchronous HTTP request failed: %s",
+ curl_multi_strerror(err));
+ return PGRES_POLLING_FAILED;
+ }
+
+ if (running)
+ {
+ /* We'll come back again. */
+ return PGRES_POLLING_READING;
+ }
+
+ done = false;
+ while ((msg = curl_multi_info_read(actx->curlm, &msgs_left)) != NULL)
+ {
+ if (msg->msg != CURLMSG_DONE)
+ {
+ /*
+ * Future cURL versions may define new message types; we don't
+ * know how to handle them, so we'll ignore them.
+ */
+ continue;
+ }
+
+ /* First check the status of the request itself. */
+ if (msg->data.result != CURLE_OK)
+ {
+ actx_error_str(actx, curl_easy_strerror(msg->data.result));
+ return PGRES_POLLING_FAILED;
+ }
+
+ /* Now remove the finished handle; we'll add it back later if needed. */
+ err = curl_multi_remove_handle(actx->curlm, msg->easy_handle);
+ if (err)
+ {
+ actx_error(actx, "cURL easy handle removal failed: %s",
+ curl_multi_strerror(err));
+ return PGRES_POLLING_FAILED;
+ }
+
+ done = true;
+ }
+
+ /* Sanity check. */
+ if (!done)
+ {
+ actx_error(actx, "no result was retrieved for the finished handle");
+ return PGRES_POLLING_FAILED;
+ }
+
+ return PGRES_POLLING_OK;
+}
+
+/*
+ * Specific HTTP Request Handlers
+ *
+ * This is finally the beginning of the actual application logic. Generally
+ * speaking, a single request consists of a start_* and a finish_* step, with
+ * drive_request() pumping the machine in between.
+ */
+
+/*
+ * Queue an OpenID Provider Configuration Request:
+ *
+ * https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderConfigurationRequest
+ * https://www.rfc-editor.org/rfc/rfc8414#section-3.1
+ *
+ * This is done first to get the endpoint URIs we need to contact and to make
+ * sure the provider provides a device authorization flow. finish_discovery()
+ * will fill in actx->provider.
+ */
+static bool
+start_discovery(struct async_ctx *actx, const char *discovery_uri)
+{
+ CHECK_SETOPT(actx, CURLOPT_HTTPGET, 1L);
+ CHECK_SETOPT(actx, CURLOPT_URL, discovery_uri);
+
+ return start_request(actx);
+}
+
+static bool
+finish_discovery(struct async_ctx *actx)
+{
+ long response_code;
+
+ /*----
+ * Now check the response. OIDC Discovery 1.0 is pretty strict:
+ *
+ * A successful response MUST use the 200 OK HTTP status code and
+ * return a JSON object using the application/json content type that
+ * contains a set of Claims as its members that are a subset of the
+ * Metadata values defined in Section 3.
+ *
+ * Compared to standard HTTP semantics, this makes life easy -- we don't
+ * need to worry about redirections (which would call the Issuer host
+ * validation into question), or non-authoritative responses, or any other
+ * complications.
+ */
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code);
+
+ if (response_code != 200)
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ actx->errctx = "failed to parse OpenID discovery document";
+ if (!parse_provider(actx, &actx->provider))
+ return false; /* error message already set */
+
+ /*
+ * Fill in any defaults for OPTIONAL/RECOMMENDED fields we care about.
+ */
+ if (!actx->provider.grant_types_supported)
+ {
+ /*
+ * Per Section 3, the default is ["authorization_code", "implicit"].
+ */
+ struct curl_slist *temp = actx->provider.grant_types_supported;
+ bool oom = false;
+
+ temp = curl_slist_append(temp, "authorization_code");
+ if (!temp)
+ oom = true;
+
+ temp = curl_slist_append(temp, "implicit");
+ if (!temp)
+ oom = true;
+
+ if (oom)
+ {
+ actx_error(actx, "out of memory");
+ return false;
+ }
+
+ actx->provider.grant_types_supported = temp;
+ }
+
+ return true;
+}
+
+#define OAUTH_GRANT_TYPE_DEVICE_CODE "urn:ietf:params:oauth:grant-type:device_code"
+
+/*
+ * Ensure that the provider supports the Device Authorization flow (i.e. it
+ * accepts the device_code grant type and provides an authorization endpoint).
+ */
+static bool
+check_for_device_flow(struct async_ctx *actx)
+{
+ const struct provider *provider = &actx->provider;
+ const struct curl_slist *grant;
+ bool device_grant_found = false;
+
+ Assert(provider->issuer); /* ensured by get_discovery_document() */
+
+ /*------
+ * First, sanity checks for discovery contents that are OPTIONAL in the
+ * spec but required for our flow:
+ * - the issuer must support the device_code grant
+ * - the issuer must have actually given us a
+ * device_authorization_endpoint
+ */
+
+ grant = provider->grant_types_supported;
+ while (grant)
+ {
+ if (strcmp(grant->data, OAUTH_GRANT_TYPE_DEVICE_CODE) == 0)
+ {
+ device_grant_found = true;
+ break;
+ }
+
+ grant = grant->next;
+ }
+
+ if (!device_grant_found)
+ {
+ actx_error(actx, "issuer \"%s\" does not support device code grants",
+ provider->issuer);
+ return false;
+ }
+
+ if (!provider->device_authorization_endpoint)
+ {
+ actx_error(actx,
+ "issuer \"%s\" does not provide a device authorization endpoint",
+ provider->issuer);
+ return false;
+ }
+
+ /* TODO: check that the endpoint uses HTTPS */
+
+ return true;
+}
+
+/*
+ * Queue a Device Authorization Request:
+ *
+ * https://www.rfc-editor.org/rfc/rfc8628#section-3.1
+ *
+ * This is the second step. We ask the provider to verify the end user out of
+ * band and authorize us to act on their behalf; it will give us the required
+ * nonces for us to later poll the request status, which we'll grab in
+ * finish_device_authz().
+ */
+static bool
+start_device_authz(struct async_ctx *actx, PGconn *conn)
+{
+ const char *device_authz_uri = actx->provider.device_authorization_endpoint;
+ PQExpBuffer work_buffer = &actx->work_data;
+
+ Assert(conn->oauth_client_id); /* ensured by get_auth_token() */
+ Assert(device_authz_uri); /* ensured by check_for_device_flow() */
+
+ /* Construct our request body. TODO: url-encode */
+ resetPQExpBuffer(work_buffer);
+ appendPQExpBuffer(work_buffer, "client_id=%s", conn->oauth_client_id);
+ if (conn->oauth_scope)
+ appendPQExpBuffer(work_buffer, "&scope=%s", conn->oauth_scope);
+ /* TODO check for broken buffer */
+
+ /* Make our request. */
+ CHECK_SETOPT(actx, CURLOPT_URL, device_authz_uri);
+ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data);
+
+ if (conn->oauth_client_secret)
+ {
+ /*----
+ * Use HTTP Basic auth to send the password. Per RFC 6749, Sec. 2.3.1,
+ *
+ * Including the client credentials in the request-body using the
+ * two parameters is NOT RECOMMENDED and SHOULD be limited to
+ * clients unable to directly utilize the HTTP Basic authentication
+ * scheme (or other password-based HTTP authentication schemes).
+ *
+ * TODO: should we omit client_id from the body in this case?
+ */
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
+ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id);
+ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret);
+ }
+ else
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE);
+
+ return start_request(actx);
+}
+
+static bool
+finish_device_authz(struct async_ctx *actx)
+{
+ long response_code;
+
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code);
+
+ /*
+ * The device authorization endpoint uses the same error response as the
+ * token endpoint, so we have to handle 400/401 here too.
+ */
+ if (response_code != 200
+ && response_code != 400
+ /* && response_code != 401 TODO */ )
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ if (response_code != 200)
+ {
+ struct token_error err = {0};
+
+ if (!parse_token_error(actx, &err))
+ {
+ free_token_error(&err);
+ return false;
+ }
+
+ if (err.error_description)
+ appendPQExpBuffer(&actx->errbuf, "%s ", err.error_description);
+
+ appendPQExpBuffer(&actx->errbuf, "(%s)", err.error);
+
+ free_token_error(&err);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ actx->errctx = "failed to parse device authorization";
+ if (!parse_device_authz(actx, &actx->authz))
+ return false; /* error message already set */
+
+ return true;
+}
+
+/*
+ * Queue an Access Token Request:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-4.1.3
+ *
+ * This is the final step. We continually poll the token endpoint to see if the
+ * user has authorized us yet. finish_token_request() will pull either the token
+ * or a (ideally temporary) error status from the provider.
+ */
+static bool
+start_token_request(struct async_ctx *actx, PGconn *conn)
+{
+ const char *token_uri = actx->provider.token_endpoint;
+ const char *device_code = actx->authz.device_code;
+ PQExpBuffer work_buffer = &actx->work_data;
+
+ Assert(conn->oauth_client_id); /* ensured by get_auth_token() */
+ Assert(token_uri); /* ensured by get_discovery_document() */
+ Assert(device_code); /* ensured by run_device_authz() */
+
+ /* Construct our request body. TODO: url-encode */
+ resetPQExpBuffer(work_buffer);
+ appendPQExpBuffer(work_buffer, "client_id=%s", conn->oauth_client_id);
+ appendPQExpBuffer(work_buffer, "&device_code=%s", device_code);
+ appendPQExpBuffer(work_buffer, "&grant_type=%s",
+ OAUTH_GRANT_TYPE_DEVICE_CODE);
+ /* TODO check for broken buffer */
+
+ /* Make our request. */
+ CHECK_SETOPT(actx, CURLOPT_URL, token_uri);
+ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data);
+
+ if (conn->oauth_client_secret)
+ {
+ /*----
+ * Use HTTP Basic auth to send the password. Per RFC 6749, Sec. 2.3.1,
+ *
+ * Including the client credentials in the request-body using the
+ * two parameters is NOT RECOMMENDED and SHOULD be limited to
+ * clients unable to directly utilize the HTTP Basic authentication
+ * scheme (or other password-based HTTP authentication schemes).
+ *
+ * TODO: should we omit client_id from the body in this case?
+ */
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
+ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id);
+ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret);
+ }
+ else
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE);
+
+ resetPQExpBuffer(work_buffer);
+ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data);
+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, work_buffer);
+
+ return start_request(actx);
+}
+
+static bool
+finish_token_request(struct async_ctx *actx, struct token *tok)
+{
+ long response_code;
+
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code);
+
+ /*
+ * Per RFC 6749, Section 5, a successful response uses 200 OK. An error
+ * response uses either 400 Bad Request or 401 Unauthorized.
+ *
+ * TODO: there are references online to 403 appearing in the wild...
+ */
+ if (response_code != 200
+ && response_code != 400
+ /* && response_code != 401 TODO */ )
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ if (response_code == 200)
+ {
+ actx->errctx = "failed to parse access token response";
+ if (!parse_access_token(actx, tok))
+ return false; /* error message already set */
+ }
+ else if (!parse_token_error(actx, &tok->err))
+ return false;
+
+ return true;
+}
+
+/*
+ * The top-level, nonblocking entry point for the cURL implementation. This will
+ * be called several times to pump the async engine.
+ *
+ * The architecture is based on PQconnectPoll(). The first half drives the
+ * connection state forward as necessary, returning if we're not ready to
+ * proceed to the next step yet. The second half performs the actual transition
+ * between states.
+ *
+ * You can trace the overall OAuth flow through the second half. It's linear
+ * until we get to the end, where we flip back and forth between
+ * OAUTH_STEP_TOKEN_REQUEST and OAUTH_STEP_WAIT_INTERVAL to regularly ping the
+ * provider.
+ */
+PostgresPollingStatusType
+pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
+{
+ fe_oauth_state *state = conn->sasl_state;
+ struct async_ctx *actx;
+
+ struct token tok = {0};
+
+ /*
+ * XXX This is not safe. cURL has stringent requirements for the thread
+ * context in which you call curl_global_init(), because it's going to try
+ * initializing a bunch of other libraries (OpenSSL, Winsock...). And we
+ * probably need to consider both the TLS backend libcurl is compiled
+ * against and what the user has asked us to do via PQinit[Open]SSL.
+ *
+ * Recent versions of libcurl have improved the thread-safety situation,
+ * but you apparently can't check at compile time whether the
+ * implementation is thread-safe, and there's a chicken-and-egg problem
+ * where you can't check the thread safety until you've initialized cURL,
+ * which you can't do before you've made sure it's thread-safe...
+ *
+ * We know we've already initialized Winsock by this point, so we should
+ * be able to safely skip that bit. But we have to tell cURL to initialize
+ * everything else, because other pieces of our client executable may
+ * already be using cURL for their own purposes. If we initialize libcurl
+ * first, with only a subset of its features, we could break those other
+ * clients nondeterministically, and that would probably be a nightmare to
+ * debug.
+ */
+ curl_global_init(CURL_GLOBAL_ALL
+ & ~CURL_GLOBAL_WIN32); /* we already initialized Winsock */
+
+ if (!state->async_ctx)
+ {
+ /*
+ * Create our asynchronous state, and hook it into the upper-level
+ * OAuth state immediately, so any failures below won't leak the
+ * context allocation.
+ */
+ actx = calloc(1, sizeof(*actx));
+ if (!actx)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return PGRES_POLLING_FAILED;
+ }
+
+ actx->mux = PGINVALID_SOCKET;
+#ifdef HAVE_SYS_EPOLL_H
+ actx->timerfd = -1;
+#endif
+
+ state->async_ctx = actx;
+ state->free_async_ctx = free_curl_async_ctx;
+
+ initPQExpBuffer(&actx->work_data);
+ initPQExpBuffer(&actx->errbuf);
+
+ if (!setup_multiplexer(actx))
+ goto error_return;
+
+ if (!setup_curl_handles(actx))
+ goto error_return;
+ }
+
+ actx = state->async_ctx;
+
+ /* By default, the multiplexer is the altsock. Reassign as desired. */
+ *altsock = actx->mux;
+
+ switch (actx->step)
+ {
+ case OAUTH_STEP_INIT:
+ break;
+
+ case OAUTH_STEP_DISCOVERY:
+ case OAUTH_STEP_DEVICE_AUTHORIZATION:
+ case OAUTH_STEP_TOKEN_REQUEST:
+ {
+ PostgresPollingStatusType status;
+
+ status = drive_request(actx);
+
+ if (status == PGRES_POLLING_FAILED)
+ goto error_return;
+ else if (status != PGRES_POLLING_OK)
+ {
+ /* not done yet */
+ free_token(&tok);
+ return status;
+ }
+ }
+
+ case OAUTH_STEP_WAIT_INTERVAL:
+ /* TODO check that the timer has expired */
+ break;
+ }
+
+ switch (actx->step)
+ {
+ case OAUTH_STEP_INIT:
+ actx->errctx = "failed to fetch OpenID discovery document";
+ if (!start_discovery(actx, conn->oauth_discovery_uri))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_DISCOVERY;
+ break;
+
+ case OAUTH_STEP_DISCOVERY:
+ if (!finish_discovery(actx))
+ goto error_return;
+
+ /* TODO: check issuer */
+
+ actx->errctx = "cannot run OAuth device authorization";
+ if (!check_for_device_flow(actx))
+ goto error_return;
+
+ actx->errctx = "failed to obtain device authorization";
+ if (!start_device_authz(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_DEVICE_AUTHORIZATION;
+ break;
+
+ case OAUTH_STEP_DEVICE_AUTHORIZATION:
+ if (!finish_device_authz(actx))
+ goto error_return;
+
+ actx->errctx = "failed to obtain access token";
+ if (!start_token_request(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_TOKEN_REQUEST;
+ break;
+
+ case OAUTH_STEP_TOKEN_REQUEST:
+ {
+ const struct token_error *err;
+#ifdef HAVE_SYS_EPOLL_H
+ struct itimerspec spec = {0};
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct kevent ev = {0};
+#endif
+
+ if (!finish_token_request(actx, &tok))
+ goto error_return;
+
+ if (!actx->user_prompted)
+ {
+ int res;
+ PQpromptOAuthDevice prompt = {
+ .verification_uri = actx->authz.verification_uri,
+ .user_code = actx->authz.user_code,
+ /* TODO: optional fields */
+ };
+
+ /*
+ * Now that we know the token endpoint isn't broken, give
+ * the user the login instructions.
+ */
+ res = PQauthDataHook(PQAUTHDATA_PROMPT_OAUTH_DEVICE, conn,
+ &prompt);
+
+ if (!res)
+ {
+ fprintf(stderr, "Visit %s and enter the code: %s",
+ prompt.verification_uri, prompt.user_code);
+ }
+ else if (res < 0)
+ {
+ actx_error(actx, "device prompt failed");
+ goto error_return;
+ }
+
+ actx->user_prompted = true;
+ }
+
+ if (tok.access_token)
+ {
+ /* Construct our Bearer token. */
+ resetPQExpBuffer(&actx->work_data);
+ appendPQExpBuffer(&actx->work_data, "Bearer %s",
+ tok.access_token);
+
+ if (PQExpBufferDataBroken(actx->work_data))
+ {
+ actx_error(actx, "out of memory");
+ goto error_return;
+ }
+
+ state->token = strdup(actx->work_data.data);
+ break;
+ }
+
+ /*
+ * authorization_pending and slow_down are the only acceptable
+ * errors; anything else and we bail.
+ */
+ err = &tok.err;
+ if (!err->error || (strcmp(err->error, "authorization_pending")
+ && strcmp(err->error, "slow_down")))
+ {
+ /* TODO handle !err->error */
+ if (err->error_description)
+ appendPQExpBuffer(&actx->errbuf, "%s ",
+ err->error_description);
+
+ appendPQExpBuffer(&actx->errbuf, "(%s)", err->error);
+
+ goto error_return;
+ }
+
+ /*
+ * A slow_down error requires us to permanently increase our
+ * retry interval by five seconds. RFC 8628, Sec. 3.5.
+ */
+ if (!strcmp(err->error, "slow_down"))
+ {
+ actx->authz.interval += 5; /* TODO check for overflow? */
+ }
+
+ /*
+ * Wait for the required interval before issuing the next
+ * request.
+ */
+ Assert(actx->authz.interval > 0);
+#ifdef HAVE_SYS_EPOLL_H
+ spec.it_value.tv_sec = actx->authz.interval;
+
+ if (timerfd_settime(actx->timerfd, 0 /* no flags */ , &spec, NULL) < 0)
+ {
+ actx_error(actx, "failed to set timerfd: %m");
+ goto error_return;
+ }
+
+ *altsock = actx->timerfd;
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ /* XXX: I guess this wants to be hidden in a routine */
+ EV_SET(&ev, 1, EVFILT_TIMER, EV_ADD, 0,
+ actx->authz.interval * 1000, 0);
+ if (kevent(actx->mux, &ev, 1, NULL, 0, NULL) < 0)
+ {
+ actx_error(actx, "failed to set kqueue timer: %m");
+ goto error_return;
+ }
+ /* XXX: why did we change the altsock in the epoll version? */
+#endif
+ actx->step = OAUTH_STEP_WAIT_INTERVAL;
+ break;
+ }
+
+ case OAUTH_STEP_WAIT_INTERVAL:
+ actx->errctx = "failed to obtain access token";
+ if (!start_token_request(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_TOKEN_REQUEST;
+ break;
+ }
+
+ free_token(&tok);
+
+ /* If we've stored a token, we're done. Otherwise come back later. */
+ return state->token ? PGRES_POLLING_OK : PGRES_POLLING_READING;
+
+error_return:
+
+ /*
+ * Assemble the three parts of our error: context, body, and detail. See
+ * also the documentation for struct async_ctx.
+ */
+ if (actx->errctx)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext(actx->errctx));
+ appendPQExpBufferStr(&conn->errorMessage, ": ");
+ }
+
+ if (PQExpBufferDataBroken(actx->errbuf))
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("out of memory"));
+ else
+ appendPQExpBufferStr(&conn->errorMessage, actx->errbuf.data);
+
+ if (actx->curl_err[0])
+ {
+ size_t len;
+
+ appendPQExpBuffer(&conn->errorMessage, " (%s)", actx->curl_err);
+
+ /* Sometimes libcurl adds a newline to the error buffer. :( */
+ len = conn->errorMessage.len;
+ if (len >= 2 && conn->errorMessage.data[len - 2] == '\n')
+ {
+ conn->errorMessage.data[len - 2] = ')';
+ conn->errorMessage.data[len - 1] = '\0';
+ conn->errorMessage.len--;
+ }
+ }
+
+ appendPQExpBufferStr(&conn->errorMessage, "\n");
+
+ free_token(&tok);
+ return PGRES_POLLING_FAILED;
+}
diff --git a/src/interfaces/libpq/fe-auth-oauth-iddawc.c b/src/interfaces/libpq/fe-auth-oauth-iddawc.c
new file mode 100644
index 0000000000..e78d4304d3
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth-iddawc.c
@@ -0,0 +1,319 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth-iddawc.c
+ * The libiddawc implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth-iddawc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <iddawc.h>
+
+#include "fe-auth.h"
+#include "fe-auth-oauth.h"
+#include "libpq-int.h"
+
+#ifdef HAVE_I_LOAD_OPENID_CONFIG
+/* Older versions of iddawc used 'load' instead of 'get' for some APIs. */
+#define i_get_openid_config i_load_openid_config
+#endif
+
+static const char *
+iddawc_error_string(int errcode)
+{
+ switch (errcode)
+ {
+ case I_OK:
+ return "I_OK";
+
+ case I_ERROR:
+ return "I_ERROR";
+
+ case I_ERROR_PARAM:
+ return "I_ERROR_PARAM";
+
+ case I_ERROR_MEMORY:
+ return "I_ERROR_MEMORY";
+
+ case I_ERROR_UNAUTHORIZED:
+ return "I_ERROR_UNAUTHORIZED";
+
+ case I_ERROR_SERVER:
+ return "I_ERROR_SERVER";
+ }
+
+ return "<unknown>";
+}
+
+static void
+iddawc_error(PGconn *conn, int errcode, const char *msg)
+{
+ appendPQExpBufferStr(&conn->errorMessage, libpq_gettext(msg));
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext(" (iddawc error %s)\n"),
+ iddawc_error_string(errcode));
+}
+
+static void
+iddawc_request_error(PGconn *conn, struct _i_session *i, int err, const char *msg)
+{
+ const char *error_code;
+ const char *desc;
+
+ appendPQExpBuffer(&conn->errorMessage, "%s: ", libpq_gettext(msg));
+
+ error_code = i_get_str_parameter(i, I_OPT_ERROR);
+ if (!error_code)
+ {
+ /*
+ * The server didn't give us any useful information, so just print the
+ * error code.
+ */
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("(iddawc error %s)\n"),
+ iddawc_error_string(err));
+ return;
+ }
+
+ /* If the server gave a string description, print that too. */
+ desc = i_get_str_parameter(i, I_OPT_ERROR_DESCRIPTION);
+ if (desc)
+ appendPQExpBuffer(&conn->errorMessage, "%s ", desc);
+
+ appendPQExpBuffer(&conn->errorMessage, "(%s)\n", error_code);
+}
+
+/*
+ * Runs the device authorization flow using libiddawc. If successful, a malloc'd
+ * token string in "Bearer xxxx..." format, suitable for sending to an
+ * OAUTHBEARER server, is returned. NULL is returned on error.
+ */
+static char *
+run_iddawc_auth_flow(PGconn *conn, const char *discovery_uri)
+{
+ struct _i_session session;
+ PQExpBuffer token_buf = NULL;
+ int err;
+ int auth_method;
+ bool user_prompted = false;
+ const char *verification_uri;
+ const char *user_code;
+ const char *access_token;
+ const char *token_type;
+ char *token = NULL;
+
+ i_init_session(&session);
+
+ token_buf = createPQExpBuffer();
+ if (!token_buf)
+ goto cleanup;
+
+ err = i_set_str_parameter(&session, I_OPT_OPENID_CONFIG_ENDPOINT, discovery_uri);
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set OpenID config endpoint");
+ goto cleanup;
+ }
+
+ err = i_get_openid_config(&session);
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to fetch OpenID discovery document");
+ goto cleanup;
+ }
+
+ if (!i_get_str_parameter(&session, I_OPT_TOKEN_ENDPOINT))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer has no token endpoint"));
+ goto cleanup;
+ }
+
+ if (!i_get_str_parameter(&session, I_OPT_DEVICE_AUTHORIZATION_ENDPOINT))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer does not support device authorization"));
+ goto cleanup;
+ }
+
+ err = i_set_response_type(&session, I_RESPONSE_TYPE_DEVICE_CODE);
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set device code response type");
+ goto cleanup;
+ }
+
+ auth_method = I_TOKEN_AUTH_METHOD_NONE;
+ if (conn->oauth_client_secret && *conn->oauth_client_secret)
+ auth_method = I_TOKEN_AUTH_METHOD_SECRET_BASIC;
+
+ err = i_set_parameter_list(&session,
+ I_OPT_CLIENT_ID, conn->oauth_client_id,
+ I_OPT_CLIENT_SECRET, conn->oauth_client_secret,
+ I_OPT_TOKEN_METHOD, auth_method,
+ I_OPT_SCOPE, conn->oauth_scope,
+ I_OPT_NONE
+ );
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set client identifier");
+ goto cleanup;
+ }
+
+ err = i_run_device_auth_request(&session);
+ if (err)
+ {
+ iddawc_request_error(conn, &session, err,
+ "failed to obtain device authorization");
+ goto cleanup;
+ }
+
+ verification_uri = i_get_str_parameter(&session, I_OPT_DEVICE_AUTH_VERIFICATION_URI);
+ if (!verification_uri)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer did not provide a verification URI"));
+ goto cleanup;
+ }
+
+ user_code = i_get_str_parameter(&session, I_OPT_DEVICE_AUTH_USER_CODE);
+ if (!user_code)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer did not provide a user code"));
+ goto cleanup;
+ }
+
+ /*
+ * Poll the token endpoint until either the user logs in and authorizes
+ * the use of a token, or a hard failure occurs. We perform one ping
+ * _before_ prompting the user, so that we don't make them do the work of
+ * logging in only to find that the token endpoint is completely
+ * unreachable.
+ */
+ err = i_run_token_request(&session);
+ while (err)
+ {
+ const char *error_code;
+ uint interval;
+
+ error_code = i_get_str_parameter(&session, I_OPT_ERROR);
+
+ /*
+ * authorization_pending and slow_down are the only acceptable errors;
+ * anything else and we bail.
+ */
+ if (!error_code || (strcmp(error_code, "authorization_pending")
+ && strcmp(error_code, "slow_down")))
+ {
+ iddawc_request_error(conn, &session, err,
+ "failed to obtain access token");
+ goto cleanup;
+ }
+
+ if (!user_prompted)
+ {
+ int res;
+ PQpromptOAuthDevice prompt = {
+ .verification_uri = verification_uri,
+ .user_code = user_code,
+ /* TODO: optional fields */
+ };
+
+ /*
+ * Now that we know the token endpoint isn't broken, give the user
+ * the login instructions.
+ */
+ res = PQauthDataHook(PQAUTHDATA_PROMPT_OAUTH_DEVICE, conn,
+ &prompt);
+
+ if (!res)
+ {
+ fprintf(stderr, "Visit %s and enter the code: %s",
+ prompt.verification_uri, prompt.user_code);
+ }
+ else if (res < 0)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("device prompt failed\n"));
+ goto cleanup;
+ }
+
+ user_prompted = true;
+ }
+
+ /*---
+ * We are required to wait between polls; the server tells us how
+ * long.
+ * TODO: if interval's not set, we need to default to five seconds
+ * TODO: sanity check the interval
+ */
+ interval = i_get_int_parameter(&session, I_OPT_DEVICE_AUTH_INTERVAL);
+
+ /*
+ * A slow_down error requires us to permanently increase our retry
+ * interval by five seconds. RFC 8628, Sec. 3.5.
+ */
+ if (!strcmp(error_code, "slow_down"))
+ {
+ interval += 5;
+ i_set_int_parameter(&session, I_OPT_DEVICE_AUTH_INTERVAL, interval);
+ }
+
+ sleep(interval);
+
+ /*
+ * XXX Reset the error code before every call, because iddawc won't do
+ * that for us. This matters if the server first sends a "pending"
+ * error code, then later hard-fails without sending an error code to
+ * overwrite the first one.
+ *
+ * That we have to do this at all seems like a bug in iddawc.
+ */
+ i_set_str_parameter(&session, I_OPT_ERROR, NULL);
+
+ err = i_run_token_request(&session);
+ }
+
+ access_token = i_get_str_parameter(&session, I_OPT_ACCESS_TOKEN);
+ token_type = i_get_str_parameter(&session, I_OPT_TOKEN_TYPE);
+
+ if (!access_token || !token_type || strcasecmp(token_type, "Bearer"))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer did not provide a bearer token"));
+ goto cleanup;
+ }
+
+ appendPQExpBufferStr(token_buf, "Bearer ");
+ appendPQExpBufferStr(token_buf, access_token);
+
+ if (PQExpBufferBroken(token_buf))
+ goto cleanup;
+
+ token = strdup(token_buf->data);
+
+cleanup:
+ if (token_buf)
+ destroyPQExpBuffer(token_buf);
+ i_clean_session(&session);
+
+ return token;
+}
+
+PostgresPollingStatusType
+pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
+{
+ fe_oauth_state *state = conn->sasl_state;
+
+ /* TODO: actually make this asynchronous */
+ state->token = run_iddawc_auth_flow(conn, conn->oauth_discovery_uri);
+ return state->token ? PGRES_POLLING_OK : PGRES_POLLING_FAILED;
+}
diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
new file mode 100644
index 0000000000..2a35c7438c
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth.c
@@ -0,0 +1,659 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth.c
+ * The front-end (client) implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "common/base64.h"
+#include "common/hmac.h"
+#include "common/jsonapi.h"
+#include "common/oauth-common.h"
+#include "fe-auth.h"
+#include "fe-auth-oauth.h"
+#include "mb/pg_wchar.h"
+
+/* The exported OAuth callback mechanism. */
+static void *oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism);
+static SASLStatus oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen);
+static bool oauth_channel_bound(void *opaq);
+static void oauth_free(void *opaq);
+
+const pg_fe_sasl_mech pg_oauth_mech = {
+ oauth_init,
+ oauth_exchange,
+ oauth_channel_bound,
+ oauth_free,
+};
+
+static void *
+oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism)
+{
+ fe_oauth_state *state;
+
+ /*
+ * We only support one SASL mechanism here; anything else is programmer
+ * error.
+ */
+ Assert(sasl_mechanism != NULL);
+ Assert(!strcmp(sasl_mechanism, OAUTHBEARER_NAME));
+
+ state = calloc(1, sizeof(*state));
+ if (!state)
+ return NULL;
+
+ state->state = FE_OAUTH_INIT;
+ state->conn = conn;
+
+ return state;
+}
+
+#define kvsep "\x01"
+
+static char *
+client_initial_response(PGconn *conn, const char *token)
+{
+ static const char *const resp_format = "n,," kvsep "auth=%s" kvsep kvsep;
+
+ PQExpBufferData buf;
+ char *response = NULL;
+
+ if (!token)
+ {
+ /*
+ * Either programmer error, or something went badly wrong during the
+ * asynchronous fetch.
+ *
+ * TODO: users shouldn't see this; what action should they take if
+ * they do?
+ */
+ libpq_append_conn_error(conn, "no OAuth token was set for the connection");
+ return NULL;
+ }
+
+ initPQExpBuffer(&buf);
+ appendPQExpBuffer(&buf, resp_format, token);
+
+ if (!PQExpBufferDataBroken(buf))
+ response = strdup(buf.data);
+
+ termPQExpBuffer(&buf);
+ return response;
+}
+
+#define ERROR_STATUS_FIELD "status"
+#define ERROR_SCOPE_FIELD "scope"
+#define ERROR_OPENID_CONFIGURATION_FIELD "openid-configuration"
+
+struct json_ctx
+{
+ char *errmsg; /* any non-NULL value stops all processing */
+ PQExpBufferData errbuf; /* backing memory for errmsg */
+ int nested; /* nesting level (zero is the top) */
+
+ const char *target_field_name; /* points to a static allocation */
+ char **target_field; /* see below */
+
+ /* target_field, if set, points to one of the following: */
+ char *status;
+ char *scope;
+ char *discovery_uri;
+};
+
+#define oauth_json_has_error(ctx) \
+ (PQExpBufferDataBroken((ctx)->errbuf) || (ctx)->errmsg)
+
+#define oauth_json_set_error(ctx, ...) \
+ do { \
+ appendPQExpBuffer(&(ctx)->errbuf, __VA_ARGS__); \
+ (ctx)->errmsg = (ctx)->errbuf.data; \
+ } while (0)
+
+static JsonParseErrorType
+oauth_json_object_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ ++ctx->nested;
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_end(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_field_start(void *state, char *name, bool isnull)
+{
+ struct json_ctx *ctx = state;
+
+ if (ctx->nested == 1)
+ {
+ if (!strcmp(name, ERROR_STATUS_FIELD))
+ {
+ ctx->target_field_name = ERROR_STATUS_FIELD;
+ ctx->target_field = &ctx->status;
+ }
+ else if (!strcmp(name, ERROR_SCOPE_FIELD))
+ {
+ ctx->target_field_name = ERROR_SCOPE_FIELD;
+ ctx->target_field = &ctx->scope;
+ }
+ else if (!strcmp(name, ERROR_OPENID_CONFIGURATION_FIELD))
+ {
+ ctx->target_field_name = ERROR_OPENID_CONFIGURATION_FIELD;
+ ctx->target_field = &ctx->discovery_uri;
+ }
+ }
+
+ free(name);
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_scalar(void *state, char *token, JsonTokenType type)
+{
+ struct json_ctx *ctx = state;
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ if (type == JSON_TOKEN_STRING)
+ {
+ *ctx->target_field = token;
+
+ ctx->target_field = NULL;
+ ctx->target_field_name = NULL;
+
+ return JSON_SUCCESS; /* don't free the token we're using */
+ }
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ free(token);
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static bool
+handle_oauth_sasl_error(PGconn *conn, char *msg, int msglen)
+{
+ JsonLexContext lex = {0};
+ JsonSemAction sem = {0};
+ JsonParseErrorType err;
+ struct json_ctx ctx = {0};
+ char *errmsg = NULL;
+
+ /* Sanity check. */
+ if (strlen(msg) != msglen)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server's error message contained an embedded NULL"));
+ return false;
+ }
+
+ makeJsonLexContextCstringLen(&lex, msg, msglen, PG_UTF8, true);
+
+ initPQExpBuffer(&ctx.errbuf);
+ sem.semstate = &ctx;
+
+ sem.object_start = oauth_json_object_start;
+ sem.object_end = oauth_json_object_end;
+ sem.object_field_start = oauth_json_object_field_start;
+ sem.array_start = oauth_json_array_start;
+ sem.scalar = oauth_json_scalar;
+
+ err = pg_parse_json(&lex, &sem);
+
+ if (err == JSON_SEM_ACTION_FAILED)
+ {
+ if (PQExpBufferDataBroken(ctx.errbuf))
+ errmsg = libpq_gettext("out of memory");
+ else if (ctx.errmsg)
+ errmsg = ctx.errmsg;
+ else
+ {
+ /*
+ * Developer error: one of the action callbacks didn't call
+ * oauth_json_set_error() before erroring out.
+ */
+ Assert(oauth_json_has_error(&ctx));
+ errmsg = "<unexpected empty error>";
+ }
+ }
+ else if (err != JSON_SUCCESS)
+ errmsg = json_errdetail(err, &lex);
+
+ if (errmsg)
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("failed to parse server's error response: %s"),
+ errmsg);
+
+ /* Don't need the error buffer or the JSON lexer anymore. */
+ termPQExpBuffer(&ctx.errbuf);
+ freeJsonLexContext(&lex);
+
+ if (errmsg)
+ return false;
+
+ /* TODO: what if these override what the user already specified? */
+ if (ctx.discovery_uri)
+ {
+ if (conn->oauth_discovery_uri)
+ free(conn->oauth_discovery_uri);
+
+ conn->oauth_discovery_uri = ctx.discovery_uri;
+ }
+
+ if (ctx.scope)
+ {
+ if (conn->oauth_scope)
+ free(conn->oauth_scope);
+
+ conn->oauth_scope = ctx.scope;
+ }
+ /* TODO: missing error scope should clear any existing connection scope */
+
+ if (!ctx.status)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("server sent error response without a status"));
+ return false;
+ }
+
+ if (!strcmp(ctx.status, "invalid_token"))
+ {
+ /*
+ * invalid_token is the only error code we'll automatically retry for,
+ * but only if we have enough information to do so.
+ */
+ if (conn->oauth_discovery_uri)
+ conn->oauth_want_retry = true;
+ }
+ /* TODO: include status in hard failure message */
+
+ return true;
+}
+
+static void
+free_request(PGconn *conn, void *vreq)
+{
+ PQoauthBearerRequest *request = vreq;
+
+ if (request->cleanup)
+ request->cleanup(conn, request);
+
+ free(request);
+}
+
+static PostgresPollingStatusType
+run_user_oauth_flow(PGconn *conn, pgsocket *altsock)
+{
+ fe_oauth_state *state = conn->sasl_state;
+ PQoauthBearerRequest *request = state->async_ctx;
+ PostgresPollingStatusType status;
+
+ if (!request->async)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow provided neither a token nor an async callback");
+ return PGRES_POLLING_FAILED;
+ }
+
+ status = request->async(conn, request, altsock);
+ if (status == PGRES_POLLING_FAILED)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow failed");
+ return status;
+ }
+ else if (status == PGRES_POLLING_OK)
+ {
+ /*
+ * We already have a token, so copy it into the state. (We can't hold
+ * onto the original string, since it may not be safe for us to free()
+ * it.)
+ */
+ PQExpBufferData token;
+
+ if (!request->token)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow did not provide a token");
+ return PGRES_POLLING_FAILED;
+ }
+
+ initPQExpBuffer(&token);
+ appendPQExpBuffer(&token, "Bearer %s", request->token);
+
+ if (PQExpBufferDataBroken(token))
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return PGRES_POLLING_FAILED;
+ }
+
+ state->token = token.data;
+ return PGRES_POLLING_OK;
+ }
+
+ /* TODO: what if no altsock was set? */
+ return status;
+}
+
+static bool
+setup_token_request(PGconn *conn, fe_oauth_state *state)
+{
+ int res;
+ PQoauthBearerRequest request = {
+ .openid_configuration = conn->oauth_discovery_uri,
+ .scope = conn->oauth_scope,
+ };
+
+ Assert(request.openid_configuration);
+
+ /* The client may have overridden the OAuth flow. */
+ res = PQauthDataHook(PQAUTHDATA_OAUTH_BEARER_TOKEN, conn, &request);
+ if (res > 0)
+ {
+ PQoauthBearerRequest *request_copy;
+
+ if (request.token)
+ {
+ /*
+ * We already have a token, so copy it into the state. (We can't
+ * hold onto the original string, since it may not be safe for us
+ * to free() it.)
+ */
+ PQExpBufferData token;
+
+ initPQExpBuffer(&token);
+ appendPQExpBuffer(&token, "Bearer %s", request.token);
+
+ if (PQExpBufferDataBroken(token))
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ goto fail;
+ }
+
+ state->token = token.data;
+
+ /* short-circuit */
+ if (request.cleanup)
+ request.cleanup(conn, &request);
+ return true;
+ }
+
+ request_copy = malloc(sizeof(*request_copy));
+ if (!request_copy)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ goto fail;
+ }
+
+ memcpy(request_copy, &request, sizeof(request));
+
+ conn->async_auth = run_user_oauth_flow;
+ state->async_ctx = request_copy;
+ state->free_async_ctx = free_request;
+ }
+ else if (res < 0)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow failed");
+ goto fail;
+ }
+ else
+ {
+ /* Use our built-in OAuth flow. */
+ conn->async_auth = pg_fe_run_oauth_flow;
+ }
+
+ return true;
+
+fail:
+ if (request.cleanup)
+ request.cleanup(conn, &request);
+ return false;
+}
+
+static bool
+derive_discovery_uri(PGconn *conn)
+{
+ PQExpBufferData discovery_buf;
+
+ if (conn->oauth_discovery_uri || !conn->oauth_issuer)
+ {
+ /*
+ * Either we already have one, or we aren't able to derive one
+ * ourselves. The latter case is not an error condition; we'll just
+ * ask the server to provide one for us.
+ */
+ return true;
+ }
+
+ initPQExpBuffer(&discovery_buf);
+
+ Assert(!conn->oauth_discovery_uri);
+ Assert(conn->oauth_issuer);
+
+ /*
+ * If we don't yet have a discovery URI, but the user gave us an explicit
+ * issuer, use the .well-known discovery URI for that issuer.
+ */
+ appendPQExpBufferStr(&discovery_buf, conn->oauth_issuer);
+ appendPQExpBufferStr(&discovery_buf, "/.well-known/openid-configuration");
+
+ if (PQExpBufferDataBroken(discovery_buf))
+ goto cleanup;
+
+ conn->oauth_discovery_uri = strdup(discovery_buf.data);
+
+cleanup:
+ termPQExpBuffer(&discovery_buf);
+ return (conn->oauth_discovery_uri != NULL);
+}
+
+static SASLStatus
+oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen)
+{
+ fe_oauth_state *state = opaq;
+ PGconn *conn = state->conn;
+
+ *output = NULL;
+ *outputlen = 0;
+
+ switch (state->state)
+ {
+ case FE_OAUTH_INIT:
+ Assert(inputlen == -1);
+
+ if (!derive_discovery_uri(conn))
+ return SASL_FAILED;
+
+ if (conn->oauth_discovery_uri)
+ {
+ if (!conn->oauth_client_id)
+ {
+ /* We can't talk to a server without a client identifier. */
+ libpq_append_conn_error(conn, "no oauth_client_id is set for the connection");
+ return SASL_FAILED;
+ }
+
+ /*
+ * Decide whether we're using a user-provided OAuth flow, or
+ * the one we have built in.
+ */
+ if (!setup_token_request(conn, state))
+ return SASL_FAILED;
+
+ if (state->token)
+ {
+ /*
+ * A really smart user implementation may have already
+ * given us the token (e.g. if there was an unexpired copy
+ * already cached). In that case, we can just fall
+ * through.
+ */
+ }
+ else
+ {
+ /*
+ * Otherwise, we have to hand the connection over to our
+ * OAuth implementation. This involves a number of HTTP
+ * connections and timed waits, so we escape the
+ * synchronous auth processing and tell PQconnectPoll to
+ * transfer control to our async implementation.
+ */
+ Assert(conn->async_auth); /* should have been set
+ * already */
+ state->state = FE_OAUTH_REQUESTING_TOKEN;
+ return SASL_ASYNC;
+ }
+ }
+ else
+ {
+ /*
+ * If we don't have a discovery URI to be able to request a
+ * token, we ask the server for one explicitly with an empty
+ * token. This doesn't require any asynchronous work.
+ */
+ state->token = strdup("");
+ if (!state->token)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return SASL_FAILED;
+ }
+ }
+
+ /* fall through */
+
+ case FE_OAUTH_REQUESTING_TOKEN:
+ /* We should still be in the initial response phase. */
+ Assert(inputlen == -1);
+
+ *output = client_initial_response(conn, state->token);
+ if (!*output)
+ return SASL_FAILED;
+
+ *outputlen = strlen(*output);
+ state->state = FE_OAUTH_BEARER_SENT;
+
+ return SASL_CONTINUE;
+
+ case FE_OAUTH_BEARER_SENT:
+ if (final)
+ {
+ /* TODO: ensure there is no message content here. */
+ return SASL_COMPLETE;
+ }
+
+ /*
+ * Error message sent by the server.
+ */
+ if (!handle_oauth_sasl_error(conn, input, inputlen))
+ return SASL_FAILED;
+
+ /*
+ * Respond with the required dummy message (RFC 7628, sec. 3.2.3).
+ */
+ *output = strdup(kvsep);
+ *outputlen = strlen(*output); /* == 1 */
+
+ state->state = FE_OAUTH_SERVER_ERROR;
+ return SASL_CONTINUE;
+
+ case FE_OAUTH_SERVER_ERROR:
+
+ /*
+ * After an error, the server should send an error response to
+ * fail the SASL handshake, which is handled in higher layers.
+ *
+ * If we get here, the server either sent *another* challenge
+ * which isn't defined in the RFC, or completed the handshake
+ * successfully after telling us it was going to fail. Neither is
+ * acceptable.
+ */
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server sent additional OAuth data after error\n"));
+ return SASL_FAILED;
+
+ default:
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("invalid OAuth exchange state\n"));
+ break;
+ }
+
+ return SASL_FAILED;
+}
+
+static bool
+oauth_channel_bound(void *opaq)
+{
+ /* This mechanism does not support channel binding. */
+ return false;
+}
+
+static void
+oauth_free(void *opaq)
+{
+ fe_oauth_state *state = opaq;
+
+ free(state->token);
+ if (state->async_ctx)
+ state->free_async_ctx(state->conn, state->async_ctx);
+
+ free(state);
+}
diff --git a/src/interfaces/libpq/fe-auth-oauth.h b/src/interfaces/libpq/fe-auth-oauth.h
new file mode 100644
index 0000000000..8d4ea45aa8
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth.h
@@ -0,0 +1,42 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth.h
+ *
+ * Definitions for OAuth authentication implementations
+ *
+ * Portions Copyright (c) 2023, PostgreSQL Global Development Group
+ *
+ * src/interfaces/libpq/fe-auth-oauth.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FE_AUTH_OAUTH_H
+#define FE_AUTH_OAUTH_H
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+
+
+typedef enum
+{
+ FE_OAUTH_INIT,
+ FE_OAUTH_REQUESTING_TOKEN,
+ FE_OAUTH_BEARER_SENT,
+ FE_OAUTH_SERVER_ERROR,
+} fe_oauth_state_enum;
+
+typedef struct
+{
+ fe_oauth_state_enum state;
+
+ PGconn *conn;
+ char *token;
+
+ void *async_ctx;
+ void (*free_async_ctx) (PGconn *conn, void *ctx);
+} fe_oauth_state;
+
+extern PostgresPollingStatusType pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock);
+
+#endif /* FE_AUTH_OAUTH_H */
diff --git a/src/interfaces/libpq/fe-auth-sasl.h b/src/interfaces/libpq/fe-auth-sasl.h
index dce16b7b8b..83bdab1f40 100644
--- a/src/interfaces/libpq/fe-auth-sasl.h
+++ b/src/interfaces/libpq/fe-auth-sasl.h
@@ -30,6 +30,7 @@ typedef enum
SASL_COMPLETE = 0,
SASL_FAILED,
SASL_CONTINUE,
+ SASL_ASYNC,
} SASLStatus;
/*
@@ -77,6 +78,8 @@ typedef struct pg_fe_sasl_mech
*
* state: The opaque mechanism state returned by init()
*
+ * final: true if the server has sent a final exchange outcome
+ *
* input: The challenge data sent by the server, or NULL when
* generating a client-first initial response (that is, when
* the server expects the client to send a message to start
@@ -101,12 +104,17 @@ typedef struct pg_fe_sasl_mech
*
* SASL_CONTINUE: The output buffer is filled with a client response.
* Additional server challenge is expected
+ * SASL_ASYNC: Some asynchronous processing external to the
+ * connection needs to be done before a response can be
+ * generated. The mechanism is responsible for setting up
+ * conn->async_auth appropriately before returning.
* SASL_COMPLETE: The SASL exchange has completed successfully.
- * SASL_FAILED: The exchance has failed and the connection should be
+ * SASL_FAILED: The exchange has failed and the connection should be
* dropped.
*--------
*/
- SASLStatus (*exchange) (void *state, char *input, int inputlen,
+ SASLStatus (*exchange) (void *state, bool final,
+ char *input, int inputlen,
char **output, int *outputlen);
/*--------
diff --git a/src/interfaces/libpq/fe-auth-scram.c b/src/interfaces/libpq/fe-auth-scram.c
index 0bb820e0d9..da168eb2f5 100644
--- a/src/interfaces/libpq/fe-auth-scram.c
+++ b/src/interfaces/libpq/fe-auth-scram.c
@@ -24,7 +24,8 @@
/* The exported SCRAM callback mechanism. */
static void *scram_init(PGconn *conn, const char *password,
const char *sasl_mechanism);
-static SASLStatus scram_exchange(void *opaq, char *input, int inputlen,
+static SASLStatus scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen);
static bool scram_channel_bound(void *opaq);
static void scram_free(void *opaq);
@@ -202,7 +203,8 @@ scram_free(void *opaq)
* Exchange a SCRAM message with backend.
*/
static SASLStatus
-scram_exchange(void *opaq, char *input, int inputlen,
+scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen)
{
fe_scram_state *state = (fe_scram_state *) opaq;
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index 9f57976b4f..9e084dd1c7 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -39,6 +39,7 @@
#endif
#include "common/md5.h"
+#include "common/oauth-common.h"
#include "common/scram-common.h"
#include "fe-auth.h"
#include "fe-auth-sasl.h"
@@ -419,13 +420,13 @@ pg_SSPI_startup(PGconn *conn, int use_negotiate, int payloadlen)
* Initialize SASL authentication exchange.
*/
static int
-pg_SASL_init(PGconn *conn, int payloadlen)
+pg_SASL_init(PGconn *conn, int payloadlen, bool *async)
{
char *initialresponse = NULL;
int initialresponselen;
const char *selected_mechanism;
PQExpBufferData mechanism_buf;
- char *password;
+ char *password = NULL;
SASLStatus status;
initPQExpBuffer(&mechanism_buf);
@@ -437,7 +438,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
goto error;
}
- if (conn->sasl_state)
+ if (conn->sasl_state && !conn->async_auth)
{
libpq_append_conn_error(conn, "duplicate SASL authentication request");
goto error;
@@ -524,6 +525,15 @@ pg_SASL_init(PGconn *conn, int payloadlen)
conn->sasl = &pg_scram_mech;
conn->password_needed = true;
}
+#ifdef USE_OAUTH
+ else if (strcmp(mechanism_buf.data, OAUTHBEARER_NAME) == 0 &&
+ !selected_mechanism)
+ {
+ selected_mechanism = OAUTHBEARER_NAME;
+ conn->sasl = &pg_oauth_mech;
+ conn->password_needed = false;
+ }
+#endif
}
if (!selected_mechanism)
@@ -563,26 +573,48 @@ pg_SASL_init(PGconn *conn, int payloadlen)
Assert(conn->sasl);
- /*
- * Initialize the SASL state information with all the information gathered
- * during the initial exchange.
- *
- * Note: Only tls-unique is supported for the moment.
- */
- conn->sasl_state = conn->sasl->init(conn,
- password,
- selected_mechanism);
if (!conn->sasl_state)
- goto oom_error;
+ {
+ /*
+ * Initialize the SASL state information with all the information
+ * gathered during the initial exchange.
+ *
+ * Note: Only tls-unique is supported for the moment.
+ */
+ conn->sasl_state = conn->sasl->init(conn,
+ password,
+ selected_mechanism);
+ if (!conn->sasl_state)
+ goto oom_error;
+ }
+ else
+ {
+ /*
+ * This is only possible if we're returning from an async loop.
+ * Disconnect it now.
+ */
+ Assert(conn->async_auth);
+ conn->async_auth = NULL;
+ }
/* Get the mechanism-specific Initial Client Response, if any */
- status = conn->sasl->exchange(conn->sasl_state,
+ status = conn->sasl->exchange(conn->sasl_state, false,
NULL, -1,
&initialresponse, &initialresponselen);
if (status == SASL_FAILED)
goto error;
+ if (status == SASL_ASYNC)
+ {
+ /*
+ * The mechanism should have set up the necessary callbacks; all we
+ * need to do is signal the caller.
+ */
+ *async = true;
+ return STATUS_OK;
+ }
+
/*
* Build a SASLInitialResponse message, and send it.
*/
@@ -625,7 +657,7 @@ oom_error:
* the protocol.
*/
static int
-pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
+pg_SASL_continue(PGconn *conn, int payloadlen, bool final, bool *async)
{
char *output;
int outputlen;
@@ -650,11 +682,21 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
/* For safety and convenience, ensure the buffer is NULL-terminated. */
challenge[payloadlen] = '\0';
- status = conn->sasl->exchange(conn->sasl_state,
+ status = conn->sasl->exchange(conn->sasl_state, final,
challenge, payloadlen,
&output, &outputlen);
free(challenge); /* don't need the input anymore */
+ if (status == SASL_ASYNC)
+ {
+ /*
+ * The mechanism should have set up the necessary callbacks; all we
+ * need to do is signal the caller.
+ */
+ *async = true;
+ return STATUS_OK;
+ }
+
if (final && !(status == SASL_FAILED || status == SASL_COMPLETE))
{
if (outputlen != 0)
@@ -955,12 +997,18 @@ check_expected_areq(AuthRequest areq, PGconn *conn)
* it. We are responsible for reading any remaining extra data, specific
* to the authentication method. 'payloadlen' is the remaining length in
* the message.
+ *
+ * If *async is set to true on return, the client doesn't yet have enough
+ * information to respond, and the caller must temporarily switch to
+ * conn->async_auth() to continue driving the exchange.
*/
int
-pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
+pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn, bool *async)
{
int oldmsglen;
+ *async = false;
+
if (!check_expected_areq(areq, conn))
return STATUS_ERROR;
@@ -1118,7 +1166,7 @@ pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
* The request contains the name (as assigned by IANA) of the
* authentication mechanism.
*/
- if (pg_SASL_init(conn, payloadlen) != STATUS_OK)
+ if (pg_SASL_init(conn, payloadlen, async) != STATUS_OK)
{
/* pg_SASL_init already set the error message */
return STATUS_ERROR;
@@ -1135,7 +1183,8 @@ pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
}
oldmsglen = conn->errorMessage.len;
if (pg_SASL_continue(conn, payloadlen,
- (areq == AUTH_REQ_SASL_FIN)) != STATUS_OK)
+ (areq == AUTH_REQ_SASL_FIN),
+ async) != STATUS_OK)
{
/* Use this message if pg_SASL_continue didn't supply one */
if (conn->errorMessage.len == oldmsglen)
@@ -1451,3 +1500,23 @@ PQchangePassword(PGconn *conn, const char *user, const char *passwd)
}
}
}
+
+PQauthDataHook_type PQauthDataHook = PQdefaultAuthDataHook;
+
+PQauthDataHook_type
+PQgetAuthDataHook(void)
+{
+ return PQauthDataHook;
+}
+
+void
+PQsetAuthDataHook(PQauthDataHook_type hook)
+{
+ PQauthDataHook = hook ? hook : PQdefaultAuthDataHook;
+}
+
+int
+PQdefaultAuthDataHook(PGAuthData type, PGconn *conn, void *data)
+{
+ return 0; /* handle nothing */
+}
diff --git a/src/interfaces/libpq/fe-auth.h b/src/interfaces/libpq/fe-auth.h
index a18c508688..3e2bc1333f 100644
--- a/src/interfaces/libpq/fe-auth.h
+++ b/src/interfaces/libpq/fe-auth.h
@@ -18,8 +18,12 @@
#include "libpq-int.h"
+extern PQauthDataHook_type PQauthDataHook;
+
+
/* Prototypes for functions in fe-auth.c */
-extern int pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn);
+extern int pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn,
+ bool *async);
extern char *pg_fe_getusername(uid_t user_id, PQExpBuffer errorMessage);
extern char *pg_fe_getauthname(PQExpBuffer errorMessage);
@@ -29,4 +33,7 @@ extern char *pg_fe_scram_build_secret(const char *password,
int iterations,
const char **errstr);
+/* Mechanisms in fe-auth-oauth.c */
+extern const pg_fe_sasl_mech pg_oauth_mech;
+
#endif /* FE_AUTH_H */
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index d4e10a0c4f..15ceb73d01 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -359,6 +359,23 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"Load-Balance-Hosts", "", 8, /* sizeof("disable") = 8 */
offsetof(struct pg_conn, load_balance_hosts)},
+ /* OAuth v2 */
+ {"oauth_issuer", NULL, NULL, NULL,
+ "OAuth-Issuer", "", 40,
+ offsetof(struct pg_conn, oauth_issuer)},
+
+ {"oauth_client_id", NULL, NULL, NULL,
+ "OAuth-Client-ID", "", 40,
+ offsetof(struct pg_conn, oauth_client_id)},
+
+ {"oauth_client_secret", NULL, NULL, NULL,
+ "OAuth-Client-Secret", "", 40,
+ offsetof(struct pg_conn, oauth_client_secret)},
+
+ {"oauth_scope", NULL, NULL, NULL,
+ "OAuth-Scope", "", 15,
+ offsetof(struct pg_conn, oauth_scope)},
+
/* Terminating entry --- MUST BE LAST */
{NULL, NULL, NULL, NULL,
NULL, NULL, 0}
@@ -618,6 +635,7 @@ pqDropServerData(PGconn *conn)
conn->write_err_msg = NULL;
conn->be_pid = 0;
conn->be_key = 0;
+ /* conn->oauth_want_retry = false; TODO */
}
@@ -2536,6 +2554,7 @@ PQconnectPoll(PGconn *conn)
case CONNECTION_NEEDED:
case CONNECTION_GSS_STARTUP:
case CONNECTION_CHECK_TARGET:
+ case CONNECTION_AUTHENTICATING:
break;
default:
@@ -3517,6 +3536,7 @@ keep_going: /* We will come back to here until there is
int avail;
AuthRequest areq;
int res;
+ bool async;
/*
* Scan the message from current point (note that if we find
@@ -3672,6 +3692,16 @@ keep_going: /* We will come back to here until there is
/* Check to see if we should mention pgpassfile */
pgpassfileWarning(conn);
+#ifdef USE_OAUTH
+ if (conn->sasl == &pg_oauth_mech
+ && conn->oauth_want_retry)
+ {
+ /* TODO: only allow retry once */
+ need_new_connection = true;
+ goto keep_going;
+ }
+#endif
+
#ifdef ENABLE_GSS
/*
@@ -3753,7 +3783,17 @@ keep_going: /* We will come back to here until there is
* Note that conn->pghost must be non-NULL if we are going to
* avoid the Kerberos code doing a hostname look-up.
*/
- res = pg_fe_sendauth(areq, msgLength, conn);
+ res = pg_fe_sendauth(areq, msgLength, conn, &async);
+
+ if (async && (res == STATUS_OK))
+ {
+ /*
+ * We'll come back later once we're ready to respond.
+ * Don't consume the request yet.
+ */
+ conn->status = CONNECTION_AUTHENTICATING;
+ goto keep_going;
+ }
/* OK, we have processed the message; mark data consumed */
conn->inStart = conn->inCursor;
@@ -3786,6 +3826,41 @@ keep_going: /* We will come back to here until there is
goto keep_going;
}
+ case CONNECTION_AUTHENTICATING:
+ {
+ PostgresPollingStatusType status;
+ pgsocket altsock = PGINVALID_SOCKET;
+
+ if (!conn->async_auth)
+ {
+ /* programmer error; should not happen */
+ libpq_append_conn_error(conn, "async authentication has no handler");
+ goto error_return;
+ }
+
+ status = conn->async_auth(conn, &altsock);
+
+ if (status == PGRES_POLLING_FAILED)
+ goto error_return;
+
+ if (status == PGRES_POLLING_OK)
+ {
+ /*
+ * Reenter the authentication exchange with the server. We
+ * didn't consume the message that started external
+ * authentication, so it'll be reprocessed as if we just
+ * received it.
+ */
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ conn->altsock = PGINVALID_SOCKET; /* TODO: what frees
+ * this? */
+ goto keep_going;
+ }
+
+ conn->altsock = altsock;
+ return status;
+ }
+
case CONNECTION_AUTH_OK:
{
/*
@@ -4285,6 +4360,7 @@ pqMakeEmptyPGconn(void)
conn->verbosity = PQERRORS_DEFAULT;
conn->show_context = PQSHOW_CONTEXT_ERRORS;
conn->sock = PGINVALID_SOCKET;
+ conn->altsock = PGINVALID_SOCKET;
conn->Pfdebug = NULL;
/*
@@ -4400,6 +4476,11 @@ freePGconn(PGconn *conn)
free(conn->rowBuf);
free(conn->target_session_attrs);
free(conn->load_balance_hosts);
+ free(conn->oauth_issuer);
+ free(conn->oauth_discovery_uri);
+ free(conn->oauth_client_id);
+ free(conn->oauth_client_secret);
+ free(conn->oauth_scope);
termPQExpBuffer(&conn->errorMessage);
termPQExpBuffer(&conn->workBuffer);
@@ -6868,6 +6949,8 @@ PQsocket(const PGconn *conn)
{
if (!conn)
return -1;
+ if (conn->altsock != PGINVALID_SOCKET)
+ return conn->altsock;
return (conn->sock != PGINVALID_SOCKET) ? conn->sock : -1;
}
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index f2fc78a481..663b1c1acf 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -1039,10 +1039,13 @@ static int
pqSocketCheck(PGconn *conn, int forRead, int forWrite, time_t end_time)
{
int result;
+ pgsocket sock;
if (!conn)
return -1;
- if (conn->sock == PGINVALID_SOCKET)
+
+ sock = (conn->altsock != PGINVALID_SOCKET) ? conn->altsock : conn->sock;
+ if (sock == PGINVALID_SOCKET)
{
libpq_append_conn_error(conn, "invalid socket");
return -1;
@@ -1059,7 +1062,7 @@ pqSocketCheck(PGconn *conn, int forRead, int forWrite, time_t end_time)
/* We will retry as long as we get EINTR */
do
- result = pqSocketPoll(conn->sock, forRead, forWrite, end_time);
+ result = pqSocketPoll(sock, forRead, forWrite, end_time);
while (result < 0 && SOCK_ERRNO == EINTR);
if (result < 0)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index defc415fa3..d095351c66 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -38,6 +38,8 @@ extern "C"
#define LIBPQ_HAS_TRACE_FLAGS 1
/* Indicates that PQsslAttribute(NULL, "library") is useful */
#define LIBPQ_HAS_SSL_LIBRARY_DETECTION 1
+/* Indicates presence of the PQAUTHDATA_PROMPT_OAUTH_DEVICE authdata hook */
+#define LIBPQ_HAS_PROMPT_OAUTH_DEVICE 1
/*
* Option flags for PQcopyResult
@@ -78,7 +80,9 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_AUTHENTICATING, /* Authentication is in progress with some
+ * external system. */
} ConnStatusType;
typedef enum
@@ -160,6 +164,13 @@ typedef enum
PQ_PIPELINE_ABORTED
} PGpipelineStatus;
+typedef enum
+{
+ PQAUTHDATA_PROMPT_OAUTH_DEVICE, /* user must visit a device-authorization
+ * URL */
+ PQAUTHDATA_OAUTH_BEARER_TOKEN, /* server requests an OAuth Bearer token */
+} PGAuthData;
+
/* PGconn encapsulates a connection to the backend.
* The contents of this struct are not supposed to be known to applications.
*/
@@ -658,10 +669,74 @@ extern int PQenv2encoding(void);
/* === in fe-auth.c === */
+typedef struct _PQpromptOAuthDevice
+{
+ const char *verification_uri; /* verification URI to visit */
+ const char *user_code; /* user code to enter */
+} PQpromptOAuthDevice;
+
+typedef struct _PQoauthBearerRequest
+{
+ /* Hook inputs (constant across all calls) */
+ const char *const openid_configuration; /* OIDC discovery URI */
+ const char *const scope; /* required scope(s), or NULL */
+
+ /* Hook outputs */
+
+ /*---------
+ * Callback implementing a custom asynchronous OAuth flow.
+ *
+ * The callback may return
+ * - PGRES_POLLING_READING/WRITING, to indicate that a file descriptor
+ * has been stored in *altsock and libpq should wait until it is
+ * readable or writable before calling back;
+ * - PGRES_POLLING_OK, to indicate that the flow is complete and
+ * request->token has been set; or
+ * - PGRES_POLLING_FAILED, to indicate that token retrieval has failed.
+ *
+ * This callback is optional. If the token can be obtained without
+ * blocking during the original call to the PQAUTHDATA_OAUTH_BEARER_TOKEN
+ * hook, it may be returned directly, but one of request->async or
+ * request->token must be set by the hook.
+ */
+ PostgresPollingStatusType (*async) (PGconn *conn,
+ struct _PQoauthBearerRequest *request,
+ int *altsock);
+
+ /*
+ * Callback to clean up custom allocations. A hook implementation may use
+ * this to free request->token and any resources in request->user.
+ *
+ * This is technically optional, but highly recommended, because there is
+ * no other indication as to when it is safe to free the token.
+ */
+ void (*cleanup) (PGconn *conn, struct _PQoauthBearerRequest *request);
+
+ /*
+ * The hook should set this to the Bearer token contents for the
+ * connection, once the flow is completed. The token contents must remain
+ * available to libpq until the hook's cleanup callback is called.
+ */
+ char *token;
+
+ /*
+ * Hook-defined data. libpq will not modify this pointer across calls to
+ * the async callback, so it can be used to keep track of
+ * application-specific state. Resources allocated here should be freed by
+ * the cleanup callback.
+ */
+ void *user;
+} PQoauthBearerRequest;
+
extern char *PQencryptPassword(const char *passwd, const char *user);
extern char *PQencryptPasswordConn(PGconn *conn, const char *passwd, const char *user, const char *algorithm);
extern PGresult *PQchangePassword(PGconn *conn, const char *user, const char *passwd);
+typedef int (*PQauthDataHook_type) (PGAuthData type, PGconn *conn, void *data);
+extern void PQsetAuthDataHook(PQauthDataHook_type hook);
+extern PQauthDataHook_type PQgetAuthDataHook(void);
+extern int PQdefaultAuthDataHook(PGAuthData type, PGconn *conn, void *data);
+
/* === in encnames.c === */
extern int pg_char_to_encoding(const char *name);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 82c18f870d..cf26c693e3 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -351,6 +351,8 @@ typedef struct pg_conn_host
* found in password file. */
} pg_conn_host;
+typedef PostgresPollingStatusType (*AsyncAuthFunc) (PGconn *conn, pgsocket *altsock);
+
/*
* PGconn stores all the state data associated with a single connection
* to a backend.
@@ -409,6 +411,15 @@ struct pg_conn
char *require_auth; /* name of the expected auth method */
char *load_balance_hosts; /* load balance over hosts */
+ /* OAuth v2 */
+ char *oauth_issuer; /* token issuer URL */
+ char *oauth_discovery_uri; /* URI of the issuer's discovery
+ * document */
+ char *oauth_client_id; /* client identifier */
+ char *oauth_client_secret; /* client secret */
+ char *oauth_scope; /* access token scope */
+ bool oauth_want_retry; /* should we retry on failure? */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -477,6 +488,9 @@ struct pg_conn
bool client_finished_auth; /* have we finished our half of the
* authentication exchange? */
+ AsyncAuthFunc async_auth; /* callback for external async authentication */
+ pgsocket altsock; /* alternative socket for client to poll */
+
/* Transient state needed while establishing connection */
PGTargetServerType target_server_type; /* desired session properties */
diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build
index a47b6f425d..753a7137d6 100644
--- a/src/interfaces/libpq/meson.build
+++ b/src/interfaces/libpq/meson.build
@@ -40,6 +40,15 @@ if gssapi.found()
)
endif
+if oauth.found()
+ libpq_sources += files('fe-auth-oauth.c')
+ if oauth_library == 'iddawc'
+ libpq_sources += files('fe-auth-oauth-iddawc.c')
+ else
+ libpq_sources += files('fe-auth-oauth-curl.c')
+ endif
+endif
+
export_file = custom_target('libpq.exports',
kwargs: gen_export_kwargs,
)
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index b0f4178b3d..f803c1200b 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -231,6 +231,7 @@ pgxs_deps = {
'llvm': llvm,
'lz4': lz4,
'nls': libintl,
+ 'oauth': oauth,
'pam': pam,
'perl': perl_dep,
'python': python3_dep,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 2461567026..6234fe66f1 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -153,6 +153,7 @@ ArrayMetaState
ArraySubWorkspace
ArrayToken
ArrayType
+AsyncAuthFunc
AsyncQueueControl
AsyncQueueEntry
AsyncRequest
@@ -354,6 +355,8 @@ CState
CTECycleClause
CTEMaterialize
CTESearchClause
+CURL
+CURLM
CV
CachedExpression
CachedPlan
@@ -1653,6 +1656,7 @@ NumericDigit
NumericSortSupport
NumericSumAccum
NumericVar
+OAuthStep
OM_uint32
OP
OSAPerGroupState
@@ -1718,6 +1722,7 @@ PFN
PGAlignedBlock
PGAlignedXLogBlock
PGAsyncStatusType
+PGAuthData
PGCALL2
PGChecksummablePage
PGContextVisibility
@@ -1877,11 +1882,14 @@ PQArgBlock
PQEnvironmentOption
PQExpBuffer
PQExpBufferData
+PQauthDataHook_type
PQcommMethods
PQconninfoOption
PQnoticeProcessor
PQnoticeReceiver
+PQoauthBearerRequest
PQprintOpt
+PQpromptOAuthDevice
PQsslKeyPassHook_OpenSSL_type
PREDICATELOCK
PREDICATELOCKTAG
@@ -3343,6 +3351,8 @@ explain_get_index_name_hook_type
f_smgr
fasthash_state
fd_set
+fe_oauth_state
+fe_oauth_state_enum
fe_scram_state
fe_scram_state_enum
fetch_range_request
--
2.34.1
v17-0001-common-jsonapi-support-FRONTEND-clients.patch (application/octet-stream, 19.6K) download
From 00976d4f750ba5d6ea5973f9edd8fe9c521ee9aa Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Mon, 3 May 2021 15:38:26 -0700
Subject: [PATCH v17 1/9] common/jsonapi: support FRONTEND clients
Based on a patch by Michael Paquier.
For frontend code, use PQExpBuffer instead of StringInfo. This requires
us to track allocation failures so that we can return JSON_OUT_OF_MEMORY
as needed. json_errdetail() now allocates its error message inside
memory owned by the JsonLexContext, so clients don't need to worry about
freeing it.
We can now partially revert b44669b2ca, now that json_errdetail() works
correctly.
---
src/bin/pg_verifybackup/t/005_bad_manifest.pl | 3 +-
src/common/Makefile | 2 +-
src/common/jsonapi.c | 270 +++++++++++++-----
src/common/meson.build | 8 +-
src/common/parse_manifest.c | 2 +-
src/include/common/jsonapi.h | 18 +-
6 files changed, 222 insertions(+), 81 deletions(-)
diff --git a/src/bin/pg_verifybackup/t/005_bad_manifest.pl b/src/bin/pg_verifybackup/t/005_bad_manifest.pl
index e278ccea5a..e2a297930e 100644
--- a/src/bin/pg_verifybackup/t/005_bad_manifest.pl
+++ b/src/bin/pg_verifybackup/t/005_bad_manifest.pl
@@ -13,7 +13,8 @@ use Test::More;
my $tempdir = PostgreSQL::Test::Utils::tempdir;
test_bad_manifest('input string ended unexpectedly',
- qr/could not parse backup manifest: parsing failed/, <<EOM);
+ qr/could not parse backup manifest: The input string ended unexpectedly/,
+ <<EOM);
{
EOM
diff --git a/src/common/Makefile b/src/common/Makefile
index 2ba5069dca..bbb5c3ab11 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -40,7 +40,7 @@ override CPPFLAGS += -DVAL_LDFLAGS_EX="\"$(LDFLAGS_EX)\""
override CPPFLAGS += -DVAL_LDFLAGS_SL="\"$(LDFLAGS_SL)\""
override CPPFLAGS += -DVAL_LIBS="\"$(LIBS)\""
-override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common $(CPPFLAGS)
+override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common -I$(libpq_srcdir) $(CPPFLAGS)
LIBS += $(PTHREAD_LIBS)
OBJS_COMMON = \
diff --git a/src/common/jsonapi.c b/src/common/jsonapi.c
index 32931ded82..479310f598 100644
--- a/src/common/jsonapi.c
+++ b/src/common/jsonapi.c
@@ -21,10 +21,41 @@
#include "mb/pg_wchar.h"
#include "port/pg_lfind.h"
-#ifndef FRONTEND
+#ifdef FRONTEND
+#include "pqexpbuffer.h"
+#else
+#include "lib/stringinfo.h"
#include "miscadmin.h"
#endif
+/*
+ * In backend, we will use palloc/pfree along with StringInfo. In frontend, use
+ * malloc and PQExpBuffer, and return JSON_OUT_OF_MEMORY on out-of-memory.
+ */
+#ifdef FRONTEND
+
+#define STRDUP(s) strdup(s)
+#define ALLOC(size) malloc(size)
+
+#define appendStrVal appendPQExpBuffer
+#define appendBinaryStrVal appendBinaryPQExpBuffer
+#define appendStrValChar appendPQExpBufferChar
+#define createStrVal createPQExpBuffer
+#define resetStrVal resetPQExpBuffer
+
+#else /* !FRONTEND */
+
+#define STRDUP(s) pstrdup(s)
+#define ALLOC(size) palloc(size)
+
+#define appendStrVal appendStringInfo
+#define appendBinaryStrVal appendBinaryStringInfo
+#define appendStrValChar appendStringInfoChar
+#define createStrVal makeStringInfo
+#define resetStrVal resetStringInfo
+
+#endif
+
/*
* The context of the parser is maintained by the recursive descent
* mechanism, but is passed explicitly to the error reporting routine
@@ -167,9 +198,16 @@ makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
lex->input_encoding = encoding;
if (need_escapes)
{
- lex->strval = makeStringInfo();
+ /*
+ * This call can fail in FRONTEND code. We defer error handling to
+ * time of use (json_lex_string()) since we might not need to parse
+ * any strings anyway.
+ */
+ lex->strval = createStrVal();
lex->flags |= JSONLEX_FREE_STRVAL;
+ lex->parse_strval = true;
}
+ lex->errormsg = NULL;
return lex;
}
@@ -182,13 +220,30 @@ makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
void
freeJsonLexContext(JsonLexContext *lex)
{
+ static const JsonLexContext empty = {0};
+
if (lex->flags & JSONLEX_FREE_STRVAL)
{
+#ifdef FRONTEND
+ destroyPQExpBuffer(lex->strval);
+#else
pfree(lex->strval->data);
pfree(lex->strval);
+#endif
+ }
+ if (lex->errormsg)
+ {
+#ifdef FRONTEND
+ destroyPQExpBuffer(lex->errormsg);
+#else
+ pfree(lex->errormsg->data);
+ pfree(lex->errormsg);
+#endif
}
if (lex->flags & JSONLEX_FREE_STRUCT)
pfree(lex);
+ else
+ *lex = empty;
}
/*
@@ -254,7 +309,7 @@ json_count_array_elements(JsonLexContext *lex, int *elements)
* etc, so doing this with a copy makes that safe.
*/
memcpy(©lex, lex, sizeof(JsonLexContext));
- copylex.strval = NULL; /* not interested in values here */
+ copylex.parse_strval = false; /* not interested in values here */
copylex.lex_level++;
count = 0;
@@ -316,14 +371,21 @@ parse_scalar(JsonLexContext *lex, JsonSemAction *sem)
/* extract the de-escaped string value, or the raw lexeme */
if (lex_peek(lex) == JSON_TOKEN_STRING)
{
- if (lex->strval != NULL)
- val = pstrdup(lex->strval->data);
+ if (lex->parse_strval)
+ {
+ val = STRDUP(lex->strval->data);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
}
else
{
int len = (lex->token_terminator - lex->token_start);
- val = palloc(len + 1);
+ val = ALLOC(len + 1);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+
memcpy(val, lex->token_start, len);
val[len] = '\0';
}
@@ -357,8 +419,12 @@ parse_object_field(JsonLexContext *lex, JsonSemAction *sem)
if (lex_peek(lex) != JSON_TOKEN_STRING)
return report_parse_error(JSON_PARSE_STRING, lex);
- if ((ostart != NULL || oend != NULL) && lex->strval != NULL)
- fname = pstrdup(lex->strval->data);
+ if ((ostart != NULL || oend != NULL) && lex->parse_strval)
+ {
+ fname = STRDUP(lex->strval->data);
+ if (fname == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
result = json_lex(lex);
if (result != JSON_SUCCESS)
return result;
@@ -414,6 +480,11 @@ parse_object(JsonLexContext *lex, JsonSemAction *sem)
JsonParseErrorType result;
#ifndef FRONTEND
+
+ /*
+ * TODO: clients need some way to put a bound on stack growth. Parse level
+ * limits maybe?
+ */
check_stack_depth();
#endif
@@ -762,8 +833,15 @@ json_lex_string(JsonLexContext *lex)
return code; \
} while (0)
- if (lex->strval != NULL)
- resetStringInfo(lex->strval);
+ if (lex->parse_strval)
+ {
+#ifdef FRONTEND
+ /* make sure initialization succeeded */
+ if (lex->strval == NULL)
+ return JSON_OUT_OF_MEMORY;
+#endif
+ resetStrVal(lex->strval);
+ }
Assert(lex->input_length > 0);
s = lex->token_start;
@@ -800,7 +878,7 @@ json_lex_string(JsonLexContext *lex)
else
FAIL_AT_CHAR_END(JSON_UNICODE_ESCAPE_FORMAT);
}
- if (lex->strval != NULL)
+ if (lex->parse_strval)
{
/*
* Combine surrogate pairs.
@@ -857,19 +935,19 @@ json_lex_string(JsonLexContext *lex)
unicode_to_utf8(ch, (unsigned char *) utf8str);
utf8len = pg_utf_mblen((unsigned char *) utf8str);
- appendBinaryStringInfo(lex->strval, utf8str, utf8len);
+ appendBinaryPQExpBuffer(lex->strval, utf8str, utf8len);
}
else if (ch <= 0x007f)
{
/* The ASCII range is the same in all encodings */
- appendStringInfoChar(lex->strval, (char) ch);
+ appendPQExpBufferChar(lex->strval, (char) ch);
}
else
FAIL_AT_CHAR_END(JSON_UNICODE_HIGH_ESCAPE);
#endif /* FRONTEND */
}
}
- else if (lex->strval != NULL)
+ else if (lex->parse_strval)
{
if (hi_surrogate != -1)
FAIL_AT_CHAR_END(JSON_UNICODE_LOW_SURROGATE);
@@ -879,22 +957,22 @@ json_lex_string(JsonLexContext *lex)
case '"':
case '\\':
case '/':
- appendStringInfoChar(lex->strval, *s);
+ appendStrValChar(lex->strval, *s);
break;
case 'b':
- appendStringInfoChar(lex->strval, '\b');
+ appendStrValChar(lex->strval, '\b');
break;
case 'f':
- appendStringInfoChar(lex->strval, '\f');
+ appendStrValChar(lex->strval, '\f');
break;
case 'n':
- appendStringInfoChar(lex->strval, '\n');
+ appendStrValChar(lex->strval, '\n');
break;
case 'r':
- appendStringInfoChar(lex->strval, '\r');
+ appendStrValChar(lex->strval, '\r');
break;
case 't':
- appendStringInfoChar(lex->strval, '\t');
+ appendStrValChar(lex->strval, '\t');
break;
default:
@@ -929,7 +1007,7 @@ json_lex_string(JsonLexContext *lex)
/*
* Skip to the first byte that requires special handling, so we
- * can batch calls to appendBinaryStringInfo.
+ * can batch calls to appendBinaryStrVal.
*/
while (p < end - sizeof(Vector8) &&
!pg_lfind8('\\', (uint8 *) p, sizeof(Vector8)) &&
@@ -953,8 +1031,8 @@ json_lex_string(JsonLexContext *lex)
}
}
- if (lex->strval != NULL)
- appendBinaryStringInfo(lex->strval, s, p - s);
+ if (lex->parse_strval)
+ appendBinaryStrVal(lex->strval, s, p - s);
/*
* s will be incremented at the top of the loop, so set it to just
@@ -970,6 +1048,11 @@ json_lex_string(JsonLexContext *lex)
return JSON_UNICODE_LOW_SURROGATE;
}
+#ifdef FRONTEND
+ if (lex->parse_strval && PQExpBufferBroken(lex->strval))
+ return JSON_OUT_OF_MEMORY;
+#endif
+
/* Hooray, we found the end of the string! */
lex->prev_token_terminator = lex->token_terminator;
lex->token_terminator = s + 1;
@@ -1145,72 +1228,93 @@ report_parse_error(JsonParseContext ctx, JsonLexContext *lex)
return JSON_SUCCESS; /* silence stupider compilers */
}
-
-#ifndef FRONTEND
-/*
- * Extract the current token from a lexing context, for error reporting.
- */
-static char *
-extract_token(JsonLexContext *lex)
-{
- int toklen = lex->token_terminator - lex->token_start;
- char *token = palloc(toklen + 1);
-
- memcpy(token, lex->token_start, toklen);
- token[toklen] = '\0';
- return token;
-}
-
/*
* Construct an (already translated) detail message for a JSON error.
*
- * Note that the error message generated by this routine may not be
- * palloc'd, making it unsafe for frontend code as there is no way to
- * know if this can be safely pfree'd or not.
+ * The returned allocation is either static or owned by the JsonLexContext and
+ * should not be freed.
*/
char *
json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
{
+ int toklen = lex->token_terminator - lex->token_start;
+
+ if (error == JSON_OUT_OF_MEMORY)
+ {
+ /* Short circuit. Allocating anything for this case is unhelpful. */
+ return _("out of memory");
+ }
+
+ if (lex->errormsg)
+ resetStrVal(lex->errormsg);
+ else
+ lex->errormsg = createStrVal();
+
switch (error)
{
case JSON_SUCCESS:
/* fall through to the error code after switch */
break;
case JSON_ESCAPING_INVALID:
- return psprintf(_("Escape sequence \"\\%s\" is invalid."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Escape sequence \"\\%.*s\" is invalid."),
+ toklen, lex->token_start);
+ break;
case JSON_ESCAPING_REQUIRED:
- return psprintf(_("Character with value 0x%02x must be escaped."),
- (unsigned char) *(lex->token_terminator));
+ appendStrVal(lex->errormsg,
+ _("Character with value 0x%02x must be escaped."),
+ (unsigned char) *(lex->token_terminator));
+ break;
case JSON_EXPECTED_END:
- return psprintf(_("Expected end of input, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected end of input, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_ARRAY_FIRST:
- return psprintf(_("Expected array element or \"]\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected array element or \"]\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_ARRAY_NEXT:
- return psprintf(_("Expected \",\" or \"]\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \",\" or \"]\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_COLON:
- return psprintf(_("Expected \":\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \":\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_JSON:
- return psprintf(_("Expected JSON value, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected JSON value, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_MORE:
return _("The input string ended unexpectedly.");
case JSON_EXPECTED_OBJECT_FIRST:
- return psprintf(_("Expected string or \"}\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected string or \"}\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_OBJECT_NEXT:
- return psprintf(_("Expected \",\" or \"}\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \",\" or \"}\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_STRING:
- return psprintf(_("Expected string, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected string, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_INVALID_TOKEN:
- return psprintf(_("Token \"%s\" is invalid."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Token \"%.*s\" is invalid."),
+ toklen, lex->token_start);
+ break;
+ case JSON_OUT_OF_MEMORY:
+ /* should have been handled above; use the error path */
+ break;
case JSON_UNICODE_CODE_POINT_ZERO:
return _("\\u0000 cannot be converted to text.");
case JSON_UNICODE_ESCAPE_FORMAT:
@@ -1219,9 +1323,19 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
/* note: this case is only reachable in frontend not backend */
return _("Unicode escape values cannot be used for code point values above 007F when the encoding is not UTF8.");
case JSON_UNICODE_UNTRANSLATABLE:
- /* note: this case is only reachable in backend not frontend */
+
+ /*
+ * note: this case is only reachable in backend not frontend.
+ * #ifdef it away so the frontend doesn't try to link against
+ * backend functionality.
+ */
+#ifndef FRONTEND
return psprintf(_("Unicode escape value could not be translated to the server's encoding %s."),
GetDatabaseEncodingName());
+#else
+ Assert(false);
+ break;
+#endif
case JSON_UNICODE_HIGH_SURROGATE:
return _("Unicode high surrogate must not follow a high surrogate.");
case JSON_UNICODE_LOW_SURROGATE:
@@ -1231,12 +1345,22 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
break;
}
- /*
- * We don't use a default: case, so that the compiler will warn about
- * unhandled enum values. But this needs to be here anyway to cover the
- * possibility of an incorrect input.
- */
- elog(ERROR, "unexpected json parse error type: %d", (int) error);
- return NULL;
-}
+ /* Note that lex->errormsg can be NULL in FRONTEND code. */
+ if (lex->errormsg && !lex->errormsg->data[0])
+ {
+ /*
+ * We don't use a default: case, so that the compiler will warn about
+ * unhandled enum values. But this needs to be here anyway to cover
+ * the possibility of an incorrect input.
+ */
+ appendStrVal(lex->errormsg,
+ "unexpected json parse error type: %d", (int) error);
+ }
+
+#ifdef FRONTEND
+ if (PQExpBufferBroken(lex->errormsg))
+ return _("out of memory while constructing error description");
#endif
+
+ return lex->errormsg->data;
+}
diff --git a/src/common/meson.build b/src/common/meson.build
index 4eb16024cb..5d2c7abaa6 100644
--- a/src/common/meson.build
+++ b/src/common/meson.build
@@ -124,13 +124,18 @@ common_sources_frontend_static += files(
# least cryptohash_openssl.c, hmac_openssl.c depend on it.
# controldata_utils.c depends on wait_event_types_h. That's arguably a
# layering violation, but ...
+#
+# XXX Frontend builds need libpq's pqexpbuffer.h, so adjust the include paths
+# appropriately. This seems completely broken.
pgcommon = {}
pgcommon_variants = {
'_srv': internal_lib_args + {
+ 'include_directories': include_directories('.'),
'sources': common_sources + [lwlocknames_h] + [wait_event_types_h],
'dependencies': [backend_common_code],
},
'': default_lib_args + {
+ 'include_directories': include_directories('../interfaces/libpq', '.'),
'sources': common_sources_frontend_static,
'dependencies': [frontend_common_code],
# Files in libpgcommon.a should use/export the "xxx_private" versions
@@ -139,6 +144,7 @@ pgcommon_variants = {
},
'_shlib': default_lib_args + {
'pic': true,
+ 'include_directories': include_directories('../interfaces/libpq', '.'),
'sources': common_sources_frontend_shlib,
'dependencies': [frontend_common_code],
},
@@ -156,7 +162,6 @@ foreach name, opts : pgcommon_variants
c_args = opts.get('c_args', []) + common_cflags[cflagname]
cflag_libs += static_library('libpgcommon@0@_@1@'.format(name, cflagname),
c_pch: pch_c_h,
- include_directories: include_directories('.'),
kwargs: opts + {
'sources': sources,
'c_args': c_args,
@@ -169,7 +174,6 @@ foreach name, opts : pgcommon_variants
lib = static_library('libpgcommon@0@'.format(name),
link_with: cflag_libs,
c_pch: pch_c_h,
- include_directories: include_directories('.'),
kwargs: opts + {
'dependencies': opts['dependencies'] + [ssl],
}
diff --git a/src/common/parse_manifest.c b/src/common/parse_manifest.c
index 92a97714f3..62d93989be 100644
--- a/src/common/parse_manifest.c
+++ b/src/common/parse_manifest.c
@@ -147,7 +147,7 @@ json_parse_manifest(JsonManifestParseContext *context, char *buffer,
/* Run the actual JSON parser. */
json_error = pg_parse_json(lex, &sem);
if (json_error != JSON_SUCCESS)
- json_manifest_parse_failure(context, "parsing failed");
+ json_manifest_parse_failure(context, json_errdetail(json_error, lex));
if (parse.state != JM_EXPECT_EOF)
json_manifest_parse_failure(context, "manifest ended unexpectedly");
diff --git a/src/include/common/jsonapi.h b/src/include/common/jsonapi.h
index 02943cdad8..75d444c17a 100644
--- a/src/include/common/jsonapi.h
+++ b/src/include/common/jsonapi.h
@@ -14,8 +14,6 @@
#ifndef JSONAPI_H
#define JSONAPI_H
-#include "lib/stringinfo.h"
-
typedef enum JsonTokenType
{
JSON_TOKEN_INVALID,
@@ -48,6 +46,7 @@ typedef enum JsonParseErrorType
JSON_EXPECTED_OBJECT_NEXT,
JSON_EXPECTED_STRING,
JSON_INVALID_TOKEN,
+ JSON_OUT_OF_MEMORY,
JSON_UNICODE_CODE_POINT_ZERO,
JSON_UNICODE_ESCAPE_FORMAT,
JSON_UNICODE_HIGH_ESCAPE,
@@ -57,6 +56,17 @@ typedef enum JsonParseErrorType
JSON_SEM_ACTION_FAILED, /* error should already be reported */
} JsonParseErrorType;
+/*
+ * Don't depend on the internal type header for strval; if callers need access
+ * then they can include the appropriate header themselves.
+ */
+#ifdef FRONTEND
+#define StrValType PQExpBufferData
+#else
+#define StrValType StringInfoData
+#endif
+
+typedef struct StrValType StrValType;
/*
* All the fields in this structure should be treated as read-only.
@@ -88,7 +98,9 @@ typedef struct JsonLexContext
bits32 flags;
int line_number; /* line number, starting from 1 */
char *line_start; /* where that line starts within input */
- StringInfo strval;
+ bool parse_strval;
+ StrValType *strval; /* only used if parse_strval == true */
+ StrValType *errormsg;
} JsonLexContext;
typedef JsonParseErrorType (*json_struct_action) (void *state);
--
2.34.1
v17-0002-Refactor-SASL-exchange-to-return-tri-state-statu.patch (application/octet-stream, 9.9K) download
From d8b567dd550aff4dda20174d64cd769c697861db Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <dgustafsson@postgresql.org>
Date: Fri, 23 Feb 2024 11:09:54 +0100
Subject: [PATCH v17 2/9] Refactor SASL exchange to return tri-state status
The SASL exchange callback returned state in to output variables:
done and success. This refactors that logic by introducing a new
return variable of type SASLStatus which makes the code easier to
read and understand, and prepares for future SASL exchanges which
operate asynchronously.
This was extracted from a larger patchset to introduce OAuthBearer
authentication and authorization.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://postgr.es/m/d1b467a78e0e36ed85a09adf979d04cf124a9d4b.camel@vmware.com
---
src/interfaces/libpq/fe-auth-sasl.h | 31 +++++++----
src/interfaces/libpq/fe-auth-scram.c | 78 +++++++++++++---------------
src/interfaces/libpq/fe-auth.c | 28 +++++-----
src/tools/pgindent/typedefs.list | 1 +
4 files changed, 71 insertions(+), 67 deletions(-)
diff --git a/src/interfaces/libpq/fe-auth-sasl.h b/src/interfaces/libpq/fe-auth-sasl.h
index ee5d1525b5..dce16b7b8b 100644
--- a/src/interfaces/libpq/fe-auth-sasl.h
+++ b/src/interfaces/libpq/fe-auth-sasl.h
@@ -21,6 +21,17 @@
#include "libpq-fe.h"
+/*
+ * Possible states for the SASL exchange, see the comment on exchange for an
+ * explanation of these.
+ */
+typedef enum
+{
+ SASL_COMPLETE = 0,
+ SASL_FAILED,
+ SASL_CONTINUE,
+} SASLStatus;
+
/*
* Frontend SASL mechanism callbacks.
*
@@ -59,7 +70,8 @@ typedef struct pg_fe_sasl_mech
* Produces a client response to a server challenge. As a special case
* for client-first SASL mechanisms, exchange() is called with a NULL
* server response once at the start of the authentication exchange to
- * generate an initial response.
+ * generate an initial response. Returns a SASLStatus indicating the
+ * state and status of the exchange.
*
* Input parameters:
*
@@ -79,22 +91,23 @@ typedef struct pg_fe_sasl_mech
*
* output: A malloc'd buffer containing the client's response to
* the server (can be empty), or NULL if the exchange should
- * be aborted. (*success should be set to false in the
+ * be aborted. (The callback should return SASL_FAILED in the
* latter case.)
*
* outputlen: The length (0 or higher) of the client response buffer,
* ignored if output is NULL.
*
- * done: Set to true if the SASL exchange should not continue,
- * because the exchange is either complete or failed
+ * Return value:
*
- * success: Set to true if the SASL exchange completed successfully.
- * Ignored if *done is false.
+ * SASL_CONTINUE: The output buffer is filled with a client response.
+ * Additional server challenge is expected
+ * SASL_COMPLETE: The SASL exchange has completed successfully.
+ * SASL_FAILED: The exchance has failed and the connection should be
+ * dropped.
*--------
*/
- void (*exchange) (void *state, char *input, int inputlen,
- char **output, int *outputlen,
- bool *done, bool *success);
+ SASLStatus (*exchange) (void *state, char *input, int inputlen,
+ char **output, int *outputlen);
/*--------
* channel_bound()
diff --git a/src/interfaces/libpq/fe-auth-scram.c b/src/interfaces/libpq/fe-auth-scram.c
index 04f0e5713d..0bb820e0d9 100644
--- a/src/interfaces/libpq/fe-auth-scram.c
+++ b/src/interfaces/libpq/fe-auth-scram.c
@@ -24,9 +24,8 @@
/* The exported SCRAM callback mechanism. */
static void *scram_init(PGconn *conn, const char *password,
const char *sasl_mechanism);
-static void scram_exchange(void *opaq, char *input, int inputlen,
- char **output, int *outputlen,
- bool *done, bool *success);
+static SASLStatus scram_exchange(void *opaq, char *input, int inputlen,
+ char **output, int *outputlen);
static bool scram_channel_bound(void *opaq);
static void scram_free(void *opaq);
@@ -202,17 +201,14 @@ scram_free(void *opaq)
/*
* Exchange a SCRAM message with backend.
*/
-static void
+static SASLStatus
scram_exchange(void *opaq, char *input, int inputlen,
- char **output, int *outputlen,
- bool *done, bool *success)
+ char **output, int *outputlen)
{
fe_scram_state *state = (fe_scram_state *) opaq;
PGconn *conn = state->conn;
const char *errstr = NULL;
- *done = false;
- *success = false;
*output = NULL;
*outputlen = 0;
@@ -225,12 +221,12 @@ scram_exchange(void *opaq, char *input, int inputlen,
if (inputlen == 0)
{
libpq_append_conn_error(conn, "malformed SCRAM message (empty message)");
- goto error;
+ return SASL_FAILED;
}
if (inputlen != strlen(input))
{
libpq_append_conn_error(conn, "malformed SCRAM message (length mismatch)");
- goto error;
+ return SASL_FAILED;
}
}
@@ -240,61 +236,59 @@ scram_exchange(void *opaq, char *input, int inputlen,
/* Begin the SCRAM handshake, by sending client nonce */
*output = build_client_first_message(state);
if (*output == NULL)
- goto error;
+ return SASL_FAILED;
*outputlen = strlen(*output);
- *done = false;
state->state = FE_SCRAM_NONCE_SENT;
- break;
+ return SASL_CONTINUE;
case FE_SCRAM_NONCE_SENT:
/* Receive salt and server nonce, send response. */
if (!read_server_first_message(state, input))
- goto error;
+ return SASL_FAILED;
*output = build_client_final_message(state);
if (*output == NULL)
- goto error;
+ return SASL_FAILED;
*outputlen = strlen(*output);
- *done = false;
state->state = FE_SCRAM_PROOF_SENT;
- break;
+ return SASL_CONTINUE;
case FE_SCRAM_PROOF_SENT:
- /* Receive server signature */
- if (!read_server_final_message(state, input))
- goto error;
-
- /*
- * Verify server signature, to make sure we're talking to the
- * genuine server.
- */
- if (!verify_server_signature(state, success, &errstr))
- {
- libpq_append_conn_error(conn, "could not verify server signature: %s", errstr);
- goto error;
- }
-
- if (!*success)
{
- libpq_append_conn_error(conn, "incorrect server signature");
+ bool match;
+
+ /* Receive server signature */
+ if (!read_server_final_message(state, input))
+ return SASL_FAILED;
+
+ /*
+ * Verify server signature, to make sure we're talking to the
+ * genuine server.
+ */
+ if (!verify_server_signature(state, &match, &errstr))
+ {
+ libpq_append_conn_error(conn, "could not verify server signature: %s", errstr);
+ return SASL_FAILED;
+ }
+
+ if (!match)
+ {
+ libpq_append_conn_error(conn, "incorrect server signature");
+ }
+ state->state = FE_SCRAM_FINISHED;
+ state->conn->client_finished_auth = true;
+ return match ? SASL_COMPLETE : SASL_FAILED;
}
- *done = true;
- state->state = FE_SCRAM_FINISHED;
- state->conn->client_finished_auth = true;
- break;
default:
/* shouldn't happen */
libpq_append_conn_error(conn, "invalid SCRAM exchange state");
- goto error;
+ break;
}
- return;
-error:
- *done = true;
- *success = false;
+ return SASL_FAILED;
}
/*
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index 1a8e4f6fbf..71dd096605 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -423,11 +423,10 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
char *initialresponse = NULL;
int initialresponselen;
- bool done;
- bool success;
const char *selected_mechanism;
PQExpBufferData mechanism_buf;
char *password;
+ SASLStatus status;
initPQExpBuffer(&mechanism_buf);
@@ -575,12 +574,11 @@ pg_SASL_init(PGconn *conn, int payloadlen)
goto oom_error;
/* Get the mechanism-specific Initial Client Response, if any */
- conn->sasl->exchange(conn->sasl_state,
- NULL, -1,
- &initialresponse, &initialresponselen,
- &done, &success);
+ status = conn->sasl->exchange(conn->sasl_state,
+ NULL, -1,
+ &initialresponse, &initialresponselen);
- if (done && !success)
+ if (status == SASL_FAILED)
goto error;
/*
@@ -629,10 +627,9 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
{
char *output;
int outputlen;
- bool done;
- bool success;
int res;
char *challenge;
+ SASLStatus status;
/* Read the SASL challenge from the AuthenticationSASLContinue message. */
challenge = malloc(payloadlen + 1);
@@ -651,13 +648,12 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
/* For safety and convenience, ensure the buffer is NULL-terminated. */
challenge[payloadlen] = '\0';
- conn->sasl->exchange(conn->sasl_state,
- challenge, payloadlen,
- &output, &outputlen,
- &done, &success);
+ status = conn->sasl->exchange(conn->sasl_state,
+ challenge, payloadlen,
+ &output, &outputlen);
free(challenge); /* don't need the input anymore */
- if (final && !done)
+ if (final && !(status == SASL_FAILED || status == SASL_COMPLETE))
{
if (outputlen != 0)
free(output);
@@ -670,7 +666,7 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
* If the exchange is not completed yet, we need to make sure that the
* SASL mechanism has generated a message to send back.
*/
- if (output == NULL && !done)
+ if (output == NULL && status == SASL_CONTINUE)
{
libpq_append_conn_error(conn, "no client response found after SASL exchange success");
return STATUS_ERROR;
@@ -692,7 +688,7 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
return STATUS_ERROR;
}
- if (done && !success)
+ if (status == SASL_FAILED)
return STATUS_ERROR;
return STATUS_OK;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index fc8b15d0cf..2461567026 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2423,6 +2423,7 @@ RuleLock
RuleStmt
RunningTransactions
RunningTransactionsData
+SASLStatus
SC_HANDLE
SECURITY_ATTRIBUTES
SECURITY_STATUS
--
2.34.1
v17-0006-Introduce-OAuth-validator-libraries.patch (application/octet-stream, 32.0K) download
From 0661817808b3b440c5fd2cbc13e47595514924ee Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Wed, 21 Feb 2024 17:04:26 +0100
Subject: [PATCH v17 6/9] Introduce OAuth validator libraries
This replaces the serverside validation code with an module API
for loading in extensions for validating bearer tokens. A lot of
code is left to be written.
Co-authored-by: Jacob Champion <jacob.champion@enterprisedb.com>
---
src/backend/libpq/auth-oauth.c | 431 +++++-------------
src/backend/utils/misc/guc_tables.c | 6 +-
src/bin/pg_combinebackup/Makefile | 2 +-
src/common/Makefile | 2 +-
src/include/libpq/oauth.h | 29 +-
src/test/modules/meson.build | 1 +
src/test/modules/oauth_validator/.gitignore | 4 +
src/test/modules/oauth_validator/Makefile | 19 +
.../oauth_validator/expected/validator.out | 6 +
src/test/modules/oauth_validator/meson.build | 33 ++
.../modules/oauth_validator/sql/validator.sql | 1 +
.../modules/oauth_validator/t/001_server.pl | 53 +++
src/test/modules/oauth_validator/validator.c | 71 +++
src/test/perl/PostgreSQL/Test/Cluster.pm | 14 +-
src/test/perl/PostgreSQL/Test/OAuthServer.pm | 158 +++++++
src/tools/pgindent/typedefs.list | 2 +
16 files changed, 500 insertions(+), 332 deletions(-)
create mode 100644 src/test/modules/oauth_validator/.gitignore
create mode 100644 src/test/modules/oauth_validator/Makefile
create mode 100644 src/test/modules/oauth_validator/expected/validator.out
create mode 100644 src/test/modules/oauth_validator/meson.build
create mode 100644 src/test/modules/oauth_validator/sql/validator.sql
create mode 100644 src/test/modules/oauth_validator/t/001_server.pl
create mode 100644 src/test/modules/oauth_validator/validator.c
create mode 100644 src/test/perl/PostgreSQL/Test/OAuthServer.pm
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
index e74c008161..765a18b9b2 100644
--- a/src/backend/libpq/auth-oauth.c
+++ b/src/backend/libpq/auth-oauth.c
@@ -6,7 +6,7 @@
* See the following RFC for more details:
* - RFC 7628: https://tools.ietf.org/html/rfc7628
*
- * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* src/backend/libpq/auth-oauth.c
@@ -19,21 +19,29 @@
#include <fcntl.h>
#include "common/oauth-common.h"
+#include "fmgr.h"
#include "lib/stringinfo.h"
#include "libpq/auth.h"
#include "libpq/hba.h"
#include "libpq/oauth.h"
#include "libpq/sasl.h"
#include "storage/fd.h"
+#include "storage/ipc.h"
/* GUC */
-char *oauth_validator_command;
+char *OAuthValidatorLibrary = "";
static void oauth_get_mechanisms(Port *port, StringInfo buf);
static void *oauth_init(Port *port, const char *selected_mech, const char *shadow_pass);
static int oauth_exchange(void *opaq, const char *input, int inputlen,
char **output, int *outputlen, const char **logdetail);
+static void load_validator_library(void);
+static void shutdown_validator_library(int code, Datum arg);
+
+static ValidatorModuleState *validator_module_state;
+static const OAuthValidatorCallbacks *ValidatorCallbacks;
+
/* Mechanism declaration */
const pg_be_sasl_mech pg_be_oauth_mech = {
oauth_get_mechanisms,
@@ -62,11 +70,7 @@ struct oauth_ctx
static char *sanitize_char(char c);
static char *parse_kvpairs_for_auth(char **input);
static void generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen);
-static bool validate(Port *port, const char *auth, const char **logdetail);
-static bool run_validator_command(Port *port, const char *token);
-static bool check_exit(FILE **fh, const char *command);
-static bool set_cloexec(int fd);
-static bool username_ok_for_shell(const char *username);
+static bool validate(Port *port, const char *auth);
#define KVSEP 0x01
#define AUTH_KEY "auth"
@@ -99,6 +103,8 @@ oauth_init(Port *port, const char *selected_mech, const char *shadow_pass)
ctx->issuer = port->hba->oauth_issuer;
ctx->scope = port->hba->oauth_scope;
+ load_validator_library();
+
return ctx;
}
@@ -249,7 +255,7 @@ oauth_exchange(void *opaq, const char *input, int inputlen,
errmsg("malformed OAUTHBEARER message"),
errdetail("Message contains additional data after the final terminator.")));
- if (!validate(ctx->port, auth, logdetail))
+ if (!validate(ctx->port, auth))
{
generate_error_response(ctx, output, outputlen);
@@ -416,70 +422,73 @@ generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
*outputlen = buf.len;
}
-static bool
-validate(Port *port, const char *auth, const char **logdetail)
+/*-----
+ * Validates the provided Authorization header and returns the token from
+ * within it. NULL is returned on validation failure.
+ *
+ * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
+ * 2.1:
+ *
+ * b64token = 1*( ALPHA / DIGIT /
+ * "-" / "." / "_" / "~" / "+" / "/" ) *"="
+ * credentials = "Bearer" 1*SP b64token
+ *
+ * The "credentials" construction is what we receive in our auth value.
+ *
+ * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
+ * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
+ * compared case-insensitively. (This is not mentioned in RFC 6750, but
+ * it's pointed out in RFC 7628 Sec. 4.)
+ *
+ * Invalid formats are technically a protocol violation, but we shouldn't
+ * reflect any information about the sensitive Bearer token back to the
+ * client; log at COMMERROR instead.
+ *
+ * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
+ */
+static const char *
+validate_token_format(const char *header)
{
- static const char *const b64_set =
+ size_t span;
+ const char *token;
+ static const char *const b64token_allowed_set =
"abcdefghijklmnopqrstuvwxyz"
"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
"0123456789-._~+/";
- const char *token;
- size_t span;
- int ret;
+ /* If the token is empty or simply too short to be correct */
+ if (!header || strlen(header) <= 7)
+ {
+ ereport(COMMERROR,
+ (errmsg("malformed OAuth bearer token 1")));
+ return NULL;
+ }
- /* TODO: handle logdetail when the test framework can check it */
-
- /*-----
- * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
- * 2.1:
- *
- * b64token = 1*( ALPHA / DIGIT /
- * "-" / "." / "_" / "~" / "+" / "/" ) *"="
- * credentials = "Bearer" 1*SP b64token
- *
- * The "credentials" construction is what we receive in our auth value.
- *
- * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
- * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
- * compared case-insensitively. (This is not mentioned in RFC 6750, but
- * it's pointed out in RFC 7628 Sec. 4.)
- *
- * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
- */
- if (pg_strncasecmp(auth, BEARER_SCHEME, strlen(BEARER_SCHEME)))
- return false;
+ if (pg_strncasecmp(header, BEARER_SCHEME, strlen(BEARER_SCHEME)))
+ return NULL;
/* Pull the bearer token out of the auth value. */
- token = auth + strlen(BEARER_SCHEME);
+ token = header + strlen(BEARER_SCHEME);
/* Swallow any additional spaces. */
while (*token == ' ')
token++;
- /*
- * Before invoking the validator command, sanity-check the token format to
- * avoid any injection attacks later in the chain. Invalid formats are
- * technically a protocol violation, but don't reflect any information
- * about the sensitive Bearer token back to the client; log at COMMERROR
- * instead.
- */
-
/* Tokens must not be empty. */
if (!*token)
{
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("malformed OAUTHBEARER message"),
+ errmsg("malformed OAuth bearer token 2"),
errdetail("Bearer token is empty.")));
- return false;
+ return NULL;
}
/*
* Make sure the token contains only allowed characters. Tokens may end
* with any number of '=' characters.
*/
- span = strspn(token, b64_set);
+ span = strspn(token, b64token_allowed_set);
while (token[span] == '=')
span++;
@@ -492,15 +501,35 @@ validate(Port *port, const char *auth, const char **logdetail)
*/
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("malformed OAUTHBEARER message"),
+ errmsg("malformed OAuth bearer token 3"),
errdetail("Bearer token is not in the correct format.")));
- return false;
+ return NULL;
}
- /* Have the validator check the token. */
- if (!run_validator_command(port, token))
+ return token;
+}
+
+static bool
+validate(Port *port, const char *auth)
+{
+ int map_status;
+ ValidatorModuleResult *ret;
+ const char *token;
+
+ /* Ensure that we have a correct token to validate */
+ if (!(token = validate_token_format(auth)))
return false;
+ /* Call the validation function from the validator module */
+ ret = ValidatorCallbacks->validate_cb(validator_module_state,
+ token, port->user_name);
+
+ if (!ret->authenticated)
+ return false;
+
+ if (ret->authn_id)
+ set_authn_id(port, ret->authn_id);
+
if (port->hba->oauth_skip_usermap)
{
/*
@@ -513,7 +542,7 @@ validate(Port *port, const char *auth, const char **logdetail)
}
/* Make sure the validator authenticated the user. */
- if (!MyClientConnectionInfo.authn_id)
+ if (ret->authn_id == NULL || ret->authn_id[0] == '\0')
{
/* TODO: use logdetail; reduce message duplication */
ereport(LOG,
@@ -523,288 +552,42 @@ validate(Port *port, const char *auth, const char **logdetail)
}
/* Finally, check the user map. */
- ret = check_usermap(port->hba->usermap, port->user_name,
- MyClientConnectionInfo.authn_id, false);
- return (ret == STATUS_OK);
+ map_status = check_usermap(port->hba->usermap, port->user_name,
+ MyClientConnectionInfo.authn_id, false);
+ return (map_status == STATUS_OK);
}
-static bool
-run_validator_command(Port *port, const char *token)
-{
- bool success = false;
- int rc;
- int pipefd[2];
- int rfd = -1;
- int wfd = -1;
-
- StringInfoData command = {0};
- char *p;
- FILE *fh = NULL;
-
- ssize_t written;
- char *line = NULL;
- size_t size = 0;
- ssize_t len;
-
- Assert(oauth_validator_command);
-
- if (!oauth_validator_command[0])
- {
- ereport(COMMERROR,
- (errmsg("oauth_validator_command is not set"),
- errhint("To allow OAuth authenticated connections, set "
- "oauth_validator_command in postgresql.conf.")));
- return false;
- }
-
- /*------
- * Since popen() is unidirectional, open up a pipe for the other
- * direction. Use CLOEXEC to ensure that our write end doesn't
- * accidentally get copied into child processes, which would prevent us
- * from closing it cleanly.
- *
- * XXX this is ugly. We should just read from the child process's stdout,
- * but that's a lot more code.
- * XXX by bypassing the popen API, we open the potential of process
- * deadlock. Clearly document child process requirements (i.e. the child
- * MUST read all data off of the pipe before writing anything).
- * TODO: port to Windows using _pipe().
- */
- rc = pipe(pipefd);
- if (rc < 0)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not create child pipe: %m")));
- return false;
- }
-
- rfd = pipefd[0];
- wfd = pipefd[1];
-
- if (!set_cloexec(wfd))
- {
- /* error message was already logged */
- goto cleanup;
- }
-
- /*----------
- * Construct the command, substituting any recognized %-specifiers:
- *
- * %f: the file descriptor of the input pipe
- * %r: the role that the client wants to assume (port->user_name)
- * %%: a literal '%'
- */
- initStringInfo(&command);
-
- for (p = oauth_validator_command; *p; p++)
- {
- if (p[0] == '%')
- {
- switch (p[1])
- {
- case 'f':
- appendStringInfo(&command, "%d", rfd);
- p++;
- break;
- case 'r':
-
- /*
- * TODO: decide how this string should be escaped. The
- * role is controlled by the client, so if we don't escape
- * it, command injections are inevitable.
- *
- * This is probably an indication that the role name needs
- * to be communicated to the validator process in some
- * other way. For this proof of concept, just be
- * incredibly strict about the characters that are allowed
- * in user names.
- */
- if (!username_ok_for_shell(port->user_name))
- goto cleanup;
-
- appendStringInfoString(&command, port->user_name);
- p++;
- break;
- case '%':
- appendStringInfoChar(&command, '%');
- p++;
- break;
- default:
- appendStringInfoChar(&command, p[0]);
- }
- }
- else
- appendStringInfoChar(&command, p[0]);
- }
-
- /* Execute the command. */
- fh = OpenPipeStream(command.data, "r");
- if (!fh)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("opening pipe to OAuth validator: %m")));
- goto cleanup;
- }
-
- /* We don't need the read end of the pipe anymore. */
- close(rfd);
- rfd = -1;
-
- /* Give the command the token to validate. */
- written = write(wfd, token, strlen(token));
- if (written != strlen(token))
- {
- /* TODO must loop for short writes, EINTR et al */
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not write token to child pipe: %m")));
- goto cleanup;
- }
-
- close(wfd);
- wfd = -1;
-
- /*-----
- * Read the command's response.
- *
- * TODO: getline() is probably too new to use, unfortunately.
- * TODO: loop over all lines
- */
- if ((len = getline(&line, &size, fh)) >= 0)
- {
- /* TODO: fail if the authn_id doesn't end with a newline */
- if (len > 0)
- line[len - 1] = '\0';
-
- set_authn_id(port, line);
- }
- else if (ferror(fh))
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not read from command \"%s\": %m",
- command.data)));
- goto cleanup;
- }
-
- /* Make sure the command exits cleanly. */
- if (!check_exit(&fh, command.data))
- {
- /* error message already logged */
- goto cleanup;
- }
-
- /* Done. */
- success = true;
-
-cleanup:
- if (line)
- free(line);
-
- /*
- * In the successful case, the pipe fds are already closed. For the error
- * case, always close out the pipe before waiting for the command, to
- * prevent deadlock.
- */
- if (rfd >= 0)
- close(rfd);
- if (wfd >= 0)
- close(wfd);
-
- if (fh)
- {
- Assert(!success);
- check_exit(&fh, command.data);
- }
-
- if (command.data)
- pfree(command.data);
-
- return success;
-}
-
-static bool
-check_exit(FILE **fh, const char *command)
+static void
+load_validator_library(void)
{
- int rc;
+ OAuthValidatorModuleInit validator_init;
- rc = ClosePipeStream(*fh);
- *fh = NULL;
-
- if (rc == -1)
- {
- /* pclose() itself failed. */
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not close pipe to command \"%s\": %m",
- command)));
- }
- else if (rc != 0)
- {
- char *reason = wait_result_to_str(rc);
-
- ereport(COMMERROR,
- (errmsg("failed to execute command \"%s\": %s",
- command, reason)));
-
- pfree(reason);
- }
+ if (OAuthValidatorLibrary[0] == '\0')
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("oauth_validator_library is not set")));
- return (rc == 0);
-}
+ validator_init = (OAuthValidatorModuleInit)
+ load_external_function(OAuthValidatorLibrary,
+ "_PG_oauth_validator_module_init", false, NULL);
-static bool
-set_cloexec(int fd)
-{
- int flags;
- int rc;
+ if (validator_init == NULL)
+ ereport(ERROR,
+ (errmsg("%s module \"%s\" have to define the symbol %s",
+ "OAuth validator", OAuthValidatorLibrary, "_PG_oauth_validator_module_init")));
- flags = fcntl(fd, F_GETFD);
- if (flags == -1)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not get fd flags for child pipe: %m")));
- return false;
- }
+ ValidatorCallbacks = (*validator_init) ();
- rc = fcntl(fd, F_SETFD, flags | FD_CLOEXEC);
- if (rc < 0)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not set FD_CLOEXEC for child pipe: %m")));
- return false;
- }
+ validator_module_state = (ValidatorModuleState *) palloc0(sizeof(ValidatorModuleState));
+ if (ValidatorCallbacks->startup_cb != NULL)
+ ValidatorCallbacks->startup_cb(validator_module_state);
- return true;
+ before_shmem_exit(shutdown_validator_library, 0);
}
-/*
- * XXX This should go away eventually and be replaced with either a proper
- * escape or a different strategy for communication with the validator command.
- */
-static bool
-username_ok_for_shell(const char *username)
+static void
+shutdown_validator_library(int code, Datum arg)
{
- /* This set is borrowed from fe_utils' appendShellStringNoError(). */
- static const char *const allowed =
- "abcdefghijklmnopqrstuvwxyz"
- "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
- "0123456789-_./:";
- size_t span;
-
- Assert(username && username[0]); /* should have already been checked */
-
- span = strspn(username, allowed);
- if (username[span] != '\0')
- {
- ereport(COMMERROR,
- (errmsg("PostgreSQL user name contains unsafe characters and cannot be passed to the OAuth validator")));
- return false;
- }
-
- return true;
+ if (ValidatorCallbacks->shutdown_cb != NULL)
+ ValidatorCallbacks->shutdown_cb(validator_module_state);
}
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 8c1b90e310..2928047a70 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -4641,12 +4641,12 @@ struct config_string ConfigureNamesString[] =
},
{
- {"oauth_validator_command", PGC_SIGHUP, CONN_AUTH_AUTH,
- gettext_noop("Command to validate OAuth v2 bearer tokens."),
+ {"oauth_validator_library", PGC_SIGHUP, CONN_AUTH_AUTH,
+ gettext_noop("Sets the library that will be called to validate OAuth v2 bearer tokens."),
NULL,
GUC_SUPERUSER_ONLY | GUC_NOT_IN_SAMPLE
},
- &oauth_validator_command,
+ &OAuthValidatorLibrary,
"",
NULL, NULL, NULL
},
diff --git a/src/bin/pg_combinebackup/Makefile b/src/bin/pg_combinebackup/Makefile
index c3729755ba..4f24b1aff6 100644
--- a/src/bin/pg_combinebackup/Makefile
+++ b/src/bin/pg_combinebackup/Makefile
@@ -31,7 +31,7 @@ OBJS = \
all: pg_combinebackup
pg_combinebackup: $(OBJS) | submake-libpgport submake-libpgfeutils
- $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(libpq_pgport) $(LIBS) -o $@$(X)
install: all installdirs
$(INSTALL_PROGRAM) pg_combinebackup$(X) '$(DESTDIR)$(bindir)/pg_combinebackup$(X)'
diff --git a/src/common/Makefile b/src/common/Makefile
index bbb5c3ab11..00e30e6bfe 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -41,7 +41,7 @@ override CPPFLAGS += -DVAL_LDFLAGS_SL="\"$(LDFLAGS_SL)\""
override CPPFLAGS += -DVAL_LIBS="\"$(LIBS)\""
override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common -I$(libpq_srcdir) $(CPPFLAGS)
-LIBS += $(PTHREAD_LIBS)
+LIBS += $(PTHREAD_LIBS) $(libpq_pgport)
OBJS_COMMON = \
archive.o \
diff --git a/src/include/libpq/oauth.h b/src/include/libpq/oauth.h
index 5edab3b25a..5c081abfae 100644
--- a/src/include/libpq/oauth.h
+++ b/src/include/libpq/oauth.h
@@ -3,7 +3,7 @@
* oauth.h
* Interface to libpq/auth-oauth.c
*
- * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* src/include/libpq/oauth.h
@@ -16,7 +16,32 @@
#include "libpq/libpq-be.h"
#include "libpq/sasl.h"
-extern char *oauth_validator_command;
+extern PGDLLIMPORT char *OAuthValidatorLibrary;
+
+typedef struct ValidatorModuleState
+{
+ void *private_data;
+} ValidatorModuleState;
+
+typedef struct ValidatorModuleResult
+{
+ bool authenticated;
+ char *authn_id;
+} ValidatorModuleResult;
+
+typedef void (*ValidatorStartupCB) (ValidatorModuleState *state);
+typedef void (*ValidatorShutdownCB) (ValidatorModuleState *state);
+typedef ValidatorModuleResult *(*ValidatorValidateCB) (ValidatorModuleState *state, const char *token, const char *role);
+
+typedef struct OAuthValidatorCallbacks
+{
+ ValidatorStartupCB startup_cb;
+ ValidatorShutdownCB shutdown_cb;
+ ValidatorValidateCB validate_cb;
+} OAuthValidatorCallbacks;
+
+typedef const OAuthValidatorCallbacks *(*OAuthValidatorModuleInit) (void);
+extern PGDLLEXPORT const OAuthValidatorCallbacks *_PG_oauth_validator_module_init(void);
/* Implementation */
extern const pg_be_sasl_mech pg_be_oauth_mech;
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index 8fbe742d38..dc54ce7189 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -9,6 +9,7 @@ subdir('gin')
subdir('injection_points')
subdir('ldap_password_func')
subdir('libpq_pipeline')
+subdir('oauth_validator')
subdir('plsample')
subdir('spgist_name_ops')
subdir('ssl_passphrase_callback')
diff --git a/src/test/modules/oauth_validator/.gitignore b/src/test/modules/oauth_validator/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/src/test/modules/oauth_validator/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/oauth_validator/Makefile b/src/test/modules/oauth_validator/Makefile
new file mode 100644
index 0000000000..1f874cd7f2
--- /dev/null
+++ b/src/test/modules/oauth_validator/Makefile
@@ -0,0 +1,19 @@
+MODULES = validator
+PGFILEDESC = "validator - test OAuth validator module"
+
+NO_INSTALLCHECK = 1
+
+TAP_TESTS = 1
+
+REGRESS = validator
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/oauth_validator
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/oauth_validator/expected/validator.out b/src/test/modules/oauth_validator/expected/validator.out
new file mode 100644
index 0000000000..360caa2cb3
--- /dev/null
+++ b/src/test/modules/oauth_validator/expected/validator.out
@@ -0,0 +1,6 @@
+SELECT 1;
+ ?column?
+----------
+ 1
+(1 row)
+
diff --git a/src/test/modules/oauth_validator/meson.build b/src/test/modules/oauth_validator/meson.build
new file mode 100644
index 0000000000..d9c1d1d577
--- /dev/null
+++ b/src/test/modules/oauth_validator/meson.build
@@ -0,0 +1,33 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+validator_sources = files(
+ 'validator.c',
+)
+
+if host_system == 'windows'
+ validator_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'validator',
+ '--FILEDESC', 'validator - test OAuth validator module',])
+endif
+
+validator = shared_module('validator',
+ validator_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += validator
+
+tests += {
+ 'name': 'oauth_validator',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'regress': {
+ 'sql': [
+ 'validator',
+ ],
+ },
+ 'tap': {
+ 'tests': [
+ 't/001_server.pl',
+ ],
+ },
+}
diff --git a/src/test/modules/oauth_validator/sql/validator.sql b/src/test/modules/oauth_validator/sql/validator.sql
new file mode 100644
index 0000000000..e0ac49d1ec
--- /dev/null
+++ b/src/test/modules/oauth_validator/sql/validator.sql
@@ -0,0 +1 @@
+SELECT 1;
diff --git a/src/test/modules/oauth_validator/t/001_server.pl b/src/test/modules/oauth_validator/t/001_server.pl
new file mode 100644
index 0000000000..49e04b0afe
--- /dev/null
+++ b/src/test/modules/oauth_validator/t/001_server.pl
@@ -0,0 +1,53 @@
+
+# Copyright (c) 2021-2024, PostgreSQL Global Development Group
+
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::OAuthServer;
+use Test::More;
+
+# Delete pg_hba.conf from the given node, add a new entry to it
+# and then execute a reload to refresh it.
+# XXX: this is copied from authentication/t/001_password and should be made
+# generic functionality if we end up using it.
+sub reset_pg_hba
+{
+ my $node = shift;
+ my $database = shift;
+ my $role = shift;
+ my $hba_method = shift;
+
+ unlink($node->data_dir . '/pg_hba.conf');
+ # just for testing purposes, use a continuation line
+ $node->append_conf('pg_hba.conf',
+ "local $database $role\\\n $hba_method");
+ $node->reload;
+ return;
+}
+
+my $node = PostgreSQL::Test::Cluster->new('primary');
+$node->init;
+$node->append_conf('postgresql.conf', "shared_preload_libraries = 'validator'\n");
+$node->append_conf('postgresql.conf', "oauth_validator_library = 'validator'\n");
+$node->start;
+
+reset_pg_hba($node, 'all', 'all', 'oauth issuer="127.0.0.1:18080" scope="openid postgres"');
+
+my $webserver = PostgreSQL::Test::OAuthServer->new(18080);
+
+my $port = $webserver->port();
+
+is($port, 18080, "Port is 18080");
+
+$webserver->setup();
+$webserver->run();
+
+$node->connect_ok("dbname=postgres oauth_client_id=f02c6361-0635", "connect",
+ expected_stderr => qr@Visit https://example\.com/ and enter the code: postgresuser@);
+
+$node->stop;
+
+done_testing();
diff --git a/src/test/modules/oauth_validator/validator.c b/src/test/modules/oauth_validator/validator.c
new file mode 100644
index 0000000000..c76d0599c5
--- /dev/null
+++ b/src/test/modules/oauth_validator/validator.c
@@ -0,0 +1,71 @@
+/*-------------------------------------------------------------------------
+ *
+ * validator.c
+ * Test module for serverside OAuth token validation callbacks
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/test/modules/oauth_validator/validator.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "libpq/oauth.h"
+#include "utils/memutils.h"
+
+PG_MODULE_MAGIC;
+
+static void validator_startup(ValidatorModuleState *state);
+static void validator_shutdown(ValidatorModuleState *state);
+static ValidatorModuleResult * validate_token(ValidatorModuleState *state,
+ const char *token,
+ const char *role);
+
+static const OAuthValidatorCallbacks validator_callbacks = {
+ .startup_cb = validator_startup,
+ .shutdown_cb = validator_shutdown,
+ .validate_cb = validate_token
+};
+
+void
+_PG_init(void)
+{
+ /* no-op */
+}
+
+const OAuthValidatorCallbacks *
+_PG_oauth_validator_module_init(void)
+{
+ return &validator_callbacks;
+}
+
+static void
+validator_startup(ValidatorModuleState *state)
+{
+ /* do nothing */
+}
+
+static void
+validator_shutdown(ValidatorModuleState *state)
+{
+ /* do nothing */
+}
+
+static ValidatorModuleResult *
+validate_token(ValidatorModuleState *state, const char *token, const char *role)
+{
+ ValidatorModuleResult *res;
+
+ res = palloc(sizeof(ValidatorModuleResult));
+
+ elog(LOG, "XXX: validating %s for %s", token, role);
+
+ res->authenticated = true;
+ res->authn_id = pstrdup(role);
+
+ return res;
+}
diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm
index 44c1bb5afd..b758ad01cc 100644
--- a/src/test/perl/PostgreSQL/Test/Cluster.pm
+++ b/src/test/perl/PostgreSQL/Test/Cluster.pm
@@ -2302,6 +2302,11 @@ instead of the default.
If this regular expression is set, matches it with the output generated.
+=item expected_stderr => B<value>
+
+If this regular expression is set, matches it against the standard error
+stream; otherwise the stderr must be empty.
+
=item log_like => [ qr/required message/ ]
=item log_unlike => [ qr/prohibited message/ ]
@@ -2345,7 +2350,14 @@ sub connect_ok
like($stdout, $params{expected_stdout}, "$test_name: stdout matches");
}
- is($stderr, "", "$test_name: no stderr");
+ if (defined($params{expected_stderr}))
+ {
+ like($stderr, $params{expected_stderr}, "$test_name: stderr matches");
+ }
+ else
+ {
+ is($stderr, "", "$test_name: no stderr");
+ }
$self->log_check($test_name, $log_location, %params);
}
diff --git a/src/test/perl/PostgreSQL/Test/OAuthServer.pm b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
new file mode 100644
index 0000000000..3ac90c3d0f
--- /dev/null
+++ b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
@@ -0,0 +1,158 @@
+#!/usr/bin/perl
+
+package PostgreSQL::Test::OAuthServer;
+
+use warnings;
+use strict;
+use threads;
+use Socket;
+use IO::Select;
+
+local *server_socket;
+
+sub new
+{
+ my $class = shift;
+ my $port = shift;
+
+ my $self = {};
+ bless($self, $class);
+
+ $self->{'port'} = $port;
+
+ return $self;
+}
+
+sub setup
+{
+ my $self = shift;
+ my $tcp = getprotobyname('tcp');
+
+ socket($self->{'socket'}, PF_INET, SOCK_STREAM, $tcp)
+ or die "no socket";
+ setsockopt($self->{'socket'}, SOL_SOCKET, SO_REUSEADDR, pack("l", 1));
+ bind($self->{'socket'}, sockaddr_in($self->{'port'}, INADDR_ANY));
+}
+
+sub port
+{
+ my $self = shift;
+
+ return $self->{'port'};
+}
+
+sub run
+{
+ my $self = shift;
+
+ my $server_thread = threads->create(\&_listen, $self);
+ $server_thread->detach();
+}
+
+sub _listen
+{
+ my $self = shift;
+
+ listen($self->{'socket'}, SOMAXCONN) or die "fail to listen: $!";
+
+ while (1)
+ {
+ my $fh;
+ my %request;
+ my $remote = accept($fh, $self->{'socket'});
+ binmode $fh;
+
+ my ($method, $object, $prot) = split(/ /, <$fh>);
+ $request{'method'} = $method;
+ $request{'object'} = $object;
+ chomp($request{'object'});
+
+ local $/ = Socket::CRLF;
+ my $c = 0;
+ while(<$fh>)
+ {
+ chomp;
+ # Headers
+ if (/:/)
+ {
+ my ($field, $value) = split(/:/, $_, 2);
+ $value =~ s/^\s+//;
+ $request{'headers'}{lc $field} = $value;
+ }
+ # POST data
+ elsif (/^$/)
+ {
+ read($fh, $request{'content'}, $request{'headers'}{'content-length'})
+ if defined $request{'headers'}{'content-length'};
+ last;
+ }
+ }
+
+ # Debug printing
+ # print ": read ".$request{'method'} . ";" . $request{'object'}.";\n";
+ # foreach my $h (keys(%{$request{'headers'}}))
+ #{
+ # printf ": headers: " . $request{'headers'}{$h} . "\n";
+ #}
+ #printf ": POST: " . $request{'content'} . "\n" if defined $request{'content'};
+
+ if ($request{'object'} eq '/.well-known/openid-configuration')
+ {
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: application/json\r\n";
+ print $fh "\r\n";
+ print $fh <<EOR;
+ {
+ "issuer": "http://localhost:$self->{'port'}",
+ "token_endpoint": "http://localhost:$self->{'port'}/token",
+ "device_authorization_endpoint": "http://localhost:$self->{'port'}/authorize",
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ "grant_types_supported": ["urn:ietf:params:oauth:grant-type:device_code"]
+ }
+EOR
+ }
+ elsif ($request{'object'} eq '/authorize')
+ {
+ print ": returning device_code\n";
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: application/json\r\n";
+ print $fh "\r\n";
+ print $fh <<EOR;
+ {
+ "device_code": "postgres",
+ "user_code" : "postgresuser",
+ "interval" : 0,
+ "verification_uri" : "https://example.com/",
+ "expires-in": 5
+ }
+EOR
+ }
+ elsif ($request{'object'} eq '/token')
+ {
+ print ": returning token\n";
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: application/json\r\n";
+ print $fh "\r\n";
+ print $fh <<EOR;
+ {
+ "access_token": "9243959234",
+ "token_type": "bearer"
+ }
+EOR
+ }
+ else
+ {
+ print ": returning default\n";
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: text/html\r\n";
+ print $fh "\r\n";
+ print $fh "Ok\n";
+ }
+
+ close($fh);
+ }
+}
+
+1;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index fe65a4222b..2bca506e16 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1657,6 +1657,7 @@ NumericSortSupport
NumericSumAccum
NumericVar
OAuthStep
+OAuthValidatorCallbacks
OM_uint32
OP
OSAPerGroupState
@@ -2980,6 +2981,7 @@ VacuumRelation
VacuumStmt
ValidIOData
ValidateIndexState
+ValidatorModuleState
ValuesScan
ValuesScanState
Var
--
2.34.1
v17-0007-Add-pytest-suite-for-OAuth.patch (application/octet-stream, 172.2K) download
From 45755e8461774f49092f79e9e0bd614f68af58ed Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Fri, 4 Jun 2021 09:06:38 -0700
Subject: [PATCH v17 7/9] Add pytest suite for OAuth
Requires Python 3. On the first run of `make installcheck` or `meson
test` the dependencies will be installed into a local virtualenv for
you. See the README for more details.
Cirrus has been updated to build OAuth support on Debian and FreeBSD.
The suite contains a --temp-instance option, analogous to pg_regress's
option of the same name, which allows an ephemeral server to be spun up
during a test run.
For iddawc, asynchronous tests still hang, as expected. Bad-interval
tests fail because iddawc apparently doesn't care that the interval is
bad.
TODOs:
- The --tap-stream option to pytest-tap is slightly broken during test
failures (it suppresses error information), which impedes debugging.
- Unsurprisingly, Windows builds fail on the Linux-/BSD-specific backend
changes. 32-bit builds on Ubuntu fail during testing as well.
- pyca/cryptography is pinned at an old version. Since we use it for
testing and not security, this isn't a critical problem yet, but it's
not ideal. Newer versions require a Rust compiler to build, and while
many platforms have precompiled wheels, some (FreeBSD) do not. Even
with the Rust pieces bypassed, compilation on FreeBSD takes a while.
- The with_oauth test skip logic should probably be integrated into the
Makefile side as well...
---
.cirrus.tasks.yml | 18 +-
meson.build | 93 ++
src/test/meson.build | 1 +
src/test/python/.gitignore | 2 +
src/test/python/Makefile | 38 +
src/test/python/README | 66 +
src/test/python/client/__init__.py | 0
src/test/python/client/conftest.py | 137 ++
src/test/python/client/test_client.py | 180 +++
src/test/python/client/test_oauth.py | 1766 +++++++++++++++++++++++++
src/test/python/conftest.py | 37 +
src/test/python/meson.build | 46 +
src/test/python/pq3.py | 728 ++++++++++
src/test/python/pytest.ini | 4 +
src/test/python/requirements.txt | 9 +
src/test/python/server/__init__.py | 0
src/test/python/server/conftest.py | 141 ++
src/test/python/server/meson.build | 22 +
src/test/python/server/oauthtest.c | 118 ++
src/test/python/server/test_oauth.py | 939 +++++++++++++
src/test/python/server/test_server.py | 21 +
src/test/python/test_internals.py | 138 ++
src/test/python/test_pq3.py | 558 ++++++++
src/test/python/tls.py | 195 +++
src/tools/make_venv | 53 +
25 files changed, 5304 insertions(+), 6 deletions(-)
create mode 100644 src/test/python/.gitignore
create mode 100644 src/test/python/Makefile
create mode 100644 src/test/python/README
create mode 100644 src/test/python/client/__init__.py
create mode 100644 src/test/python/client/conftest.py
create mode 100644 src/test/python/client/test_client.py
create mode 100644 src/test/python/client/test_oauth.py
create mode 100644 src/test/python/conftest.py
create mode 100644 src/test/python/meson.build
create mode 100644 src/test/python/pq3.py
create mode 100644 src/test/python/pytest.ini
create mode 100644 src/test/python/requirements.txt
create mode 100644 src/test/python/server/__init__.py
create mode 100644 src/test/python/server/conftest.py
create mode 100644 src/test/python/server/meson.build
create mode 100644 src/test/python/server/oauthtest.c
create mode 100644 src/test/python/server/test_oauth.py
create mode 100644 src/test/python/server/test_server.py
create mode 100644 src/test/python/test_internals.py
create mode 100644 src/test/python/test_pq3.py
create mode 100644 src/test/python/tls.py
create mode 100755 src/tools/make_venv
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 3b5b54df58..f02da9a2b0 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -20,7 +20,7 @@ env:
MTEST_ARGS: --print-errorlogs --no-rebuild -C build
PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests
TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf
- PG_TEST_EXTRA: kerberos ldap ssl load_balance
+ PG_TEST_EXTRA: kerberos ldap ssl load_balance python
# What files to preserve in case tests fail
@@ -165,7 +165,7 @@ task:
chown root:postgres /tmp/cores
sysctl kern.corefile='/tmp/cores/%N.%P.core'
setup_additional_packages_script: |
- #pkg install -y ...
+ pkg install -y curl
# NB: Intentionally build without -Dllvm. The freebsd image size is already
# large enough to make VM startup slow, and even without llvm freebsd
@@ -177,6 +177,7 @@ task:
-Dcassert=true -Dinjection_points=true \
-Duuid=bsd -Dtcl_version=tcl86 -Ddtrace=auto \
-DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
+ -Doauth=curl \
-Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \
build
EOF
@@ -225,6 +226,7 @@ LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
--with-libxslt
--with-llvm
--with-lz4
+ --with-oauth=curl
--with-pam
--with-perl
--with-python
@@ -237,6 +239,7 @@ LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
LINUX_MESON_FEATURES: &LINUX_MESON_FEATURES >-
-Dllvm=enabled
+ -Doauth=curl
-Duuid=e2fs
@@ -312,8 +315,11 @@ task:
EOF
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install \
+ libcurl4-openssl-dev \
+ libcurl4-openssl-dev:i386 \
+ python3-venv \
matrix:
- name: Linux - Debian Bullseye - Autoconf
@@ -676,8 +682,8 @@ task:
folder: $CCACHE_DIR
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install libcurl4-openssl-dev
###
# Test that code can be built with gcc/clang without warnings
diff --git a/meson.build b/meson.build
index 8c83eac03d..0cf45f1764 100644
--- a/meson.build
+++ b/meson.build
@@ -3194,6 +3194,9 @@ else
endif
testwrap = files('src/tools/testwrap')
+make_venv = files('src/tools/make_venv')
+
+checked_working_venv = false
foreach test_dir : tests
testwrap_base = [
@@ -3355,6 +3358,96 @@ foreach test_dir : tests
)
endforeach
install_suites += test_group
+ elif kind == 'pytest'
+ venv_name = test_dir['name'] + '_venv'
+ venv_path = meson.build_root() / venv_name
+
+ # The Python tests require a working venv module. This is part of the
+ # standard library, but some platforms disable it until a separate package
+ # is installed. Those same platforms don't provide an easy way to check
+ # whether the venv command will work until the first time you try it, so
+ # we decide whether or not to enable these tests on the fly.
+ if not checked_working_venv
+ cmd = run_command(python, '-m', 'venv', venv_path, check: false)
+
+ have_working_venv = (cmd.returncode() == 0)
+ if not have_working_venv
+ warning('A working Python venv module is required to run Python tests.')
+ endif
+
+ checked_working_venv = true
+ endif
+
+ if not have_working_venv
+ continue
+ endif
+
+ # Make sure the temporary installation is in PATH (necessary both for
+ # --temp-instance and for any pip modules compiling against libpq, like
+ # psycopg2).
+ env = test_env
+ env.prepend('PATH', temp_install_bindir, test_dir['bd'])
+
+ foreach name, value : t.get('env', {})
+ env.set(name, value)
+ endforeach
+
+ reqs = files(t['requirements'])
+ test('install_' + venv_name,
+ python,
+ args: [ make_venv, '--requirements', reqs, venv_path ],
+ env: env,
+ priority: setup_tests_priority - 1, # must run after tmp_install
+ is_parallel: false,
+ suite: ['setup'],
+ timeout: 60, # 30s is too short for the cryptography package compile
+ )
+
+ test_group = test_dir['name']
+ test_output = test_result_dir / test_group / kind
+ test_kwargs = {
+ #'protocol': 'tap',
+ 'suite': test_group,
+ 'timeout': 1000,
+ 'depends': test_deps,
+ 'env': env,
+ }
+
+ pytest = venv_path / 'bin' / 'py.test'
+ test_command = [
+ pytest,
+ # Avoid running these tests against an existing database.
+ '--temp-instance', test_output / 'data',
+
+ # FIXME pytest-tap's stream feature accidentally suppresses errors that
+ # are critical for debugging:
+ # https://github.com/python-tap/pytest-tap/issues/30
+ # Don't use the meson TAP protocol for now...
+ #'--tap-stream',
+ ]
+
+ foreach pyt : t['tests']
+ # Similarly to TAP, strip ./ and .py to make the names prettier
+ pyt_p = pyt
+ if pyt_p.startswith('./')
+ pyt_p = pyt_p.split('./')[1]
+ endif
+ if pyt_p.endswith('.py')
+ pyt_p = fs.stem(pyt_p)
+ endif
+
+ test(test_group / pyt_p,
+ python,
+ kwargs: test_kwargs,
+ args: testwrap_base + [
+ '--testgroup', test_group,
+ '--testname', pyt_p,
+ '--', test_command,
+ test_dir['sd'] / pyt,
+ ],
+ )
+ endforeach
+ install_suites += test_group
else
error('unknown kind @0@ of test in @1@'.format(kind, test_dir['sd']))
endif
diff --git a/src/test/meson.build b/src/test/meson.build
index c3d0dfedf1..f401ec179e 100644
--- a/src/test/meson.build
+++ b/src/test/meson.build
@@ -7,6 +7,7 @@ subdir('authentication')
subdir('recovery')
subdir('subscription')
subdir('modules')
+subdir('python')
if ssl.found()
subdir('ssl')
diff --git a/src/test/python/.gitignore b/src/test/python/.gitignore
new file mode 100644
index 0000000000..0e8f027b2e
--- /dev/null
+++ b/src/test/python/.gitignore
@@ -0,0 +1,2 @@
+__pycache__/
+/venv/
diff --git a/src/test/python/Makefile b/src/test/python/Makefile
new file mode 100644
index 0000000000..b0695b6287
--- /dev/null
+++ b/src/test/python/Makefile
@@ -0,0 +1,38 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+# Only Python 3 is supported, but if it's named something different on your
+# system you can override it with the PYTHON3 variable.
+PYTHON3 := python3
+
+# All dependencies are placed into this directory. The default is .gitignored
+# for you, but you can override it if you'd like.
+VENV := ./venv
+
+override VBIN := $(VENV)/bin
+override PIP := $(VBIN)/pip
+override PYTEST := $(VBIN)/py.test
+override ISORT := $(VBIN)/isort
+override BLACK := $(VBIN)/black
+
+.PHONY: installcheck indent
+
+installcheck: $(PYTEST)
+ $(PYTEST) -v -rs
+
+indent: $(ISORT) $(BLACK)
+ $(ISORT) --profile black *.py client/*.py server/*.py
+ $(BLACK) *.py client/*.py server/*.py
+
+$(PYTEST) $(ISORT) $(BLACK) &: requirements.txt | $(PIP)
+ $(PIP) install --force-reinstall -r $<
+
+$(PIP):
+ $(PYTHON3) -m venv $(VENV)
+
+# A convenience recipe to rebuild psycopg2 against the local libpq.
+.PHONY: rebuild-psycopg2
+rebuild-psycopg2: | $(PIP)
+ $(PIP) install --force-reinstall --no-binary :all: $(shell grep psycopg2 requirements.txt)
diff --git a/src/test/python/README b/src/test/python/README
new file mode 100644
index 0000000000..acf339a589
--- /dev/null
+++ b/src/test/python/README
@@ -0,0 +1,66 @@
+A test suite for exercising both the libpq client and the server backend at the
+protocol level, based on pytest and Construct.
+
+WARNING! This suite takes superuser-level control of the cluster under test,
+writing to the server config, creating and destroying databases, etc. It also
+spins up various ephemeral TCP services. This is not safe for production servers
+and therefore must be explicitly opted into by setting PG_TEST_EXTRA=python in
+the environment.
+
+The test suite currently assumes that the standard PG* environment variables
+point to the database under test and are sufficient to log in a superuser on
+that system. In other words, a bare `psql` needs to Just Work before the test
+suite can do its thing. For a newly built dev cluster, typically all that I need
+to do is a
+
+ export PGDATABASE=postgres
+
+but you can adjust as needed for your setup. See also 'Advanced Usage' below.
+
+## Requirements
+
+A supported version (3.6+) of Python.
+
+The first run of
+
+ make installcheck PG_TEST_EXTRA=python
+
+will install a local virtual environment and all needed dependencies. During
+development, if libpq changes incompatibly, you can issue
+
+ $ make rebuild-psycopg2
+
+to force a rebuild of the client library.
+
+## Hacking
+
+The code style is enforced by a _very_ opinionated autoformatter. Running the
+
+ make indent
+
+recipe will invoke it for you automatically. Don't fight the tool; part of the
+zen is in knowing that if the formatter makes your code ugly, there's probably a
+cleaner way to write your code.
+
+## Advanced Usage
+
+The Makefile is there for convenience, but you don't have to use it. Activate
+the virtualenv to be able to use pytest directly:
+
+ $ export PG_TEST_EXTRA=python
+ $ source venv/bin/activate
+ $ py.test -k oauth
+ ...
+ $ py.test ./server/test_server.py
+ ...
+ $ deactivate # puts the PATH et al back the way it was before
+
+To make quick smoke tests possible, slow tests have been marked explicitly. You
+can skip them by saying e.g.
+
+ $ py.test -m 'not slow'
+
+If you'd rather not test against an existing server, you can have the suite spin
+up a temporary one using whatever pg_ctl it finds in PATH:
+
+ $ py.test --temp-instance=./tmp_check
diff --git a/src/test/python/client/__init__.py b/src/test/python/client/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/src/test/python/client/conftest.py b/src/test/python/client/conftest.py
new file mode 100644
index 0000000000..94f3620af3
--- /dev/null
+++ b/src/test/python/client/conftest.py
@@ -0,0 +1,137 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import socket
+import sys
+import threading
+
+import psycopg2
+import psycopg2.extras
+import pytest
+
+import pq3
+
+BLOCKING_TIMEOUT = 2 # the number of seconds to wait for blocking calls
+
+
+@pytest.fixture
+def server_socket(unused_tcp_port_factory):
+ """
+ Returns a listening socket bound to an ephemeral port.
+ """
+ with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+ s.bind(("127.0.0.1", unused_tcp_port_factory()))
+ s.listen(1)
+ s.settimeout(BLOCKING_TIMEOUT)
+ yield s
+
+
+class ClientHandshake(threading.Thread):
+ """
+ A thread that connects to a local Postgres server using psycopg2. Once the
+ opening handshake completes, the connection will be immediately closed.
+ """
+
+ def __init__(self, *, port, **kwargs):
+ super().__init__()
+
+ kwargs["port"] = port
+ self._kwargs = kwargs
+
+ self.exception = None
+
+ def run(self):
+ try:
+ conn = psycopg2.connect(host="127.0.0.1", **self._kwargs)
+ self._pump_async(conn)
+ conn.close()
+ except Exception as e:
+ self.exception = e
+
+ def check_completed(self, timeout=BLOCKING_TIMEOUT):
+ """
+ Joins the client thread. Raises an exception if the thread could not be
+ joined, or if it threw an exception itself. (The exception will be
+ cleared, so future calls to check_completed will succeed.)
+ """
+ self.join(timeout)
+
+ if self.is_alive():
+ raise TimeoutError("client thread did not handshake within the timeout")
+ elif self.exception:
+ e = self.exception
+ self.exception = None
+ raise e
+
+ def _pump_async(self, conn):
+ """
+ Polls a psycopg2 connection until it's completed. (Synchronous
+ connections will work here too; they'll just immediately return OK.)
+ """
+ psycopg2.extras.wait_select(conn)
+
+
+@pytest.fixture
+def accept(server_socket):
+ """
+ Returns a factory function that, when called, returns a pair (sock, client)
+ where sock is a server socket that has accepted a connection from client,
+ and client is an instance of ClientHandshake. Clients will complete their
+ handshakes and cleanly disconnect.
+
+ The default connstring options may be extended or overridden by passing
+ arbitrary keyword arguments. Keep in mind that you generally should not
+ override the host or port, since they point to the local test server.
+
+ For situations where a client needs to connect more than once to complete a
+ handshake, the accept function may be called more than once. (The client
+ returned for subsequent calls will always be the same client that was
+ returned for the first call.)
+
+ Tests must either complete the handshake so that the client thread can be
+ automatically joined during teardown, or else call client.check_completed()
+ and manually handle any expected errors.
+ """
+ _, port = server_socket.getsockname()
+
+ client = None
+ default_opts = dict(
+ port=port,
+ user=pq3.pguser(),
+ sslmode="disable",
+ )
+
+ def factory(**kwargs):
+ nonlocal client
+
+ if client is None:
+ opts = dict(default_opts)
+ opts.update(kwargs)
+
+ # The server_socket is already listening, so the client thread can
+ # be safely started; it'll block on the connection until we accept.
+ client = ClientHandshake(**opts)
+ client.start()
+
+ sock, _ = server_socket.accept()
+ return sock, client
+
+ yield factory
+
+ if client is not None:
+ client.check_completed()
+
+
+@pytest.fixture
+def conn(accept):
+ """
+ Returns an accepted, wrapped pq3 connection to a psycopg2 client. The socket
+ will be closed when the test finishes, and the client will be checked for a
+ cleanly completed handshake.
+ """
+ sock, client = accept()
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ yield conn
diff --git a/src/test/python/client/test_client.py b/src/test/python/client/test_client.py
new file mode 100644
index 0000000000..c4c946fda4
--- /dev/null
+++ b/src/test/python/client/test_client.py
@@ -0,0 +1,180 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import sys
+
+import psycopg2
+import pytest
+from cryptography.hazmat.primitives import hashes, hmac
+
+import pq3
+
+
+def finish_handshake(conn):
+ """
+ Sends the AuthenticationOK message and the standard opening salvo of server
+ messages, then asserts that the client immediately sends a Terminate message
+ to close the connection cleanly.
+ """
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.OK)
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"client_encoding", value=b"UTF-8")
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"DateStyle", value=b"ISO, MDY")
+ pq3.send(conn, pq3.types.ReadyForQuery, status=b"I")
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.Terminate
+
+
+def test_handshake(conn):
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ finish_handshake(conn)
+
+
+def test_aborted_connection(accept):
+ """
+ Make sure the client correctly reports an early close during handshakes.
+ """
+ sock, client = accept()
+ sock.close()
+
+ expected = "server closed the connection unexpectedly"
+ with pytest.raises(psycopg2.OperationalError, match=expected):
+ client.check_completed()
+
+
+#
+# SCRAM-SHA-256 (see RFC 5802: https://tools.ietf.org/html/rfc5802)
+#
+
+
+@pytest.fixture
+def password():
+ """
+ Returns a password for use by both client and server.
+ """
+ # TODO: parameterize this with passwords that require SASLprep.
+ return "secret"
+
+
+@pytest.fixture
+def pwconn(accept, password):
+ """
+ Like the conn fixture, but uses a password in the connection.
+ """
+ sock, client = accept(password=password)
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ yield conn
+
+
+def sha256(data):
+ """The H(str) function from Section 2.2."""
+ digest = hashes.Hash(hashes.SHA256())
+ digest.update(data)
+ return digest.finalize()
+
+
+def hmac_256(key, data):
+ """The HMAC(key, str) function from Section 2.2."""
+ h = hmac.HMAC(key, hashes.SHA256())
+ h.update(data)
+ return h.finalize()
+
+
+def xor(a, b):
+ """The XOR operation from Section 2.2."""
+ res = bytearray(a)
+ for i, byte in enumerate(b):
+ res[i] ^= byte
+ return bytes(res)
+
+
+def h_i(data, salt, i):
+ """The Hi(str, salt, i) function from Section 2.2."""
+ assert i > 0
+
+ acc = hmac_256(data, salt + b"\x00\x00\x00\x01")
+ last = acc
+ i -= 1
+
+ while i:
+ u = hmac_256(data, last)
+ acc = xor(acc, u)
+
+ last = u
+ i -= 1
+
+ return acc
+
+
+def test_scram(pwconn, password):
+ startup = pq3.recv1(pwconn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ pwconn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"SCRAM-SHA-256", b""],
+ )
+
+ # Get the client-first-message.
+ pkt = pq3.recv1(pwconn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ initial = pq3.SASLInitialResponse.parse(pkt.payload)
+ assert initial.name == b"SCRAM-SHA-256"
+
+ c_bind, authzid, c_name, c_nonce = initial.data.split(b",")
+ assert c_bind == b"n" # no channel bindings on a plaintext connection
+ assert authzid == b"" # we don't support authzid currently
+ assert c_name == b"n=" # libpq doesn't honor the GS2 username
+ assert c_nonce.startswith(b"r=")
+
+ # Send the server-first-message.
+ salt = b"12345"
+ iterations = 2
+
+ s_nonce = c_nonce + b"somenonce"
+ s_salt = b"s=" + base64.b64encode(salt)
+ s_iterations = b"i=%d" % iterations
+
+ msg = b",".join([s_nonce, s_salt, s_iterations])
+ pq3.send(pwconn, pq3.types.AuthnRequest, type=pq3.authn.SASLContinue, body=msg)
+
+ # Get the client-final-message.
+ pkt = pq3.recv1(pwconn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ c_bind_final, c_nonce_final, c_proof = pkt.payload.split(b",")
+ assert c_bind_final == b"c=" + base64.b64encode(c_bind + b"," + authzid + b",")
+ assert c_nonce_final == s_nonce
+
+ # Calculate what the client proof should be.
+ salted_password = h_i(password.encode("ascii"), salt, iterations)
+ client_key = hmac_256(salted_password, b"Client Key")
+ stored_key = sha256(client_key)
+
+ auth_message = b",".join(
+ [c_name, c_nonce, s_nonce, s_salt, s_iterations, c_bind_final, c_nonce_final]
+ )
+ client_signature = hmac_256(stored_key, auth_message)
+ client_proof = xor(client_key, client_signature)
+
+ expected = b"p=" + base64.b64encode(client_proof)
+ assert c_proof == expected
+
+ # Send the correct server signature.
+ server_key = hmac_256(salted_password, b"Server Key")
+ server_signature = hmac_256(server_key, auth_message)
+
+ s_verify = b"v=" + base64.b64encode(server_signature)
+ pq3.send(pwconn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal, body=s_verify)
+
+ # Done!
+ finish_handshake(pwconn)
diff --git a/src/test/python/client/test_oauth.py b/src/test/python/client/test_oauth.py
new file mode 100644
index 0000000000..c4b7f91ff4
--- /dev/null
+++ b/src/test/python/client/test_oauth.py
@@ -0,0 +1,1766 @@
+#
+# Copyright 2021 VMware, Inc.
+# Portions Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import collections
+import ctypes
+import http.server
+import json
+import logging
+import os
+import platform
+import secrets
+import sys
+import threading
+import time
+import traceback
+import types
+import urllib.parse
+from numbers import Number
+
+import psycopg2
+import pytest
+
+import pq3
+
+from .conftest import BLOCKING_TIMEOUT
+
+# The client tests need libpq to have been compiled with OAuth support; skip
+# them otherwise.
+pytestmark = pytest.mark.skipif(
+ os.getenv("with_oauth") == "none",
+ reason="OAuth client tests require --with-oauth support",
+)
+
+if platform.system() == "Darwin":
+ libpq = ctypes.cdll.LoadLibrary("libpq.5.dylib")
+else:
+ libpq = ctypes.cdll.LoadLibrary("libpq.so.5")
+
+
+def finish_handshake(conn):
+ """
+ Sends the AuthenticationOK message and the standard opening salvo of server
+ messages, then asserts that the client immediately sends a Terminate message
+ to close the connection cleanly.
+ """
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.OK)
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"client_encoding", value=b"UTF-8")
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"DateStyle", value=b"ISO, MDY")
+ pq3.send(conn, pq3.types.ReadyForQuery, status=b"I")
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.Terminate
+
+
+#
+# OAUTHBEARER (see RFC 7628: https://tools.ietf.org/html/rfc7628)
+#
+
+
+def start_oauth_handshake(conn):
+ """
+ Negotiates an OAUTHBEARER SASL challenge. Returns the client's initial
+ response data.
+ """
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn, pq3.types.AuthnRequest, type=pq3.authn.SASL, body=[b"OAUTHBEARER", b""]
+ )
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ initial = pq3.SASLInitialResponse.parse(pkt.payload)
+ assert initial.name == b"OAUTHBEARER"
+
+ return initial.data
+
+
+def get_auth_value(initial):
+ """
+ Finds the auth value (e.g. "Bearer somedata..." in the client's initial SASL
+ response.
+ """
+ kvpairs = initial.split(b"\x01")
+ assert kvpairs[0] == b"n,," # no channel binding or authzid
+ assert kvpairs[2] == b"" # ends with an empty kvpair
+ assert kvpairs[3] == b"" # ...and there's nothing after it
+ assert len(kvpairs) == 4
+
+ key, value = kvpairs[1].split(b"=", 2)
+ assert key == b"auth"
+
+ return value
+
+
+def xtest_oauth_success(conn): # TODO
+ initial = start_oauth_handshake(conn)
+
+ auth = get_auth_value(initial)
+ assert auth.startswith(b"Bearer ")
+
+ # Accept the token. TODO actually validate
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+class RawResponse(str):
+ """
+ Returned by registered endpoint callbacks to take full control of the
+ response. Usually, return values are converted to JSON; a RawResponse body
+ will be passed to the client as-is, allowing endpoint implementations to
+ issue invalid JSON.
+ """
+
+ pass
+
+
+class OpenIDProvider(threading.Thread):
+ """
+ A thread that runs a mock OpenID provider server.
+ """
+
+ def __init__(self, *, port):
+ super().__init__()
+
+ self.exception = None
+
+ addr = ("", port)
+ self.server = self._Server(addr, self._Handler)
+
+ # TODO: allow HTTPS only, somehow
+ oauth = self._OAuthState()
+ oauth.host = f"localhost:{port}"
+ oauth.issuer = f"http://localhost:{port}"
+
+ # The following endpoints are required to be advertised by providers,
+ # even though our chosen client implementation does not actually make
+ # use of them.
+ oauth.register_endpoint(
+ "authorization_endpoint", "POST", "/authorize", self._authorization_handler
+ )
+ oauth.register_endpoint("jwks_uri", "GET", "/keys", self._jwks_handler)
+
+ self.server.oauth = oauth
+
+ def run(self):
+ try:
+ # XXX socketserver.serve_forever() has a serious architectural
+ # issue: its select loop wakes up every `poll_interval` seconds to
+ # see if the server is shutting down. The default, 500 ms, only lets
+ # us run two tests every second. But the faster we go, the more CPU
+ # we burn unnecessarily...
+ self.server.serve_forever(poll_interval=0.01)
+ except Exception as e:
+ self.exception = e
+
+ def stop(self, timeout=BLOCKING_TIMEOUT):
+ """
+ Shuts down the server and joins its thread. Raises an exception if the
+ thread could not be joined, or if it threw an exception itself. Must
+ only be called once, after start().
+ """
+ self.server.shutdown()
+ self.join(timeout)
+
+ if self.is_alive():
+ raise TimeoutError("client thread did not handshake within the timeout")
+ elif self.exception:
+ e = self.exception
+ raise e
+
+ class _OAuthState(object):
+ def __init__(self):
+ self.endpoint_paths = {}
+ self._endpoints = {}
+
+ # Provide a standard discovery document by default; tests can
+ # override it.
+ self.register_endpoint(
+ None,
+ "GET",
+ "/.well-known/openid-configuration",
+ self._default_discovery_handler,
+ )
+
+ def register_endpoint(self, name, method, path, func):
+ if method not in self._endpoints:
+ self._endpoints[method] = {}
+
+ self._endpoints[method][path] = func
+
+ if name is not None:
+ self.endpoint_paths[name] = path
+
+ def endpoint(self, method, path):
+ if method not in self._endpoints:
+ return None
+
+ return self._endpoints[method].get(path)
+
+ def _default_discovery_handler(self, headers, params):
+ doc = {
+ "issuer": self.issuer,
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ "grant_types_supported": [
+ "urn:ietf:params:oauth:grant-type:device_code"
+ ],
+ }
+
+ for name, path in self.endpoint_paths.items():
+ doc[name] = self.issuer + path
+
+ return 200, doc
+
+ class _Server(http.server.HTTPServer):
+ def handle_error(self, request, addr):
+ self.shutdown_request(request)
+ raise
+
+ @staticmethod
+ def _jwks_handler(headers, params):
+ return 200, {"keys": []}
+
+ @staticmethod
+ def _authorization_handler(headers, params):
+ # We don't actually want this to be called during these tests -- we
+ # should be using the device authorization endpoint instead.
+ assert (
+ False
+ ), "authorization handler called instead of device authorization handler"
+
+ class _Handler(http.server.BaseHTTPRequestHandler):
+ timeout = BLOCKING_TIMEOUT
+
+ def _handle(self, *, params=None, handler=None):
+ oauth = self.server.oauth
+ assert self.headers["Host"] == oauth.host
+
+ if handler is None:
+ handler = oauth.endpoint(self.command, self.path)
+ assert (
+ handler is not None
+ ), f"no registered endpoint for {self.command} {self.path}"
+
+ result = handler(self.headers, params)
+
+ if len(result) == 2:
+ headers = {"Content-Type": "application/json"}
+ code, resp = result
+ else:
+ code, headers, resp = result
+
+ self.send_response(code)
+ for h, v in headers.items():
+ self.send_header(h, v)
+ self.end_headers()
+
+ if resp is not None:
+ if not isinstance(resp, RawResponse):
+ resp = json.dumps(resp)
+ resp = resp.encode("utf-8")
+ self.wfile.write(resp)
+
+ self.close_connection = True
+
+ def do_GET(self):
+ self._handle()
+
+ def _request_body(self):
+ length = self.headers["Content-Length"]
+
+ # Handle only an explicit content-length.
+ assert length is not None
+ length = int(length)
+
+ return self.rfile.read(length).decode("utf-8")
+
+ def do_POST(self):
+ assert self.headers["Content-Type"] == "application/x-www-form-urlencoded"
+
+ body = self._request_body()
+ params = urllib.parse.parse_qs(body)
+
+ self._handle(params=params)
+
+
+@pytest.fixture
+def openid_provider(unused_tcp_port_factory):
+ """
+ A fixture that returns the OAuth state of a running OpenID provider server. The
+ server will be stopped when the fixture is torn down.
+ """
+ thread = OpenIDProvider(port=unused_tcp_port_factory())
+ thread.start()
+
+ try:
+ yield thread.server.oauth
+ finally:
+ thread.stop()
+
+
+#
+# PQAuthDataHook implementation, matching libpq.h
+#
+
+
+PQAUTHDATA_PROMPT_OAUTH_DEVICE = 0
+PQAUTHDATA_OAUTH_BEARER_TOKEN = 1
+
+PGRES_POLLING_FAILED = 0
+PGRES_POLLING_READING = 1
+PGRES_POLLING_WRITING = 2
+PGRES_POLLING_OK = 3
+
+
+class PQPromptOAuthDevice(ctypes.Structure):
+ _fields_ = [
+ ("verification_uri", ctypes.c_char_p),
+ ("user_code", ctypes.c_char_p),
+ ]
+
+
+class PQOAuthBearerRequest(ctypes.Structure):
+ pass
+
+
+PQOAuthBearerRequest._fields_ = [
+ ("openid_configuration", ctypes.c_char_p),
+ ("scope", ctypes.c_char_p),
+ (
+ "async_",
+ ctypes.CFUNCTYPE(
+ ctypes.c_int,
+ ctypes.c_void_p,
+ ctypes.POINTER(PQOAuthBearerRequest),
+ ctypes.POINTER(ctypes.c_int),
+ ),
+ ),
+ (
+ "cleanup",
+ ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(PQOAuthBearerRequest)),
+ ),
+ ("token", ctypes.c_char_p),
+ ("user", ctypes.c_void_p),
+]
+
+
+@pytest.fixture
+def auth_data_cb():
+ """
+ Tracks calls to the libpq authdata hook. The yielded object contains a calls
+ member that records the data sent to the hook. If a test needs to perform
+ custom actions during a call, it can set the yielded object's impl callback;
+ beware that the callback takes place on a different thread.
+
+ This is done differently from the other callback implementations on purpose.
+ For the others, we can declare test-specific callbacks and have them perform
+ direct assertions on the data they receive. But that won't work for a C
+ callback, because there's no way for us to bubble up the assertion through
+ libpq. Instead, this mock-style approach is taken, where we just record the
+ calls and let the test examine them later.
+ """
+
+ class _Call:
+ pass
+
+ class _cb(object):
+ def __init__(self):
+ self.calls = []
+
+ cb = _cb()
+ cb.impl = None
+
+ # The callback will occur on a different thread, so protect the cb object.
+ cb_lock = threading.Lock()
+
+ @ctypes.CFUNCTYPE(ctypes.c_int, ctypes.c_byte, ctypes.c_void_p, ctypes.c_void_p)
+ def auth_data_cb(typ, pgconn, data):
+ handle_by_default = 0 # does an implementation have to be provided?
+
+ if typ == PQAUTHDATA_PROMPT_OAUTH_DEVICE:
+ cls = PQPromptOAuthDevice
+ handle_by_default = 1
+ elif typ == PQAUTHDATA_OAUTH_BEARER_TOKEN:
+ cls = PQOAuthBearerRequest
+ else:
+ return 0
+
+ call = _Call()
+ call.type = typ
+
+ # The lifetime of the underlying data being pointed to doesn't
+ # necessarily match the lifetime of the Python object, so we can't
+ # reference a Structure's fields after returning. Explicitly copy the
+ # contents over, field by field.
+ data = ctypes.cast(data, ctypes.POINTER(cls))
+ for name, _ in cls._fields_:
+ setattr(call, name, getattr(data.contents, name))
+
+ with cb_lock:
+ cb.calls.append(call)
+
+ if cb.impl:
+ # Pass control back to the test.
+ try:
+ return cb.impl(typ, pgconn, data.contents)
+ except Exception:
+ # This can't escape into the C stack, but we can fail the flow
+ # and hope the traceback gives us enough detail.
+ logging.error(
+ "Exception during authdata hook callback:\n"
+ + traceback.format_exc()
+ )
+ return -1
+
+ return handle_by_default
+
+ libpq.PQsetAuthDataHook(auth_data_cb)
+ try:
+ yield cb
+ finally:
+ # The callback is about to go out of scope, so make sure libpq is
+ # disconnected from it. (We wouldn't want to accidentally influence
+ # later tests anyway.)
+ libpq.PQsetAuthDataHook(None)
+
+
+@pytest.mark.parametrize("secret", [None, "", "hunter2"])
+@pytest.mark.parametrize("scope", [None, "", "openid email"])
+@pytest.mark.parametrize("retries", [0, 1])
+@pytest.mark.parametrize(
+ "asynchronous",
+ [
+ pytest.param(False, id="synchronous"),
+ pytest.param(True, id="asynchronous"),
+ ],
+)
+def test_oauth_with_explicit_issuer(
+ accept, openid_provider, asynchronous, retries, scope, secret, auth_data_cb
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ oauth_client_secret=secret,
+ oauth_scope=scope,
+ async_=asynchronous,
+ )
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+ verification_url = "https://example.com/device"
+
+ access_token = secrets.token_urlsafe()
+
+ def check_client_authn(headers, params):
+ if not secret:
+ assert params["client_id"] == [client_id]
+ return
+
+ # Require the client to use Basic authn; request-body credentials are
+ # NOT RECOMMENDED (RFC 6749, Sec. 2.3.1).
+ assert "Authorization" in headers
+
+ method, creds = headers["Authorization"].split()
+ assert method == "Basic"
+
+ expected = f"{client_id}:{secret}"
+ assert base64.b64decode(creds) == expected.encode("ascii")
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ check_client_authn(headers, params)
+
+ if scope:
+ assert params["scope"] == [scope]
+ else:
+ assert "scope" not in params
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": verification_url,
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ attempts = 0
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ check_client_authn(headers, params)
+
+ assert params["grant_type"] == ["urn:ietf:params:oauth:grant-type:device_code"]
+ assert params["device_code"] == [device_code]
+
+ now = time.monotonic()
+
+ with retry_lock:
+ nonlocal attempts
+
+ # If the test wants to force the client to retry, return an
+ # authorization_pending response and decrement the retry count.
+ if attempts < retries:
+ attempts += 1
+ return 400, {"error": "authorization_pending"}
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+ if retries:
+ # Finally, make sure that the client prompted the user once with the
+ # expected authorization URL and user code.
+ assert len(auth_data_cb.calls) == 2
+
+ # First call should have been for a custom flow, which we ignored.
+ assert auth_data_cb.calls[0].type == PQAUTHDATA_OAUTH_BEARER_TOKEN
+
+ # Second call is for our user prompt.
+ call = auth_data_cb.calls[1]
+ assert call.type == PQAUTHDATA_PROMPT_OAUTH_DEVICE
+ assert call.verification_uri.decode() == verification_url
+ assert call.user_code.decode() == user_code
+
+
+def expect_disconnected_handshake(sock):
+ """
+ Helper for any tests that expect the client to disconnect immediately after
+ being sent the OAUTHBEARER SASL method. Generally speaking, this requires
+ the client to have an oauth_issuer set so that it doesn't try to go through
+ discovery.
+ """
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake.
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"OAUTHBEARER", b""],
+ )
+
+ # The client should disconnect at this point.
+ assert not conn.read()
+
+
+def test_oauth_requires_client_id(accept, openid_provider):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ # Do not set a client ID; this should cause a client error after the
+ # server asks for OAUTHBEARER and the client tries to contact the
+ # issuer.
+ )
+
+ expect_disconnected_handshake(sock)
+
+ expected_error = "no oauth_client_id is set"
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.slow
+@pytest.mark.parametrize("error_code", ["authorization_pending", "slow_down"])
+@pytest.mark.parametrize("retries", [1, 2])
+def test_oauth_retry_interval(accept, openid_provider, retries, error_code):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id="some-id",
+ )
+
+ expected_retry_interval = 1
+ access_token = secrets.token_urlsafe()
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ resp = {
+ "device_code": "my-device-code",
+ "user_code": "my-user-code",
+ "interval": expected_retry_interval,
+ "verification_uri": "https://example.com",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ attempts = 0
+ last_retry = None
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ now = time.monotonic()
+
+ with retry_lock:
+ nonlocal attempts, last_retry, expected_retry_interval
+
+ # Make sure the retry interval is being respected by the client.
+ if last_retry is not None:
+ interval = now - last_retry
+ assert interval >= expected_retry_interval
+
+ last_retry = now
+
+ # If the test wants to force the client to retry, return the desired
+ # error response and decrement the retry count.
+ if attempts < retries:
+ attempts += 1
+
+ # A slow_down code requires the client to additionally increase
+ # its interval by five seconds.
+ if error_code == "slow_down":
+ expected_retry_interval += 5
+
+ return 400, {"error": error_code}
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+@pytest.fixture
+def self_pipe():
+ """
+ Yields a pipe fd pair.
+ """
+
+ class _Pipe:
+ pass
+
+ p = _Pipe()
+ p.readfd, p.writefd = os.pipe()
+
+ try:
+ yield p
+ finally:
+ os.close(p.readfd)
+ os.close(p.writefd)
+
+
+@pytest.mark.parametrize("scope", [None, "", "openid email"])
+@pytest.mark.parametrize(
+ "retries",
+ [
+ -1, # no async callback
+ 0, # async callback immediately returns token
+ 1, # async callback waits on altsock once
+ 2, # async callback waits on altsock twice
+ ],
+)
+@pytest.mark.parametrize(
+ "asynchronous",
+ [
+ pytest.param(False, id="synchronous"),
+ pytest.param(True, id="asynchronous"),
+ ],
+)
+def test_user_defined_flow(
+ accept, auth_data_cb, self_pipe, scope, retries, asynchronous
+):
+ issuer = "http://localhost"
+ discovery_uri = issuer + "/.well-known/openid-configuration"
+ access_token = secrets.token_urlsafe()
+
+ sock, _ = accept(
+ oauth_issuer=issuer,
+ oauth_client_id="some-id",
+ oauth_scope=scope,
+ async_=asynchronous,
+ )
+
+ # Track callbacks.
+ attempts = 0
+ wakeup_called = False
+ cleanup_calls = 0
+ lock = threading.Lock()
+
+ def wakeup():
+ """Writes a byte to the wakeup pipe."""
+ nonlocal wakeup_called
+ with lock:
+ wakeup_called = True
+ os.write(self_pipe.writefd, b"\0")
+
+ def get_token(pgconn, request, p_altsock):
+ """
+ Async token callback. While attempts < retries, libpq will be instructed
+ to wait on the self_pipe. When attempts == retries, the token will be
+ set.
+
+ Note that assertions and exceptions raised here are allowed but not very
+ helpful, since they can't bubble through the libpq stack to be collected
+ by the test suite. Try not to rely too heavily on them.
+ """
+ # Make sure libpq passed our user data through.
+ assert request.user == 42
+
+ with lock:
+ nonlocal attempts, wakeup_called
+
+ if attempts:
+ # If we've already started the timer, we shouldn't get a
+ # call back before it trips.
+ assert wakeup_called, "authdata hook was called before the timer"
+
+ # Drain the wakeup byte.
+ os.read(self_pipe.readfd, 1)
+
+ if attempts < retries:
+ attempts += 1
+
+ # Wake up the client in a little bit of time.
+ wakeup_called = False
+ threading.Timer(0.1, wakeup).start()
+
+ # Tell libpq to wait on the other end of the wakeup pipe.
+ p_altsock[0] = self_pipe.readfd
+ return PGRES_POLLING_READING
+
+ # Done!
+ request.token = access_token.encode()
+ return PGRES_POLLING_OK
+
+ @ctypes.CFUNCTYPE(
+ ctypes.c_int,
+ ctypes.c_void_p,
+ ctypes.POINTER(PQOAuthBearerRequest),
+ ctypes.POINTER(ctypes.c_int),
+ )
+ def get_token_wrapper(pgconn, p_request, p_altsock):
+ """
+ Translation layer between C and Python for the async callback.
+ Assertions and exceptions will be swallowed at the boundary, so make
+ sure they don't escape here.
+ """
+ try:
+ return get_token(pgconn, p_request.contents, p_altsock)
+ except Exception:
+ logging.error("Exception during async callback:\n" + traceback.format_exc())
+ return PGRES_POLLING_FAILED
+
+ @ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(PQOAuthBearerRequest))
+ def cleanup(pgconn, p_request):
+ """
+ Should be called exactly once per connection.
+ """
+ nonlocal cleanup_calls
+ with lock:
+ cleanup_calls += 1
+
+ def bearer_hook(typ, pgconn, request):
+ """
+ Implementation of the PQAuthDataHook, which either sets up an async
+ callback or returns the token directly, depending on the value of
+ retries.
+
+ As above, try not to rely too much on assertions/exceptions here.
+ """
+ assert typ == PQAUTHDATA_OAUTH_BEARER_TOKEN
+ request.cleanup = cleanup
+
+ if retries < 0:
+ # Special case: return a token immediately without a callback.
+ request.token = access_token.encode()
+ return 1
+
+ # Tell libpq to call us back.
+ request.async_ = get_token_wrapper
+ request.user = ctypes.c_void_p(42) # will be checked in the callback
+ return 1
+
+ auth_data_cb.impl = bearer_hook
+
+ # Now drive the server side.
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in our custom callback
+ # being invoked to fetch the token.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+ # Check the data provided to the hook.
+ assert len(auth_data_cb.calls) == 1
+
+ call = auth_data_cb.calls[0]
+ assert call.type == PQAUTHDATA_OAUTH_BEARER_TOKEN
+ assert call.openid_configuration.decode() == discovery_uri
+ assert call.scope == (None if scope is None else scope.encode())
+
+ # Make sure we cleaned up after ourselves.
+ assert cleanup_calls == 1
+
+
+def alt_patterns(*patterns):
+ """
+ Just combines multiple alternative regexes into one. It's not very efficient
+ but IMO it's easier to read and maintain.
+ """
+ pat = ""
+
+ for p in patterns:
+ if pat:
+ pat += "|"
+ pat += f"({p})"
+
+ return pat
+
+
+@pytest.mark.parametrize(
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
+ (
+ 400,
+ {
+ "error": "invalid_client",
+ "error_description": "client authentication failed",
+ },
+ ),
+ r"failed to obtain device authorization: client authentication failed \(invalid_client\)",
+ id="authentication failure with description",
+ ),
+ pytest.param(
+ (400, {"error": "invalid_request"}),
+ r"failed to obtain device authorization: \(invalid_request\)",
+ id="invalid request without description",
+ ),
+ pytest.param(
+ (400, {}),
+ alt_patterns(
+ r'failed to parse token error response: field "error" is missing',
+ r"failed to obtain device authorization: \(iddawc error I_ERROR_PARAM\)",
+ ),
+ id="broken error response",
+ ),
+ pytest.param(
+ (200, RawResponse(r'{ "interval": 3.5.8 }')),
+ alt_patterns(
+ r"failed to parse device authorization: Token .* is invalid",
+ r"failed to obtain device authorization: \(iddawc error I_ERROR\)",
+ ),
+ id="non-numeric interval",
+ ),
+ pytest.param(
+ (200, RawResponse(r'{ "interval": 08 }')),
+ alt_patterns(
+ r"failed to parse device authorization: Token .* is invalid",
+ r"failed to obtain device authorization: \(iddawc error I_ERROR\)",
+ ),
+ id="invalid numeric interval",
+ ),
+ ],
+)
+def test_oauth_device_authorization_failures(
+ accept, openid_provider, failure_mode, error_pattern
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ )
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ return failure_mode
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert False, "token endpoint was invoked unexpectedly"
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+Missing = object() # sentinel for test_oauth_device_authorization_bad_json()
+
+
+@pytest.mark.parametrize(
+ "bad_value",
+ [
+ pytest.param({"device_code": 3}, id="object"),
+ pytest.param([1, 2, 3], id="array"),
+ pytest.param("some string", id="string"),
+ pytest.param(4, id="numeric"),
+ pytest.param(False, id="boolean"),
+ pytest.param(None, id="null"),
+ pytest.param(Missing, id="missing"),
+ ],
+)
+@pytest.mark.parametrize(
+ "field_name,ok_type,required",
+ [
+ ("device_code", str, True),
+ ("user_code", str, True),
+ ("verification_uri", str, True),
+ ("interval", int, False),
+ ],
+)
+def test_oauth_device_authorization_bad_json_schema(
+ accept, openid_provider, field_name, ok_type, required, bad_value
+):
+ # To make the test matrix easy, just skip the tests that aren't actually
+ # interesting (field of the correct type, missing optional field).
+ if bad_value is Missing and not required:
+ pytest.skip("not interesting: optional field")
+ elif type(bad_value) == ok_type: # not isinstance(), because bool is an int
+ pytest.skip("not interesting: correct type")
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=secrets.token_hex(),
+ )
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ # Begin with an acceptable base response...
+ resp = {
+ "device_code": "my-device-code",
+ "user_code": "my-user-code",
+ "interval": 0,
+ "verification_uri": "https://example.com",
+ "expires_in": 5,
+ }
+
+ # ...then tweak it so the client fails.
+ if bad_value is Missing:
+ del resp[field_name]
+ else:
+ resp[field_name] = bad_value
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert False, "token endpoint was invoked unexpectedly"
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ if bad_value is Missing:
+ error_pattern = f'field "{field_name}" is missing'
+ elif ok_type == str:
+ error_pattern = f'field "{field_name}" must be a string'
+ elif ok_type == int:
+ error_pattern = f'field "{field_name}" must be a number'
+ else:
+ assert False, "update error_pattern for new failure mode"
+
+ # XXX iddawc doesn't really check for problems in the device authorization
+ # response, leading to this patchwork:
+ if field_name == "verification_uri":
+ error_pattern = alt_patterns(
+ error_pattern,
+ "issuer did not provide a verification URI",
+ )
+ elif field_name == "user_code":
+ error_pattern = alt_patterns(
+ error_pattern,
+ "issuer did not provide a user code",
+ )
+ else:
+ error_pattern = alt_patterns(
+ error_pattern,
+ r"failed to obtain access token: \(iddawc error I_ERROR_PARAM\)",
+ )
+
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
+ (
+ 400,
+ {
+ "error": "expired_token",
+ "error_description": "the device code has expired",
+ },
+ ),
+ r"failed to obtain access token: the device code has expired \(expired_token\)",
+ id="expired token with description",
+ ),
+ pytest.param(
+ (400, {"error": "access_denied"}),
+ r"failed to obtain access token: \(access_denied\)",
+ id="access denied without description",
+ ),
+ pytest.param(
+ (400, {}),
+ alt_patterns(
+ r'failed to parse token error response: field "error" is missing',
+ r"failed to obtain access token: \(iddawc error I_ERROR_PARAM\)",
+ ),
+ id="empty error response",
+ ),
+ pytest.param(
+ (200, {}, {}),
+ alt_patterns(
+ r"failed to parse access token response: no content type was provided",
+ r"failed to obtain access token: \(iddawc error I_ERROR\)",
+ ),
+ id="missing content type",
+ ),
+ pytest.param(
+ (200, {"Content-Type": "text/plain"}, {}),
+ alt_patterns(
+ r"failed to parse access token response: unexpected content type",
+ r"failed to obtain access token: \(iddawc error I_ERROR\)",
+ ),
+ id="wrong content type",
+ ),
+ ],
+)
+@pytest.mark.parametrize("retries", [0, 1])
+def test_oauth_token_failures(
+ accept, openid_provider, retries, failure_mode, error_pattern
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ )
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ assert params["client_id"] == [client_id]
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": "https://example.com/device",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ retry_lock = threading.Lock()
+ final_sent = False
+
+ def token_endpoint(headers, params):
+ with retry_lock:
+ nonlocal retries, final_sent
+
+ # If the test wants to force the client to retry, return an
+ # authorization_pending response and decrement the retry count.
+ if retries > 0:
+ retries -= 1
+ return 400, {"error": "authorization_pending"}
+
+ # We should only return our failure_mode response once; any further
+ # requests indicate that the client isn't correctly bailing out.
+ assert not final_sent, "client continued after token error"
+
+ final_sent = True
+
+ return failure_mode
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "bad_value",
+ [
+ pytest.param({"device_code": 3}, id="object"),
+ pytest.param([1, 2, 3], id="array"),
+ pytest.param("some string", id="string"),
+ pytest.param(4, id="numeric"),
+ pytest.param(False, id="boolean"),
+ pytest.param(None, id="null"),
+ pytest.param(Missing, id="missing"),
+ ],
+)
+@pytest.mark.parametrize(
+ "field_name,ok_type,required",
+ [
+ ("access_token", str, True),
+ ("token_type", str, True),
+ ],
+)
+def test_oauth_token_bad_json_schema(
+ accept, openid_provider, field_name, ok_type, required, bad_value
+):
+ # To make the test matrix easy, just skip the tests that aren't actually
+ # interesting (field of the correct type, missing optional field).
+ if bad_value is Missing and not required:
+ pytest.skip("not interesting: optional field")
+ elif type(bad_value) == ok_type: # not isinstance(), because bool is an int
+ pytest.skip("not interesting: correct type")
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=secrets.token_hex(),
+ )
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ resp = {
+ "device_code": "my-device-code",
+ "user_code": "my-user-code",
+ "interval": 0,
+ "verification_uri": "https://example.com",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ # Begin with an acceptable base response...
+ resp = {
+ "access_token": secrets.token_urlsafe(),
+ "token_type": "bearer",
+ }
+
+ # ...then tweak it so the client fails.
+ if bad_value is Missing:
+ del resp[field_name]
+ else:
+ resp[field_name] = bad_value
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ error_pattern = "failed to parse access token response: "
+ if bad_value is Missing:
+ error_pattern += f'field "{field_name}" is missing'
+ elif ok_type == str:
+ error_pattern += f'field "{field_name}" must be a string'
+ elif ok_type == int:
+ error_pattern += f'field "{field_name}" must be a number'
+ else:
+ assert False, "update error_pattern for new failure mode"
+
+ # XXX iddawc is fairly silent on the topic.
+ error_pattern = alt_patterns(
+ error_pattern,
+ r"failed to obtain access token: \(iddawc error I_ERROR_PARAM\)",
+ )
+
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize("scope", [None, "openid email"])
+@pytest.mark.parametrize(
+ "base_response",
+ [
+ {"status": "invalid_token"},
+ {"extra_object": {"key": "value"}, "status": "invalid_token"},
+ {"extra_object": {"status": 1}, "status": "invalid_token"},
+ ],
+)
+def test_oauth_discovery(accept, openid_provider, base_response, scope):
+ sock, client = accept(oauth_client_id=secrets.token_hex())
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+ verification_url = "https://example.com/device"
+
+ access_token = secrets.token_urlsafe()
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ if scope:
+ assert params["scope"] == [scope]
+ else:
+ assert "scope" not in params
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": verification_url,
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert params["grant_type"] == ["urn:ietf:params:oauth:grant-type:device_code"]
+ assert params["device_code"] == [device_code]
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # For discovery, the client should send an empty auth header. See
+ # RFC 7628, Sec. 4.3.
+ auth = get_auth_value(initial)
+ assert auth == b""
+
+ # We will fail the first SASL exchange. First return a link to the
+ # discovery document, pointing to the test provider server.
+ resp = dict(base_response)
+
+ discovery_uri = f"{openid_provider.issuer}/.well-known/openid-configuration"
+ resp["openid-configuration"] = discovery_uri
+
+ if scope:
+ resp["scope"] = scope
+
+ resp = json.dumps(resp)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASLContinue,
+ body=resp.encode("ascii"),
+ )
+
+ # Per RFC, the client is required to send a dummy ^A response.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+ assert pkt.payload == b"\x01"
+
+ # Now fail the SASL exchange.
+ pq3.send(
+ conn,
+ pq3.types.ErrorResponse,
+ fields=[
+ b"SFATAL",
+ b"C28000",
+ b"Mdoesn't matter",
+ b"",
+ ],
+ )
+
+ # The client will connect to us a second time, using the parameters we sent
+ # it.
+ sock, _ = accept()
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+@pytest.mark.parametrize(
+ "response,expected_error",
+ [
+ pytest.param(
+ "abcde",
+ 'Token "abcde" is invalid',
+ id="bad JSON: invalid syntax",
+ ),
+ pytest.param(
+ '"abcde"',
+ "top-level element must be an object",
+ id="bad JSON: top-level element is a string",
+ ),
+ pytest.param(
+ "[]",
+ "top-level element must be an object",
+ id="bad JSON: top-level element is an array",
+ ),
+ pytest.param(
+ "{}",
+ "server sent error response without a status",
+ id="bad JSON: no status member",
+ ),
+ pytest.param(
+ '{ "status": null }',
+ 'field "status" must be a string',
+ id="bad JSON: null status member",
+ ),
+ pytest.param(
+ '{ "status": 0 }',
+ 'field "status" must be a string',
+ id="bad JSON: int status member",
+ ),
+ pytest.param(
+ '{ "status": [ "bad" ] }',
+ 'field "status" must be a string',
+ id="bad JSON: array status member",
+ ),
+ pytest.param(
+ '{ "status": { "bad": "bad" } }',
+ 'field "status" must be a string',
+ id="bad JSON: object status member",
+ ),
+ pytest.param(
+ '{ "nested": { "status": "bad" } }',
+ "server sent error response without a status",
+ id="bad JSON: nested status",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token" ',
+ "The input string ended unexpectedly",
+ id="bad JSON: unterminated object",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token" } { }',
+ 'Expected end of input, but found "{"',
+ id="bad JSON: trailing data",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "openid-configuration": 1 }',
+ 'field "openid-configuration" must be a string',
+ id="bad JSON: int openid-configuration member",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "openid-configuration": 1 }',
+ 'field "openid-configuration" must be a string',
+ id="bad JSON: int openid-configuration member",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "scope": 1 }',
+ 'field "scope" must be a string',
+ id="bad JSON: int scope member",
+ ),
+ ],
+)
+def test_oauth_discovery_server_error(accept, response, expected_error):
+ sock, client = accept(oauth_client_id=secrets.token_hex())
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # Fail the SASL exchange with an invalid JSON response.
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASLContinue,
+ body=response.encode("utf-8"),
+ )
+
+ # The client should disconnect, so the socket is closed here. (If
+ # the client doesn't disconnect, it will report a different error
+ # below and the test will fail.)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "bad_response,expected_error",
+ [
+ pytest.param(
+ (200, {"Content-Type": "text/plain"}, {}),
+ r'failed to parse OpenID discovery document: unexpected content type "text/plain"',
+ id="not JSON",
+ ),
+ pytest.param(
+ (200, {}, {}),
+ r"failed to parse OpenID discovery document: no content type was provided",
+ id="no Content-Type",
+ ),
+ pytest.param(
+ (204, {}, None),
+ r"failed to fetch OpenID discovery document: unexpected response code 204",
+ id="no content",
+ ),
+ pytest.param(
+ (301, {"Location": "https://localhost/"}, None),
+ r"failed to fetch OpenID discovery document: unexpected response code 301",
+ id="redirection",
+ ),
+ pytest.param(
+ (404, {}),
+ r"failed to fetch OpenID discovery document: unexpected response code 404",
+ id="not found",
+ ),
+ pytest.param(
+ (200, RawResponse("blah\x00blah")),
+ r"failed to parse OpenID discovery document: response contains embedded NULLs",
+ id="NULL bytes in document",
+ ),
+ pytest.param(
+ (200, 123),
+ r"failed to parse OpenID discovery document: top-level element must be an object",
+ id="scalar at top level",
+ ),
+ pytest.param(
+ (200, []),
+ r"failed to parse OpenID discovery document: top-level element must be an object",
+ id="array at top level",
+ ),
+ pytest.param(
+ (200, RawResponse("{")),
+ r"failed to parse OpenID discovery document.* input string ended unexpectedly",
+ id="unclosed object",
+ ),
+ pytest.param(
+ (200, RawResponse(r'{ "hello": ] }')),
+ r"failed to parse OpenID discovery document.* Expected JSON value",
+ id="bad array",
+ ),
+ pytest.param(
+ (200, {"issuer": 123}),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="non-string issuer",
+ ),
+ pytest.param(
+ (200, {"issuer": ["something"]}),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="issuer array",
+ ),
+ pytest.param(
+ (200, {"issuer": {}}),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="issuer object",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": 123}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="scalar grant types field",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": {}}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="object grant types field",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": [123]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="non-string grant types",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": ["something", 123]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="non-string grant types later in the list",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": ["something", {}]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="object grant types later in the list",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": ["something", ["something"]]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="embedded array grant types later in the list",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "grant_types_supported": ["something"],
+ "token_endpoint": "https://example.com/",
+ "issuer": 123,
+ },
+ ),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="non-string issuer after other valid fields",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "ignored": {"grant_types_supported": 123, "token_endpoint": 123},
+ "issuer": 123,
+ },
+ ),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="non-string issuer after other ignored fields",
+ ),
+ pytest.param(
+ (200, {"token_endpoint": "https://example.com/"}),
+ r'failed to parse OpenID discovery document: field "issuer" is missing',
+ id="missing issuer",
+ ),
+ pytest.param(
+ (200, {"issuer": "https://example.com/"}),
+ r'failed to parse OpenID discovery document: field "token_endpoint" is missing',
+ id="missing token endpoint",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "issuer": "https://example.com",
+ "token_endpoint": "https://example.com/token",
+ "device_authorization_endpoint": "https://example.com/dev",
+ },
+ ),
+ r'cannot run OAuth device authorization: issuer "https://example.com" does not support device code grants',
+ id="missing device code grants",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "issuer": "https://example.com",
+ "token_endpoint": "https://example.com/token",
+ "grant_types_supported": [
+ "urn:ietf:params:oauth:grant-type:device_code"
+ ],
+ },
+ ),
+ r'cannot run OAuth device authorization: issuer "https://example.com" does not provide a device authorization endpoint',
+ id="missing device_authorization_endpoint",
+ ),
+ #
+ # Exercise HTTP-level failures by breaking the protocol. Note that the
+ # error messages here are implementation-dependent.
+ #
+ pytest.param(
+ (1000, {}),
+ r"failed to fetch OpenID discovery document: Unsupported protocol \(.*\)",
+ id="invalid HTTP response code",
+ ),
+ pytest.param(
+ (200, {"Content-Length": -1}, {}),
+ r"failed to fetch OpenID discovery document: Weird server reply \(.*Content-Length.*\)",
+ id="bad HTTP Content-Length",
+ ),
+ ],
+)
+def test_oauth_discovery_provider_failure(
+ accept, openid_provider, bad_response, expected_error
+):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=secrets.token_hex(),
+ )
+
+ def failing_discovery_handler(headers, params):
+ return bad_response
+
+ openid_provider.register_endpoint(
+ None,
+ "GET",
+ "/.well-known/openid-configuration",
+ failing_discovery_handler,
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # XXX iddawc doesn't differentiate...
+ expected_error = alt_patterns(
+ expected_error,
+ r"failed to fetch OpenID discovery document \(iddawc error I_ERROR(_PARAM)?\)",
+ )
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "sasl_err,resp_type,resp_payload,expected_error",
+ [
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.ErrorResponse,
+ dict(
+ fields=[b"SFATAL", b"C28000", b"Mexpected error message", b""],
+ ),
+ "expected error message",
+ id="standard server error: invalid_request",
+ ),
+ pytest.param(
+ {"status": "invalid_token"},
+ pq3.types.ErrorResponse,
+ dict(
+ fields=[b"SFATAL", b"C28000", b"Mexpected error message", b""],
+ ),
+ "expected error message",
+ id="standard server error: invalid_token without discovery URI",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASLContinue, body=b""),
+ "server sent additional OAuth data",
+ id="broken server: additional challenge after error",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASLFinal),
+ "server sent additional OAuth data",
+ id="broken server: SASL success after error",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASL, body=[b"OAUTHBEARER", b""]),
+ "duplicate SASL authentication request",
+ id="broken server: SASL reinitialization after error",
+ ),
+ ],
+)
+def test_oauth_server_error(accept, sasl_err, resp_type, resp_payload, expected_error):
+ sock, client = accept()
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ start_oauth_handshake(conn)
+
+ # Ignore the client data. Return an error "challenge".
+ resp = json.dumps(sasl_err)
+ resp = resp.encode("utf-8")
+
+ pq3.send(
+ conn, pq3.types.AuthnRequest, type=pq3.authn.SASLContinue, body=resp
+ )
+
+ # Per RFC, the client is required to send a dummy ^A response.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+ assert pkt.payload == b"\x01"
+
+ # Now fail the SASL exchange (in either a valid way, or an invalid
+ # one, depending on the test).
+ pq3.send(conn, resp_type, **resp_payload)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
diff --git a/src/test/python/conftest.py b/src/test/python/conftest.py
new file mode 100644
index 0000000000..71e167d7e7
--- /dev/null
+++ b/src/test/python/conftest.py
@@ -0,0 +1,37 @@
+#
+# Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import os
+
+import pytest
+
+
+def pytest_addoption(parser):
+ """
+ Adds custom command line options to py.test. We add one to signal temporary
+ Postgres instance creation for the server tests.
+
+ Per pytest documentation, this must live in the top level test directory.
+ """
+ parser.addoption(
+ "--temp-instance",
+ metavar="DIR",
+ help="create a temporary Postgres instance in DIR",
+ )
+
+
+@pytest.fixture(scope="session", autouse=True)
+def _check_PG_TEST_EXTRA(request):
+ """
+ Automatically skips the whole suite if PG_TEST_EXTRA doesn't contain
+ 'python'. pytestmark doesn't seem to work in a top-level conftest.py, so
+ I've made this an autoused fixture instead.
+
+ TODO: there are tests here that are probably safe, but until I do a full
+ analysis on which are and which are not, I've made the entire thing opt-in.
+ """
+ extra_tests = os.getenv("PG_TEST_EXTRA", "").split()
+ if "python" not in extra_tests:
+ pytest.skip("Potentially unsafe test 'python' not enabled in PG_TEST_EXTRA")
diff --git a/src/test/python/meson.build b/src/test/python/meson.build
new file mode 100644
index 0000000000..045526a43a
--- /dev/null
+++ b/src/test/python/meson.build
@@ -0,0 +1,46 @@
+# Copyright (c) 2023, PostgreSQL Global Development Group
+
+subdir('server')
+
+pytest_env = {
+ 'with_oauth': oauth_library,
+
+ # Point to the default database; the tests will create their own databases as
+ # needed.
+ 'PGDATABASE': 'postgres',
+
+ # Avoid the need for a Rust compiler on platforms without prebuilt wheels for
+ # pyca/cryptography.
+ 'CRYPTOGRAPHY_DONT_BUILD_RUST': '1',
+}
+
+# Some modules (psycopg2) need OpenSSL at compile time; for platforms where we
+# might have multiple implementations installed (macOS+brew), try to use the
+# same one that libpq is using.
+if ssl.found()
+ pytest_incdir = ssl.get_variable(pkgconfig: 'includedir', default_value: '')
+ if pytest_incdir != ''
+ pytest_env += { 'CPPFLAGS': '-I@0@'.format(pytest_incdir) }
+ endif
+
+ pytest_libdir = ssl.get_variable(pkgconfig: 'libdir', default_value: '')
+ if pytest_libdir != ''
+ pytest_env += { 'LDFLAGS': '-L@0@'.format(pytest_libdir) }
+ endif
+endif
+
+tests += {
+ 'name': 'python',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'pytest': {
+ 'requirements': meson.current_source_dir() / 'requirements.txt',
+ 'tests': [
+ './client',
+ './server',
+ './test_internals.py',
+ './test_pq3.py',
+ ],
+ 'env': pytest_env,
+ },
+}
diff --git a/src/test/python/pq3.py b/src/test/python/pq3.py
new file mode 100644
index 0000000000..b51ac96c71
--- /dev/null
+++ b/src/test/python/pq3.py
@@ -0,0 +1,728 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import getpass
+import io
+import os
+import ssl
+import sys
+import textwrap
+
+from construct import *
+
+import tls
+
+
+def protocol(major, minor):
+ """
+ Returns the protocol version, in integer format, corresponding to the given
+ major and minor version numbers.
+ """
+ return (major << 16) | minor
+
+
+# Startup
+
+StringList = GreedyRange(NullTerminated(GreedyBytes))
+
+
+class KeyValueAdapter(Adapter):
+ """
+ Turns a key-value store into a null-terminated list of null-terminated
+ strings, as presented on the wire in the startup packet.
+ """
+
+ def _encode(self, obj, context, path):
+ if isinstance(obj, list):
+ return obj
+
+ l = []
+
+ for k, v in obj.items():
+ if isinstance(k, str):
+ k = k.encode("utf-8")
+ l.append(k)
+
+ if isinstance(v, str):
+ v = v.encode("utf-8")
+ l.append(v)
+
+ l.append(b"")
+ return l
+
+ def _decode(self, obj, context, path):
+ # TODO: turn a list back into a dict
+ return obj
+
+
+KeyValues = KeyValueAdapter(StringList)
+
+_startup_payload = Switch(
+ this.proto,
+ {
+ protocol(3, 0): KeyValues,
+ },
+ default=GreedyBytes,
+)
+
+
+def _default_protocol(this):
+ try:
+ if isinstance(this.payload, (list, dict)):
+ return protocol(3, 0)
+ except AttributeError:
+ pass # no payload passed during build
+
+ return 0
+
+
+def _startup_payload_len(this):
+ """
+ The payload field has a fixed size based on the length of the packet. But
+ if the caller hasn't supplied an explicit length at build time, we have to
+ build the payload to figure out how long it is, which requires us to know
+ the length first... This function exists solely to break the cycle.
+ """
+ assert this._building, "_startup_payload_len() cannot be called during parsing"
+
+ try:
+ payload = this.payload
+ except AttributeError:
+ return 0 # no payload
+
+ if isinstance(payload, bytes):
+ # already serialized; just use the given length
+ return len(payload)
+
+ try:
+ proto = this.proto
+ except AttributeError:
+ proto = _default_protocol(this)
+
+ data = _startup_payload.build(payload, proto=proto)
+ return len(data)
+
+
+Startup = Struct(
+ "len" / Default(Int32sb, lambda this: _startup_payload_len(this) + 8),
+ "proto" / Default(Hex(Int32sb), _default_protocol),
+ "payload" / FixedSized(this.len - 8, Default(_startup_payload, b"")),
+)
+
+# Pq3
+
+
+# Adapted from construct.core.EnumIntegerString
+class EnumNamedByte:
+ def __init__(self, val, name):
+ self._val = val
+ self._name = name
+
+ def __int__(self):
+ return ord(self._val)
+
+ def __str__(self):
+ return "(enum) %s %r" % (self._name, self._val)
+
+ def __repr__(self):
+ return "EnumNamedByte(%r)" % self._val
+
+ def __eq__(self, other):
+ if isinstance(other, EnumNamedByte):
+ other = other._val
+ if not isinstance(other, bytes):
+ return NotImplemented
+
+ return self._val == other
+
+ def __hash__(self):
+ return hash(self._val)
+
+
+# Adapted from construct.core.Enum
+class ByteEnum(Adapter):
+ def __init__(self, **mapping):
+ super(ByteEnum, self).__init__(Byte)
+ self.namemapping = {k: EnumNamedByte(v, k) for k, v in mapping.items()}
+ self.decmapping = {v: EnumNamedByte(v, k) for k, v in mapping.items()}
+
+ def __getattr__(self, name):
+ if name in self.namemapping:
+ return self.decmapping[self.namemapping[name]]
+ raise AttributeError
+
+ def _decode(self, obj, context, path):
+ b = bytes([obj])
+ try:
+ return self.decmapping[b]
+ except KeyError:
+ return EnumNamedByte(b, "(unknown)")
+
+ def _encode(self, obj, context, path):
+ if isinstance(obj, int):
+ return obj
+ elif isinstance(obj, bytes):
+ return ord(obj)
+ return int(obj)
+
+
+types = ByteEnum(
+ ErrorResponse=b"E",
+ ReadyForQuery=b"Z",
+ Query=b"Q",
+ EmptyQueryResponse=b"I",
+ AuthnRequest=b"R",
+ PasswordMessage=b"p",
+ BackendKeyData=b"K",
+ CommandComplete=b"C",
+ ParameterStatus=b"S",
+ DataRow=b"D",
+ Terminate=b"X",
+)
+
+
+authn = Enum(
+ Int32ub,
+ OK=0,
+ SASL=10,
+ SASLContinue=11,
+ SASLFinal=12,
+)
+
+
+_authn_body = Switch(
+ this.type,
+ {
+ authn.OK: Terminated,
+ authn.SASL: StringList,
+ },
+ default=GreedyBytes,
+)
+
+
+def _data_len(this):
+ assert this._building, "_data_len() cannot be called during parsing"
+
+ if not hasattr(this, "data") or this.data is None:
+ return -1
+
+ return len(this.data)
+
+
+# The protocol reuses the PasswordMessage for several authentication response
+# types, and there's no good way to figure out which is which without keeping
+# state for the entire stream. So this is a separate Construct that can be
+# explicitly parsed/built by code that knows it's needed.
+SASLInitialResponse = Struct(
+ "name" / NullTerminated(GreedyBytes),
+ "len" / Default(Int32sb, lambda this: _data_len(this)),
+ "data"
+ / IfThenElse(
+ # Allow tests to explicitly pass an incorrect length during testing, by
+ # not enforcing a FixedSized during build. (The len calculation above
+ # defaults to the correct size.)
+ this._building,
+ Optional(GreedyBytes),
+ If(this.len != -1, Default(FixedSized(this.len, GreedyBytes), b"")),
+ ),
+ Terminated, # make sure the entire response is consumed
+)
+
+
+_column = FocusedSeq(
+ "data",
+ "len" / Default(Int32sb, lambda this: _data_len(this)),
+ "data" / If(this.len != -1, FixedSized(this.len, GreedyBytes)),
+)
+
+
+_payload_map = {
+ types.ErrorResponse: Struct("fields" / StringList),
+ types.ReadyForQuery: Struct("status" / Bytes(1)),
+ types.Query: Struct("query" / NullTerminated(GreedyBytes)),
+ types.EmptyQueryResponse: Terminated,
+ types.AuthnRequest: Struct("type" / authn, "body" / Default(_authn_body, b"")),
+ types.BackendKeyData: Struct("pid" / Int32ub, "key" / Hex(Int32ub)),
+ types.CommandComplete: Struct("tag" / NullTerminated(GreedyBytes)),
+ types.ParameterStatus: Struct(
+ "name" / NullTerminated(GreedyBytes), "value" / NullTerminated(GreedyBytes)
+ ),
+ types.DataRow: Struct("columns" / Default(PrefixedArray(Int16sb, _column), b"")),
+ types.Terminate: Terminated,
+}
+
+
+_payload = FocusedSeq(
+ "_payload",
+ "_payload"
+ / Switch(
+ this._.type,
+ _payload_map,
+ default=GreedyBytes,
+ ),
+ Terminated, # make sure every payload consumes the entire packet
+)
+
+
+def _payload_len(this):
+ """
+ See _startup_payload_len() for an explanation.
+ """
+ assert this._building, "_payload_len() cannot be called during parsing"
+
+ try:
+ payload = this.payload
+ except AttributeError:
+ return 0 # no payload
+
+ if isinstance(payload, bytes):
+ # already serialized; just use the given length
+ return len(payload)
+
+ data = _payload.build(payload, type=this.type)
+ return len(data)
+
+
+Pq3 = Struct(
+ "type" / types,
+ "len" / Default(Int32ub, lambda this: _payload_len(this) + 4),
+ "payload" / FixedSized(this.len - 4, Default(_payload, b"")),
+)
+
+
+# Environment
+
+
+def pghost():
+ return os.environ.get("PGHOST", default="localhost")
+
+
+def pgport():
+ return int(os.environ.get("PGPORT", default=5432))
+
+
+def pguser():
+ try:
+ return os.environ["PGUSER"]
+ except KeyError:
+ return getpass.getuser()
+
+
+def pgdatabase():
+ return os.environ.get("PGDATABASE", default="postgres")
+
+
+# Connections
+
+
+def _hexdump_translation_map():
+ """
+ For hexdumps. Translates any unprintable or non-ASCII bytes into '.'.
+ """
+ input = bytearray()
+
+ for i in range(128):
+ c = chr(i)
+
+ if not c.isprintable():
+ input += bytes([i])
+
+ input += bytes(range(128, 256))
+
+ return bytes.maketrans(input, b"." * len(input))
+
+
+class _DebugStream(object):
+ """
+ Wraps a file-like object and adds hexdumps of the read and write data. Call
+ end_packet() on a _DebugStream to write the accumulated hexdumps to the
+ output stream, along with the packet that was sent.
+ """
+
+ _translation_map = _hexdump_translation_map()
+
+ def __init__(self, stream, out=sys.stdout):
+ """
+ Creates a new _DebugStream wrapping the given stream (which must have
+ been created by wrap()). All attributes not provided by the _DebugStream
+ are delegated to the wrapped stream. out is the text stream to which
+ hexdumps are written.
+ """
+ self.raw = stream
+ self._out = out
+ self._rbuf = io.BytesIO()
+ self._wbuf = io.BytesIO()
+
+ def __getattr__(self, name):
+ return getattr(self.raw, name)
+
+ def __setattr__(self, name, value):
+ if name in ("raw", "_out", "_rbuf", "_wbuf"):
+ return object.__setattr__(self, name, value)
+
+ setattr(self.raw, name, value)
+
+ def read(self, *args, **kwargs):
+ buf = self.raw.read(*args, **kwargs)
+
+ self._rbuf.write(buf)
+ return buf
+
+ def write(self, b):
+ self._wbuf.write(b)
+ return self.raw.write(b)
+
+ def recv(self, *args):
+ buf = self.raw.recv(*args)
+
+ self._rbuf.write(buf)
+ return buf
+
+ def _flush(self, buf, prefix):
+ width = 16
+ hexwidth = width * 3 - 1
+
+ count = 0
+ buf.seek(0)
+
+ while True:
+ line = buf.read(16)
+
+ if not line:
+ if count:
+ self._out.write("\n") # separate the output block with a newline
+ return
+
+ self._out.write("%s %04X:\t" % (prefix, count))
+ self._out.write("%*s\t" % (-hexwidth, line.hex(" ")))
+ self._out.write(line.translate(self._translation_map).decode("ascii"))
+ self._out.write("\n")
+
+ count += 16
+
+ def print_debug(self, obj, *, prefix=""):
+ contents = ""
+ if obj is not None:
+ contents = str(obj)
+
+ for line in contents.splitlines():
+ self._out.write("%s%s\n" % (prefix, line))
+
+ self._out.write("\n")
+
+ def flush_debug(self, *, prefix=""):
+ self._flush(self._rbuf, prefix + "<")
+ self._rbuf = io.BytesIO()
+
+ self._flush(self._wbuf, prefix + ">")
+ self._wbuf = io.BytesIO()
+
+ def end_packet(self, pkt, *, read=False, prefix="", indent=" "):
+ """
+ Marks the end of a logical "packet" of data. A string representation of
+ pkt will be printed, and the debug buffers will be flushed with an
+ indent. All lines can be optionally prefixed.
+
+ If read is True, the packet representation is written after the debug
+ buffers; otherwise the default of False (meaning write) causes the
+ packet representation to be dumped first. This is meant to capture the
+ logical flow of layer translation.
+ """
+ write = not read
+
+ if write:
+ self.print_debug(pkt, prefix=prefix + "> ")
+
+ self.flush_debug(prefix=prefix + indent)
+
+ if read:
+ self.print_debug(pkt, prefix=prefix + "< ")
+
+
+@contextlib.contextmanager
+def wrap(socket, *, debug_stream=None):
+ """
+ Transforms a raw socket into a connection that can be used for Construct
+ building and parsing. The return value is a context manager and can be used
+ in a with statement.
+ """
+ # It is critical that buffering be disabled here, so that we can still
+ # manipulate the raw socket without desyncing the stream.
+ with socket.makefile("rwb", buffering=0) as sfile:
+ # Expose the original socket's recv() on the SocketIO object we return.
+ def recv(self, *args):
+ return socket.recv(*args)
+
+ sfile.recv = recv.__get__(sfile)
+
+ conn = sfile
+ if debug_stream:
+ conn = _DebugStream(conn, debug_stream)
+
+ try:
+ yield conn
+ finally:
+ if debug_stream:
+ conn.flush_debug(prefix="? ")
+
+
+def _send(stream, cls, obj):
+ debugging = hasattr(stream, "flush_debug")
+ out = io.BytesIO()
+
+ # Ideally we would build directly to the passed stream, but because we need
+ # to reparse the generated output for the debugging case, build to an
+ # intermediate BytesIO and send it instead.
+ cls.build_stream(obj, out)
+ buf = out.getvalue()
+
+ stream.write(buf)
+ if debugging:
+ pkt = cls.parse(buf)
+ stream.end_packet(pkt)
+
+ stream.flush()
+
+
+def send(stream, packet_type, payload_data=None, **payloadkw):
+ """
+ Sends a packet on the given pq3 connection. type is the pq3.types member
+ that should be assigned to the packet. If payload_data is given, it will be
+ used as the packet payload; otherwise the key/value pairs in payloadkw will
+ be the payload contents.
+ """
+ data = payloadkw
+
+ if payload_data is not None:
+ if payloadkw:
+ raise ValueError(
+ "payload_data and payload keywords may not be used simultaneously"
+ )
+
+ data = payload_data
+
+ _send(stream, Pq3, dict(type=packet_type, payload=data))
+
+
+def send_startup(stream, proto=None, **kwargs):
+ """
+ Sends a startup packet on the given pq3 connection. In most cases you should
+ use the handshake functions instead, which will do this for you.
+
+ By default, a protocol version 3 packet will be sent. This can be overridden
+ with the proto parameter.
+ """
+ pkt = {}
+
+ if proto is not None:
+ pkt["proto"] = proto
+ if kwargs:
+ pkt["payload"] = kwargs
+
+ _send(stream, Startup, pkt)
+
+
+def recv1(stream, *, cls=Pq3):
+ """
+ Receives a single pq3 packet from the given stream and returns it.
+ """
+ resp = cls.parse_stream(stream)
+
+ debugging = hasattr(stream, "flush_debug")
+ if debugging:
+ stream.end_packet(resp, read=True)
+
+ return resp
+
+
+def handshake(stream, **kwargs):
+ """
+ Performs a libpq v3 startup handshake. kwargs should contain the key/value
+ parameters to send to the server in the startup packet.
+ """
+ # Send our startup parameters.
+ send_startup(stream, **kwargs)
+
+ # Receive and dump packets until the server indicates it's ready for our
+ # first query.
+ while True:
+ resp = recv1(stream)
+ if resp is None:
+ raise RuntimeError("server closed connection during handshake")
+
+ if resp.type == types.ReadyForQuery:
+ return
+ elif resp.type == types.ErrorResponse:
+ raise RuntimeError(
+ f"received error response from peer: {resp.payload.fields!r}"
+ )
+
+
+# TLS
+
+
+class _TLSStream(object):
+ """
+ A file-like object that performs TLS encryption/decryption on a wrapped
+ stream. Differs from ssl.SSLSocket in that we have full visibility and
+ control over the TLS layer.
+ """
+
+ def __init__(self, stream, context):
+ self._stream = stream
+ self._debugging = hasattr(stream, "flush_debug")
+
+ self._in = ssl.MemoryBIO()
+ self._out = ssl.MemoryBIO()
+ self._ssl = context.wrap_bio(self._in, self._out)
+
+ def handshake(self):
+ try:
+ self._pump(lambda: self._ssl.do_handshake())
+ finally:
+ self._flush_debug(prefix="? ")
+
+ def read(self, *args):
+ return self._pump(lambda: self._ssl.read(*args))
+
+ def write(self, *args):
+ return self._pump(lambda: self._ssl.write(*args))
+
+ def _decode(self, buf):
+ """
+ Attempts to decode a buffer of TLS data into a packet representation
+ that can be printed.
+
+ TODO: handle buffers (and record fragments) that don't align with packet
+ boundaries.
+ """
+ end = len(buf)
+ bio = io.BytesIO(buf)
+
+ ret = io.StringIO()
+
+ while bio.tell() < end:
+ record = tls.Plaintext.parse_stream(bio)
+
+ if ret.tell() > 0:
+ ret.write("\n")
+ ret.write("[Record] ")
+ ret.write(str(record))
+ ret.write("\n")
+
+ if record.type == tls.ContentType.handshake:
+ record_cls = tls.Handshake
+ else:
+ continue
+
+ innerlen = len(record.fragment)
+ inner = io.BytesIO(record.fragment)
+
+ while inner.tell() < innerlen:
+ msg = record_cls.parse_stream(inner)
+
+ indented = "[Message] " + str(msg)
+ indented = textwrap.indent(indented, " ")
+
+ ret.write("\n")
+ ret.write(indented)
+ ret.write("\n")
+
+ return ret.getvalue()
+
+ def flush(self):
+ if not self._out.pending:
+ self._stream.flush()
+ return
+
+ buf = self._out.read()
+ self._stream.write(buf)
+
+ if self._debugging:
+ pkt = self._decode(buf)
+ self._stream.end_packet(pkt, prefix=" ")
+
+ self._stream.flush()
+
+ def _pump(self, operation):
+ while True:
+ try:
+ return operation()
+ except (ssl.SSLWantReadError, ssl.SSLWantWriteError) as e:
+ want = e
+ self._read_write(want)
+
+ def _recv(self, maxsize):
+ buf = self._stream.recv(4096)
+ if not buf:
+ self._in.write_eof()
+ return
+
+ self._in.write(buf)
+
+ if not self._debugging:
+ return
+
+ pkt = self._decode(buf)
+ self._stream.end_packet(pkt, read=True, prefix=" ")
+
+ def _read_write(self, want):
+ # XXX This needs work. So many corner cases yet to handle. For one,
+ # doing blocking writes in flush may lead to distributed deadlock if the
+ # peer is already blocking on its writes.
+
+ if isinstance(want, ssl.SSLWantWriteError):
+ assert self._out.pending, "SSL backend wants write without data"
+
+ self.flush()
+
+ if isinstance(want, ssl.SSLWantReadError):
+ self._recv(4096)
+
+ def _flush_debug(self, prefix):
+ if not self._debugging:
+ return
+
+ self._stream.flush_debug(prefix=prefix)
+
+
+@contextlib.contextmanager
+def tls_handshake(stream, context):
+ """
+ Performs a TLS handshake over the given stream (which must have been created
+ via a call to wrap()), and returns a new stream which transparently tunnels
+ data over the TLS connection.
+
+ If the passed stream has debugging enabled, the returned stream will also
+ have debugging, using the same output IO.
+ """
+ debugging = hasattr(stream, "flush_debug")
+
+ # Send our startup parameters.
+ send_startup(stream, proto=protocol(1234, 5679))
+
+ # Look at the SSL response.
+ resp = stream.read(1)
+ if debugging:
+ stream.flush_debug(prefix=" ")
+
+ if resp == b"N":
+ raise RuntimeError("server does not support SSLRequest")
+ if resp != b"S":
+ raise RuntimeError(f"unexpected response of type {resp!r} during TLS startup")
+
+ tls = _TLSStream(stream, context)
+ tls.handshake()
+
+ if debugging:
+ tls = _DebugStream(tls, stream._out)
+
+ try:
+ yield tls
+ # TODO: teardown/unwrap the connection?
+ finally:
+ if debugging:
+ tls.flush_debug(prefix="? ")
diff --git a/src/test/python/pytest.ini b/src/test/python/pytest.ini
new file mode 100644
index 0000000000..ab7a6e7fb9
--- /dev/null
+++ b/src/test/python/pytest.ini
@@ -0,0 +1,4 @@
+[pytest]
+
+markers =
+ slow: mark test as slow
diff --git a/src/test/python/requirements.txt b/src/test/python/requirements.txt
new file mode 100644
index 0000000000..57ba1ced94
--- /dev/null
+++ b/src/test/python/requirements.txt
@@ -0,0 +1,9 @@
+black
+# cryptography 35.x and later add many platform/toolchain restrictions, beware
+cryptography~=3.4.8
+construct~=2.10.61
+isort~=5.6
+# TODO: update to psycopg[c] 3.1
+psycopg2~=2.9.7
+pytest~=7.3
+pytest-asyncio~=0.21.0
diff --git a/src/test/python/server/__init__.py b/src/test/python/server/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/src/test/python/server/conftest.py b/src/test/python/server/conftest.py
new file mode 100644
index 0000000000..c89f0756ae
--- /dev/null
+++ b/src/test/python/server/conftest.py
@@ -0,0 +1,141 @@
+#
+# Portions Copyright 2021 VMware, Inc.
+# Portions Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import collections
+import contextlib
+import os
+import shutil
+import socket
+import subprocess
+import sys
+
+import pytest
+
+import pq3
+
+BLOCKING_TIMEOUT = 2 # the number of seconds to wait for blocking calls
+
+
+def cleanup_prior_instance(datadir):
+ """
+ Clean up an existing data directory, but make sure it actually looks like a
+ data directory first. (Empty folders will remain untouched, since initdb can
+ populate them.)
+ """
+ required_entries = set(["base", "PG_VERSION", "postgresql.conf"])
+ empty = True
+
+ try:
+ with os.scandir(datadir) as entries:
+ for e in entries:
+ empty = False
+ required_entries.discard(e.name)
+
+ except FileNotFoundError:
+ return # nothing to clean up
+
+ if empty:
+ return # initdb can handle an empty datadir
+
+ if required_entries:
+ pytest.fail(
+ f"--temp-instance directory \"{datadir}\" is not empty and doesn't look like a data directory (missing {', '.join(required_entries)})"
+ )
+
+ # Okay, seems safe enough now.
+ shutil.rmtree(datadir)
+
+
+@pytest.fixture(scope="session")
+def postgres_instance(pytestconfig, unused_tcp_port_factory):
+ """
+ If --temp-instance has been passed to pytest, this fixture runs a temporary
+ Postgres instance on an available port. Otherwise, the fixture will attempt
+ to contact a running Postgres server on (PGHOST, PGPORT); dependent tests
+ will be skipped if the connection fails.
+
+ Yields a (host, port) tuple for connecting to the server.
+ """
+ PGInstance = collections.namedtuple("PGInstance", ["addr", "temporary"])
+
+ datadir = pytestconfig.getoption("temp_instance")
+ if datadir:
+ # We were told to create a temporary instance. Use pg_ctl to set it up
+ # on an unused port.
+ cleanup_prior_instance(datadir)
+ subprocess.run(["pg_ctl", "-D", datadir, "init"], check=True)
+
+ # The CI looks for *.log files to upload, so the file name here isn't
+ # completely arbitrary.
+ log = os.path.join(datadir, "postmaster.log")
+ port = unused_tcp_port_factory()
+
+ subprocess.run(
+ [
+ "pg_ctl",
+ "-D",
+ datadir,
+ "-l",
+ log,
+ "-o",
+ " ".join(
+ [
+ f"-c port={port}",
+ "-c listen_addresses=localhost",
+ "-c log_connections=on",
+ "-c shared_preload_libraries=oauthtest",
+ "-c oauth_validator_library=oauthtest",
+ ]
+ ),
+ "start",
+ ],
+ check=True,
+ )
+
+ yield ("localhost", port)
+
+ subprocess.run(["pg_ctl", "-D", datadir, "stop"], check=True)
+
+ else:
+ # Try to contact an already running server; skip the suite if we can't
+ # find one.
+ addr = (pq3.pghost(), pq3.pgport())
+
+ try:
+ with socket.create_connection(addr, timeout=BLOCKING_TIMEOUT):
+ pass
+ except ConnectionError as e:
+ pytest.skip(f"unable to connect to Postgres server at {addr}: {e}")
+
+ yield addr
+
+
+@pytest.fixture
+def connect(postgres_instance):
+ """
+ A factory fixture that, when called, returns a socket connected to a
+ Postgres server, wrapped in a pq3 connection. Dependent tests will be
+ skipped if no server is available.
+ """
+ addr = postgres_instance
+
+ # Set up an ExitStack to handle safe cleanup of all of the moving pieces.
+ with contextlib.ExitStack() as stack:
+
+ def conn_factory():
+ sock = socket.create_connection(addr, timeout=BLOCKING_TIMEOUT)
+
+ # Have ExitStack close our socket.
+ stack.enter_context(sock)
+
+ # Wrap the connection in a pq3 layer and have ExitStack clean it up
+ # too.
+ wrap_ctx = pq3.wrap(sock, debug_stream=sys.stdout)
+ conn = stack.enter_context(wrap_ctx)
+
+ return conn
+
+ yield conn_factory
diff --git a/src/test/python/server/meson.build b/src/test/python/server/meson.build
new file mode 100644
index 0000000000..7946c971cb
--- /dev/null
+++ b/src/test/python/server/meson.build
@@ -0,0 +1,22 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+if not oauth.found()
+ subdir_done()
+endif
+
+oauthtest_sources = files(
+ 'oauthtest.c',
+)
+
+if host_system == 'windows'
+ oauthtest_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'oauthtest',
+ '--FILEDESC', 'passthrough module to validate OAuth tests',
+ ])
+endif
+
+oauthtest = shared_module('oauthtest',
+ oauthtest_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += oauthtest
diff --git a/src/test/python/server/oauthtest.c b/src/test/python/server/oauthtest.c
new file mode 100644
index 0000000000..f8e6c1651b
--- /dev/null
+++ b/src/test/python/server/oauthtest.c
@@ -0,0 +1,118 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauthtest.c
+ * Test module for serverside OAuth token validation callbacks
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/test/python/server/oauthtest.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "libpq/oauth.h"
+#include "utils/guc.h"
+#include "utils/memutils.h"
+
+PG_MODULE_MAGIC;
+
+static void test_startup(ValidatorModuleState *state);
+static void test_shutdown(ValidatorModuleState *state);
+static ValidatorModuleResult * test_validate(ValidatorModuleState *state,
+ const char *token,
+ const char *role);
+
+static const OAuthValidatorCallbacks callbacks = {
+ .startup_cb = test_startup,
+ .shutdown_cb = test_shutdown,
+ .validate_cb = test_validate,
+};
+
+static char *expected_bearer = "";
+static bool set_authn_id = false;
+static char *authn_id = "";
+static bool reflect_role = false;
+
+void
+_PG_init(void)
+{
+ DefineCustomStringVariable("oauthtest.expected_bearer",
+ "Expected Bearer token for future connections",
+ NULL,
+ &expected_bearer,
+ "",
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("oauthtest.set_authn_id",
+ "Whether to set an authenticated identity",
+ NULL,
+ &set_authn_id,
+ false,
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+ DefineCustomStringVariable("oauthtest.authn_id",
+ "Authenticated identity to use for future connections",
+ NULL,
+ &authn_id,
+ "",
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("oauthtest.reflect_role",
+ "Ignore the bearer token; use the requested role as the authn_id",
+ NULL,
+ &reflect_role,
+ false,
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+
+ MarkGUCPrefixReserved("oauthtest");
+}
+
+const OAuthValidatorCallbacks *
+_PG_oauth_validator_module_init(void)
+{
+ return &callbacks;
+}
+
+static void
+test_startup(ValidatorModuleState *state)
+{
+}
+
+static void
+test_shutdown(ValidatorModuleState *state)
+{
+}
+
+static ValidatorModuleResult *
+test_validate(ValidatorModuleState *state, const char *token, const char *role)
+{
+ ValidatorModuleResult *res;
+
+ res = palloc0(sizeof(ValidatorModuleResult)); /* TODO: palloc context? */
+
+ if (reflect_role)
+ {
+ res->authenticated = true;
+ res->authn_id = pstrdup(role); /* TODO: constify? */
+ }
+ else
+ {
+ if (*expected_bearer && !strcmp(token, expected_bearer))
+ res->authenticated = true;
+ if (set_authn_id)
+ res->authn_id = authn_id;
+ }
+
+ return res;
+}
diff --git a/src/test/python/server/test_oauth.py b/src/test/python/server/test_oauth.py
new file mode 100644
index 0000000000..96f5e1e1b5
--- /dev/null
+++ b/src/test/python/server/test_oauth.py
@@ -0,0 +1,939 @@
+#
+# Copyright 2021 VMware, Inc.
+# Portions Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import contextlib
+import json
+import os
+import pathlib
+import secrets
+import shlex
+import shutil
+import socket
+import struct
+from multiprocessing import shared_memory
+
+import psycopg2
+import pytest
+from psycopg2 import sql
+
+import pq3
+
+from .conftest import BLOCKING_TIMEOUT
+
+MAX_SASL_MESSAGE_LENGTH = 65535
+
+INVALID_AUTHORIZATION_ERRCODE = b"28000"
+PROTOCOL_VIOLATION_ERRCODE = b"08P01"
+FEATURE_NOT_SUPPORTED_ERRCODE = b"0A000"
+
+SHARED_MEM_NAME = "oauth-pytest"
+MAX_UINT16 = 2**16 - 1
+
+
+@contextlib.contextmanager
+def prepend_file(path, lines):
+ """
+ A context manager that prepends a file on disk with the desired lines of
+ text. When the context manager is exited, the file will be restored to its
+ original contents.
+ """
+ # First make a backup of the original file.
+ bak = path + ".bak"
+ shutil.copy2(path, bak)
+
+ try:
+ # Write the new lines, followed by the original file content.
+ with open(path, "w") as new, open(bak, "r") as orig:
+ new.writelines(lines)
+ shutil.copyfileobj(orig, new)
+
+ # Return control to the calling code.
+ yield
+
+ finally:
+ # Put the backup back into place.
+ os.replace(bak, path)
+
+
+@pytest.fixture(scope="module")
+def oauth_ctx(postgres_instance):
+ """
+ Creates a database and user that use the oauth auth method. The context
+ object contains the dbname and user attributes as strings to be used during
+ connection, as well as the issuer and scope that have been set in the HBA
+ configuration.
+
+ This fixture assumes that the standard PG* environment variables point to a
+ server running on a local machine, and that the PGUSER has rights to create
+ databases and roles.
+ """
+ id = secrets.token_hex(4)
+
+ class Context:
+ dbname = "oauth_test_" + id
+
+ user = "oauth_user_" + id
+ punct_user = "oauth_\"'? ;&!_user_" + id # username w/ punctuation
+ map_user = "oauth_map_user_" + id
+ authz_user = "oauth_authz_user_" + id
+
+ issuer = "https://example.com/" + id
+ scope = "openid " + id
+
+ ctx = Context()
+ hba_lines = (
+ f'host {ctx.dbname} {ctx.map_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" map=oauth\n',
+ f'host {ctx.dbname} {ctx.authz_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" trust_validator_authz=1\n',
+ f'host {ctx.dbname} all samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}"\n',
+ )
+ ident_lines = (r"oauth /^(.*)@example\.com$ \1",)
+
+ host, port = postgres_instance
+ conn = psycopg2.connect(host=host, port=port)
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+
+ # Create our roles and database.
+ user = sql.Identifier(ctx.user)
+ punct_user = sql.Identifier(ctx.punct_user)
+ map_user = sql.Identifier(ctx.map_user)
+ authz_user = sql.Identifier(ctx.authz_user)
+ dbname = sql.Identifier(ctx.dbname)
+
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(punct_user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(map_user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(authz_user))
+ c.execute(sql.SQL("CREATE DATABASE {};").format(dbname))
+
+ # Replace pg_hba and pg_ident.
+ c.execute("SHOW hba_file;")
+ hba = c.fetchone()[0]
+
+ c.execute("SHOW ident_file;")
+ ident = c.fetchone()[0]
+
+ with prepend_file(hba, hba_lines), prepend_file(ident, ident_lines):
+ c.execute("SELECT pg_reload_conf();")
+
+ # Use the new database and user.
+ yield ctx
+
+ # Put things back the way they were.
+ c.execute("SELECT pg_reload_conf();")
+
+ c.execute(sql.SQL("DROP DATABASE {};").format(dbname))
+ c.execute(sql.SQL("DROP ROLE {};").format(authz_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(map_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(punct_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(user))
+
+
+@pytest.fixture()
+def conn(oauth_ctx, connect):
+ """
+ A convenience wrapper for connect(). The main purpose of this fixture is to
+ make sure oauth_ctx runs its setup code before the connection is made.
+ """
+ return connect()
+
+
+def bearer_token(*, size=16):
+ """
+ Generates a Bearer token using secrets.token_urlsafe(). The generated token
+ size in bytes may be specified; if unset, a small 16-byte token will be
+ generated.
+ """
+
+ if size % 4:
+ raise ValueError(f"requested token size {size} is not a multiple of 4")
+
+ token = secrets.token_urlsafe(size // 4 * 3)
+ assert len(token) == size
+
+ return token
+
+
+def begin_oauth_handshake(conn, oauth_ctx, *, user=None):
+ if user is None:
+ user = oauth_ctx.authz_user
+
+ pq3.send_startup(conn, user=user, database=oauth_ctx.dbname)
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.AuthnRequest
+
+ # The server should advertise exactly one mechanism.
+ assert resp.payload.type == pq3.authn.SASL
+ assert resp.payload.body == [b"OAUTHBEARER", b""]
+
+
+def send_initial_response(conn, *, auth=None, bearer=None):
+ """
+ Sends the OAUTHBEARER initial response on the connection, using the given
+ bearer token. Alternatively to a bearer token, the initial response's auth
+ field may be explicitly specified to test corner cases.
+ """
+ if bearer is not None and auth is not None:
+ raise ValueError("exactly one of the auth and bearer kwargs must be set")
+
+ if bearer is not None:
+ auth = b"Bearer " + bearer
+
+ if auth is None:
+ raise ValueError("exactly one of the auth and bearer kwargs must be set")
+
+ initial = pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"n,,\x01auth=" + auth + b"\x01\x01",
+ )
+ )
+ pq3.send(conn, pq3.types.PasswordMessage, initial)
+
+
+def expect_handshake_success(conn):
+ """
+ Validates that the server responds with an AuthnOK message, and then drains
+ the connection until a ReadyForQuery message is received.
+ """
+ resp = pq3.recv1(conn)
+
+ assert resp.type == pq3.types.AuthnRequest
+ assert resp.payload.type == pq3.authn.OK
+ assert not resp.payload.body
+
+ receive_until(conn, pq3.types.ReadyForQuery)
+
+
+def expect_handshake_failure(conn, oauth_ctx):
+ """
+ Performs the OAUTHBEARER SASL failure "handshake" and validates the server's
+ side of the conversation, including the final ErrorResponse.
+ """
+
+ # We expect a discovery "challenge" back from the server before the authn
+ # failure message.
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.AuthnRequest
+
+ req = resp.payload
+ assert req.type == pq3.authn.SASLContinue
+
+ body = json.loads(req.body)
+ assert body["status"] == "invalid_token"
+ assert body["scope"] == oauth_ctx.scope
+
+ expected_config = oauth_ctx.issuer + "/.well-known/openid-configuration"
+ assert body["openid-configuration"] == expected_config
+
+ # Send the dummy response to complete the failed handshake.
+ pq3.send(conn, pq3.types.PasswordMessage, b"\x01")
+ resp = pq3.recv1(conn)
+
+ err = ExpectedError(INVALID_AUTHORIZATION_ERRCODE, "bearer authentication failed")
+ err.match(resp)
+
+
+def receive_until(conn, type):
+ """
+ receive_until pulls packets off the pq3 connection until a packet with the
+ desired type is found, or an error response is received.
+ """
+ while True:
+ pkt = pq3.recv1(conn)
+
+ if pkt.type == type:
+ return pkt
+ elif pkt.type == pq3.types.ErrorResponse:
+ raise RuntimeError(
+ f"received error response from peer: {pkt.payload.fields!r}"
+ )
+
+
+@pytest.fixture()
+def setup_validator(postgres_instance):
+ """
+ A per-test fixture that sets up the test validator with expected behavior.
+ The setting will be reverted during teardown.
+ """
+ host, port = postgres_instance
+ conn = psycopg2.connect(host=host, port=port)
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+ prev = dict()
+
+ def setter(**gucs):
+ for guc, val in gucs.items():
+ # Save the previous value.
+ c.execute(sql.SQL("SHOW oauthtest.{};").format(sql.Identifier(guc)))
+ prev[guc] = c.fetchone()[0]
+
+ c.execute(
+ sql.SQL("ALTER SYSTEM SET oauthtest.{} TO %s;").format(
+ sql.Identifier(guc)
+ ),
+ (val,),
+ )
+ c.execute("SELECT pg_reload_conf();")
+
+ yield setter
+
+ # Restore the previous values.
+ for guc, val in prev.items():
+ c.execute(
+ sql.SQL("ALTER SYSTEM SET oauthtest.{} TO %s;").format(
+ sql.Identifier(guc)
+ ),
+ (val,),
+ )
+ c.execute("SELECT pg_reload_conf();")
+
+
+@pytest.mark.parametrize("token_len", [16, 1024, 4096])
+@pytest.mark.parametrize(
+ "auth_prefix",
+ [
+ b"Bearer ",
+ b"bearer ",
+ b"Bearer ",
+ ],
+)
+def test_oauth(setup_validator, connect, oauth_ctx, auth_prefix, token_len):
+ # Generate our bearer token with the desired length.
+ token = bearer_token(size=token_len)
+ setup_validator(expected_bearer=token)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ auth = auth_prefix + token.encode("ascii")
+ send_initial_response(conn, auth=auth)
+ expect_handshake_success(conn)
+
+ # Make sure that the server has not set an authenticated ID.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT system_user;")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ assert row.columns == [None]
+
+
+@pytest.mark.parametrize(
+ "token_value",
+ [
+ "abcdzA==",
+ "123456M=",
+ "x-._~+/x",
+ ],
+)
+def test_oauth_bearer_corner_cases(setup_validator, connect, oauth_ctx, token_value):
+ setup_validator(expected_bearer=token_value)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ send_initial_response(conn, bearer=token_value.encode("ascii"))
+
+ expect_handshake_success(conn)
+
+
+@pytest.mark.parametrize(
+ "user,authn_id,should_succeed",
+ [
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: ctx.user,
+ True,
+ id="validator authn: succeeds when authn_id == username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: None,
+ False,
+ id="validator authn: fails when authn_id is not set",
+ ),
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: ctx.authz_user,
+ False,
+ id="validator authn: fails when authn_id != username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: ctx.map_user + "@example.com",
+ True,
+ id="validator with map: succeeds when authn_id matches map",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: None,
+ False,
+ id="validator with map: fails when authn_id is not set",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: ctx.map_user + "@example.net",
+ False,
+ id="validator with map: fails when authn_id doesn't match map",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: None,
+ True,
+ id="validator authz: succeeds with no authn_id",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "",
+ True,
+ id="validator authz: succeeds with empty authn_id",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "postgres",
+ True,
+ id="validator authz: succeeds with basic username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "me@example.com",
+ True,
+ id="validator authz: succeeds with email address",
+ ),
+ ],
+)
+def test_oauth_authn_id(
+ setup_validator, connect, oauth_ctx, user, authn_id, should_succeed
+):
+ token = bearer_token()
+ authn_id = authn_id(oauth_ctx)
+
+ # Set up the validator appropriately.
+ gucs = dict(expected_bearer=token)
+ if authn_id is not None:
+ gucs["set_authn_id"] = True
+ gucs["authn_id"] = authn_id
+ setup_validator(**gucs)
+
+ conn = connect()
+ username = user(oauth_ctx)
+ begin_oauth_handshake(conn, oauth_ctx, user=username)
+ send_initial_response(conn, bearer=token.encode("ascii"))
+
+ if not should_succeed:
+ expect_handshake_failure(conn, oauth_ctx)
+ return
+
+ expect_handshake_success(conn)
+
+ # Check the reported authn_id.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT system_user;")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ expected = authn_id
+ if expected is not None:
+ expected = b"oauth:" + expected.encode("ascii")
+
+ row = resp.payload
+ assert row.columns == [expected]
+
+
+class ExpectedError(object):
+ def __init__(self, code, msg=None, detail=None):
+ self.code = code
+ self.msg = msg
+ self.detail = detail
+
+ # Protect against the footgun of an accidental empty string, which will
+ # "match" anything. If you don't want to match message or detail, just
+ # don't pass them.
+ if self.msg == "":
+ raise ValueError("msg must be non-empty or None")
+ if self.detail == "":
+ raise ValueError("detail must be non-empty or None")
+
+ def _getfield(self, resp, type):
+ """
+ Searches an ErrorResponse for a single field of the given type (e.g.
+ "M", "C", "D") and returns its value. Asserts if it doesn't find exactly
+ one field.
+ """
+ prefix = type.encode("ascii")
+ fields = [f for f in resp.payload.fields if f.startswith(prefix)]
+
+ assert len(fields) == 1
+ return fields[0][1:] # strip off the type byte
+
+ def match(self, resp):
+ """
+ Checks that the given response matches the expected code, message, and
+ detail (if given). The error code must match exactly. The expected
+ message and detail must be contained within the actual strings.
+ """
+ assert resp.type == pq3.types.ErrorResponse
+
+ code = self._getfield(resp, "C")
+ assert code == self.code
+
+ if self.msg:
+ msg = self._getfield(resp, "M")
+ expected = self.msg.encode("utf-8")
+ assert expected in msg
+
+ if self.detail:
+ detail = self._getfield(resp, "D")
+ expected = self.detail.encode("utf-8")
+ assert expected in detail
+
+
+def test_oauth_rejected_bearer(conn, oauth_ctx):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send a bearer token that doesn't match what the validator expects. It
+ # should fail the connection.
+ send_initial_response(conn, bearer=b"xxxxxx")
+
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.parametrize(
+ "bad_bearer",
+ [
+ b"Bearer ",
+ b"Bearer a===b",
+ b"Bearer hello!",
+ b"Bearer trailingspace ",
+ b"Bearer trailingtab\t",
+ b"Bearer me@example.com",
+ b"Beare abcd",
+ b'OAuth realm="Example"',
+ b"",
+ ],
+)
+def test_oauth_invalid_bearer(setup_validator, connect, oauth_ctx, bad_bearer):
+ # Tell the validator to accept any token. This ensures that the invalid
+ # bearer tokens are rejected before the validation step.
+ setup_validator(reflect_role=True)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+ send_initial_response(conn, auth=bad_bearer)
+
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.slow
+@pytest.mark.parametrize(
+ "resp_type,resp,err",
+ [
+ pytest.param(
+ None,
+ None,
+ None,
+ marks=pytest.mark.slow,
+ id="no response (expect timeout)",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"hello",
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not send a kvsep response",
+ ),
+ id="bad dummy response",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"\x01\x01",
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not send a kvsep response",
+ ),
+ id="multiple kvseps",
+ ),
+ pytest.param(
+ pq3.types.Query,
+ dict(query=b""),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "expected SASL response"),
+ id="bad response message type",
+ ),
+ ],
+)
+def test_oauth_bad_response_to_error_challenge(conn, oauth_ctx, resp_type, resp, err):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send an empty auth initial response, which will force an authn failure.
+ send_initial_response(conn, auth=b"")
+
+ # We expect a discovery "challenge" back from the server before the authn
+ # failure message.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.AuthnRequest
+
+ req = pkt.payload
+ assert req.type == pq3.authn.SASLContinue
+
+ body = json.loads(req.body)
+ assert body["status"] == "invalid_token"
+
+ if resp_type is None:
+ # Do not send the dummy response. We should time out and not get a
+ # response from the server.
+ with pytest.raises(socket.timeout):
+ conn.read(1)
+
+ # Done with the test.
+ return
+
+ # Send the bad response.
+ pq3.send(conn, resp_type, resp)
+
+ # Make sure the server fails the connection correctly.
+ pkt = pq3.recv1(conn)
+ err.match(pkt)
+
+
+@pytest.mark.parametrize(
+ "type,payload,err",
+ [
+ pytest.param(
+ pq3.types.ErrorResponse,
+ dict(fields=[b""]),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "expected SASL response"),
+ id="error response in initial message",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"x" * (MAX_SASL_MESSAGE_LENGTH + 1),
+ ExpectedError(
+ INVALID_AUTHORIZATION_ERRCODE, "bearer authentication failed"
+ ),
+ id="overlong initial response data",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"SCRAM-SHA-256")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE, "invalid SASL authentication mechanism"
+ ),
+ id="bad SASL mechanism selection",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", len=2, data=b"x")),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "insufficient data"),
+ id="SASL data underflow",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", len=0, data=b"x")),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "invalid message format"),
+ id="SASL data overflow",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "message is empty",
+ ),
+ id="empty",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"n,,\x01auth=\x01\x01\0")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "length does not match input length",
+ ),
+ id="contains null byte",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected channel-binding flag", # XXX this is a bit strange
+ ),
+ id="initial error response",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"p=tls-server-end-point,,\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "server does not support channel binding",
+ ),
+ id="uses channel binding",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"x,,\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected channel-binding flag",
+ ),
+ id="invalid channel binding specifier",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Comma expected",
+ ),
+ id="bad GS2 header: missing channel binding terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,a")),
+ ExpectedError(
+ FEATURE_NOT_SUPPORTED_ERRCODE,
+ "client uses authorization identity",
+ ),
+ id="bad GS2 header: authzid in use",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,b,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected attribute",
+ ),
+ id="bad GS2 header: extra attribute",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected attribute 0x00", # XXX this is a bit strange
+ ),
+ id="bad GS2 header: missing authzid terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Key-value separator expected",
+ ),
+ id="missing initial kvsep",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Key-value separator expected",
+ ),
+ id="missing initial kvsep",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "does not contain an auth value",
+ ),
+ id="missing auth value: empty key-value list",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01host=example.com\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "does not contain an auth value",
+ ),
+ id="missing auth value: other keys present",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01host=example.com")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "unterminated key/value pair",
+ ),
+ id="missing value terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not contain a final terminator",
+ ),
+ id="missing list terminator: empty list",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01auth=Bearer 0\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not contain a final terminator",
+ ),
+ id="missing list terminator: with auth value",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01auth=Bearer 0\x01\x01blah")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "additional data after the final terminator",
+ ),
+ id="additional key after terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01key\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "key without a value",
+ ),
+ id="key without value",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"y,,\x01auth=Bearer 0\x01auth=Bearer 1\x01\x01",
+ )
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "contains multiple auth values",
+ ),
+ id="multiple auth values",
+ ),
+ ],
+)
+def test_oauth_bad_initial_response(conn, oauth_ctx, type, payload, err):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # The server expects a SASL response; give it something else instead.
+ if not isinstance(payload, dict):
+ payload = dict(payload_data=payload)
+ pq3.send(conn, type, **payload)
+
+ resp = pq3.recv1(conn)
+ err.match(resp)
+
+
+def test_oauth_empty_initial_response(setup_validator, connect, oauth_ctx):
+ token = bearer_token()
+ setup_validator(expected_bearer=token)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send an initial response without data.
+ initial = pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER"))
+ pq3.send(conn, pq3.types.PasswordMessage, initial)
+
+ # The server should respond with an empty challenge so we can send the data
+ # it wants.
+ pkt = pq3.recv1(conn)
+
+ assert pkt.type == pq3.types.AuthnRequest
+ assert pkt.payload.type == pq3.authn.SASLContinue
+ assert not pkt.payload.body
+
+ # Now send the initial data.
+ data = b"n,,\x01auth=Bearer " + token.encode("ascii") + b"\x01\x01"
+ pq3.send(conn, pq3.types.PasswordMessage, data)
+
+ # Server should now complete the handshake.
+ expect_handshake_success(conn)
+
+
+# TODO: see if there's a way to test this easily after the API switch
+def xtest_oauth_no_validator(setup_validator, oauth_ctx, connect):
+ # Clear out our validator command, then establish a new connection.
+ set_validator("")
+ conn = connect()
+
+ begin_oauth_handshake(conn, oauth_ctx)
+ send_initial_response(conn, bearer=bearer_token())
+
+ # The server should fail the connection.
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.parametrize(
+ "user",
+ [
+ pytest.param(
+ lambda ctx: ctx.user,
+ id="basic username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.punct_user,
+ id="'unsafe' characters are passed through correctly",
+ ),
+ ],
+)
+def test_oauth_validator_role(setup_validator, oauth_ctx, connect, user):
+ username = user(oauth_ctx)
+
+ # Tell the validator to reflect the PGUSER as the authenticated identity.
+ setup_validator(reflect_role=True)
+ conn = connect()
+
+ # Log in. Note that reflection ignores the bearer token.
+ begin_oauth_handshake(conn, oauth_ctx, user=username)
+ send_initial_response(conn, bearer=b"dontcare")
+ expect_handshake_success(conn)
+
+ # Check the user identity.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT system_user;")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ expected = b"oauth:" + username.encode("utf-8")
+ assert row.columns == [expected]
diff --git a/src/test/python/server/test_server.py b/src/test/python/server/test_server.py
new file mode 100644
index 0000000000..02126dba79
--- /dev/null
+++ b/src/test/python/server/test_server.py
@@ -0,0 +1,21 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import pq3
+
+
+def test_handshake(connect):
+ """Basic sanity check."""
+ conn = connect()
+
+ pq3.handshake(conn, user=pq3.pguser(), database=pq3.pgdatabase())
+
+ pq3.send(conn, pq3.types.Query, query=b"")
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.EmptyQueryResponse
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.ReadyForQuery
diff --git a/src/test/python/test_internals.py b/src/test/python/test_internals.py
new file mode 100644
index 0000000000..dee4855fc0
--- /dev/null
+++ b/src/test/python/test_internals.py
@@ -0,0 +1,138 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import io
+
+from pq3 import _DebugStream
+
+
+def test_DebugStream_read():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+
+ stream = _DebugStream(under, out)
+
+ res = stream.read(5)
+ assert res == b"abcde"
+
+ res = stream.read(16)
+ assert res == b"fghijklmnopqrstu"
+
+ stream.flush_debug()
+
+ res = stream.read()
+ assert res == b"vwxyz"
+
+ stream.flush_debug()
+
+ expected = (
+ "< 0000:\t61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 70\tabcdefghijklmnop\n"
+ "< 0010:\t71 72 73 74 75 \tqrstu\n"
+ "\n"
+ "< 0000:\t76 77 78 79 7a \tvwxyz\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_write():
+ under = io.BytesIO()
+ out = io.StringIO()
+
+ stream = _DebugStream(under, out)
+
+ stream.write(b"\x00\x01\x02")
+ stream.flush()
+
+ assert under.getvalue() == b"\x00\x01\x02"
+
+ stream.write(b"\xc0\xc1\xc2")
+ stream.flush()
+
+ assert under.getvalue() == b"\x00\x01\x02\xc0\xc1\xc2"
+
+ stream.flush_debug()
+
+ expected = "> 0000:\t00 01 02 c0 c1 c2 \t......\n\n"
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_read_write():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+ stream = _DebugStream(under, out)
+
+ res = stream.read(5)
+ assert res == b"abcde"
+
+ stream.write(b"xxxxx")
+ stream.flush()
+
+ assert under.getvalue() == b"abcdexxxxxklmnopqrstuvwxyz"
+
+ res = stream.read(5)
+ assert res == b"klmno"
+
+ stream.write(b"xxxxx")
+ stream.flush()
+
+ assert under.getvalue() == b"abcdexxxxxklmnoxxxxxuvwxyz"
+
+ stream.flush_debug()
+
+ expected = (
+ "< 0000:\t61 62 63 64 65 6b 6c 6d 6e 6f \tabcdeklmno\n"
+ "\n"
+ "> 0000:\t78 78 78 78 78 78 78 78 78 78 \txxxxxxxxxx\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_end_packet():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+ stream = _DebugStream(under, out)
+
+ stream.read(5)
+ stream.end_packet("read description", read=True, indent=" ")
+
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("write description", indent=" ")
+
+ stream.read(5)
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("read/write combo for read", read=True, indent=" ")
+
+ stream.read(5)
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("read/write combo for write", indent=" ")
+
+ expected = (
+ " < 0000:\t61 62 63 64 65 \tabcde\n"
+ "\n"
+ "< read description\n"
+ "\n"
+ "> write description\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ " < 0000:\t6b 6c 6d 6e 6f \tklmno\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ "< read/write combo for read\n"
+ "\n"
+ "> read/write combo for write\n"
+ "\n"
+ " < 0000:\t75 76 77 78 79 \tuvwxy\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
diff --git a/src/test/python/test_pq3.py b/src/test/python/test_pq3.py
new file mode 100644
index 0000000000..e0c0e0568d
--- /dev/null
+++ b/src/test/python/test_pq3.py
@@ -0,0 +1,558 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import getpass
+import io
+import struct
+import sys
+
+import pytest
+from construct import Container, PaddingError, StreamError, TerminatedError
+
+import pq3
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"\x00\x00\x00\x10\x00\x04\x00\x00abcdefgh",
+ Container(len=16, proto=0x40000, payload=b"abcdefgh"),
+ b"",
+ id="8-byte payload",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x08\x00\x04\x00\x00",
+ Container(len=8, proto=0x40000, payload=b""),
+ b"",
+ id="no payload",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x09\x00\x04\x00\x00abcde",
+ Container(len=9, proto=0x40000, payload=b"a"),
+ b"bcde",
+ id="1-byte payload and extra padding",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x0B\x00\x03\x00\x00hi\x00",
+ Container(len=11, proto=pq3.protocol(3, 0), payload=[b"hi"]),
+ b"",
+ id="implied parameter list when using proto version 3.0",
+ ),
+ ],
+)
+def test_Startup_parse(raw, expected, extra):
+ with io.BytesIO(raw) as stream:
+ actual = pq3.Startup.parse_stream(stream)
+
+ assert actual == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "packet,expected_bytes",
+ [
+ pytest.param(
+ dict(),
+ b"\x00\x00\x00\x08\x00\x00\x00\x00",
+ id="nothing set",
+ ),
+ pytest.param(
+ dict(len=10, proto=0x12345678),
+ b"\x00\x00\x00\x0A\x12\x34\x56\x78\x00\x00",
+ id="len and proto set explicitly",
+ ),
+ pytest.param(
+ dict(proto=0x12345678),
+ b"\x00\x00\x00\x08\x12\x34\x56\x78",
+ id="implied len with no payload",
+ ),
+ pytest.param(
+ dict(proto=0x12345678, payload=b"abcd"),
+ b"\x00\x00\x00\x0C\x12\x34\x56\x78abcd",
+ id="implied len with payload",
+ ),
+ pytest.param(
+ dict(payload=[b""]),
+ b"\x00\x00\x00\x09\x00\x03\x00\x00\x00",
+ id="implied proto version 3 when sending parameters",
+ ),
+ pytest.param(
+ dict(payload=[b"hi", b""]),
+ b"\x00\x00\x00\x0C\x00\x03\x00\x00hi\x00\x00",
+ id="implied proto version 3 and len when sending more than one parameter",
+ ),
+ pytest.param(
+ dict(payload=dict(user="jsmith", database="postgres")),
+ b"\x00\x00\x00\x27\x00\x03\x00\x00user\x00jsmith\x00database\x00postgres\x00\x00",
+ id="auto-serialization of dict parameters",
+ ),
+ ],
+)
+def test_Startup_build(packet, expected_bytes):
+ actual = pq3.Startup.build(packet)
+ assert actual == expected_bytes
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"*\x00\x00\x00\x08abcd",
+ dict(type=b"*", len=8, payload=b"abcd"),
+ b"",
+ id="4-byte payload",
+ ),
+ pytest.param(
+ b"*\x00\x00\x00\x04",
+ dict(type=b"*", len=4, payload=b""),
+ b"",
+ id="no payload",
+ ),
+ pytest.param(
+ b"*\x00\x00\x00\x05xabcd",
+ dict(type=b"*", len=5, payload=b"x"),
+ b"abcd",
+ id="1-byte payload with extra padding",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x08\x00\x00\x00\x00",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=8,
+ payload=dict(type=pq3.authn.OK, body=None),
+ ),
+ b"",
+ id="AuthenticationOk",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x12\x00\x00\x00\x0AEXTERNAL\x00\x00",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=18,
+ payload=dict(type=pq3.authn.SASL, body=[b"EXTERNAL", b""]),
+ ),
+ b"",
+ id="AuthenticationSASL",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0B12345",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=13,
+ payload=dict(type=pq3.authn.SASLContinue, body=b"12345"),
+ ),
+ b"",
+ id="AuthenticationSASLContinue",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0C12345",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=13,
+ payload=dict(type=pq3.authn.SASLFinal, body=b"12345"),
+ ),
+ b"",
+ id="AuthenticationSASLFinal",
+ ),
+ pytest.param(
+ b"p\x00\x00\x00\x0Bhunter2",
+ dict(
+ type=pq3.types.PasswordMessage,
+ len=11,
+ payload=b"hunter2",
+ ),
+ b"",
+ id="PasswordMessage",
+ ),
+ pytest.param(
+ b"K\x00\x00\x00\x0C\x00\x00\x00\x00\x12\x34\x56\x78",
+ dict(
+ type=pq3.types.BackendKeyData,
+ len=12,
+ payload=dict(pid=0, key=0x12345678),
+ ),
+ b"",
+ id="BackendKeyData",
+ ),
+ pytest.param(
+ b"C\x00\x00\x00\x08SET\x00",
+ dict(
+ type=pq3.types.CommandComplete,
+ len=8,
+ payload=dict(tag=b"SET"),
+ ),
+ b"",
+ id="CommandComplete",
+ ),
+ pytest.param(
+ b"E\x00\x00\x00\x11Mbad!\x00Mdog!\x00\x00",
+ dict(type=b"E", len=17, payload=dict(fields=[b"Mbad!", b"Mdog!", b""])),
+ b"",
+ id="ErrorResponse",
+ ),
+ pytest.param(
+ b"S\x00\x00\x00\x08a\x00b\x00",
+ dict(
+ type=pq3.types.ParameterStatus,
+ len=8,
+ payload=dict(name=b"a", value=b"b"),
+ ),
+ b"",
+ id="ParameterStatus",
+ ),
+ pytest.param(
+ b"Z\x00\x00\x00\x05x",
+ dict(type=b"Z", len=5, payload=dict(status=b"x")),
+ b"",
+ id="ReadyForQuery",
+ ),
+ pytest.param(
+ b"Q\x00\x00\x00\x06!\x00",
+ dict(type=pq3.types.Query, len=6, payload=dict(query=b"!")),
+ b"",
+ id="Query",
+ ),
+ pytest.param(
+ b"D\x00\x00\x00\x0B\x00\x01\x00\x00\x00\x01!",
+ dict(type=pq3.types.DataRow, len=11, payload=dict(columns=[b"!"])),
+ b"",
+ id="DataRow",
+ ),
+ pytest.param(
+ b"D\x00\x00\x00\x06\x00\x00extra",
+ dict(type=pq3.types.DataRow, len=6, payload=dict(columns=[])),
+ b"extra",
+ id="DataRow with extra data",
+ ),
+ pytest.param(
+ b"I\x00\x00\x00\x04",
+ dict(type=pq3.types.EmptyQueryResponse, len=4, payload=None),
+ b"",
+ id="EmptyQueryResponse",
+ ),
+ pytest.param(
+ b"I\x00\x00\x00\x04\xFF",
+ dict(type=b"I", len=4, payload=None),
+ b"\xFF",
+ id="EmptyQueryResponse with extra bytes",
+ ),
+ pytest.param(
+ b"X\x00\x00\x00\x04",
+ dict(type=pq3.types.Terminate, len=4, payload=None),
+ b"",
+ id="Terminate",
+ ),
+ ],
+)
+def test_Pq3_parse(raw, expected, extra):
+ with io.BytesIO(raw) as stream:
+ actual = pq3.Pq3.parse_stream(stream)
+
+ assert actual == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(type=b"*", len=5),
+ b"*\x00\x00\x00\x05\x00",
+ id="type and len set explicitly",
+ ),
+ pytest.param(
+ dict(type=b"*"),
+ b"*\x00\x00\x00\x04",
+ id="implied len with no payload",
+ ),
+ pytest.param(
+ dict(type=b"*", payload=b"1234"),
+ b"*\x00\x00\x00\x081234",
+ id="implied len with payload",
+ ),
+ pytest.param(
+ dict(type=pq3.types.AuthnRequest, payload=dict(type=pq3.authn.OK)),
+ b"R\x00\x00\x00\x08\x00\x00\x00\x00",
+ id="implied len/type for AuthenticationOK",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(
+ type=pq3.authn.SASL,
+ body=[b"SCRAM-SHA-256-PLUS", b"SCRAM-SHA-256", b""],
+ ),
+ ),
+ b"R\x00\x00\x00\x2A\x00\x00\x00\x0ASCRAM-SHA-256-PLUS\x00SCRAM-SHA-256\x00\x00",
+ id="implied len/type for AuthenticationSASL",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(type=pq3.authn.SASLContinue, body=b"12345"),
+ ),
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0B12345",
+ id="implied len/type for AuthenticationSASLContinue",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(type=pq3.authn.SASLFinal, body=b"12345"),
+ ),
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0C12345",
+ id="implied len/type for AuthenticationSASLFinal",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.PasswordMessage,
+ payload=b"hunter2",
+ ),
+ b"p\x00\x00\x00\x0Bhunter2",
+ id="implied len/type for PasswordMessage",
+ ),
+ pytest.param(
+ dict(type=pq3.types.BackendKeyData, payload=dict(pid=1, key=7)),
+ b"K\x00\x00\x00\x0C\x00\x00\x00\x01\x00\x00\x00\x07",
+ id="implied len/type for BackendKeyData",
+ ),
+ pytest.param(
+ dict(type=pq3.types.CommandComplete, payload=dict(tag=b"SET")),
+ b"C\x00\x00\x00\x08SET\x00",
+ id="implied len/type for CommandComplete",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ErrorResponse, payload=dict(fields=[b"error", b""])),
+ b"E\x00\x00\x00\x0Berror\x00\x00",
+ id="implied len/type for ErrorResponse",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ParameterStatus, payload=dict(name=b"a", value=b"b")),
+ b"S\x00\x00\x00\x08a\x00b\x00",
+ id="implied len/type for ParameterStatus",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ReadyForQuery, payload=dict(status=b"I")),
+ b"Z\x00\x00\x00\x05I",
+ id="implied len/type for ReadyForQuery",
+ ),
+ pytest.param(
+ dict(type=pq3.types.Query, payload=dict(query=b"SELECT 1;")),
+ b"Q\x00\x00\x00\x0eSELECT 1;\x00",
+ id="implied len/type for Query",
+ ),
+ pytest.param(
+ dict(type=pq3.types.DataRow, payload=dict(columns=[b"abcd"])),
+ b"D\x00\x00\x00\x0E\x00\x01\x00\x00\x00\x04abcd",
+ id="implied len/type for DataRow",
+ ),
+ pytest.param(
+ dict(type=pq3.types.EmptyQueryResponse),
+ b"I\x00\x00\x00\x04",
+ id="implied len for EmptyQueryResponse",
+ ),
+ pytest.param(
+ dict(type=pq3.types.Terminate),
+ b"X\x00\x00\x00\x04",
+ id="implied len for Terminate",
+ ),
+ ],
+)
+def test_Pq3_build(fields, expected):
+ actual = pq3.Pq3.build(fields)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"\x00\x00",
+ dict(columns=[]),
+ b"",
+ id="no columns",
+ ),
+ pytest.param(
+ b"\x00\x01\x00\x00\x00\x04abcd",
+ dict(columns=[b"abcd"]),
+ b"",
+ id="one column",
+ ),
+ pytest.param(
+ b"\x00\x02\x00\x00\x00\x04abcd\x00\x00\x00\x01x",
+ dict(columns=[b"abcd", b"x"]),
+ b"",
+ id="multiple columns",
+ ),
+ pytest.param(
+ b"\x00\x02\x00\x00\x00\x00\x00\x00\x00\x01x",
+ dict(columns=[b"", b"x"]),
+ b"",
+ id="empty column value",
+ ),
+ pytest.param(
+ b"\x00\x02\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF",
+ dict(columns=[None, None]),
+ b"",
+ id="null columns",
+ ),
+ ],
+)
+def test_DataRow_parse(raw, expected, extra):
+ pkt = b"D" + struct.pack("!i", len(raw) + 4) + raw
+ with io.BytesIO(pkt) as stream:
+ actual = pq3.Pq3.parse_stream(stream)
+
+ assert actual.type == pq3.types.DataRow
+ assert actual.payload == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(),
+ b"\x00\x00",
+ id="no columns",
+ ),
+ pytest.param(
+ dict(columns=[None, None]),
+ b"\x00\x02\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF",
+ id="null columns",
+ ),
+ ],
+)
+def test_DataRow_build(fields, expected):
+ actual = pq3.Pq3.build(dict(type=pq3.types.DataRow, payload=fields))
+
+ expected = b"D" + struct.pack("!i", len(expected) + 4) + expected
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "raw,expected,exception",
+ [
+ pytest.param(
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ dict(name=b"EXTERNAL", len=-1, data=None),
+ None,
+ id="no initial response",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\x02me",
+ dict(name=b"EXTERNAL", len=2, data=b"me"),
+ None,
+ id="initial response",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\x02meextra",
+ None,
+ TerminatedError,
+ id="extra data",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\xFFme",
+ None,
+ StreamError,
+ id="underflow",
+ ),
+ ],
+)
+def test_SASLInitialResponse_parse(raw, expected, exception):
+ ctx = contextlib.nullcontext()
+ if exception:
+ ctx = pytest.raises(exception)
+
+ with ctx:
+ actual = pq3.SASLInitialResponse.parse(raw)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(name=b"EXTERNAL"),
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ id="no initial response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=None),
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ id="no initial response (explicit None)",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=b""),
+ b"EXTERNAL\x00\x00\x00\x00\x00",
+ id="empty response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=b"me@example.com"),
+ b"EXTERNAL\x00\x00\x00\x00\x0Eme@example.com",
+ id="initial response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", len=2, data=b"me@example.com"),
+ b"EXTERNAL\x00\x00\x00\x00\x02me@example.com",
+ id="data overflow",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", len=14, data=b"me"),
+ b"EXTERNAL\x00\x00\x00\x00\x0Eme",
+ id="data underflow",
+ ),
+ ],
+)
+def test_SASLInitialResponse_build(fields, expected):
+ actual = pq3.SASLInitialResponse.build(fields)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "version,expected_bytes",
+ [
+ pytest.param((3, 0), b"\x00\x03\x00\x00", id="version 3"),
+ pytest.param((1234, 5679), b"\x04\xd2\x16\x2f", id="SSLRequest"),
+ ],
+)
+def test_protocol(version, expected_bytes):
+ # Make sure the integer returned by protocol is correctly serialized on the
+ # wire.
+ assert struct.pack("!i", pq3.protocol(*version)) == expected_bytes
+
+
+@pytest.mark.parametrize(
+ "envvar,func,expected",
+ [
+ ("PGHOST", pq3.pghost, "localhost"),
+ ("PGPORT", pq3.pgport, 5432),
+ ("PGUSER", pq3.pguser, getpass.getuser()),
+ ("PGDATABASE", pq3.pgdatabase, "postgres"),
+ ],
+)
+def test_env_defaults(monkeypatch, envvar, func, expected):
+ monkeypatch.delenv(envvar, raising=False)
+
+ actual = func()
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "envvars,func,expected",
+ [
+ (dict(PGHOST="otherhost"), pq3.pghost, "otherhost"),
+ (dict(PGPORT="6789"), pq3.pgport, 6789),
+ (dict(PGUSER="postgres"), pq3.pguser, "postgres"),
+ (dict(PGDATABASE="template1"), pq3.pgdatabase, "template1"),
+ ],
+)
+def test_env(monkeypatch, envvars, func, expected):
+ for k, v in envvars.items():
+ monkeypatch.setenv(k, v)
+
+ actual = func()
+ assert actual == expected
diff --git a/src/test/python/tls.py b/src/test/python/tls.py
new file mode 100644
index 0000000000..075c02c1ca
--- /dev/null
+++ b/src/test/python/tls.py
@@ -0,0 +1,195 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+from construct import *
+
+#
+# TLS 1.3
+#
+# Most of the types below are transcribed from RFC 8446:
+#
+# https://tools.ietf.org/html/rfc8446
+#
+
+
+def _Vector(size_field, element):
+ return Prefixed(size_field, GreedyRange(element))
+
+
+# Alerts
+
+AlertLevel = Enum(
+ Byte,
+ warning=1,
+ fatal=2,
+)
+
+AlertDescription = Enum(
+ Byte,
+ close_notify=0,
+ unexpected_message=10,
+ bad_record_mac=20,
+ decryption_failed_RESERVED=21,
+ record_overflow=22,
+ decompression_failure=30,
+ handshake_failure=40,
+ no_certificate_RESERVED=41,
+ bad_certificate=42,
+ unsupported_certificate=43,
+ certificate_revoked=44,
+ certificate_expired=45,
+ certificate_unknown=46,
+ illegal_parameter=47,
+ unknown_ca=48,
+ access_denied=49,
+ decode_error=50,
+ decrypt_error=51,
+ export_restriction_RESERVED=60,
+ protocol_version=70,
+ insufficient_security=71,
+ internal_error=80,
+ user_canceled=90,
+ no_renegotiation=100,
+ unsupported_extension=110,
+)
+
+Alert = Struct(
+ "level" / AlertLevel,
+ "description" / AlertDescription,
+)
+
+
+# Extensions
+
+ExtensionType = Enum(
+ Int16ub,
+ server_name=0,
+ max_fragment_length=1,
+ status_request=5,
+ supported_groups=10,
+ signature_algorithms=13,
+ use_srtp=14,
+ heartbeat=15,
+ application_layer_protocol_negotiation=16,
+ signed_certificate_timestamp=18,
+ client_certificate_type=19,
+ server_certificate_type=20,
+ padding=21,
+ pre_shared_key=41,
+ early_data=42,
+ supported_versions=43,
+ cookie=44,
+ psk_key_exchange_modes=45,
+ certificate_authorities=47,
+ oid_filters=48,
+ post_handshake_auth=49,
+ signature_algorithms_cert=50,
+ key_share=51,
+)
+
+Extension = Struct(
+ "extension_type" / ExtensionType,
+ "extension_data" / Prefixed(Int16ub, GreedyBytes),
+)
+
+
+# ClientHello
+
+
+class _CipherSuiteAdapter(Adapter):
+ class _hextuple(tuple):
+ def __repr__(self):
+ return f"(0x{self[0]:02X}, 0x{self[1]:02X})"
+
+ def _encode(self, obj, context, path):
+ return bytes(obj)
+
+ def _decode(self, obj, context, path):
+ assert len(obj) == 2
+ return self._hextuple(obj)
+
+
+ProtocolVersion = Hex(Int16ub)
+
+Random = Hex(Bytes(32))
+
+CipherSuite = _CipherSuiteAdapter(Byte[2])
+
+ClientHello = Struct(
+ "legacy_version" / ProtocolVersion,
+ "random" / Random,
+ "legacy_session_id" / Prefixed(Byte, Hex(GreedyBytes)),
+ "cipher_suites" / _Vector(Int16ub, CipherSuite),
+ "legacy_compression_methods" / Prefixed(Byte, GreedyBytes),
+ "extensions" / _Vector(Int16ub, Extension),
+)
+
+# ServerHello
+
+ServerHello = Struct(
+ "legacy_version" / ProtocolVersion,
+ "random" / Random,
+ "legacy_session_id_echo" / Prefixed(Byte, Hex(GreedyBytes)),
+ "cipher_suite" / CipherSuite,
+ "legacy_compression_method" / Hex(Byte),
+ "extensions" / _Vector(Int16ub, Extension),
+)
+
+# Handshake
+
+HandshakeType = Enum(
+ Byte,
+ client_hello=1,
+ server_hello=2,
+ new_session_ticket=4,
+ end_of_early_data=5,
+ encrypted_extensions=8,
+ certificate=11,
+ certificate_request=13,
+ certificate_verify=15,
+ finished=20,
+ key_update=24,
+ message_hash=254,
+)
+
+Handshake = Struct(
+ "msg_type" / HandshakeType,
+ "length" / Int24ub,
+ "payload"
+ / Switch(
+ this.msg_type,
+ {
+ HandshakeType.client_hello: ClientHello,
+ HandshakeType.server_hello: ServerHello,
+ # HandshakeType.end_of_early_data: EndOfEarlyData,
+ # HandshakeType.encrypted_extensions: EncryptedExtensions,
+ # HandshakeType.certificate_request: CertificateRequest,
+ # HandshakeType.certificate: Certificate,
+ # HandshakeType.certificate_verify: CertificateVerify,
+ # HandshakeType.finished: Finished,
+ # HandshakeType.new_session_ticket: NewSessionTicket,
+ # HandshakeType.key_update: KeyUpdate,
+ },
+ default=FixedSized(this.length, GreedyBytes),
+ ),
+)
+
+# Records
+
+ContentType = Enum(
+ Byte,
+ invalid=0,
+ change_cipher_spec=20,
+ alert=21,
+ handshake=22,
+ application_data=23,
+)
+
+Plaintext = Struct(
+ "type" / ContentType,
+ "legacy_record_version" / ProtocolVersion,
+ "length" / Int16ub,
+ "fragment" / FixedSized(this.length, GreedyBytes),
+)
diff --git a/src/tools/make_venv b/src/tools/make_venv
new file mode 100755
index 0000000000..a8fd530f46
--- /dev/null
+++ b/src/tools/make_venv
@@ -0,0 +1,53 @@
+#!/usr/bin/env python3
+
+import argparse
+import subprocess
+import os
+import sys
+
+parser = argparse.ArgumentParser()
+
+parser.add_argument('--requirements', help='path to pip requirements file', type=str)
+parser.add_argument('--privatedir', help='private directory for target', type=str)
+parser.add_argument('venv_path', help='desired venv location')
+
+args = parser.parse_args()
+
+# Decide whether or not to capture stdout into a log file. We only do this if
+# we've been given our own private directory.
+#
+# FIXME Unfortunately this interferes with debugging on Cirrus, because the
+# private directory isn't uploaded in the sanity check's artifacts. When we
+# don't capture the log file, it gets spammed to stdout during build... Is there
+# a way to push this into the meson-log somehow? For now, the capture
+# implementation is commented out.
+logfile = None
+
+if args.privatedir:
+ if not os.path.isdir(args.privatedir):
+ os.mkdir(args.privatedir)
+
+ # FIXME see above comment
+ # logpath = os.path.join(args.privatedir, 'stdout.txt')
+ # logfile = open(logpath, 'w')
+
+def run(*args):
+ kwargs = dict(check=True)
+ if logfile:
+ kwargs.update(stdout=logfile)
+
+ subprocess.run(args, **kwargs)
+
+# Create the virtualenv first.
+run(sys.executable, '-m', 'venv', args.venv_path)
+
+# Update pip next. This helps avoid old pip bugs; the version inside system
+# Pythons tends to be pretty out of date.
+pip = os.path.join(args.venv_path, 'bin', 'pip')
+run(pip, 'install', '-U', 'pip')
+
+# Finally, install the test's requirements. We need pytest and pytest-tap, no
+# matter what the test needs.
+run(pip, 'install', 'pytest', 'pytest-tap')
+if args.requirements:
+ run(pip, 'install', '-r', args.requirements)
--
2.34.1
v17-0008-XXX-temporary-patches-to-build-and-test.patch (application/octet-stream, 3.8K) download
From 0f9f8848568d650a1bc6519d16724bfa57075f02 Mon Sep 17 00:00:00 2001
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Tue, 20 Feb 2024 11:35:29 -0800
Subject: [PATCH v17 8/9] XXX temporary patches to build and test
- the new pg_combinebackup utility uses JSON in the frontend without
0001; has something changed?
- construct 2.10.70 has some incompatibilities with the current tests
- temporarily skip the exit check (from Daniel Gustafsson); this needs
to be turned into an exception for curl rather than a plain exit call
---
src/bin/pg_combinebackup/Makefile | 6 ++++--
src/bin/pg_combinebackup/meson.build | 3 ++-
src/bin/pg_verifybackup/Makefile | 2 +-
src/interfaces/libpq/Makefile | 2 +-
src/test/python/requirements.txt | 4 +++-
5 files changed, 11 insertions(+), 6 deletions(-)
diff --git a/src/bin/pg_combinebackup/Makefile b/src/bin/pg_combinebackup/Makefile
index 4f24b1aff6..2f7dc1ed87 100644
--- a/src/bin/pg_combinebackup/Makefile
+++ b/src/bin/pg_combinebackup/Makefile
@@ -18,6 +18,8 @@ include $(top_builddir)/src/Makefile.global
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils
+# TODO: fix this properly
+LDFLAGS_INTERNAL += -lpgcommon $(libpq_pgport)
OBJS = \
$(WIN32RES) \
@@ -30,8 +32,8 @@ OBJS = \
all: pg_combinebackup
-pg_combinebackup: $(OBJS) | submake-libpgport submake-libpgfeutils
- $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(libpq_pgport) $(LIBS) -o $@$(X)
+pg_combinebackup: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
install: all installdirs
$(INSTALL_PROGRAM) pg_combinebackup$(X) '$(DESTDIR)$(bindir)/pg_combinebackup$(X)'
diff --git a/src/bin/pg_combinebackup/meson.build b/src/bin/pg_combinebackup/meson.build
index 30dbbaa6cf..926f63f365 100644
--- a/src/bin/pg_combinebackup/meson.build
+++ b/src/bin/pg_combinebackup/meson.build
@@ -17,7 +17,8 @@ endif
pg_combinebackup = executable('pg_combinebackup',
pg_combinebackup_sources,
- dependencies: [frontend_code],
+ # XXX linking against libpq isn't good, but how was JSON working?
+ dependencies: [frontend_code, libpq],
kwargs: default_bin_args,
)
bin_targets += pg_combinebackup
diff --git a/src/bin/pg_verifybackup/Makefile b/src/bin/pg_verifybackup/Makefile
index 7c045f142e..3372fada01 100644
--- a/src/bin/pg_verifybackup/Makefile
+++ b/src/bin/pg_verifybackup/Makefile
@@ -17,7 +17,7 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
# We need libpq only because fe_utils does.
-LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils -lpgcommon $(libpq_pgport)
OBJS = \
$(WIN32RES) \
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 5dab88f095..7603c54340 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -126,7 +126,7 @@ libpq-refs-stamp: $(shlib)
ifneq ($(enable_coverage), yes)
ifeq (,$(filter aix solaris,$(PORTNAME)))
@if nm -A -u $< 2>/dev/null | grep -v -e __cxa_atexit -e __tsan_func_exit | grep exit; then \
- echo 'libpq must not be calling any function which invokes exit'; exit 1; \
+ echo 'libpq must not be calling any function which invokes exit'; \
fi
endif
endif
diff --git a/src/test/python/requirements.txt b/src/test/python/requirements.txt
index 57ba1ced94..0dfcffb83e 100644
--- a/src/test/python/requirements.txt
+++ b/src/test/python/requirements.txt
@@ -1,7 +1,9 @@
black
# cryptography 35.x and later add many platform/toolchain restrictions, beware
cryptography~=3.4.8
-construct~=2.10.61
+# TODO: figure out why 2.10.70 broke things
+# (probably https://github.com/construct/construct/pull/1015)
+construct==2.10.69
isort~=5.6
# TODO: update to psycopg[c] 3.1
psycopg2~=2.9.7
--
2.34.1
v17-0005-backend-add-OAUTHBEARER-SASL-mechanism.patch (application/octet-stream, 39.8K) download
From 29d7e3cbeda15cb618c4dd7c364c95924b208a23 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 4 May 2021 16:21:11 -0700
Subject: [PATCH v17 5/9] backend: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) on the server side. This adds a new
auth method, oauth, to pg_hba.
Because OAuth implementations vary so wildly, and bearer token
validation is heavily dependent on the issuing party, authn/z is done by
communicating with an external program: the oauth_validator_command.
This command must do the following:
1. Receive the bearer token by reading its contents from a file
descriptor passed from the server. (The numeric value of this
descriptor may be inserted into the oauth_validator_command using the
%f specifier.)
This MUST be the first action the command performs. The server will
not begin reading stdout from the command until the token has been
read in full, so if the command tries to print anything and hits a
buffer limit, the backend will deadlock and time out.
2. Validate the bearer token. The correct way to do this depends on the
issuer, but it generally involves either cryptographic operations to
prove that the token was issued by a trusted party, or the
presentation of the bearer token to some other party so that _it_ can
perform validation.
The command MUST maintain confidentiality of the bearer token, since
in most cases it can be used just like a password. (There are ways to
cryptographically bind tokens to client certificates, but they are
way beyond the scope of this commit message.)
If the token cannot be validated, the command must exit with a
non-zero status. Further authentication/authorization is pointless if
the bearer token wasn't issued by someone you trust.
3. Authenticate the user, authorize the user, or both:
a. To authenticate the user, use the bearer token to retrieve some
trusted identifier string for the end user. The exact process for
this is, again, issuer-dependent. The command should print the
authenticated identity string to stdout, followed by a newline.
If the user cannot be authenticated, the validator should not
print anything to stdout. It should also exit with a non-zero
status, unless the token may be used to authorize the connection
through some other means (see below).
On a success, the command may then exit with a zero success code.
By default, the server will then check to make sure the identity
string matches the role that is being used (or matches a usermap
entry, if one is in use).
b. To optionally authorize the user, in combination with the HBA
option trust_validator_authz=1 (see below), the validator simply
returns a zero exit code if the client should be allowed to
connect with its presented role (which can be passed to the
command using the %r specifier), or a non-zero code otherwise.
The hard part is in determining whether the given token truly
authorizes the client to use the given role, which must
unfortunately be left as an exercise to the reader.
This obviously requires some care, as a poorly implemented token
validator may silently open the entire database to anyone with a
bearer token. But it may be a more portable approach, since OAuth
is designed as an authorization framework, not an authentication
framework. For example, the user's bearer token could carry an
"allow_superuser_access" claim, which would authorize pseudonymous
database access as any role. It's then up to the OAuth system
administrators to ensure that allow_superuser_access is doled out
only to the proper users.
c. It's possible that the user can be successfully authenticated but
isn't authorized to connect. In this case, the command may print
the authenticated ID and then fail with a non-zero exit code.
(This makes it easier to see what's going on in the Postgres
logs.)
4. Token validators may optionally log to stderr. This will be printed
verbatim into the Postgres server logs.
The oauth method supports the following HBA options (but note that two
of them are not optional, since we have no way of choosing sensible
defaults):
issuer: Required. The URL of the OAuth issuing party, which the client
must contact to receive a bearer token.
Some real-world examples as of time of writing:
- https://accounts.google.com
- https://login.microsoft.com/[tenant-id]/v2.0
scope: Required. The OAuth scope(s) required for the server to
authenticate and/or authorize the user. This is heavily
deployment-specific, but a simple example is "openid email".
map: Optional. Specify a standard PostgreSQL user map; this works
the same as with other auth methods such as peer. If a map is
not specified, the user ID returned by the token validator
must exactly match the role that's being requested (but see
trust_validator_authz, below).
trust_validator_authz:
Optional. When set to 1, this allows the token validator to
take full control of the authorization process. Standard user
mapping is skipped: if the validator command succeeds, the
client is allowed to connect under its desired role and no
further checks are done.
Unlike the client, servers support OAuth without needing to be built
against libiddawc (since the responsibility for "speaking" OAuth/OIDC
correctly is delegated entirely to the oauth_validator_command).
Several TODOs:
- port to platforms other than "modern Linux/BSD"
- overhaul the communication with oauth_validator_command, which is
currently a bad hack on OpenPipeStream()
- implement more sanity checks on the OAUTHBEARER message format and
tokens sent by the client
- implement more helpful handling of HBA misconfigurations
- properly interpolate JSON when generating error responses
- use logdetail during auth failures
- deal with role names that can't be safely passed to system() without
shell-escaping
- allow passing the configured issuer to the oauth_validator_command, to
deal with multi-issuer setups
- ...and more.
---
src/backend/libpq/Makefile | 1 +
src/backend/libpq/auth-oauth.c | 810 ++++++++++++++++++++++++++++
src/backend/libpq/auth-sasl.c | 10 +-
src/backend/libpq/auth-scram.c | 4 +-
src/backend/libpq/auth.c | 26 +-
src/backend/libpq/hba.c | 31 +-
src/backend/libpq/meson.build | 1 +
src/backend/utils/misc/guc_tables.c | 12 +
src/include/libpq/auth.h | 17 +
src/include/libpq/hba.h | 6 +-
src/include/libpq/oauth.h | 24 +
src/include/libpq/sasl.h | 11 +
src/tools/pgindent/typedefs.list | 1 +
13 files changed, 922 insertions(+), 32 deletions(-)
create mode 100644 src/backend/libpq/auth-oauth.c
create mode 100644 src/include/libpq/oauth.h
diff --git a/src/backend/libpq/Makefile b/src/backend/libpq/Makefile
index 6d385fd6a4..98eb2a8242 100644
--- a/src/backend/libpq/Makefile
+++ b/src/backend/libpq/Makefile
@@ -15,6 +15,7 @@ include $(top_builddir)/src/Makefile.global
# be-fsstubs is here for historical reasons, probably belongs elsewhere
OBJS = \
+ auth-oauth.o \
auth-sasl.o \
auth-scram.o \
auth.o \
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
new file mode 100644
index 0000000000..e74c008161
--- /dev/null
+++ b/src/backend/libpq/auth-oauth.c
@@ -0,0 +1,810 @@
+/*-------------------------------------------------------------------------
+ *
+ * auth-oauth.c
+ * Server-side implementation of the SASL OAUTHBEARER mechanism.
+ *
+ * See the following RFC for more details:
+ * - RFC 7628: https://tools.ietf.org/html/rfc7628
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/backend/libpq/auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <unistd.h>
+#include <fcntl.h>
+
+#include "common/oauth-common.h"
+#include "lib/stringinfo.h"
+#include "libpq/auth.h"
+#include "libpq/hba.h"
+#include "libpq/oauth.h"
+#include "libpq/sasl.h"
+#include "storage/fd.h"
+
+/* GUC */
+char *oauth_validator_command;
+
+static void oauth_get_mechanisms(Port *port, StringInfo buf);
+static void *oauth_init(Port *port, const char *selected_mech, const char *shadow_pass);
+static int oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, const char **logdetail);
+
+/* Mechanism declaration */
+const pg_be_sasl_mech pg_be_oauth_mech = {
+ oauth_get_mechanisms,
+ oauth_init,
+ oauth_exchange,
+
+ PG_MAX_AUTH_TOKEN_LENGTH,
+};
+
+
+typedef enum
+{
+ OAUTH_STATE_INIT = 0,
+ OAUTH_STATE_ERROR,
+ OAUTH_STATE_FINISHED,
+} oauth_state;
+
+struct oauth_ctx
+{
+ oauth_state state;
+ Port *port;
+ const char *issuer;
+ const char *scope;
+};
+
+static char *sanitize_char(char c);
+static char *parse_kvpairs_for_auth(char **input);
+static void generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen);
+static bool validate(Port *port, const char *auth, const char **logdetail);
+static bool run_validator_command(Port *port, const char *token);
+static bool check_exit(FILE **fh, const char *command);
+static bool set_cloexec(int fd);
+static bool username_ok_for_shell(const char *username);
+
+#define KVSEP 0x01
+#define AUTH_KEY "auth"
+#define BEARER_SCHEME "Bearer "
+
+static void
+oauth_get_mechanisms(Port *port, StringInfo buf)
+{
+ /* Only OAUTHBEARER is supported. */
+ appendStringInfoString(buf, OAUTHBEARER_NAME);
+ appendStringInfoChar(buf, '\0');
+}
+
+static void *
+oauth_init(Port *port, const char *selected_mech, const char *shadow_pass)
+{
+ struct oauth_ctx *ctx;
+
+ if (strcmp(selected_mech, OAUTHBEARER_NAME))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("client selected an invalid SASL authentication mechanism")));
+
+ ctx = palloc0(sizeof(*ctx));
+
+ ctx->state = OAUTH_STATE_INIT;
+ ctx->port = port;
+
+ Assert(port->hba);
+ ctx->issuer = port->hba->oauth_issuer;
+ ctx->scope = port->hba->oauth_scope;
+
+ return ctx;
+}
+
+static int
+oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, const char **logdetail)
+{
+ char *p;
+ char cbind_flag;
+ char *auth;
+
+ struct oauth_ctx *ctx = opaq;
+
+ *output = NULL;
+ *outputlen = -1;
+
+ /*
+ * If the client didn't include an "Initial Client Response" in the
+ * SASLInitialResponse message, send an empty challenge, to which the
+ * client will respond with the same data that usually comes in the
+ * Initial Client Response.
+ */
+ if (input == NULL)
+ {
+ Assert(ctx->state == OAUTH_STATE_INIT);
+
+ *output = pstrdup("");
+ *outputlen = 0;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ /*
+ * Check that the input length agrees with the string length of the input.
+ */
+ if (inputlen == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The message is empty.")));
+ if (inputlen != strlen(input))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message length does not match input length.")));
+
+ switch (ctx->state)
+ {
+ case OAUTH_STATE_INIT:
+ /* Handle this case below. */
+ break;
+
+ case OAUTH_STATE_ERROR:
+
+ /*
+ * Only one response is valid for the client during authentication
+ * failure: a single kvsep.
+ */
+ if (inputlen != 1 || *input != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Client did not send a kvsep response.")));
+
+ /* The (failed) handshake is now complete. */
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_FAILURE;
+
+ default:
+ elog(ERROR, "invalid OAUTHBEARER exchange state");
+ return PG_SASL_EXCHANGE_FAILURE;
+ }
+
+ /* Handle the client's initial message. */
+ p = pstrdup(input);
+
+ /*
+ * OAUTHBEARER does not currently define a channel binding (so there is no
+ * OAUTHBEARER-PLUS, and we do not accept a 'p' specifier). We accept a
+ * 'y' specifier purely for the remote chance that a future specification
+ * could define one; then future clients can still interoperate with this
+ * server implementation. 'n' is the expected case.
+ */
+ cbind_flag = *p;
+ switch (cbind_flag)
+ {
+ case 'p':
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The server does not support channel binding for OAuth, but the client message includes channel binding data.")));
+ break;
+
+ case 'y': /* fall through */
+ case 'n':
+ p++;
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Comma expected, but found character %s.",
+ sanitize_char(*p))));
+ p++;
+ break;
+
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected channel-binding flag %s.",
+ sanitize_char(cbind_flag))));
+ }
+
+ /*
+ * Forbid optional authzid (authorization identity). We don't support it.
+ */
+ if (*p == 'a')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("client uses authorization identity, but it is not supported")));
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected attribute %s in client-first-message.",
+ sanitize_char(*p))));
+ p++;
+
+ /* All remaining fields are separated by the RFC's kvsep (\x01). */
+ if (*p != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Key-value separator expected, but found character %s.",
+ sanitize_char(*p))));
+ p++;
+
+ auth = parse_kvpairs_for_auth(&p);
+ if (!auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message does not contain an auth value.")));
+
+ /* We should be at the end of our message. */
+ if (*p)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains additional data after the final terminator.")));
+
+ if (!validate(ctx->port, auth, logdetail))
+ {
+ generate_error_response(ctx, output, outputlen);
+
+ ctx->state = OAUTH_STATE_ERROR;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_SUCCESS;
+}
+
+/*
+ * Convert an arbitrary byte to printable form. For error messages.
+ *
+ * If it's a printable ASCII character, print it as a single character.
+ * otherwise, print it in hex.
+ *
+ * The returned pointer points to a static buffer.
+ */
+static char *
+sanitize_char(char c)
+{
+ static char buf[5];
+
+ if (c >= 0x21 && c <= 0x7E)
+ snprintf(buf, sizeof(buf), "'%c'", c);
+ else
+ snprintf(buf, sizeof(buf), "0x%02x", (unsigned char) c);
+ return buf;
+}
+
+/*
+ * Consumes all kvpairs in an OAUTHBEARER exchange message. If the "auth" key is
+ * found, its value is returned.
+ */
+static char *
+parse_kvpairs_for_auth(char **input)
+{
+ char *pos = *input;
+ char *auth = NULL;
+
+ /*----
+ * The relevant ABNF, from Sec. 3.1:
+ *
+ * kvsep = %x01
+ * key = 1*(ALPHA)
+ * value = *(VCHAR / SP / HTAB / CR / LF )
+ * kvpair = key "=" value kvsep
+ * ;;gs2-header = See RFC 5801
+ * client-resp = (gs2-header kvsep *kvpair kvsep) / kvsep
+ *
+ * By the time we reach this code, the gs2-header and initial kvsep have
+ * already been validated. We start at the beginning of the first kvpair.
+ */
+
+ while (*pos)
+ {
+ char *end;
+ char *sep;
+ char *key;
+ char *value;
+
+ /*
+ * Find the end of this kvpair. Note that input is null-terminated by
+ * the SASL code, so the strchr() is bounded.
+ */
+ end = strchr(pos, KVSEP);
+ if (!end)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an unterminated key/value pair.")));
+ *end = '\0';
+
+ if (pos == end)
+ {
+ /* Empty kvpair, signifying the end of the list. */
+ *input = pos + 1;
+ return auth;
+ }
+
+ /*
+ * Find the end of the key name.
+ *
+ * TODO further validate the key/value grammar? empty keys, bad
+ * chars...
+ */
+ sep = strchr(pos, '=');
+ if (!sep)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains a key without a value.")));
+ *sep = '\0';
+
+ /* Both key and value are now safely terminated. */
+ key = pos;
+ value = sep + 1;
+
+ if (!strcmp(key, AUTH_KEY))
+ {
+ if (auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains multiple auth values.")));
+
+ auth = value;
+ }
+ else
+ {
+ /*
+ * The RFC also defines the host and port keys, but they are not
+ * required for OAUTHBEARER and we do not use them. Also, per Sec.
+ * 3.1, any key/value pairs we don't recognize must be ignored.
+ */
+ }
+
+ /* Move to the next pair. */
+ pos = end + 1;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message did not contain a final terminator.")));
+
+ return NULL; /* unreachable */
+}
+
+static void
+generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
+{
+ StringInfoData buf;
+
+ /*
+ * The admin needs to set an issuer and scope for OAuth to work. There's
+ * not really a way to hide this from the user, either, because we can't
+ * choose a "default" issuer, so be honest in the failure message.
+ *
+ * TODO: see if there's a better place to fail, earlier than this.
+ */
+ if (!ctx->issuer || !ctx->scope)
+ ereport(FATAL,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("OAuth is not properly configured for this user"),
+ errdetail_log("The issuer and scope parameters must be set in pg_hba.conf.")));
+
+
+ initStringInfo(&buf);
+
+ /*
+ * TODO: JSON escaping
+ */
+ appendStringInfo(&buf,
+ "{ "
+ "\"status\": \"invalid_token\", "
+ "\"openid-configuration\": \"%s/.well-known/openid-configuration\", "
+ "\"scope\": \"%s\" "
+ "}",
+ ctx->issuer, ctx->scope);
+
+ *output = buf.data;
+ *outputlen = buf.len;
+}
+
+static bool
+validate(Port *port, const char *auth, const char **logdetail)
+{
+ static const char *const b64_set =
+ "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-._~+/";
+
+ const char *token;
+ size_t span;
+ int ret;
+
+ /* TODO: handle logdetail when the test framework can check it */
+
+ /*-----
+ * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
+ * 2.1:
+ *
+ * b64token = 1*( ALPHA / DIGIT /
+ * "-" / "." / "_" / "~" / "+" / "/" ) *"="
+ * credentials = "Bearer" 1*SP b64token
+ *
+ * The "credentials" construction is what we receive in our auth value.
+ *
+ * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
+ * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
+ * compared case-insensitively. (This is not mentioned in RFC 6750, but
+ * it's pointed out in RFC 7628 Sec. 4.)
+ *
+ * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
+ */
+ if (pg_strncasecmp(auth, BEARER_SCHEME, strlen(BEARER_SCHEME)))
+ return false;
+
+ /* Pull the bearer token out of the auth value. */
+ token = auth + strlen(BEARER_SCHEME);
+
+ /* Swallow any additional spaces. */
+ while (*token == ' ')
+ token++;
+
+ /*
+ * Before invoking the validator command, sanity-check the token format to
+ * avoid any injection attacks later in the chain. Invalid formats are
+ * technically a protocol violation, but don't reflect any information
+ * about the sensitive Bearer token back to the client; log at COMMERROR
+ * instead.
+ */
+
+ /* Tokens must not be empty. */
+ if (!*token)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Bearer token is empty.")));
+ return false;
+ }
+
+ /*
+ * Make sure the token contains only allowed characters. Tokens may end
+ * with any number of '=' characters.
+ */
+ span = strspn(token, b64_set);
+ while (token[span] == '=')
+ span++;
+
+ if (token[span] != '\0')
+ {
+ /*
+ * This error message could be more helpful by printing the
+ * problematic character(s), but that'd be a bit like printing a piece
+ * of someone's password into the logs.
+ */
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Bearer token is not in the correct format.")));
+ return false;
+ }
+
+ /* Have the validator check the token. */
+ if (!run_validator_command(port, token))
+ return false;
+
+ if (port->hba->oauth_skip_usermap)
+ {
+ /*
+ * If the validator is our authorization authority, we're done.
+ * Authentication may or may not have been performed depending on the
+ * validator implementation; all that matters is that the validator
+ * says the user can log in with the target role.
+ */
+ return true;
+ }
+
+ /* Make sure the validator authenticated the user. */
+ if (!MyClientConnectionInfo.authn_id)
+ {
+ /* TODO: use logdetail; reduce message duplication */
+ ereport(LOG,
+ (errmsg("OAuth bearer authentication failed for user \"%s\": validator provided no identity",
+ port->user_name)));
+ return false;
+ }
+
+ /* Finally, check the user map. */
+ ret = check_usermap(port->hba->usermap, port->user_name,
+ MyClientConnectionInfo.authn_id, false);
+ return (ret == STATUS_OK);
+}
+
+static bool
+run_validator_command(Port *port, const char *token)
+{
+ bool success = false;
+ int rc;
+ int pipefd[2];
+ int rfd = -1;
+ int wfd = -1;
+
+ StringInfoData command = {0};
+ char *p;
+ FILE *fh = NULL;
+
+ ssize_t written;
+ char *line = NULL;
+ size_t size = 0;
+ ssize_t len;
+
+ Assert(oauth_validator_command);
+
+ if (!oauth_validator_command[0])
+ {
+ ereport(COMMERROR,
+ (errmsg("oauth_validator_command is not set"),
+ errhint("To allow OAuth authenticated connections, set "
+ "oauth_validator_command in postgresql.conf.")));
+ return false;
+ }
+
+ /*------
+ * Since popen() is unidirectional, open up a pipe for the other
+ * direction. Use CLOEXEC to ensure that our write end doesn't
+ * accidentally get copied into child processes, which would prevent us
+ * from closing it cleanly.
+ *
+ * XXX this is ugly. We should just read from the child process's stdout,
+ * but that's a lot more code.
+ * XXX by bypassing the popen API, we open the potential of process
+ * deadlock. Clearly document child process requirements (i.e. the child
+ * MUST read all data off of the pipe before writing anything).
+ * TODO: port to Windows using _pipe().
+ */
+ rc = pipe(pipefd);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create child pipe: %m")));
+ return false;
+ }
+
+ rfd = pipefd[0];
+ wfd = pipefd[1];
+
+ if (!set_cloexec(wfd))
+ {
+ /* error message was already logged */
+ goto cleanup;
+ }
+
+ /*----------
+ * Construct the command, substituting any recognized %-specifiers:
+ *
+ * %f: the file descriptor of the input pipe
+ * %r: the role that the client wants to assume (port->user_name)
+ * %%: a literal '%'
+ */
+ initStringInfo(&command);
+
+ for (p = oauth_validator_command; *p; p++)
+ {
+ if (p[0] == '%')
+ {
+ switch (p[1])
+ {
+ case 'f':
+ appendStringInfo(&command, "%d", rfd);
+ p++;
+ break;
+ case 'r':
+
+ /*
+ * TODO: decide how this string should be escaped. The
+ * role is controlled by the client, so if we don't escape
+ * it, command injections are inevitable.
+ *
+ * This is probably an indication that the role name needs
+ * to be communicated to the validator process in some
+ * other way. For this proof of concept, just be
+ * incredibly strict about the characters that are allowed
+ * in user names.
+ */
+ if (!username_ok_for_shell(port->user_name))
+ goto cleanup;
+
+ appendStringInfoString(&command, port->user_name);
+ p++;
+ break;
+ case '%':
+ appendStringInfoChar(&command, '%');
+ p++;
+ break;
+ default:
+ appendStringInfoChar(&command, p[0]);
+ }
+ }
+ else
+ appendStringInfoChar(&command, p[0]);
+ }
+
+ /* Execute the command. */
+ fh = OpenPipeStream(command.data, "r");
+ if (!fh)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("opening pipe to OAuth validator: %m")));
+ goto cleanup;
+ }
+
+ /* We don't need the read end of the pipe anymore. */
+ close(rfd);
+ rfd = -1;
+
+ /* Give the command the token to validate. */
+ written = write(wfd, token, strlen(token));
+ if (written != strlen(token))
+ {
+ /* TODO must loop for short writes, EINTR et al */
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write token to child pipe: %m")));
+ goto cleanup;
+ }
+
+ close(wfd);
+ wfd = -1;
+
+ /*-----
+ * Read the command's response.
+ *
+ * TODO: getline() is probably too new to use, unfortunately.
+ * TODO: loop over all lines
+ */
+ if ((len = getline(&line, &size, fh)) >= 0)
+ {
+ /* TODO: fail if the authn_id doesn't end with a newline */
+ if (len > 0)
+ line[len - 1] = '\0';
+
+ set_authn_id(port, line);
+ }
+ else if (ferror(fh))
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from command \"%s\": %m",
+ command.data)));
+ goto cleanup;
+ }
+
+ /* Make sure the command exits cleanly. */
+ if (!check_exit(&fh, command.data))
+ {
+ /* error message already logged */
+ goto cleanup;
+ }
+
+ /* Done. */
+ success = true;
+
+cleanup:
+ if (line)
+ free(line);
+
+ /*
+ * In the successful case, the pipe fds are already closed. For the error
+ * case, always close out the pipe before waiting for the command, to
+ * prevent deadlock.
+ */
+ if (rfd >= 0)
+ close(rfd);
+ if (wfd >= 0)
+ close(wfd);
+
+ if (fh)
+ {
+ Assert(!success);
+ check_exit(&fh, command.data);
+ }
+
+ if (command.data)
+ pfree(command.data);
+
+ return success;
+}
+
+static bool
+check_exit(FILE **fh, const char *command)
+{
+ int rc;
+
+ rc = ClosePipeStream(*fh);
+ *fh = NULL;
+
+ if (rc == -1)
+ {
+ /* pclose() itself failed. */
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close pipe to command \"%s\": %m",
+ command)));
+ }
+ else if (rc != 0)
+ {
+ char *reason = wait_result_to_str(rc);
+
+ ereport(COMMERROR,
+ (errmsg("failed to execute command \"%s\": %s",
+ command, reason)));
+
+ pfree(reason);
+ }
+
+ return (rc == 0);
+}
+
+static bool
+set_cloexec(int fd)
+{
+ int flags;
+ int rc;
+
+ flags = fcntl(fd, F_GETFD);
+ if (flags == -1)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not get fd flags for child pipe: %m")));
+ return false;
+ }
+
+ rc = fcntl(fd, F_SETFD, flags | FD_CLOEXEC);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not set FD_CLOEXEC for child pipe: %m")));
+ return false;
+ }
+
+ return true;
+}
+
+/*
+ * XXX This should go away eventually and be replaced with either a proper
+ * escape or a different strategy for communication with the validator command.
+ */
+static bool
+username_ok_for_shell(const char *username)
+{
+ /* This set is borrowed from fe_utils' appendShellStringNoError(). */
+ static const char *const allowed =
+ "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-_./:";
+ size_t span;
+
+ Assert(username && username[0]); /* should have already been checked */
+
+ span = strspn(username, allowed);
+ if (username[span] != '\0')
+ {
+ ereport(COMMERROR,
+ (errmsg("PostgreSQL user name contains unsafe characters and cannot be passed to the OAuth validator")));
+ return false;
+ }
+
+ return true;
+}
diff --git a/src/backend/libpq/auth-sasl.c b/src/backend/libpq/auth-sasl.c
index 08b24d90b4..4039e7fa3e 100644
--- a/src/backend/libpq/auth-sasl.c
+++ b/src/backend/libpq/auth-sasl.c
@@ -20,14 +20,6 @@
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
-/*
- * Maximum accepted size of SASL messages.
- *
- * The messages that the server or libpq generate are much smaller than this,
- * but have some headroom.
- */
-#define PG_MAX_SASL_MESSAGE_LENGTH 1024
-
/*
* Perform a SASL exchange with a libpq client, using a specific mechanism
* implementation.
@@ -103,7 +95,7 @@ CheckSASLAuth(const pg_be_sasl_mech *mech, Port *port, char *shadow_pass,
/* Get the actual SASL message */
initStringInfo(&buf);
- if (pq_getmessage(&buf, PG_MAX_SASL_MESSAGE_LENGTH))
+ if (pq_getmessage(&buf, mech->max_message_length))
{
/* EOF - pq_getmessage already logged error */
pfree(buf.data);
diff --git a/src/backend/libpq/auth-scram.c b/src/backend/libpq/auth-scram.c
index 2abb1a9b3a..aa6b5020dc 100644
--- a/src/backend/libpq/auth-scram.c
+++ b/src/backend/libpq/auth-scram.c
@@ -118,7 +118,9 @@ static int scram_exchange(void *opaq, const char *input, int inputlen,
const pg_be_sasl_mech pg_be_scram_mech = {
scram_get_mechanisms,
scram_init,
- scram_exchange
+ scram_exchange,
+
+ PG_MAX_SASL_MESSAGE_LENGTH
};
/*
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 9bbdc4beb0..db7c77da86 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -29,6 +29,7 @@
#include "libpq/auth.h"
#include "libpq/crypt.h"
#include "libpq/libpq.h"
+#include "libpq/oauth.h"
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
#include "libpq/scram.h"
@@ -47,7 +48,6 @@
*/
static void auth_failed(Port *port, int status, const char *logdetail);
static char *recv_password_packet(Port *port);
-static void set_authn_id(Port *port, const char *id);
/*----------------------------------------------------------------
@@ -203,22 +203,6 @@ static int CheckRADIUSAuth(Port *port);
static int PerformRadiusTransaction(const char *server, const char *secret, const char *portstr, const char *identifier, const char *user_name, const char *passwd);
-/*
- * Maximum accepted size of GSS and SSPI authentication tokens.
- * We also use this as a limit on ordinary password packet lengths.
- *
- * Kerberos tickets are usually quite small, but the TGTs issued by Windows
- * domain controllers include an authorization field known as the Privilege
- * Attribute Certificate (PAC), which contains the user's Windows permissions
- * (group memberships etc.). The PAC is copied into all tickets obtained on
- * the basis of this TGT (even those issued by Unix realms which the Windows
- * realm trusts), and can be several kB in size. The maximum token size
- * accepted by Windows systems is determined by the MaxAuthToken Windows
- * registry setting. Microsoft recommends that it is not set higher than
- * 65535 bytes, so that seems like a reasonable limit for us as well.
- */
-#define PG_MAX_AUTH_TOKEN_LENGTH 65535
-
/*----------------------------------------------------------------
* Global authentication functions
*----------------------------------------------------------------
@@ -307,6 +291,9 @@ auth_failed(Port *port, int status, const char *logdetail)
case uaRADIUS:
errstr = gettext_noop("RADIUS authentication failed for user \"%s\"");
break;
+ case uaOAuth:
+ errstr = gettext_noop("OAuth bearer authentication failed for user \"%s\"");
+ break;
default:
errstr = gettext_noop("authentication failed for user \"%s\": invalid authentication method");
break;
@@ -342,7 +329,7 @@ auth_failed(Port *port, int status, const char *logdetail)
* lifetime of MyClientConnectionInfo, so it is safe to pass a string that is
* managed by an external library.
*/
-static void
+void
set_authn_id(Port *port, const char *id)
{
Assert(id);
@@ -629,6 +616,9 @@ ClientAuthentication(Port *port)
case uaTrust:
status = STATUS_OK;
break;
+ case uaOAuth:
+ status = CheckSASLAuth(&pg_be_oauth_mech, port, NULL, NULL);
+ break;
}
if ((status == STATUS_OK && port->hba->clientcert == clientCertFull)
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index 8004d102ad..03c3f038c7 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -119,7 +119,8 @@ static const char *const UserAuthName[] =
"ldap",
"cert",
"radius",
- "peer"
+ "peer",
+ "oauth",
};
/*
@@ -1748,6 +1749,8 @@ parse_hba_line(TokenizedAuthLine *tok_line, int elevel)
#endif
else if (strcmp(token->string, "radius") == 0)
parsedline->auth_method = uaRADIUS;
+ else if (strcmp(token->string, "oauth") == 0)
+ parsedline->auth_method = uaOAuth;
else
{
ereport(elevel,
@@ -2067,8 +2070,9 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->auth_method != uaPeer &&
hbaline->auth_method != uaGSS &&
hbaline->auth_method != uaSSPI &&
- hbaline->auth_method != uaCert)
- INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, and cert"));
+ hbaline->auth_method != uaCert &&
+ hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, cert, and oauth"));
hbaline->usermap = pstrdup(val);
}
else if (strcmp(name, "clientcert") == 0)
@@ -2451,6 +2455,27 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->radiusidentifiers = parsed_identifiers;
hbaline->radiusidentifiers_s = pstrdup(val);
}
+ else if (strcmp(name, "issuer") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("issuer", gettext_noop("oauth"));
+ hbaline->oauth_issuer = pstrdup(val);
+ }
+ else if (strcmp(name, "scope") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("scope", gettext_noop("oauth"));
+ hbaline->oauth_scope = pstrdup(val);
+ }
+ else if (strcmp(name, "trust_validator_authz") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("trust_validator_authz", gettext_noop("oauth"));
+ if (strcmp(val, "1") == 0)
+ hbaline->oauth_skip_usermap = true;
+ else
+ hbaline->oauth_skip_usermap = false;
+ }
else
{
ereport(elevel,
diff --git a/src/backend/libpq/meson.build b/src/backend/libpq/meson.build
index 7c65314512..c85527fb01 100644
--- a/src/backend/libpq/meson.build
+++ b/src/backend/libpq/meson.build
@@ -1,6 +1,7 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
backend_sources += files(
+ 'auth-oauth.c',
'auth-sasl.c',
'auth-scram.c',
'auth.c',
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 527a2b2734..8c1b90e310 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -47,6 +47,7 @@
#include "jit/jit.h"
#include "libpq/auth.h"
#include "libpq/libpq.h"
+#include "libpq/oauth.h"
#include "libpq/scram.h"
#include "nodes/queryjumble.h"
#include "optimizer/cost.h"
@@ -4639,6 +4640,17 @@ struct config_string ConfigureNamesString[] =
check_debug_io_direct, assign_debug_io_direct, NULL
},
+ {
+ {"oauth_validator_command", PGC_SIGHUP, CONN_AUTH_AUTH,
+ gettext_noop("Command to validate OAuth v2 bearer tokens."),
+ NULL,
+ GUC_SUPERUSER_ONLY | GUC_NOT_IN_SAMPLE
+ },
+ &oauth_validator_command,
+ "",
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, NULL, NULL, NULL, NULL
diff --git a/src/include/libpq/auth.h b/src/include/libpq/auth.h
index 227b41daf6..22f6ab9f1d 100644
--- a/src/include/libpq/auth.h
+++ b/src/include/libpq/auth.h
@@ -16,6 +16,22 @@
#include "libpq/libpq-be.h"
+/*
+ * Maximum accepted size of GSS and SSPI authentication tokens.
+ * We also use this as a limit on ordinary password packet lengths.
+ *
+ * Kerberos tickets are usually quite small, but the TGTs issued by Windows
+ * domain controllers include an authorization field known as the Privilege
+ * Attribute Certificate (PAC), which contains the user's Windows permissions
+ * (group memberships etc.). The PAC is copied into all tickets obtained on
+ * the basis of this TGT (even those issued by Unix realms which the Windows
+ * realm trusts), and can be several kB in size. The maximum token size
+ * accepted by Windows systems is determined by the MaxAuthToken Windows
+ * registry setting. Microsoft recommends that it is not set higher than
+ * 65535 bytes, so that seems like a reasonable limit for us as well.
+ */
+#define PG_MAX_AUTH_TOKEN_LENGTH 65535
+
extern PGDLLIMPORT char *pg_krb_server_keyfile;
extern PGDLLIMPORT bool pg_krb_caseins_users;
extern PGDLLIMPORT bool pg_gss_accept_delegation;
@@ -23,6 +39,7 @@ extern PGDLLIMPORT bool pg_gss_accept_delegation;
extern void ClientAuthentication(Port *port);
extern void sendAuthRequest(Port *port, AuthRequest areq, const char *extradata,
int extralen);
+extern void set_authn_id(Port *port, const char *id);
/* Hook for plugins to get control in ClientAuthentication() */
typedef void (*ClientAuthentication_hook_type) (Port *, int);
diff --git a/src/include/libpq/hba.h b/src/include/libpq/hba.h
index 8ea837ae82..53c999a69f 100644
--- a/src/include/libpq/hba.h
+++ b/src/include/libpq/hba.h
@@ -39,7 +39,8 @@ typedef enum UserAuth
uaCert,
uaRADIUS,
uaPeer,
-#define USER_AUTH_LAST uaPeer /* Must be last value of this enum */
+ uaOAuth,
+#define USER_AUTH_LAST uaOAuth /* Must be last value of this enum */
} UserAuth;
/*
@@ -135,6 +136,9 @@ typedef struct HbaLine
char *radiusidentifiers_s;
List *radiusports;
char *radiusports_s;
+ char *oauth_issuer;
+ char *oauth_scope;
+ bool oauth_skip_usermap;
} HbaLine;
typedef struct IdentLine
diff --git a/src/include/libpq/oauth.h b/src/include/libpq/oauth.h
new file mode 100644
index 0000000000..5edab3b25a
--- /dev/null
+++ b/src/include/libpq/oauth.h
@@ -0,0 +1,24 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth.h
+ * Interface to libpq/auth-oauth.c
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/libpq/oauth.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_OAUTH_H
+#define PG_OAUTH_H
+
+#include "libpq/libpq-be.h"
+#include "libpq/sasl.h"
+
+extern char *oauth_validator_command;
+
+/* Implementation */
+extern const pg_be_sasl_mech pg_be_oauth_mech;
+
+#endif /* PG_OAUTH_H */
diff --git a/src/include/libpq/sasl.h b/src/include/libpq/sasl.h
index 7a1f970cca..3f2c02b8f2 100644
--- a/src/include/libpq/sasl.h
+++ b/src/include/libpq/sasl.h
@@ -26,6 +26,14 @@
#define PG_SASL_EXCHANGE_SUCCESS 1
#define PG_SASL_EXCHANGE_FAILURE 2
+/*
+ * Maximum accepted size of SASL messages.
+ *
+ * The messages that the server or libpq generate are much smaller than this,
+ * but have some headroom.
+ */
+#define PG_MAX_SASL_MESSAGE_LENGTH 1024
+
/*
* Backend SASL mechanism callbacks.
*
@@ -127,6 +135,9 @@ typedef struct pg_be_sasl_mech
const char *input, int inputlen,
char **output, int *outputlen,
const char **logdetail);
+
+ /* The maximum size allowed for client SASLResponses. */
+ int max_message_length;
} pg_be_sasl_mech;
/* Common implementation for auth.c */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 6234fe66f1..fe65a4222b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3543,6 +3543,7 @@ normal_rand_fctx
nsphash_hash
ntile_context
numeric
+oauth_state
object_access_hook_type
object_access_hook_type_str
off_t
--
2.34.1
v17-0009-WIP-Python-OAuth-provider-implementation.patch (application/octet-stream, 9.7K) download
From de8f81bd7d792e7739381d15bf2afa000729476f Mon Sep 17 00:00:00 2001
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Mon, 26 Feb 2024 16:24:32 -0800
Subject: [PATCH v17 9/9] WIP: Python OAuth provider implementation
---
src/test/modules/oauth_validator/Makefile | 2 +
src/test/modules/oauth_validator/meson.build | 3 +
.../modules/oauth_validator/t/001_server.pl | 12 +-
.../modules/oauth_validator/t/oauth_server.py | 92 ++++++++++++
src/test/perl/PostgreSQL/Test/OAuthServer.pm | 141 +++---------------
5 files changed, 125 insertions(+), 125 deletions(-)
create mode 100755 src/test/modules/oauth_validator/t/oauth_server.py
diff --git a/src/test/modules/oauth_validator/Makefile b/src/test/modules/oauth_validator/Makefile
index 1f874cd7f2..e93e01455a 100644
--- a/src/test/modules/oauth_validator/Makefile
+++ b/src/test/modules/oauth_validator/Makefile
@@ -1,3 +1,5 @@
+export PYTHON
+
MODULES = validator
PGFILEDESC = "validator - test OAuth validator module"
diff --git a/src/test/modules/oauth_validator/meson.build b/src/test/modules/oauth_validator/meson.build
index d9c1d1d577..3feba6f826 100644
--- a/src/test/modules/oauth_validator/meson.build
+++ b/src/test/modules/oauth_validator/meson.build
@@ -29,5 +29,8 @@ tests += {
'tests': [
't/001_server.pl',
],
+ 'env': {
+ 'PYTHON': python.path(),
+ },
},
}
diff --git a/src/test/modules/oauth_validator/t/001_server.pl b/src/test/modules/oauth_validator/t/001_server.pl
index 49e04b0afe..bbfa69e442 100644
--- a/src/test/modules/oauth_validator/t/001_server.pl
+++ b/src/test/modules/oauth_validator/t/001_server.pl
@@ -34,20 +34,16 @@ $node->append_conf('postgresql.conf', "shared_preload_libraries = 'validator'\n"
$node->append_conf('postgresql.conf', "oauth_validator_library = 'validator'\n");
$node->start;
-reset_pg_hba($node, 'all', 'all', 'oauth issuer="127.0.0.1:18080" scope="openid postgres"');
-
-my $webserver = PostgreSQL::Test::OAuthServer->new(18080);
+my $webserver = PostgreSQL::Test::OAuthServer->new();
+$webserver->run();
my $port = $webserver->port();
-
-is($port, 18080, "Port is 18080");
-
-$webserver->setup();
-$webserver->run();
+reset_pg_hba($node, 'all', 'all', 'oauth issuer="127.0.0.1:' . $port . '" scope="openid postgres"');
$node->connect_ok("dbname=postgres oauth_client_id=f02c6361-0635", "connect",
expected_stderr => qr@Visit https://example\.com/ and enter the code: postgresuser@);
+$webserver->stop();
$node->stop;
done_testing();
diff --git a/src/test/modules/oauth_validator/t/oauth_server.py b/src/test/modules/oauth_validator/t/oauth_server.py
new file mode 100755
index 0000000000..60d2f68f29
--- /dev/null
+++ b/src/test/modules/oauth_validator/t/oauth_server.py
@@ -0,0 +1,92 @@
+#! /usr/bin/env python3
+
+import http.server
+import json
+import os
+import sys
+from typing import TypeAlias
+
+
+class OAuthHandler(http.server.BaseHTTPRequestHandler):
+ JsonObject: TypeAlias = dict[str, object]
+
+ def do_GET(self):
+ if self.path == "/.well-known/openid-configuration":
+ resp = self.config()
+ else:
+ self.send_error(404, "Not Found")
+ return
+
+ self._send_json(resp)
+
+ def do_POST(self):
+ if self.path == "/authorize":
+ resp = self.authorization()
+ elif self.path == "/token":
+ resp = self.token()
+ else:
+ self.send_error(404, "Not Found")
+ return
+
+ self._send_json(resp)
+
+ def _send_json(self, js: JsonObject) -> None:
+ """
+ Sends the provided JSON dict as an application/json response.
+ """
+
+ resp = json.dumps(js).encode("ascii")
+
+ self.send_response(200, "OK")
+ self.send_header("Content-Type", "application/json")
+ self.send_header("Content-Length", str(len(resp)))
+ self.end_headers()
+
+ self.wfile.write(resp)
+
+ def config(self) -> JsonObject:
+ port = self.server.socket.getsockname()[1]
+
+ return {
+ "issuer": f"http://localhost:{port}",
+ "token_endpoint": f"http://localhost:{port}/token",
+ "device_authorization_endpoint": f"http://localhost:{port}/authorize",
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ "grant_types_supported": ["urn:ietf:params:oauth:grant-type:device_code"],
+ }
+
+ def authorization(self) -> JsonObject:
+ return {
+ "device_code": "postgres",
+ "user_code": "postgresuser",
+ "interval": 0,
+ "verification_uri": "https://example.com/",
+ "expires-in": 5,
+ }
+
+ def token(self) -> JsonObject:
+ return {
+ "access_token": "9243959234",
+ "token_type": "bearer",
+ }
+
+
+def main():
+ s = http.server.HTTPServer(("127.0.0.1", 0), OAuthHandler)
+
+ # Give the parent the port number to contact (this is also the signal that
+ # we're ready to receive requests).
+ port = s.socket.getsockname()[1]
+ print(port)
+
+ stdout = sys.stdout.fileno()
+ sys.stdout.close()
+ os.close(stdout)
+
+ s.serve_forever() # we expect our parent to send a termination signal
+
+
+if __name__ == "__main__":
+ main()
diff --git a/src/test/perl/PostgreSQL/Test/OAuthServer.pm b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
index 3ac90c3d0f..d96733f531 100644
--- a/src/test/perl/PostgreSQL/Test/OAuthServer.pm
+++ b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
@@ -5,6 +5,7 @@ package PostgreSQL::Test::OAuthServer;
use warnings;
use strict;
use threads;
+use Scalar::Util;
use Socket;
use IO::Select;
@@ -13,27 +14,13 @@ local *server_socket;
sub new
{
my $class = shift;
- my $port = shift;
my $self = {};
bless($self, $class);
- $self->{'port'} = $port;
-
return $self;
}
-sub setup
-{
- my $self = shift;
- my $tcp = getprotobyname('tcp');
-
- socket($self->{'socket'}, PF_INET, SOCK_STREAM, $tcp)
- or die "no socket";
- setsockopt($self->{'socket'}, SOL_SOCKET, SO_REUSEADDR, pack("l", 1));
- bind($self->{'socket'}, sockaddr_in($self->{'port'}, INADDR_ANY));
-}
-
sub port
{
my $self = shift;
@@ -44,115 +31,35 @@ sub port
sub run
{
my $self = shift;
+ my $port;
- my $server_thread = threads->create(\&_listen, $self);
- $server_thread->detach();
+ my $pid = open(my $read_fh, "-|", $ENV{PYTHON}, "t/oauth_server.py")
+ // die "failed to start OAuth server: $!";
+
+ read($read_fh, $port, 7) // die "failed to read port number: $!";
+ chomp $port;
+ die "server did not advertise a valid port"
+ unless Scalar::Util::looks_like_number($port);
+
+ $self->{'pid'} = $pid;
+ $self->{'port'} = $port;
+ $self->{'child'} = $read_fh;
+
+ print("# OAuth provider (PID $pid) is listening on port $port\n");
}
-sub _listen
+sub stop
{
my $self = shift;
- listen($self->{'socket'}, SOMAXCONN) or die "fail to listen: $!";
-
- while (1)
- {
- my $fh;
- my %request;
- my $remote = accept($fh, $self->{'socket'});
- binmode $fh;
-
- my ($method, $object, $prot) = split(/ /, <$fh>);
- $request{'method'} = $method;
- $request{'object'} = $object;
- chomp($request{'object'});
-
- local $/ = Socket::CRLF;
- my $c = 0;
- while(<$fh>)
- {
- chomp;
- # Headers
- if (/:/)
- {
- my ($field, $value) = split(/:/, $_, 2);
- $value =~ s/^\s+//;
- $request{'headers'}{lc $field} = $value;
- }
- # POST data
- elsif (/^$/)
- {
- read($fh, $request{'content'}, $request{'headers'}{'content-length'})
- if defined $request{'headers'}{'content-length'};
- last;
- }
- }
-
- # Debug printing
- # print ": read ".$request{'method'} . ";" . $request{'object'}.";\n";
- # foreach my $h (keys(%{$request{'headers'}}))
- #{
- # printf ": headers: " . $request{'headers'}{$h} . "\n";
- #}
- #printf ": POST: " . $request{'content'} . "\n" if defined $request{'content'};
-
- if ($request{'object'} eq '/.well-known/openid-configuration')
- {
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: application/json\r\n";
- print $fh "\r\n";
- print $fh <<EOR;
- {
- "issuer": "http://localhost:$self->{'port'}",
- "token_endpoint": "http://localhost:$self->{'port'}/token",
- "device_authorization_endpoint": "http://localhost:$self->{'port'}/authorize",
- "response_types_supported": ["token"],
- "subject_types_supported": ["public"],
- "id_token_signing_alg_values_supported": ["RS256"],
- "grant_types_supported": ["urn:ietf:params:oauth:grant-type:device_code"]
- }
-EOR
- }
- elsif ($request{'object'} eq '/authorize')
- {
- print ": returning device_code\n";
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: application/json\r\n";
- print $fh "\r\n";
- print $fh <<EOR;
- {
- "device_code": "postgres",
- "user_code" : "postgresuser",
- "interval" : 0,
- "verification_uri" : "https://example.com/",
- "expires-in": 5
- }
-EOR
- }
- elsif ($request{'object'} eq '/token')
- {
- print ": returning token\n";
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: application/json\r\n";
- print $fh "\r\n";
- print $fh <<EOR;
- {
- "access_token": "9243959234",
- "token_type": "bearer"
- }
-EOR
- }
- else
- {
- print ": returning default\n";
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: text/html\r\n";
- print $fh "\r\n";
- print $fh "Ok\n";
- }
-
- close($fh);
- }
+ print("# Sending SIGTERM to OAuth provider PID: $self->{'pid'}\n");
+
+ kill(15, $self->{'pid'});
+ $self->{'pid'} = undef;
+
+ # Closing the popen() handle waits for the process to exit.
+ close($self->{'child'});
+ $self->{'child'} = undef;
}
1;
--
2.34.1
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-02-28 17:40 Daniel Gustafsson <daniel@yesql.se>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
1 sibling, 1 reply; 424+ messages in thread
From: Daniel Gustafsson @ 2024-02-28 17:40 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: pgsql-hackers
> On 28 Feb 2024, at 15:05, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
>
> [Trying again, with all patches unzipped and the CC list temporarily
> removed to avoid flooding people's inboxes. Original message follows.]
>
> On Fri, Feb 23, 2024 at 5:01 PM Jacob Champion
> <jacob.champion@enterprisedb.com> wrote:
>> The
>> patchset is now carrying a lot of squash-cruft, and I plan to flatten
>> it in the next version.
>
> This is done in v17, which is also now based on the two patches pulled
> out by Daniel in [1]. Besides the squashes, which make up most of the
> range-diff, I've fixed a call to strncasecmp() which is not available
> on Windows.
>
> Daniel and I discussed trying a Python version of the test server,
> since the standard library there should give us more goodies to work
> with. A proof of concept is in 0009. I think the big question I have
> for it is, how would we communicate what we want the server to do for
> the test? (We could perhaps switch on magic values of the client ID?)
> In the end I'd like to be testing close to 100% of the failure modes,
> and that's likely to mean a lot of back-and-forth if the server
> implementation isn't in the Perl process.
Thanks for the new version, I'm digesting the test patches but for now I have a
few smaller comments:
+#define ALLOC(size) malloc(size)
I wonder if we should use pg_malloc_extended(size, MCXT_ALLOC_NO_OOM) instead
to self document the code. We clearly don't want feature-parity with server-
side palloc here. I know we use malloc in similar ALLOC macros so it's not
unique in that regard, but maybe?
+#ifdef FRONTEND
+ destroyPQExpBuffer(lex->errormsg);
+#else
+ pfree(lex->errormsg->data);
+ pfree(lex->errormsg);
+#endif
Wouldn't it be nicer if we abstracted this into a destroyStrVal function to a)
avoid the ifdefs and b) make it more like the rest of the new API? While it's
only used in two places (close to each other) it's a shame to let the
underlying API bleed through the abstraction.
+ CURLM *curlm; /* top-level multi handle for cURL operations */
Nitpick, but curl is not capitalized cURL anymore (for some value of "anymore"
since it changed in 2016 [0]). I do wonder if we should consistently write
"libcurl" as well since we don't use curl but libcurl.
+ PQExpBufferData work_data; /* scratch buffer for general use (remember
+ to clear out prior contents first!) */
This seems like asking for subtle bugs due to uncleared buffers bleeding into
another operation (especially since we are writing this data across the wire).
How about having an array the size of OAuthStep of unallocated buffers where
each step use it's own? Storing the content of each step could also be useful
for debugging. Looking at the statemachine here it's not an obvious change but
also not impossible.
+ * TODO: This disables DNS resolution timeouts unless libcurl has been
+ * compiled against alternative resolution support. We should check that.
curl_version_info() can be used to check for c-ares support.
+ * so you don't have to write out the error handling every time. They assume
+ * that they're embedded in a function returning bool, however.
It feels a bit iffy to encode the returntype in the macro, we can use the same
trick that DISABLE_SIGPIPE employs where a failaction is passed in.
+ if (!strcmp(name, field->name))
Project style is to test for (strcmp(x,y) == 0) rather than (!strcmp()) to
improve readability.
+ libpq_append_conn_error(conn, "out of memory");
While not introduced in this patch, it's not an ideal pattern to report "out of
memory" errors via a function which may allocate memory.
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server's error message contained an embedded NULL"));
We should maybe add ", discarding" or something similar after this string to
indicate that there was an actual error which has been thrown away, the error
wasn't that the server passed an embedded NULL.
+#ifdef USE_OAUTH
+ else if (strcmp(mechanism_buf.data, OAUTHBEARER_NAME) == 0 &&
+ !selected_mechanism)
I wonder if we instead should move the guards inside the statement and error
out with "not built with OAuth support" or something similar like how we do
with TLS and other optional components?
+ errdetail("Comma expected, but found character %s.",
+ sanitize_char(*p))));
The %s formatter should be wrapped like '%s' to indicate that the message part
is the character in question (and we can then reuse the translation since the
error message already exist for SCRAM).
+ temp = curl_slist_append(temp, "authorization_code");
+ if (!temp)
+ oom = true;
+
+ temp = curl_slist_append(temp, "implicit");
While not a bug per se, it reads a bit odd to call another operation that can
allocate memory when the oom flag has been set. I think we can move some
things around a little to make it clearer.
The attached diff contains some (most?) of the above as a patch on top of your
v17, but as a .txt to keep the CFBot from munging on it.
--
Daniel Gustafsson
Attachments:
v17_review_suggestions.txt (text/plain, 15.8K) download
commit 62b6bda852d09458ffcd5854b7810b3b316e2b1f
Author: Daniel Gustafsson <daniel@yesql.se>
Date: Wed Feb 28 18:26:03 2024 +0100
Suggested changes
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
index 765a18b9b2..f5cf271566 100644
--- a/src/backend/libpq/auth-oauth.c
+++ b/src/backend/libpq/auth-oauth.c
@@ -204,7 +204,7 @@ oauth_exchange(void *opaq, const char *input, int inputlen,
ereport(ERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
errmsg("malformed OAUTHBEARER message"),
- errdetail("Comma expected, but found character %s.",
+ errdetail("Comma expected, but found character \"%s\".",
sanitize_char(*p))));
p++;
break;
diff --git a/src/common/jsonapi.c b/src/common/jsonapi.c
index 479310f598..2d1f30353a 100644
--- a/src/common/jsonapi.c
+++ b/src/common/jsonapi.c
@@ -42,6 +42,7 @@
#define appendStrValChar appendPQExpBufferChar
#define createStrVal createPQExpBuffer
#define resetStrVal resetPQExpBuffer
+#define destroyStrVal destroyPQExpBuffer
#else /* !FRONTEND */
@@ -53,6 +54,7 @@
#define appendStrValChar appendStringInfoChar
#define createStrVal makeStringInfo
#define resetStrVal resetStringInfo
+#define destroyStrVal destroyStringInfo
#endif
@@ -223,23 +225,11 @@ freeJsonLexContext(JsonLexContext *lex)
static const JsonLexContext empty = {0};
if (lex->flags & JSONLEX_FREE_STRVAL)
- {
-#ifdef FRONTEND
- destroyPQExpBuffer(lex->strval);
-#else
- pfree(lex->strval->data);
- pfree(lex->strval);
-#endif
- }
+ destroyStrVal(lex->strval);
+
if (lex->errormsg)
- {
-#ifdef FRONTEND
- destroyPQExpBuffer(lex->errormsg);
-#else
- pfree(lex->errormsg->data);
- pfree(lex->errormsg);
-#endif
- }
+ destroyStrVal(lex->errormsg);
+
if (lex->flags & JSONLEX_FREE_STRUCT)
pfree(lex);
else
diff --git a/src/common/stringinfo.c b/src/common/stringinfo.c
index c61d5c58f3..09419f6042 100644
--- a/src/common/stringinfo.c
+++ b/src/common/stringinfo.c
@@ -350,3 +350,10 @@ enlargeStringInfo(StringInfo str, int needed)
str->maxlen = newlen;
}
+
+void
+destroyStringInfo(StringInfo str)
+{
+ pfree(str->data);
+ pfree(str);
+}
diff --git a/src/include/lib/stringinfo.h b/src/include/lib/stringinfo.h
index 2cd636b01c..64ec6419af 100644
--- a/src/include/lib/stringinfo.h
+++ b/src/include/lib/stringinfo.h
@@ -233,4 +233,6 @@ extern void appendBinaryStringInfoNT(StringInfo str,
*/
extern void enlargeStringInfo(StringInfo str, int needed);
+
+extern void destroyStringInfo(StringInfo str);
#endif /* STRINGINFO_H */
diff --git a/src/interfaces/libpq/fe-auth-oauth-curl.c b/src/interfaces/libpq/fe-auth-oauth-curl.c
index 3e20ba5818..7a27198d66 100644
--- a/src/interfaces/libpq/fe-auth-oauth-curl.c
+++ b/src/interfaces/libpq/fe-auth-oauth-curl.c
@@ -293,40 +293,39 @@ free_curl_async_ctx(PGconn *conn, void *ctx)
/*
* Macros for getting and setting state for the connection's two cURL handles,
- * so you don't have to write out the error handling every time. They assume
- * that they're embedded in a function returning bool, however.
+ * so you don't have to write out the error handling every time.
*/
-#define CHECK_MSETOPT(ACTX, OPT, VAL) \
+#define CHECK_MSETOPT(ACTX, OPT, VAL, FAILACTION) \
do { \
struct async_ctx *_actx = (ACTX); \
CURLMcode _setopterr = curl_multi_setopt(_actx->curlm, OPT, VAL); \
if (_setopterr) { \
actx_error(_actx, "failed to set %s on OAuth connection: %s",\
#OPT, curl_multi_strerror(_setopterr)); \
- return false; \
+ FAILACTION; \
} \
} while (0)
-#define CHECK_SETOPT(ACTX, OPT, VAL) \
+#define CHECK_SETOPT(ACTX, OPT, VAL, FAILACTION) \
do { \
struct async_ctx *_actx = (ACTX); \
CURLcode _setopterr = curl_easy_setopt(_actx->curl, OPT, VAL); \
if (_setopterr) { \
actx_error(_actx, "failed to set %s on OAuth connection: %s",\
#OPT, curl_easy_strerror(_setopterr)); \
- return false; \
+ FAILACTION; \
} \
} while (0)
-#define CHECK_GETINFO(ACTX, INFO, OUT) \
+#define CHECK_GETINFO(ACTX, INFO, OUT, FAILACTION) \
do { \
struct async_ctx *_actx = (ACTX); \
CURLcode _getinfoerr = curl_easy_getinfo(_actx->curl, INFO, OUT); \
if (_getinfoerr) { \
actx_error(_actx, "failed to get %s from OAuth response: %s",\
#INFO, curl_easy_strerror(_getinfoerr)); \
- return false; \
+ FAILACTION; \
} \
} while (0)
@@ -450,7 +449,7 @@ oauth_json_object_field_start(void *state, char *name, bool isnull)
while (field->name)
{
- if (!strcmp(name, field->name))
+ if (strcmp(name, field->name) == 0)
{
ctx->active = field;
break;
@@ -616,7 +615,7 @@ parse_oauth_json(struct async_ctx *actx, const struct json_field *fields)
bool success = false;
/* Make sure the server thinks it's given us JSON. */
- CHECK_GETINFO(actx, CURLINFO_CONTENT_TYPE, &content_type);
+ CHECK_GETINFO(actx, CURLINFO_CONTENT_TYPE, &content_type, return false);
if (!content_type)
{
@@ -1109,6 +1108,8 @@ register_timer(CURLM *curlm, long timeout, void *ctx)
static bool
setup_curl_handles(struct async_ctx *actx)
{
+ curl_version_info_data *curl_info;
+
/*
* Create our multi handle. This encapsulates the entire conversation with
* cURL for this connection.
@@ -1121,14 +1122,19 @@ setup_curl_handles(struct async_ctx *actx)
return false;
}
+ /*
+ * Extract information about the libcurl we are linked against.
+ */
+ curl_info = curl_version_info(CURLVERSION_NOW);
+
/*
* The multi handle tells us what to wait on using two callbacks. These
* will manipulate actx->mux as needed.
*/
- CHECK_MSETOPT(actx, CURLMOPT_SOCKETFUNCTION, register_socket);
- CHECK_MSETOPT(actx, CURLMOPT_SOCKETDATA, actx);
- CHECK_MSETOPT(actx, CURLMOPT_TIMERFUNCTION, register_timer);
- CHECK_MSETOPT(actx, CURLMOPT_TIMERDATA, actx);
+ CHECK_MSETOPT(actx, CURLMOPT_SOCKETFUNCTION, register_socket, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_SOCKETDATA, actx, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_TIMERFUNCTION, register_timer, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_TIMERDATA, actx, return false);
/*
* Set up an easy handle. All of our requests are made serially, so we
@@ -1145,25 +1151,26 @@ setup_curl_handles(struct async_ctx *actx)
* Multi-threaded applications must set CURLOPT_NOSIGNAL. This requires us
* to handle the possibility of SIGPIPE ourselves.
*
- * TODO: This disables DNS resolution timeouts unless libcurl has been
- * compiled against alternative resolution support. We should check that.
- *
* TODO: handle SIGPIPE via pq_block_sigpipe(), or via a
* CURLOPT_SOCKOPTFUNCTION maybe...
*/
- CHECK_SETOPT(actx, CURLOPT_NOSIGNAL, 1L);
+ CHECK_SETOPT(actx, CURLOPT_NOSIGNAL, 1L, return false);
+ if (!curl_info->ares_num)
+ {
+ /* No alternative resolver, TODO: warn about timeouts */
+ }
/* TODO investigate using conn->Pfdebug and CURLOPT_DEBUGFUNCTION here */
- CHECK_SETOPT(actx, CURLOPT_VERBOSE, 1L);
- CHECK_SETOPT(actx, CURLOPT_ERRORBUFFER, actx->curl_err);
+ CHECK_SETOPT(actx, CURLOPT_VERBOSE, 1L, return false);
+ CHECK_SETOPT(actx, CURLOPT_ERRORBUFFER, actx->curl_err, return false);
/* TODO */
- CHECK_SETOPT(actx, CURLOPT_WRITEDATA, stderr);
+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, stderr, return false);
/*
* Only HTTP[S] is allowed. TODO: disallow HTTP without user opt-in
*/
- CHECK_SETOPT(actx, CURLOPT_PROTOCOLS, CURLPROTO_HTTP | CURLPROTO_HTTPS);
+ CHECK_SETOPT(actx, CURLOPT_PROTOCOLS, CURLPROTO_HTTP | CURLPROTO_HTTPS, return false);
/*
* Suppress the Accept header to make our request as minimal as possible.
@@ -1172,7 +1179,7 @@ setup_curl_handles(struct async_ctx *actx)
* what comes back anyway.)
*/
actx->headers = curl_slist_append(actx->headers, "Accept:"); /* TODO: check result */
- CHECK_SETOPT(actx, CURLOPT_HTTPHEADER, actx->headers);
+ CHECK_SETOPT(actx, CURLOPT_HTTPHEADER, actx->headers, return false);
return true;
}
@@ -1213,8 +1220,8 @@ start_request(struct async_ctx *actx)
int running;
resetPQExpBuffer(&actx->work_data);
- CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data);
- CHECK_SETOPT(actx, CURLOPT_WRITEDATA, &actx->work_data);
+ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data, return false);
+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, &actx->work_data, return false);
err = curl_multi_add_handle(actx->curlm, actx->curl);
if (err)
@@ -1339,8 +1346,8 @@ drive_request(struct async_ctx *actx)
static bool
start_discovery(struct async_ctx *actx, const char *discovery_uri)
{
- CHECK_SETOPT(actx, CURLOPT_HTTPGET, 1L);
- CHECK_SETOPT(actx, CURLOPT_URL, discovery_uri);
+ CHECK_SETOPT(actx, CURLOPT_HTTPGET, 1L, return false);
+ CHECK_SETOPT(actx, CURLOPT_URL, discovery_uri, return false);
return start_request(actx);
}
@@ -1363,7 +1370,7 @@ finish_discovery(struct async_ctx *actx)
* validation into question), or non-authoritative responses, or any other
* complications.
*/
- CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code);
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
if (response_code != 200)
{
@@ -1387,20 +1394,17 @@ finish_discovery(struct async_ctx *actx)
* Per Section 3, the default is ["authorization_code", "implicit"].
*/
struct curl_slist *temp = actx->provider.grant_types_supported;
- bool oom = false;
temp = curl_slist_append(temp, "authorization_code");
- if (!temp)
- oom = true;
-
- temp = curl_slist_append(temp, "implicit");
- if (!temp)
- oom = true;
-
- if (oom)
+ if (temp)
{
- actx_error(actx, "out of memory");
- return false;
+ temp = curl_slist_append(temp, "implicit");
+ if (!temp)
+ {
+ curl_slist_free_all(temp);
+ actx_error(actx, "out of memory");
+ return false;
+ }
}
actx->provider.grant_types_supported = temp;
@@ -1491,8 +1495,8 @@ start_device_authz(struct async_ctx *actx, PGconn *conn)
/* TODO check for broken buffer */
/* Make our request. */
- CHECK_SETOPT(actx, CURLOPT_URL, device_authz_uri);
- CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data);
+ CHECK_SETOPT(actx, CURLOPT_URL, device_authz_uri, return false);
+ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data, return false);
if (conn->oauth_client_secret)
{
@@ -1506,12 +1510,12 @@ start_device_authz(struct async_ctx *actx, PGconn *conn)
*
* TODO: should we omit client_id from the body in this case?
*/
- CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
- CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id);
- CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret);
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC, return false);
+ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id, return false);
+ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret, return false);
}
else
- CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE);
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE, return false);
return start_request(actx);
}
@@ -1521,7 +1525,7 @@ finish_device_authz(struct async_ctx *actx)
{
long response_code;
- CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code);
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
/*
* The device authorization endpoint uses the same error response as the
@@ -1593,8 +1597,8 @@ start_token_request(struct async_ctx *actx, PGconn *conn)
/* TODO check for broken buffer */
/* Make our request. */
- CHECK_SETOPT(actx, CURLOPT_URL, token_uri);
- CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data);
+ CHECK_SETOPT(actx, CURLOPT_URL, token_uri, return false);
+ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data, return false);
if (conn->oauth_client_secret)
{
@@ -1608,16 +1612,16 @@ start_token_request(struct async_ctx *actx, PGconn *conn)
*
* TODO: should we omit client_id from the body in this case?
*/
- CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
- CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id);
- CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret);
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC, return false);
+ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id, return false);
+ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret, return false);
}
else
- CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE);
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE, return false);
resetPQExpBuffer(work_buffer);
- CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data);
- CHECK_SETOPT(actx, CURLOPT_WRITEDATA, work_buffer);
+ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data, return false);
+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, work_buffer, return false);
return start_request(actx);
}
@@ -1627,7 +1631,7 @@ finish_token_request(struct async_ctx *actx, struct token *tok)
{
long response_code;
- CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code);
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
/*
* Per RFC 6749, Section 5, a successful response uses 200 OK. An error
@@ -1889,7 +1893,7 @@ pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
* A slow_down error requires us to permanently increase our
* retry interval by five seconds. RFC 8628, Sec. 3.5.
*/
- if (!strcmp(err->error, "slow_down"))
+ if (strcmp(err->error, "slow_down") == 0)
{
actx->authz.interval += 5; /* TODO check for overflow? */
}
diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
index 2a35c7438c..66ee8ff076 100644
--- a/src/interfaces/libpq/fe-auth-oauth.c
+++ b/src/interfaces/libpq/fe-auth-oauth.c
@@ -49,7 +49,7 @@ oauth_init(PGconn *conn, const char *password,
* error.
*/
Assert(sasl_mechanism != NULL);
- Assert(!strcmp(sasl_mechanism, OAUTHBEARER_NAME));
+ Assert(strcmp(sasl_mechanism, OAUTHBEARER_NAME) == 0);
state = calloc(1, sizeof(*state));
if (!state)
@@ -156,17 +156,17 @@ oauth_json_object_field_start(void *state, char *name, bool isnull)
if (ctx->nested == 1)
{
- if (!strcmp(name, ERROR_STATUS_FIELD))
+ if (strcmp(name, ERROR_STATUS_FIELD) == 0)
{
ctx->target_field_name = ERROR_STATUS_FIELD;
ctx->target_field = &ctx->status;
}
- else if (!strcmp(name, ERROR_SCOPE_FIELD))
+ else if (strcmp(name, ERROR_SCOPE_FIELD) == 0)
{
ctx->target_field_name = ERROR_SCOPE_FIELD;
ctx->target_field = &ctx->scope;
}
- else if (!strcmp(name, ERROR_OPENID_CONFIGURATION_FIELD))
+ else if (strcmp(name, ERROR_OPENID_CONFIGURATION_FIELD) == 0)
{
ctx->target_field_name = ERROR_OPENID_CONFIGURATION_FIELD;
ctx->target_field = &ctx->discovery_uri;
@@ -243,7 +243,7 @@ handle_oauth_sasl_error(PGconn *conn, char *msg, int msglen)
if (strlen(msg) != msglen)
{
appendPQExpBufferStr(&conn->errorMessage,
- libpq_gettext("server's error message contained an embedded NULL"));
+ libpq_gettext("server's error message contained an embedded NULL, and was discarded"));
return false;
}
@@ -316,7 +316,7 @@ handle_oauth_sasl_error(PGconn *conn, char *msg, int msglen)
return false;
}
- if (!strcmp(ctx.status, "invalid_token"))
+ if (strcmp(ctx.status, "invalid_token") == 0)
{
/*
* invalid_token is the only error code we'll automatically retry for,
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index 9e084dd1c7..6e3538c9fd 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -525,15 +525,18 @@ pg_SASL_init(PGconn *conn, int payloadlen, bool *async)
conn->sasl = &pg_scram_mech;
conn->password_needed = true;
}
-#ifdef USE_OAUTH
else if (strcmp(mechanism_buf.data, OAUTHBEARER_NAME) == 0 &&
!selected_mechanism)
{
+#ifdef USE_OAUTH
selected_mechanism = OAUTHBEARER_NAME;
conn->sasl = &pg_oauth_mech;
conn->password_needed = false;
- }
+#else
+ libpq_append_conn_error(conn, "OAuth is required, but client does not support it");
+ goto error;
#endif
+ }
}
if (!selected_mechanism)
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-02-28 21:50 Andrew Dunstan <andrew@dunslane.net>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
1 sibling, 1 reply; 424+ messages in thread
From: Andrew Dunstan @ 2024-02-28 21:50 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; pgsql-hackers
On 2024-02-28 We 09:05, Jacob Champion wrote:
>
> Daniel and I discussed trying a Python version of the test server,
> since the standard library there should give us more goodies to work
> with. A proof of concept is in 0009. I think the big question I have
> for it is, how would we communicate what we want the server to do for
> the test? (We could perhaps switch on magic values of the client ID?)
> In the end I'd like to be testing close to 100% of the failure modes,
> and that's likely to mean a lot of back-and-forth if the server
> implementation isn't in the Perl process.
Can you give some more details about what this python gadget would buy
us? I note that there are a couple of CPAN modules that provide OAuth2
servers, not sure if they would be of any use.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-02-28 21:52 Daniel Gustafsson <daniel@yesql.se>
parent: Andrew Dunstan <andrew@dunslane.net>
0 siblings, 1 reply; 424+ messages in thread
From: Daniel Gustafsson @ 2024-02-28 21:52 UTC (permalink / raw)
To: Andrew Dunstan <andrew@dunslane.net>; +Cc: Jacob Champion <jacob.champion@enterprisedb.com>; pgsql-hackers
> On 28 Feb 2024, at 22:50, Andrew Dunstan <andrew@dunslane.net> wrote:
>
> On 2024-02-28 We 09:05, Jacob Champion wrote:
>>
>> Daniel and I discussed trying a Python version of the test server,
>> since the standard library there should give us more goodies to work
>> with. A proof of concept is in 0009. I think the big question I have
>> for it is, how would we communicate what we want the server to do for
>> the test? (We could perhaps switch on magic values of the client ID?)
>> In the end I'd like to be testing close to 100% of the failure modes,
>> and that's likely to mean a lot of back-and-forth if the server
>> implementation isn't in the Perl process.
>
> Can you give some more details about what this python gadget would buy us? I note that there are a couple of CPAN modules that provide OAuth2 servers, not sure if they would be of any use.
The main benefit would be to be able to provide a full testharness without
adding any additional dependencies over what we already have (Python being
required by meson). That should ideally make it easy to get good coverage from
BF animals as no installation is needed.
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-02-29 14:49 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Daniel Gustafsson <daniel@yesql.se>
0 siblings, 0 replies; 424+ messages in thread
From: Jacob Champion @ 2024-02-29 14:49 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: Andrew Dunstan <andrew@dunslane.net>; pgsql-hackers; Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
[re-adding the CC list I dropped earlier]
On Wed, Feb 28, 2024 at 1:52 PM Daniel Gustafsson <daniel@yesql.se> wrote:
>
> > On 28 Feb 2024, at 22:50, Andrew Dunstan <andrew@dunslane.net> wrote:
> > Can you give some more details about what this python gadget would buy us? I note that there are a couple of CPAN modules that provide OAuth2 servers, not sure if they would be of any use.
>
> The main benefit would be to be able to provide a full testharness without
> adding any additional dependencies over what we already have (Python being
> required by meson). That should ideally make it easy to get good coverage from
> BF animals as no installation is needed.
As an additional note, the test suite ideally needs to be able to
exercise failure modes where the provider itself is malfunctioning. So
we hand-roll responses rather than deferring to an external
OAuth/OpenID implementation, which adds HTTP and JSON dependencies at
minimum, and Python includes both. See also the discussion with
Stephen upthread [1].
(I do think it'd be nice to eventually include a prepackaged OAuth
server in the test suite, to stack coverage for the happy path and
further test interoperability.)
Thanks,
--Jacob
[1] https://postgr.es/m/CAAWbhmh%2B6q4t3P%2BwDmS%3DJuHBpcgF-VM2cXNft8XV02yk-cHCpQ%40mail.gmail.com
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-02-29 21:08 Daniel Gustafsson <daniel@yesql.se>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
2 siblings, 1 reply; 424+ messages in thread
From: Daniel Gustafsson @ 2024-02-29 21:08 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: pgsql-hackers; Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
> On 27 Feb 2024, at 20:20, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
>
> On Fri, Feb 23, 2024 at 5:01 PM Jacob Champion
> <jacob.champion@enterprisedb.com> wrote:
>> The
>> patchset is now carrying a lot of squash-cruft, and I plan to flatten
>> it in the next version.
>
> This is done in v17, which is also now based on the two patches pulled
> out by Daniel in [1]. Besides the squashes, which make up most of the
> range-diff, I've fixed a call to strncasecmp() which is not available
> on Windows.
Two quick questions:
+ /* TODO */
+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, stderr);
I might be missing something, but what this is intended for in
setup_curl_handles()?
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth-iddawc.c
As discussed off-list I think we should leave iddawc support for later and
focus on getting one library properly supported to start with. If you agree,
let's drop this from the patchset to make it easier to digest. We should make
sure we keep pluggability such that another library can be supported though,
much like the libpq TLS support.
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-03-01 00:04 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Daniel Gustafsson <daniel@yesql.se>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2024-03-01 00:04 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: pgsql-hackers; Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Wed, Feb 28, 2024 at 9:40 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> +#define ALLOC(size) malloc(size)
> I wonder if we should use pg_malloc_extended(size, MCXT_ALLOC_NO_OOM) instead
> to self document the code. We clearly don't want feature-parity with server-
> side palloc here. I know we use malloc in similar ALLOC macros so it's not
> unique in that regard, but maybe?
I have a vague recollection that linking fe_memutils into libpq
tripped the exit() checks, but I can try again and see.
> +#ifdef FRONTEND
> + destroyPQExpBuffer(lex->errormsg);
> +#else
> + pfree(lex->errormsg->data);
> + pfree(lex->errormsg);
> +#endif
> Wouldn't it be nicer if we abstracted this into a destroyStrVal function to a)
> avoid the ifdefs and b) make it more like the rest of the new API? While it's
> only used in two places (close to each other) it's a shame to let the
> underlying API bleed through the abstraction.
Good idea. I'll fold this from your patch into the next set (and do
the same for the ones I've marked +1 below).
> + CURLM *curlm; /* top-level multi handle for cURL operations */
> Nitpick, but curl is not capitalized cURL anymore (for some value of "anymore"
> since it changed in 2016 [0]). I do wonder if we should consistently write
> "libcurl" as well since we don't use curl but libcurl.
Huh, I missed that memo. Thanks -- that makes it much easier to type!
> + PQExpBufferData work_data; /* scratch buffer for general use (remember
> + to clear out prior contents first!) */
> This seems like asking for subtle bugs due to uncleared buffers bleeding into
> another operation (especially since we are writing this data across the wire).
> How about having an array the size of OAuthStep of unallocated buffers where
> each step use it's own? Storing the content of each step could also be useful
> for debugging. Looking at the statemachine here it's not an obvious change but
> also not impossible.
I like that idea; I'll give it a look.
> + * TODO: This disables DNS resolution timeouts unless libcurl has been
> + * compiled against alternative resolution support. We should check that.
> curl_version_info() can be used to check for c-ares support.
+1
> + * so you don't have to write out the error handling every time. They assume
> + * that they're embedded in a function returning bool, however.
> It feels a bit iffy to encode the returntype in the macro, we can use the same
> trick that DISABLE_SIGPIPE employs where a failaction is passed in.
+1
> + if (!strcmp(name, field->name))
> Project style is to test for (strcmp(x,y) == 0) rather than (!strcmp()) to
> improve readability.
+1
> + libpq_append_conn_error(conn, "out of memory");
> While not introduced in this patch, it's not an ideal pattern to report "out of
> memory" errors via a function which may allocate memory.
Does trying (and failing) to allocate more memory cause any harm? Best
case, we still have enough room in the errorMessage to fit the whole
error; worst case, we mark the errorMessage broken and then
PQerrorMessage() can handle it correctly.
> + appendPQExpBufferStr(&conn->errorMessage,
> + libpq_gettext("server's error message contained an embedded NULL"));
> We should maybe add ", discarding" or something similar after this string to
> indicate that there was an actual error which has been thrown away, the error
> wasn't that the server passed an embedded NULL.
+1
> +#ifdef USE_OAUTH
> + else if (strcmp(mechanism_buf.data, OAUTHBEARER_NAME) == 0 &&
> + !selected_mechanism)
> I wonder if we instead should move the guards inside the statement and error
> out with "not built with OAuth support" or something similar like how we do
> with TLS and other optional components?
This one seems like a step backwards. IIUC, the client can currently
handle a situation where the server returns multiple mechanisms
(though the server doesn't support that yet), and I'd really like to
make use of that property without making users upgrade libpq.
That said, it'd be good to have a more specific error message in the
case where we don't have a match...
> + errdetail("Comma expected, but found character %s.",
> + sanitize_char(*p))));
> The %s formatter should be wrapped like '%s' to indicate that the message part
> is the character in question (and we can then reuse the translation since the
> error message already exist for SCRAM).
+1
> + temp = curl_slist_append(temp, "authorization_code");
> + if (!temp)
> + oom = true;
> +
> + temp = curl_slist_append(temp, "implicit");
> While not a bug per se, it reads a bit odd to call another operation that can
> allocate memory when the oom flag has been set. I think we can move some
> things around a little to make it clearer.
I'm not a huge fan of nested happy paths/pyramids of doom, but in this
case it's small enough that I'm not opposed. :D
> The attached diff contains some (most?) of the above as a patch on top of your
> v17, but as a .txt to keep the CFBot from munging on it.
Thanks very much! I plan to apply all but the USE_OAUTH guard change
(but let me know if you feel strongly about it).
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-03-01 00:11 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Daniel Gustafsson <daniel@yesql.se>
0 siblings, 0 replies; 424+ messages in thread
From: Jacob Champion @ 2024-03-01 00:11 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: pgsql-hackers; Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>
On Thu, Feb 29, 2024 at 1:08 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> + /* TODO */
> + CHECK_SETOPT(actx, CURLOPT_WRITEDATA, stderr);
> I might be missing something, but what this is intended for in
> setup_curl_handles()?
Ah, that's cruft left over from early debugging, just so that I could
see what was going on. I'll remove it.
> --- /dev/null
> +++ b/src/interfaces/libpq/fe-auth-oauth-iddawc.c
> As discussed off-list I think we should leave iddawc support for later and
> focus on getting one library properly supported to start with. If you agree,
> let's drop this from the patchset to make it easier to digest. We should make
> sure we keep pluggability such that another library can be supported though,
> much like the libpq TLS support.
Agreed. The number of changes being folded into the next set is
already pretty big so I think this will wait until next+1.
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-03-01 01:08 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2024-03-01 01:08 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: pgsql-hackers; Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>; Andrew Dunstan <andrew@dunslane.net>
On Thu, Feb 29, 2024 at 4:04 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> On Wed, Feb 28, 2024 at 9:40 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> > + temp = curl_slist_append(temp, "authorization_code");
> > + if (!temp)
> > + oom = true;
> > +
> > + temp = curl_slist_append(temp, "implicit");
> > While not a bug per se, it reads a bit odd to call another operation that can
> > allocate memory when the oom flag has been set. I think we can move some
> > things around a little to make it clearer.
>
> I'm not a huge fan of nested happy paths/pyramids of doom, but in this
> case it's small enough that I'm not opposed. :D
I ended up rewriting this patch hunk a bit to handle earlier OOM
failures; let me know what you think.
--
v18 is the result of plenty of yak shaving now that the Windows build
is working. In addition to Daniel's changes as discussed upthread,
- I have rebased over v2 of the SASL-refactoring patches
- the last CompilerWarnings failure has been fixed
- the py.test suite now runs on Windows (but does not yet completely pass)
- py.test has been completely disabled for the 32-bit Debian test in
Cirrus; I don't know if there's a way to install 32-bit Python
side-by-side with 64-bit
We are now very, very close to green.
The new oauth_validator tests can't work on Windows, since the client
doesn't support OAuth there. The python/server tests can handle this
case, since they emulate the client behavior; do we want to try
something similar in Perl?
--Jacob
Attachments: download all (10 files, .tar.gz)
since-v17.diff.txt (text/plain, 30.6K) download
1: 00976d4f75 ! 1: e2a0b48561 common/jsonapi: support FRONTEND clients
@@ Commit message
We can now partially revert b44669b2ca, now that json_errdetail() works
correctly.
+ Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
+
## src/bin/pg_verifybackup/t/005_bad_manifest.pl ##
@@ src/bin/pg_verifybackup/t/005_bad_manifest.pl: use Test::More;
my $tempdir = PostgreSQL::Test::Utils::tempdir;
@@ src/common/jsonapi.c
+#define appendStrValChar appendPQExpBufferChar
+#define createStrVal createPQExpBuffer
+#define resetStrVal resetPQExpBuffer
++#define destroyStrVal destroyPQExpBuffer
+
+#else /* !FRONTEND */
+
@@ src/common/jsonapi.c
+#define appendStrValChar appendStringInfoChar
+#define createStrVal makeStringInfo
+#define resetStrVal resetStringInfo
++#define destroyStrVal destroyStringInfo
+
+#endif
+
@@ src/common/jsonapi.c: makeJsonLexContextCstringLen(JsonLexContext *lex, char *js
+ static const JsonLexContext empty = {0};
+
if (lex->flags & JSONLEX_FREE_STRVAL)
- {
-+#ifdef FRONTEND
-+ destroyPQExpBuffer(lex->strval);
-+#else
- pfree(lex->strval->data);
- pfree(lex->strval);
-+#endif
-+ }
+- {
+- pfree(lex->strval->data);
+- pfree(lex->strval);
+- }
++ destroyStrVal(lex->strval);
++
+ if (lex->errormsg)
-+ {
-+#ifdef FRONTEND
-+ destroyPQExpBuffer(lex->errormsg);
-+#else
-+ pfree(lex->errormsg->data);
-+ pfree(lex->errormsg);
-+#endif
- }
++ destroyStrVal(lex->errormsg);
++
if (lex->flags & JSONLEX_FREE_STRUCT)
pfree(lex);
+ else
@@ src/common/parse_manifest.c: json_parse_manifest(JsonManifestParseContext *conte
json_manifest_parse_failure(context, "manifest ended unexpectedly");
+ ## src/common/stringinfo.c ##
+@@ src/common/stringinfo.c: enlargeStringInfo(StringInfo str, int needed)
+
+ str->maxlen = newlen;
+ }
++
++void
++destroyStringInfo(StringInfo str)
++{
++ pfree(str->data);
++ pfree(str);
++}
+
## src/include/common/jsonapi.h ##
@@
#ifndef JSONAPI_H
@@ src/include/common/jsonapi.h: typedef struct JsonLexContext
} JsonLexContext;
typedef JsonParseErrorType (*json_struct_action) (void *state);
+
+ ## src/include/lib/stringinfo.h ##
+@@ src/include/lib/stringinfo.h: extern void appendBinaryStringInfoNT(StringInfo str,
+ */
+ extern void enlargeStringInfo(StringInfo str, int needed);
+
++
++extern void destroyStringInfo(StringInfo str);
+ #endif /* STRINGINFO_H */
2: d8b567dd55 ! 2: db625e1d01 Refactor SASL exchange to return tri-state status
@@ src/interfaces/libpq/fe-auth-sasl.h: typedef struct pg_fe_sasl_mech
+ * SASL_CONTINUE: The output buffer is filled with a client response.
+ * Additional server challenge is expected
+ * SASL_COMPLETE: The SASL exchange has completed successfully.
-+ * SASL_FAILED: The exchance has failed and the connection should be
++ * SASL_FAILED: The exchange has failed and the connection should be
+ * dropped.
*--------
*/
@@ src/interfaces/libpq/fe-auth.c: pg_SASL_continue(PGconn *conn, int payloadlen, b
free(challenge); /* don't need the input anymore */
- if (final && !done)
-+ if (final && !(status == SASL_FAILED || status == SASL_COMPLETE))
++ if (final && status == SASL_CONTINUE)
{
if (outputlen != 0)
free(output);
3: 83d78f598c ! 3: e4ad0260d5 Explicitly require password for SCRAM exchange
@@ Commit message
Discussion: https://postgr.es/m/d1b467a78e0e36ed85a09adf979d04cf124a9d4b.camel@vmware.com
## src/interfaces/libpq/fe-auth.c ##
+@@ src/interfaces/libpq/fe-auth.c: pg_SASL_init(PGconn *conn, int payloadlen)
+ int initialresponselen;
+ const char *selected_mechanism;
+ PQExpBufferData mechanism_buf;
+- char *password;
++ char *password = NULL;
+ SASLStatus status;
+
+ initPQExpBuffer(&mechanism_buf);
@@ src/interfaces/libpq/fe-auth.c: pg_SASL_init(PGconn *conn, int payloadlen)
/*
* Parse the list of SASL authentication mechanisms in the
4: 00c8073807 ! 4: 229f602d5c libpq: add OAUTHBEARER SASL mechanism
@@ Commit message
- figure out pgsocket/int difference on Windows
- fix intermittent failure in the cleanup callback tests (race
condition?)
+ - support require_auth
- ...and more.
+ Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
+
## configure ##
@@ configure: with_uuid
with_readline
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+
+/*
+ * Macros for getting and setting state for the connection's two cURL handles,
-+ * so you don't have to write out the error handling every time. They assume
-+ * that they're embedded in a function returning bool, however.
++ * so you don't have to write out the error handling every time.
+ */
+
-+#define CHECK_MSETOPT(ACTX, OPT, VAL) \
++#define CHECK_MSETOPT(ACTX, OPT, VAL, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLMcode _setopterr = curl_multi_setopt(_actx->curlm, OPT, VAL); \
+ if (_setopterr) { \
+ actx_error(_actx, "failed to set %s on OAuth connection: %s",\
+ #OPT, curl_multi_strerror(_setopterr)); \
-+ return false; \
++ FAILACTION; \
+ } \
+ } while (0)
+
-+#define CHECK_SETOPT(ACTX, OPT, VAL) \
++#define CHECK_SETOPT(ACTX, OPT, VAL, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLcode _setopterr = curl_easy_setopt(_actx->curl, OPT, VAL); \
+ if (_setopterr) { \
+ actx_error(_actx, "failed to set %s on OAuth connection: %s",\
+ #OPT, curl_easy_strerror(_setopterr)); \
-+ return false; \
++ FAILACTION; \
+ } \
+ } while (0)
+
-+#define CHECK_GETINFO(ACTX, INFO, OUT) \
++#define CHECK_GETINFO(ACTX, INFO, OUT, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLcode _getinfoerr = curl_easy_getinfo(_actx->curl, INFO, OUT); \
+ if (_getinfoerr) { \
+ actx_error(_actx, "failed to get %s from OAuth response: %s",\
+ #INFO, curl_easy_strerror(_getinfoerr)); \
-+ return false; \
++ FAILACTION; \
+ } \
+ } while (0)
+
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+
+ while (field->name)
+ {
-+ if (!strcmp(name, field->name))
++ if (strcmp(name, field->name) == 0)
+ {
+ ctx->active = field;
+ break;
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ bool success = false;
+
+ /* Make sure the server thinks it's given us JSON. */
-+ CHECK_GETINFO(actx, CURLINFO_CONTENT_TYPE, &content_type);
++ CHECK_GETINFO(actx, CURLINFO_CONTENT_TYPE, &content_type, return false);
+
+ if (!content_type)
+ {
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+static int
+parse_interval(const char *interval_str)
+{
-+ float parsed;
++ double parsed;
+ int cnt;
+
+ /*
+ * The JSON lexer has already validated the number, which is stricter than
+ * the %f format, so we should be good to use sscanf().
+ */
-+ cnt = sscanf(interval_str, "%f", &parsed);
++ cnt = sscanf(interval_str, "%lf", &parsed);
+
+ if (cnt != 1)
+ {
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ return 1; /* don't fall through in release builds */
+ }
+
-+ parsed = ceilf(parsed);
++ parsed = ceil(parsed);
+
+ if (parsed < 1)
+ return 1; /* TODO this slows down the tests
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+static bool
+setup_curl_handles(struct async_ctx *actx)
+{
++ curl_version_info_data *curl_info;
++
+ /*
+ * Create our multi handle. This encapsulates the entire conversation with
+ * cURL for this connection.
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ }
+
+ /*
++ * Extract information about the libcurl we are linked against.
++ */
++ curl_info = curl_version_info(CURLVERSION_NOW);
++
++ /*
+ * The multi handle tells us what to wait on using two callbacks. These
+ * will manipulate actx->mux as needed.
+ */
-+ CHECK_MSETOPT(actx, CURLMOPT_SOCKETFUNCTION, register_socket);
-+ CHECK_MSETOPT(actx, CURLMOPT_SOCKETDATA, actx);
-+ CHECK_MSETOPT(actx, CURLMOPT_TIMERFUNCTION, register_timer);
-+ CHECK_MSETOPT(actx, CURLMOPT_TIMERDATA, actx);
++ CHECK_MSETOPT(actx, CURLMOPT_SOCKETFUNCTION, register_socket, return false);
++ CHECK_MSETOPT(actx, CURLMOPT_SOCKETDATA, actx, return false);
++ CHECK_MSETOPT(actx, CURLMOPT_TIMERFUNCTION, register_timer, return false);
++ CHECK_MSETOPT(actx, CURLMOPT_TIMERDATA, actx, return false);
+
+ /*
+ * Set up an easy handle. All of our requests are made serially, so we
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ * Multi-threaded applications must set CURLOPT_NOSIGNAL. This requires us
+ * to handle the possibility of SIGPIPE ourselves.
+ *
-+ * TODO: This disables DNS resolution timeouts unless libcurl has been
-+ * compiled against alternative resolution support. We should check that.
-+ *
+ * TODO: handle SIGPIPE via pq_block_sigpipe(), or via a
+ * CURLOPT_SOCKOPTFUNCTION maybe...
+ */
-+ CHECK_SETOPT(actx, CURLOPT_NOSIGNAL, 1L);
++ CHECK_SETOPT(actx, CURLOPT_NOSIGNAL, 1L, return false);
++ if (!curl_info->ares_num)
++ {
++ /* No alternative resolver, TODO: warn about timeouts */
++ }
+
+ /* TODO investigate using conn->Pfdebug and CURLOPT_DEBUGFUNCTION here */
-+ CHECK_SETOPT(actx, CURLOPT_VERBOSE, 1L);
-+ CHECK_SETOPT(actx, CURLOPT_ERRORBUFFER, actx->curl_err);
-+
-+ /* TODO */
-+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, stderr);
++ CHECK_SETOPT(actx, CURLOPT_VERBOSE, 1L, return false);
++ CHECK_SETOPT(actx, CURLOPT_ERRORBUFFER, actx->curl_err, return false);
+
+ /*
+ * Only HTTP[S] is allowed. TODO: disallow HTTP without user opt-in
+ */
-+ CHECK_SETOPT(actx, CURLOPT_PROTOCOLS, CURLPROTO_HTTP | CURLPROTO_HTTPS);
++ CHECK_SETOPT(actx, CURLOPT_PROTOCOLS, CURLPROTO_HTTP | CURLPROTO_HTTPS, return false);
+
+ /*
+ * Suppress the Accept header to make our request as minimal as possible.
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ * what comes back anyway.)
+ */
+ actx->headers = curl_slist_append(actx->headers, "Accept:"); /* TODO: check result */
-+ CHECK_SETOPT(actx, CURLOPT_HTTPHEADER, actx->headers);
++ CHECK_SETOPT(actx, CURLOPT_HTTPHEADER, actx->headers, return false);
+
+ return true;
+}
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ int running;
+
+ resetPQExpBuffer(&actx->work_data);
-+ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data);
-+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, &actx->work_data);
++ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data, return false);
++ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, &actx->work_data, return false);
+
+ err = curl_multi_add_handle(actx->curlm, actx->curl);
+ if (err)
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+static bool
+start_discovery(struct async_ctx *actx, const char *discovery_uri)
+{
-+ CHECK_SETOPT(actx, CURLOPT_HTTPGET, 1L);
-+ CHECK_SETOPT(actx, CURLOPT_URL, discovery_uri);
++ CHECK_SETOPT(actx, CURLOPT_HTTPGET, 1L, return false);
++ CHECK_SETOPT(actx, CURLOPT_URL, discovery_uri, return false);
+
+ return start_request(actx);
+}
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ * validation into question), or non-authoritative responses, or any other
+ * complications.
+ */
-+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code);
++ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ if (response_code != 200)
+ {
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ * Per Section 3, the default is ["authorization_code", "implicit"].
+ */
+ struct curl_slist *temp = actx->provider.grant_types_supported;
-+ bool oom = false;
+
+ temp = curl_slist_append(temp, "authorization_code");
-+ if (!temp)
-+ oom = true;
++ if (temp)
++ {
++ temp = curl_slist_append(temp, "implicit");
++ }
+
-+ temp = curl_slist_append(temp, "implicit");
+ if (!temp)
-+ oom = true;
-+
-+ if (oom)
+ {
+ actx_error(actx, "out of memory");
+ return false;
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ /* TODO check for broken buffer */
+
+ /* Make our request. */
-+ CHECK_SETOPT(actx, CURLOPT_URL, device_authz_uri);
-+ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data);
++ CHECK_SETOPT(actx, CURLOPT_URL, device_authz_uri, return false);
++ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data, return false);
+
+ if (conn->oauth_client_secret)
+ {
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ *
+ * TODO: should we omit client_id from the body in this case?
+ */
-+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
-+ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id);
-+ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret);
++ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC, return false);
++ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id, return false);
++ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret, return false);
+ }
+ else
-+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE);
++ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE, return false);
+
+ return start_request(actx);
+}
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+{
+ long response_code;
+
-+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code);
++ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ /*
+ * The device authorization endpoint uses the same error response as the
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ /* TODO check for broken buffer */
+
+ /* Make our request. */
-+ CHECK_SETOPT(actx, CURLOPT_URL, token_uri);
-+ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data);
++ CHECK_SETOPT(actx, CURLOPT_URL, token_uri, return false);
++ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data, return false);
+
+ if (conn->oauth_client_secret)
+ {
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ *
+ * TODO: should we omit client_id from the body in this case?
+ */
-+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
-+ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id);
-+ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret);
++ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC, return false);
++ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id, return false);
++ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret, return false);
+ }
+ else
-+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE);
++ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE, return false);
+
+ resetPQExpBuffer(work_buffer);
-+ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data);
-+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, work_buffer);
++ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data, return false);
++ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, work_buffer, return false);
+
+ return start_request(actx);
+}
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+{
+ long response_code;
+
-+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code);
++ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ /*
+ * Per RFC 6749, Section 5, a successful response uses 200 OK. An error
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+ * A slow_down error requires us to permanently increase our
+ * retry interval by five seconds. RFC 8628, Sec. 3.5.
+ */
-+ if (!strcmp(err->error, "slow_down"))
++ if (strcmp(err->error, "slow_down") == 0)
+ {
+ actx->authz.interval += 5; /* TODO check for overflow? */
+ }
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+ * error.
+ */
+ Assert(sasl_mechanism != NULL);
-+ Assert(!strcmp(sasl_mechanism, OAUTHBEARER_NAME));
++ Assert(strcmp(sasl_mechanism, OAUTHBEARER_NAME) == 0);
+
+ state = calloc(1, sizeof(*state));
+ if (!state)
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+
+ if (ctx->nested == 1)
+ {
-+ if (!strcmp(name, ERROR_STATUS_FIELD))
++ if (strcmp(name, ERROR_STATUS_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_STATUS_FIELD;
+ ctx->target_field = &ctx->status;
+ }
-+ else if (!strcmp(name, ERROR_SCOPE_FIELD))
++ else if (strcmp(name, ERROR_SCOPE_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_SCOPE_FIELD;
+ ctx->target_field = &ctx->scope;
+ }
-+ else if (!strcmp(name, ERROR_OPENID_CONFIGURATION_FIELD))
++ else if (strcmp(name, ERROR_OPENID_CONFIGURATION_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_OPENID_CONFIGURATION_FIELD;
+ ctx->target_field = &ctx->discovery_uri;
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+ if (strlen(msg) != msglen)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
-+ libpq_gettext("server's error message contained an embedded NULL"));
++ libpq_gettext("server's error message contained an embedded NULL, and was discarded"));
+ return false;
+ }
+
@@ src/interfaces/libpq/fe-auth-oauth.c (new)
+ return false;
+ }
+
-+ if (!strcmp(ctx.status, "invalid_token"))
++ if (strcmp(ctx.status, "invalid_token") == 0)
+ {
+ /*
+ * invalid_token is the only error code we'll automatically retry for,
@@ src/interfaces/libpq/fe-auth-sasl.h: typedef struct pg_fe_sasl_mech
+ * generated. The mechanism is responsible for setting up
+ * conn->async_auth appropriately before returning.
* SASL_COMPLETE: The SASL exchange has completed successfully.
-- * SASL_FAILED: The exchance has failed and the connection should be
-+ * SASL_FAILED: The exchange has failed and the connection should be
+ * SASL_FAILED: The exchange has failed and the connection should be
* dropped.
*--------
*/
@@ src/interfaces/libpq/fe-auth.c: pg_SSPI_startup(PGconn *conn, int use_negotiate,
{
char *initialresponse = NULL;
int initialresponselen;
- const char *selected_mechanism;
- PQExpBufferData mechanism_buf;
-- char *password;
-+ char *password = NULL;
- SASLStatus status;
-
- initPQExpBuffer(&mechanism_buf);
@@ src/interfaces/libpq/fe-auth.c: pg_SASL_init(PGconn *conn, int payloadlen)
goto error;
}
@@ src/interfaces/libpq/fe-auth.c: pg_SASL_continue(PGconn *conn, int payloadlen, b
+ return STATUS_OK;
+ }
+
- if (final && !(status == SASL_FAILED || status == SASL_COMPLETE))
+ if (final && status == SASL_CONTINUE)
{
if (outputlen != 0)
@@ src/interfaces/libpq/fe-auth.c: check_expected_areq(AuthRequest areq, PGconn *conn)
5: 29d7e3cbed ! 5: 5c5a83e44e backend: add OAUTHBEARER SASL mechanism
@@ Commit message
deal with multi-issuer setups
- ...and more.
+ Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
+
## src/backend/libpq/Makefile ##
@@ src/backend/libpq/Makefile: include $(top_builddir)/src/Makefile.global
# be-fsstubs is here for historical reasons, probably belongs elsewhere
@@ src/backend/libpq/auth-oauth.c (new)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
-+ errdetail("Comma expected, but found character %s.",
++ errdetail("Comma expected, but found character \"%s\".",
+ sanitize_char(*p))));
+ p++;
+ break;
6: 0661817808 = 6: 7a42365d62 Introduce OAuth validator libraries
7: 45755e8461 ! 7: 9c46ea6cf9 Add pytest suite for OAuth
@@ Commit message
TODOs:
- The --tap-stream option to pytest-tap is slightly broken during test
failures (it suppresses error information), which impedes debugging.
- - Unsurprisingly, Windows builds fail on the Linux-/BSD-specific backend
- changes. 32-bit builds on Ubuntu fail during testing as well.
- pyca/cryptography is pinned at an old version. Since we use it for
testing and not security, this isn't a critical problem yet, but it's
not ideal. Newer versions require a Rust compiler to build, and while
@@ Commit message
with the Rust pieces bypassed, compilation on FreeBSD takes a while.
- The with_oauth test skip logic should probably be integrated into the
Makefile side as well...
+ - See if 32-bit tests can be enabled with a 32-bit Python.
## .cirrus.tasks.yml ##
@@ .cirrus.tasks.yml: env:
@@ .cirrus.tasks.yml: task:
matrix:
- name: Linux - Debian Bullseye - Autoconf
+@@ .cirrus.tasks.yml: task:
+
+ # Also build & test in a 32bit build - it's gotten rare to test that
+ # locally.
++ # XXX 32-bit Python tests are currently disabled, as the system's 64-bit
++ # Python modules can't link against libpq.
+ configure_32_script: |
+ su postgres <<-EOF
+ export CC='ccache gcc -m32'
+@@ .cirrus.tasks.yml: task:
+ -Dllvm=disabled \
+ --pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
+ -DPERL=perl5.32-i386-linux-gnu \
+- -DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
++ -DPG_TEST_EXTRA="${PG_TEST_EXTRA//"python"}" \
+ build-32
+ EOF
+
@@ .cirrus.tasks.yml: task:
folder: $CCACHE_DIR
@@ meson.build: foreach test_dir : tests
+ env.set(name, value)
+ endforeach
+
-+ reqs = files(t['requirements'])
-+ test('install_' + venv_name,
-+ python,
-+ args: [ make_venv, '--requirements', reqs, venv_path ],
-+ env: env,
-+ priority: setup_tests_priority - 1, # must run after tmp_install
-+ is_parallel: false,
-+ suite: ['setup'],
-+ timeout: 60, # 30s is too short for the cryptography package compile
-+ )
++ if get_option('PG_TEST_EXTRA').contains('python')
++ reqs = files(t['requirements'])
++ test('install_' + venv_name,
++ python,
++ args: [ make_venv, '--requirements', reqs, venv_path ],
++ env: env,
++ priority: setup_tests_priority - 1, # must run after tmp_install
++ is_parallel: false,
++ suite: ['setup'],
++ timeout: 60, # 30s is too short for the cryptography package compile
++ )
++ endif
+
+ test_group = test_dir['name']
+ test_output = test_result_dir / test_group / kind
@@ meson.build: foreach test_dir : tests
+ 'env': env,
+ }
+
-+ pytest = venv_path / 'bin' / 'py.test'
++ if fs.is_dir(venv_path / 'Scripts')
++ # Windows virtualenv layout
++ pytest = venv_path / 'Scripts' / 'py.test'
++ else
++ pytest = venv_path / 'bin' / 'py.test'
++ endif
++
+ test_command = [
+ pytest,
+ # Avoid running these tests against an existing database.
@@ meson.build: foreach test_dir : tests
+ pyt_p = fs.stem(pyt_p)
+ endif
+
++ testwrap_pytest = testwrap_base + [
++ '--testgroup', test_group,
++ '--testname', pyt_p,
++ ]
++ if not get_option('PG_TEST_EXTRA').contains('python')
++ testwrap_pytest += ['--skip', '"python" tests not enabled in PG_TEST_EXTRA']
++ endif
++
+ test(test_group / pyt_p,
+ python,
+ kwargs: test_kwargs,
-+ args: testwrap_base + [
-+ '--testgroup', test_group,
-+ '--testname', pyt_p,
++ args: testwrap_pytest + [
+ '--', test_command,
+ test_dir['sd'] / pyt,
+ ],
@@ src/test/python/client/test_oauth.py (new)
+
+if platform.system() == "Darwin":
+ libpq = ctypes.cdll.LoadLibrary("libpq.5.dylib")
++elif platform.system() == "Windows":
++ pass # TODO
+else:
+ libpq = ctypes.cdll.LoadLibrary("libpq.so.5")
+
@@ src/test/python/conftest.py (new)
+ Automatically skips the whole suite if PG_TEST_EXTRA doesn't contain
+ 'python'. pytestmark doesn't seem to work in a top-level conftest.py, so
+ I've made this an autoused fixture instead.
-+
-+ TODO: there are tests here that are probably safe, but until I do a full
-+ analysis on which are and which are not, I've made the entire thing opt-in.
+ """
+ extra_tests = os.getenv("PG_TEST_EXTRA", "").split()
+ if "python" not in extra_tests:
@@ src/test/python/pq3.py (new)
+import getpass
+import io
+import os
++import platform
+import ssl
+import sys
+import textwrap
@@ src/test/python/pq3.py (new)
+ try:
+ return os.environ["PGUSER"]
+ except KeyError:
++ if platform.system() == "Windows":
++ # libpq defaults to GetUserName() on Windows.
++ return os.getlogin()
+ return getpass.getuser()
+
+
@@ src/test/python/server/meson.build (new)
@@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
-+if not oauth.found()
-+ subdir_done()
-+endif
-+
+oauthtest_sources = files(
+ 'oauthtest.c',
+)
@@ src/test/python/server/test_oauth.py (new)
+import json
+import os
+import pathlib
++import platform
+import secrets
+import shlex
+import shutil
@@ src/test/python/server/test_oauth.py (new)
+ scope = "openid " + id
+
+ ctx = Context()
-+ hba_lines = (
++ hba_lines = [
+ f'host {ctx.dbname} {ctx.map_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" map=oauth\n',
+ f'host {ctx.dbname} {ctx.authz_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" trust_validator_authz=1\n',
+ f'host {ctx.dbname} all samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}"\n',
-+ )
-+ ident_lines = (r"oauth /^(.*)@example\.com$ \1",)
++ ]
++ ident_lines = [r"oauth /^(.*)@example\.com$ \1"]
++
++ if platform.system() == "Windows":
++ # XXX why is 'samehost' not behaving as expected on Windows?
++ for l in list(hba_lines):
++ hba_lines.append(l.replace("samehost", "::1/128"))
+
+ host, port = postgres_instance
+ conn = psycopg2.connect(host=host, port=port)
@@ src/test/python/test_pq3.py (new)
+import contextlib
+import getpass
+import io
++import platform
+import struct
+import sys
+
@@ src/test/python/test_pq3.py (new)
+ [
+ ("PGHOST", pq3.pghost, "localhost"),
+ ("PGPORT", pq3.pgport, 5432),
-+ ("PGUSER", pq3.pguser, getpass.getuser()),
++ (
++ "PGUSER",
++ pq3.pguser,
++ os.getlogin() if platform.system() == "Windows" else getpass.getuser(),
++ ),
+ ("PGDATABASE", pq3.pgdatabase, "postgres"),
+ ],
+)
@@ src/tools/make_venv (new)
+import argparse
+import subprocess
+import os
++import platform
+import sys
+
+parser = argparse.ArgumentParser()
@@ src/tools/make_venv (new)
+
+# Update pip next. This helps avoid old pip bugs; the version inside system
+# Pythons tends to be pretty out of date.
-+pip = os.path.join(args.venv_path, 'bin', 'pip')
-+run(pip, 'install', '-U', 'pip')
++bindir = 'Scripts' if platform.system() == 'Windows' else 'bin'
++python = os.path.join(args.venv_path, bindir, 'python3')
++run(python, '-m', 'pip', 'install', '-U', 'pip')
+
+# Finally, install the test's requirements. We need pytest and pytest-tap, no
+# matter what the test needs.
++pip = os.path.join(args.venv_path, bindir, 'pip')
+run(pip, 'install', 'pytest', 'pytest-tap')
+if args.requirements:
+ run(pip, 'install', '-r', args.requirements)
8: 0f9f884856 ! 8: 8ad4ce3068 XXX temporary patches to build and test
@@ src/bin/pg_verifybackup/Makefile: top_builddir = ../../..
## src/interfaces/libpq/Makefile ##
@@ src/interfaces/libpq/Makefile: libpq-refs-stamp: $(shlib)
ifneq ($(enable_coverage), yes)
- ifeq (,$(filter aix solaris,$(PORTNAME)))
+ ifeq (,$(filter solaris,$(PORTNAME)))
@if nm -A -u $< 2>/dev/null | grep -v -e __cxa_atexit -e __tsan_func_exit | grep exit; then \
- echo 'libpq must not be calling any function which invokes exit'; exit 1; \
+ echo 'libpq must not be calling any function which invokes exit'; \
9: de8f81bd7d ! 9: 5630465578 WIP: Python OAuth provider implementation
@@ src/test/modules/oauth_validator/t/oauth_server.py (new)
+import json
+import os
+import sys
-+from typing import TypeAlias
+
+
+class OAuthHandler(http.server.BaseHTTPRequestHandler):
-+ JsonObject: TypeAlias = dict[str, object]
++ JsonObject = dict[str, object] # TypeAlias is not available until 3.10
+
+ def do_GET(self):
+ if self.path == "/.well-known/openid-configuration":
v18-0005-backend-add-OAUTHBEARER-SASL-mechanism.patch (application/octet-stream, 39.9K) download
From 5c5a83e44e74d022b65aa884e369653bb6a0c175 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 4 May 2021 16:21:11 -0700
Subject: [PATCH v18 5/9] backend: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) on the server side. This adds a new
auth method, oauth, to pg_hba.
Because OAuth implementations vary so wildly, and bearer token
validation is heavily dependent on the issuing party, authn/z is done by
communicating with an external program: the oauth_validator_command.
This command must do the following:
1. Receive the bearer token by reading its contents from a file
descriptor passed from the server. (The numeric value of this
descriptor may be inserted into the oauth_validator_command using the
%f specifier.)
This MUST be the first action the command performs. The server will
not begin reading stdout from the command until the token has been
read in full, so if the command tries to print anything and hits a
buffer limit, the backend will deadlock and time out.
2. Validate the bearer token. The correct way to do this depends on the
issuer, but it generally involves either cryptographic operations to
prove that the token was issued by a trusted party, or the
presentation of the bearer token to some other party so that _it_ can
perform validation.
The command MUST maintain confidentiality of the bearer token, since
in most cases it can be used just like a password. (There are ways to
cryptographically bind tokens to client certificates, but they are
way beyond the scope of this commit message.)
If the token cannot be validated, the command must exit with a
non-zero status. Further authentication/authorization is pointless if
the bearer token wasn't issued by someone you trust.
3. Authenticate the user, authorize the user, or both:
a. To authenticate the user, use the bearer token to retrieve some
trusted identifier string for the end user. The exact process for
this is, again, issuer-dependent. The command should print the
authenticated identity string to stdout, followed by a newline.
If the user cannot be authenticated, the validator should not
print anything to stdout. It should also exit with a non-zero
status, unless the token may be used to authorize the connection
through some other means (see below).
On a success, the command may then exit with a zero success code.
By default, the server will then check to make sure the identity
string matches the role that is being used (or matches a usermap
entry, if one is in use).
b. To optionally authorize the user, in combination with the HBA
option trust_validator_authz=1 (see below), the validator simply
returns a zero exit code if the client should be allowed to
connect with its presented role (which can be passed to the
command using the %r specifier), or a non-zero code otherwise.
The hard part is in determining whether the given token truly
authorizes the client to use the given role, which must
unfortunately be left as an exercise to the reader.
This obviously requires some care, as a poorly implemented token
validator may silently open the entire database to anyone with a
bearer token. But it may be a more portable approach, since OAuth
is designed as an authorization framework, not an authentication
framework. For example, the user's bearer token could carry an
"allow_superuser_access" claim, which would authorize pseudonymous
database access as any role. It's then up to the OAuth system
administrators to ensure that allow_superuser_access is doled out
only to the proper users.
c. It's possible that the user can be successfully authenticated but
isn't authorized to connect. In this case, the command may print
the authenticated ID and then fail with a non-zero exit code.
(This makes it easier to see what's going on in the Postgres
logs.)
4. Token validators may optionally log to stderr. This will be printed
verbatim into the Postgres server logs.
The oauth method supports the following HBA options (but note that two
of them are not optional, since we have no way of choosing sensible
defaults):
issuer: Required. The URL of the OAuth issuing party, which the client
must contact to receive a bearer token.
Some real-world examples as of time of writing:
- https://accounts.google.com
- https://login.microsoft.com/[tenant-id]/v2.0
scope: Required. The OAuth scope(s) required for the server to
authenticate and/or authorize the user. This is heavily
deployment-specific, but a simple example is "openid email".
map: Optional. Specify a standard PostgreSQL user map; this works
the same as with other auth methods such as peer. If a map is
not specified, the user ID returned by the token validator
must exactly match the role that's being requested (but see
trust_validator_authz, below).
trust_validator_authz:
Optional. When set to 1, this allows the token validator to
take full control of the authorization process. Standard user
mapping is skipped: if the validator command succeeds, the
client is allowed to connect under its desired role and no
further checks are done.
Unlike the client, servers support OAuth without needing to be built
against libiddawc (since the responsibility for "speaking" OAuth/OIDC
correctly is delegated entirely to the oauth_validator_command).
Several TODOs:
- port to platforms other than "modern Linux/BSD"
- overhaul the communication with oauth_validator_command, which is
currently a bad hack on OpenPipeStream()
- implement more sanity checks on the OAUTHBEARER message format and
tokens sent by the client
- implement more helpful handling of HBA misconfigurations
- properly interpolate JSON when generating error responses
- use logdetail during auth failures
- deal with role names that can't be safely passed to system() without
shell-escaping
- allow passing the configured issuer to the oauth_validator_command, to
deal with multi-issuer setups
- ...and more.
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
---
src/backend/libpq/Makefile | 1 +
src/backend/libpq/auth-oauth.c | 810 ++++++++++++++++++++++++++++
src/backend/libpq/auth-sasl.c | 10 +-
src/backend/libpq/auth-scram.c | 4 +-
src/backend/libpq/auth.c | 26 +-
src/backend/libpq/hba.c | 31 +-
src/backend/libpq/meson.build | 1 +
src/backend/utils/misc/guc_tables.c | 12 +
src/include/libpq/auth.h | 17 +
src/include/libpq/hba.h | 6 +-
src/include/libpq/oauth.h | 24 +
src/include/libpq/sasl.h | 11 +
src/tools/pgindent/typedefs.list | 1 +
13 files changed, 922 insertions(+), 32 deletions(-)
create mode 100644 src/backend/libpq/auth-oauth.c
create mode 100644 src/include/libpq/oauth.h
diff --git a/src/backend/libpq/Makefile b/src/backend/libpq/Makefile
index 6d385fd6a4..98eb2a8242 100644
--- a/src/backend/libpq/Makefile
+++ b/src/backend/libpq/Makefile
@@ -15,6 +15,7 @@ include $(top_builddir)/src/Makefile.global
# be-fsstubs is here for historical reasons, probably belongs elsewhere
OBJS = \
+ auth-oauth.o \
auth-sasl.o \
auth-scram.o \
auth.o \
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
new file mode 100644
index 0000000000..a9d2646023
--- /dev/null
+++ b/src/backend/libpq/auth-oauth.c
@@ -0,0 +1,810 @@
+/*-------------------------------------------------------------------------
+ *
+ * auth-oauth.c
+ * Server-side implementation of the SASL OAUTHBEARER mechanism.
+ *
+ * See the following RFC for more details:
+ * - RFC 7628: https://tools.ietf.org/html/rfc7628
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/backend/libpq/auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <unistd.h>
+#include <fcntl.h>
+
+#include "common/oauth-common.h"
+#include "lib/stringinfo.h"
+#include "libpq/auth.h"
+#include "libpq/hba.h"
+#include "libpq/oauth.h"
+#include "libpq/sasl.h"
+#include "storage/fd.h"
+
+/* GUC */
+char *oauth_validator_command;
+
+static void oauth_get_mechanisms(Port *port, StringInfo buf);
+static void *oauth_init(Port *port, const char *selected_mech, const char *shadow_pass);
+static int oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, const char **logdetail);
+
+/* Mechanism declaration */
+const pg_be_sasl_mech pg_be_oauth_mech = {
+ oauth_get_mechanisms,
+ oauth_init,
+ oauth_exchange,
+
+ PG_MAX_AUTH_TOKEN_LENGTH,
+};
+
+
+typedef enum
+{
+ OAUTH_STATE_INIT = 0,
+ OAUTH_STATE_ERROR,
+ OAUTH_STATE_FINISHED,
+} oauth_state;
+
+struct oauth_ctx
+{
+ oauth_state state;
+ Port *port;
+ const char *issuer;
+ const char *scope;
+};
+
+static char *sanitize_char(char c);
+static char *parse_kvpairs_for_auth(char **input);
+static void generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen);
+static bool validate(Port *port, const char *auth, const char **logdetail);
+static bool run_validator_command(Port *port, const char *token);
+static bool check_exit(FILE **fh, const char *command);
+static bool set_cloexec(int fd);
+static bool username_ok_for_shell(const char *username);
+
+#define KVSEP 0x01
+#define AUTH_KEY "auth"
+#define BEARER_SCHEME "Bearer "
+
+static void
+oauth_get_mechanisms(Port *port, StringInfo buf)
+{
+ /* Only OAUTHBEARER is supported. */
+ appendStringInfoString(buf, OAUTHBEARER_NAME);
+ appendStringInfoChar(buf, '\0');
+}
+
+static void *
+oauth_init(Port *port, const char *selected_mech, const char *shadow_pass)
+{
+ struct oauth_ctx *ctx;
+
+ if (strcmp(selected_mech, OAUTHBEARER_NAME))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("client selected an invalid SASL authentication mechanism")));
+
+ ctx = palloc0(sizeof(*ctx));
+
+ ctx->state = OAUTH_STATE_INIT;
+ ctx->port = port;
+
+ Assert(port->hba);
+ ctx->issuer = port->hba->oauth_issuer;
+ ctx->scope = port->hba->oauth_scope;
+
+ return ctx;
+}
+
+static int
+oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, const char **logdetail)
+{
+ char *p;
+ char cbind_flag;
+ char *auth;
+
+ struct oauth_ctx *ctx = opaq;
+
+ *output = NULL;
+ *outputlen = -1;
+
+ /*
+ * If the client didn't include an "Initial Client Response" in the
+ * SASLInitialResponse message, send an empty challenge, to which the
+ * client will respond with the same data that usually comes in the
+ * Initial Client Response.
+ */
+ if (input == NULL)
+ {
+ Assert(ctx->state == OAUTH_STATE_INIT);
+
+ *output = pstrdup("");
+ *outputlen = 0;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ /*
+ * Check that the input length agrees with the string length of the input.
+ */
+ if (inputlen == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The message is empty.")));
+ if (inputlen != strlen(input))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message length does not match input length.")));
+
+ switch (ctx->state)
+ {
+ case OAUTH_STATE_INIT:
+ /* Handle this case below. */
+ break;
+
+ case OAUTH_STATE_ERROR:
+
+ /*
+ * Only one response is valid for the client during authentication
+ * failure: a single kvsep.
+ */
+ if (inputlen != 1 || *input != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Client did not send a kvsep response.")));
+
+ /* The (failed) handshake is now complete. */
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_FAILURE;
+
+ default:
+ elog(ERROR, "invalid OAUTHBEARER exchange state");
+ return PG_SASL_EXCHANGE_FAILURE;
+ }
+
+ /* Handle the client's initial message. */
+ p = pstrdup(input);
+
+ /*
+ * OAUTHBEARER does not currently define a channel binding (so there is no
+ * OAUTHBEARER-PLUS, and we do not accept a 'p' specifier). We accept a
+ * 'y' specifier purely for the remote chance that a future specification
+ * could define one; then future clients can still interoperate with this
+ * server implementation. 'n' is the expected case.
+ */
+ cbind_flag = *p;
+ switch (cbind_flag)
+ {
+ case 'p':
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The server does not support channel binding for OAuth, but the client message includes channel binding data.")));
+ break;
+
+ case 'y': /* fall through */
+ case 'n':
+ p++;
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Comma expected, but found character \"%s\".",
+ sanitize_char(*p))));
+ p++;
+ break;
+
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected channel-binding flag %s.",
+ sanitize_char(cbind_flag))));
+ }
+
+ /*
+ * Forbid optional authzid (authorization identity). We don't support it.
+ */
+ if (*p == 'a')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("client uses authorization identity, but it is not supported")));
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected attribute %s in client-first-message.",
+ sanitize_char(*p))));
+ p++;
+
+ /* All remaining fields are separated by the RFC's kvsep (\x01). */
+ if (*p != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Key-value separator expected, but found character %s.",
+ sanitize_char(*p))));
+ p++;
+
+ auth = parse_kvpairs_for_auth(&p);
+ if (!auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message does not contain an auth value.")));
+
+ /* We should be at the end of our message. */
+ if (*p)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains additional data after the final terminator.")));
+
+ if (!validate(ctx->port, auth, logdetail))
+ {
+ generate_error_response(ctx, output, outputlen);
+
+ ctx->state = OAUTH_STATE_ERROR;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_SUCCESS;
+}
+
+/*
+ * Convert an arbitrary byte to printable form. For error messages.
+ *
+ * If it's a printable ASCII character, print it as a single character.
+ * otherwise, print it in hex.
+ *
+ * The returned pointer points to a static buffer.
+ */
+static char *
+sanitize_char(char c)
+{
+ static char buf[5];
+
+ if (c >= 0x21 && c <= 0x7E)
+ snprintf(buf, sizeof(buf), "'%c'", c);
+ else
+ snprintf(buf, sizeof(buf), "0x%02x", (unsigned char) c);
+ return buf;
+}
+
+/*
+ * Consumes all kvpairs in an OAUTHBEARER exchange message. If the "auth" key is
+ * found, its value is returned.
+ */
+static char *
+parse_kvpairs_for_auth(char **input)
+{
+ char *pos = *input;
+ char *auth = NULL;
+
+ /*----
+ * The relevant ABNF, from Sec. 3.1:
+ *
+ * kvsep = %x01
+ * key = 1*(ALPHA)
+ * value = *(VCHAR / SP / HTAB / CR / LF )
+ * kvpair = key "=" value kvsep
+ * ;;gs2-header = See RFC 5801
+ * client-resp = (gs2-header kvsep *kvpair kvsep) / kvsep
+ *
+ * By the time we reach this code, the gs2-header and initial kvsep have
+ * already been validated. We start at the beginning of the first kvpair.
+ */
+
+ while (*pos)
+ {
+ char *end;
+ char *sep;
+ char *key;
+ char *value;
+
+ /*
+ * Find the end of this kvpair. Note that input is null-terminated by
+ * the SASL code, so the strchr() is bounded.
+ */
+ end = strchr(pos, KVSEP);
+ if (!end)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an unterminated key/value pair.")));
+ *end = '\0';
+
+ if (pos == end)
+ {
+ /* Empty kvpair, signifying the end of the list. */
+ *input = pos + 1;
+ return auth;
+ }
+
+ /*
+ * Find the end of the key name.
+ *
+ * TODO further validate the key/value grammar? empty keys, bad
+ * chars...
+ */
+ sep = strchr(pos, '=');
+ if (!sep)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains a key without a value.")));
+ *sep = '\0';
+
+ /* Both key and value are now safely terminated. */
+ key = pos;
+ value = sep + 1;
+
+ if (!strcmp(key, AUTH_KEY))
+ {
+ if (auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains multiple auth values.")));
+
+ auth = value;
+ }
+ else
+ {
+ /*
+ * The RFC also defines the host and port keys, but they are not
+ * required for OAUTHBEARER and we do not use them. Also, per Sec.
+ * 3.1, any key/value pairs we don't recognize must be ignored.
+ */
+ }
+
+ /* Move to the next pair. */
+ pos = end + 1;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message did not contain a final terminator.")));
+
+ return NULL; /* unreachable */
+}
+
+static void
+generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
+{
+ StringInfoData buf;
+
+ /*
+ * The admin needs to set an issuer and scope for OAuth to work. There's
+ * not really a way to hide this from the user, either, because we can't
+ * choose a "default" issuer, so be honest in the failure message.
+ *
+ * TODO: see if there's a better place to fail, earlier than this.
+ */
+ if (!ctx->issuer || !ctx->scope)
+ ereport(FATAL,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("OAuth is not properly configured for this user"),
+ errdetail_log("The issuer and scope parameters must be set in pg_hba.conf.")));
+
+
+ initStringInfo(&buf);
+
+ /*
+ * TODO: JSON escaping
+ */
+ appendStringInfo(&buf,
+ "{ "
+ "\"status\": \"invalid_token\", "
+ "\"openid-configuration\": \"%s/.well-known/openid-configuration\", "
+ "\"scope\": \"%s\" "
+ "}",
+ ctx->issuer, ctx->scope);
+
+ *output = buf.data;
+ *outputlen = buf.len;
+}
+
+static bool
+validate(Port *port, const char *auth, const char **logdetail)
+{
+ static const char *const b64_set =
+ "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-._~+/";
+
+ const char *token;
+ size_t span;
+ int ret;
+
+ /* TODO: handle logdetail when the test framework can check it */
+
+ /*-----
+ * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
+ * 2.1:
+ *
+ * b64token = 1*( ALPHA / DIGIT /
+ * "-" / "." / "_" / "~" / "+" / "/" ) *"="
+ * credentials = "Bearer" 1*SP b64token
+ *
+ * The "credentials" construction is what we receive in our auth value.
+ *
+ * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
+ * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
+ * compared case-insensitively. (This is not mentioned in RFC 6750, but
+ * it's pointed out in RFC 7628 Sec. 4.)
+ *
+ * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
+ */
+ if (pg_strncasecmp(auth, BEARER_SCHEME, strlen(BEARER_SCHEME)))
+ return false;
+
+ /* Pull the bearer token out of the auth value. */
+ token = auth + strlen(BEARER_SCHEME);
+
+ /* Swallow any additional spaces. */
+ while (*token == ' ')
+ token++;
+
+ /*
+ * Before invoking the validator command, sanity-check the token format to
+ * avoid any injection attacks later in the chain. Invalid formats are
+ * technically a protocol violation, but don't reflect any information
+ * about the sensitive Bearer token back to the client; log at COMMERROR
+ * instead.
+ */
+
+ /* Tokens must not be empty. */
+ if (!*token)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Bearer token is empty.")));
+ return false;
+ }
+
+ /*
+ * Make sure the token contains only allowed characters. Tokens may end
+ * with any number of '=' characters.
+ */
+ span = strspn(token, b64_set);
+ while (token[span] == '=')
+ span++;
+
+ if (token[span] != '\0')
+ {
+ /*
+ * This error message could be more helpful by printing the
+ * problematic character(s), but that'd be a bit like printing a piece
+ * of someone's password into the logs.
+ */
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Bearer token is not in the correct format.")));
+ return false;
+ }
+
+ /* Have the validator check the token. */
+ if (!run_validator_command(port, token))
+ return false;
+
+ if (port->hba->oauth_skip_usermap)
+ {
+ /*
+ * If the validator is our authorization authority, we're done.
+ * Authentication may or may not have been performed depending on the
+ * validator implementation; all that matters is that the validator
+ * says the user can log in with the target role.
+ */
+ return true;
+ }
+
+ /* Make sure the validator authenticated the user. */
+ if (!MyClientConnectionInfo.authn_id)
+ {
+ /* TODO: use logdetail; reduce message duplication */
+ ereport(LOG,
+ (errmsg("OAuth bearer authentication failed for user \"%s\": validator provided no identity",
+ port->user_name)));
+ return false;
+ }
+
+ /* Finally, check the user map. */
+ ret = check_usermap(port->hba->usermap, port->user_name,
+ MyClientConnectionInfo.authn_id, false);
+ return (ret == STATUS_OK);
+}
+
+static bool
+run_validator_command(Port *port, const char *token)
+{
+ bool success = false;
+ int rc;
+ int pipefd[2];
+ int rfd = -1;
+ int wfd = -1;
+
+ StringInfoData command = {0};
+ char *p;
+ FILE *fh = NULL;
+
+ ssize_t written;
+ char *line = NULL;
+ size_t size = 0;
+ ssize_t len;
+
+ Assert(oauth_validator_command);
+
+ if (!oauth_validator_command[0])
+ {
+ ereport(COMMERROR,
+ (errmsg("oauth_validator_command is not set"),
+ errhint("To allow OAuth authenticated connections, set "
+ "oauth_validator_command in postgresql.conf.")));
+ return false;
+ }
+
+ /*------
+ * Since popen() is unidirectional, open up a pipe for the other
+ * direction. Use CLOEXEC to ensure that our write end doesn't
+ * accidentally get copied into child processes, which would prevent us
+ * from closing it cleanly.
+ *
+ * XXX this is ugly. We should just read from the child process's stdout,
+ * but that's a lot more code.
+ * XXX by bypassing the popen API, we open the potential of process
+ * deadlock. Clearly document child process requirements (i.e. the child
+ * MUST read all data off of the pipe before writing anything).
+ * TODO: port to Windows using _pipe().
+ */
+ rc = pipe(pipefd);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create child pipe: %m")));
+ return false;
+ }
+
+ rfd = pipefd[0];
+ wfd = pipefd[1];
+
+ if (!set_cloexec(wfd))
+ {
+ /* error message was already logged */
+ goto cleanup;
+ }
+
+ /*----------
+ * Construct the command, substituting any recognized %-specifiers:
+ *
+ * %f: the file descriptor of the input pipe
+ * %r: the role that the client wants to assume (port->user_name)
+ * %%: a literal '%'
+ */
+ initStringInfo(&command);
+
+ for (p = oauth_validator_command; *p; p++)
+ {
+ if (p[0] == '%')
+ {
+ switch (p[1])
+ {
+ case 'f':
+ appendStringInfo(&command, "%d", rfd);
+ p++;
+ break;
+ case 'r':
+
+ /*
+ * TODO: decide how this string should be escaped. The
+ * role is controlled by the client, so if we don't escape
+ * it, command injections are inevitable.
+ *
+ * This is probably an indication that the role name needs
+ * to be communicated to the validator process in some
+ * other way. For this proof of concept, just be
+ * incredibly strict about the characters that are allowed
+ * in user names.
+ */
+ if (!username_ok_for_shell(port->user_name))
+ goto cleanup;
+
+ appendStringInfoString(&command, port->user_name);
+ p++;
+ break;
+ case '%':
+ appendStringInfoChar(&command, '%');
+ p++;
+ break;
+ default:
+ appendStringInfoChar(&command, p[0]);
+ }
+ }
+ else
+ appendStringInfoChar(&command, p[0]);
+ }
+
+ /* Execute the command. */
+ fh = OpenPipeStream(command.data, "r");
+ if (!fh)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("opening pipe to OAuth validator: %m")));
+ goto cleanup;
+ }
+
+ /* We don't need the read end of the pipe anymore. */
+ close(rfd);
+ rfd = -1;
+
+ /* Give the command the token to validate. */
+ written = write(wfd, token, strlen(token));
+ if (written != strlen(token))
+ {
+ /* TODO must loop for short writes, EINTR et al */
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write token to child pipe: %m")));
+ goto cleanup;
+ }
+
+ close(wfd);
+ wfd = -1;
+
+ /*-----
+ * Read the command's response.
+ *
+ * TODO: getline() is probably too new to use, unfortunately.
+ * TODO: loop over all lines
+ */
+ if ((len = getline(&line, &size, fh)) >= 0)
+ {
+ /* TODO: fail if the authn_id doesn't end with a newline */
+ if (len > 0)
+ line[len - 1] = '\0';
+
+ set_authn_id(port, line);
+ }
+ else if (ferror(fh))
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from command \"%s\": %m",
+ command.data)));
+ goto cleanup;
+ }
+
+ /* Make sure the command exits cleanly. */
+ if (!check_exit(&fh, command.data))
+ {
+ /* error message already logged */
+ goto cleanup;
+ }
+
+ /* Done. */
+ success = true;
+
+cleanup:
+ if (line)
+ free(line);
+
+ /*
+ * In the successful case, the pipe fds are already closed. For the error
+ * case, always close out the pipe before waiting for the command, to
+ * prevent deadlock.
+ */
+ if (rfd >= 0)
+ close(rfd);
+ if (wfd >= 0)
+ close(wfd);
+
+ if (fh)
+ {
+ Assert(!success);
+ check_exit(&fh, command.data);
+ }
+
+ if (command.data)
+ pfree(command.data);
+
+ return success;
+}
+
+static bool
+check_exit(FILE **fh, const char *command)
+{
+ int rc;
+
+ rc = ClosePipeStream(*fh);
+ *fh = NULL;
+
+ if (rc == -1)
+ {
+ /* pclose() itself failed. */
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close pipe to command \"%s\": %m",
+ command)));
+ }
+ else if (rc != 0)
+ {
+ char *reason = wait_result_to_str(rc);
+
+ ereport(COMMERROR,
+ (errmsg("failed to execute command \"%s\": %s",
+ command, reason)));
+
+ pfree(reason);
+ }
+
+ return (rc == 0);
+}
+
+static bool
+set_cloexec(int fd)
+{
+ int flags;
+ int rc;
+
+ flags = fcntl(fd, F_GETFD);
+ if (flags == -1)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not get fd flags for child pipe: %m")));
+ return false;
+ }
+
+ rc = fcntl(fd, F_SETFD, flags | FD_CLOEXEC);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not set FD_CLOEXEC for child pipe: %m")));
+ return false;
+ }
+
+ return true;
+}
+
+/*
+ * XXX This should go away eventually and be replaced with either a proper
+ * escape or a different strategy for communication with the validator command.
+ */
+static bool
+username_ok_for_shell(const char *username)
+{
+ /* This set is borrowed from fe_utils' appendShellStringNoError(). */
+ static const char *const allowed =
+ "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-_./:";
+ size_t span;
+
+ Assert(username && username[0]); /* should have already been checked */
+
+ span = strspn(username, allowed);
+ if (username[span] != '\0')
+ {
+ ereport(COMMERROR,
+ (errmsg("PostgreSQL user name contains unsafe characters and cannot be passed to the OAuth validator")));
+ return false;
+ }
+
+ return true;
+}
diff --git a/src/backend/libpq/auth-sasl.c b/src/backend/libpq/auth-sasl.c
index 08b24d90b4..4039e7fa3e 100644
--- a/src/backend/libpq/auth-sasl.c
+++ b/src/backend/libpq/auth-sasl.c
@@ -20,14 +20,6 @@
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
-/*
- * Maximum accepted size of SASL messages.
- *
- * The messages that the server or libpq generate are much smaller than this,
- * but have some headroom.
- */
-#define PG_MAX_SASL_MESSAGE_LENGTH 1024
-
/*
* Perform a SASL exchange with a libpq client, using a specific mechanism
* implementation.
@@ -103,7 +95,7 @@ CheckSASLAuth(const pg_be_sasl_mech *mech, Port *port, char *shadow_pass,
/* Get the actual SASL message */
initStringInfo(&buf);
- if (pq_getmessage(&buf, PG_MAX_SASL_MESSAGE_LENGTH))
+ if (pq_getmessage(&buf, mech->max_message_length))
{
/* EOF - pq_getmessage already logged error */
pfree(buf.data);
diff --git a/src/backend/libpq/auth-scram.c b/src/backend/libpq/auth-scram.c
index 2abb1a9b3a..aa6b5020dc 100644
--- a/src/backend/libpq/auth-scram.c
+++ b/src/backend/libpq/auth-scram.c
@@ -118,7 +118,9 @@ static int scram_exchange(void *opaq, const char *input, int inputlen,
const pg_be_sasl_mech pg_be_scram_mech = {
scram_get_mechanisms,
scram_init,
- scram_exchange
+ scram_exchange,
+
+ PG_MAX_SASL_MESSAGE_LENGTH
};
/*
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 9bbdc4beb0..db7c77da86 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -29,6 +29,7 @@
#include "libpq/auth.h"
#include "libpq/crypt.h"
#include "libpq/libpq.h"
+#include "libpq/oauth.h"
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
#include "libpq/scram.h"
@@ -47,7 +48,6 @@
*/
static void auth_failed(Port *port, int status, const char *logdetail);
static char *recv_password_packet(Port *port);
-static void set_authn_id(Port *port, const char *id);
/*----------------------------------------------------------------
@@ -203,22 +203,6 @@ static int CheckRADIUSAuth(Port *port);
static int PerformRadiusTransaction(const char *server, const char *secret, const char *portstr, const char *identifier, const char *user_name, const char *passwd);
-/*
- * Maximum accepted size of GSS and SSPI authentication tokens.
- * We also use this as a limit on ordinary password packet lengths.
- *
- * Kerberos tickets are usually quite small, but the TGTs issued by Windows
- * domain controllers include an authorization field known as the Privilege
- * Attribute Certificate (PAC), which contains the user's Windows permissions
- * (group memberships etc.). The PAC is copied into all tickets obtained on
- * the basis of this TGT (even those issued by Unix realms which the Windows
- * realm trusts), and can be several kB in size. The maximum token size
- * accepted by Windows systems is determined by the MaxAuthToken Windows
- * registry setting. Microsoft recommends that it is not set higher than
- * 65535 bytes, so that seems like a reasonable limit for us as well.
- */
-#define PG_MAX_AUTH_TOKEN_LENGTH 65535
-
/*----------------------------------------------------------------
* Global authentication functions
*----------------------------------------------------------------
@@ -307,6 +291,9 @@ auth_failed(Port *port, int status, const char *logdetail)
case uaRADIUS:
errstr = gettext_noop("RADIUS authentication failed for user \"%s\"");
break;
+ case uaOAuth:
+ errstr = gettext_noop("OAuth bearer authentication failed for user \"%s\"");
+ break;
default:
errstr = gettext_noop("authentication failed for user \"%s\": invalid authentication method");
break;
@@ -342,7 +329,7 @@ auth_failed(Port *port, int status, const char *logdetail)
* lifetime of MyClientConnectionInfo, so it is safe to pass a string that is
* managed by an external library.
*/
-static void
+void
set_authn_id(Port *port, const char *id)
{
Assert(id);
@@ -629,6 +616,9 @@ ClientAuthentication(Port *port)
case uaTrust:
status = STATUS_OK;
break;
+ case uaOAuth:
+ status = CheckSASLAuth(&pg_be_oauth_mech, port, NULL, NULL);
+ break;
}
if ((status == STATUS_OK && port->hba->clientcert == clientCertFull)
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index 8004d102ad..03c3f038c7 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -119,7 +119,8 @@ static const char *const UserAuthName[] =
"ldap",
"cert",
"radius",
- "peer"
+ "peer",
+ "oauth",
};
/*
@@ -1748,6 +1749,8 @@ parse_hba_line(TokenizedAuthLine *tok_line, int elevel)
#endif
else if (strcmp(token->string, "radius") == 0)
parsedline->auth_method = uaRADIUS;
+ else if (strcmp(token->string, "oauth") == 0)
+ parsedline->auth_method = uaOAuth;
else
{
ereport(elevel,
@@ -2067,8 +2070,9 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->auth_method != uaPeer &&
hbaline->auth_method != uaGSS &&
hbaline->auth_method != uaSSPI &&
- hbaline->auth_method != uaCert)
- INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, and cert"));
+ hbaline->auth_method != uaCert &&
+ hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, cert, and oauth"));
hbaline->usermap = pstrdup(val);
}
else if (strcmp(name, "clientcert") == 0)
@@ -2451,6 +2455,27 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->radiusidentifiers = parsed_identifiers;
hbaline->radiusidentifiers_s = pstrdup(val);
}
+ else if (strcmp(name, "issuer") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("issuer", gettext_noop("oauth"));
+ hbaline->oauth_issuer = pstrdup(val);
+ }
+ else if (strcmp(name, "scope") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("scope", gettext_noop("oauth"));
+ hbaline->oauth_scope = pstrdup(val);
+ }
+ else if (strcmp(name, "trust_validator_authz") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("trust_validator_authz", gettext_noop("oauth"));
+ if (strcmp(val, "1") == 0)
+ hbaline->oauth_skip_usermap = true;
+ else
+ hbaline->oauth_skip_usermap = false;
+ }
else
{
ereport(elevel,
diff --git a/src/backend/libpq/meson.build b/src/backend/libpq/meson.build
index 7c65314512..c85527fb01 100644
--- a/src/backend/libpq/meson.build
+++ b/src/backend/libpq/meson.build
@@ -1,6 +1,7 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
backend_sources += files(
+ 'auth-oauth.c',
'auth-sasl.c',
'auth-scram.c',
'auth.c',
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 93ded31ed9..0f83f0d870 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -48,6 +48,7 @@
#include "jit/jit.h"
#include "libpq/auth.h"
#include "libpq/libpq.h"
+#include "libpq/oauth.h"
#include "libpq/scram.h"
#include "nodes/queryjumble.h"
#include "optimizer/cost.h"
@@ -4670,6 +4671,17 @@ struct config_string ConfigureNamesString[] =
check_debug_io_direct, assign_debug_io_direct, NULL
},
+ {
+ {"oauth_validator_command", PGC_SIGHUP, CONN_AUTH_AUTH,
+ gettext_noop("Command to validate OAuth v2 bearer tokens."),
+ NULL,
+ GUC_SUPERUSER_ONLY | GUC_NOT_IN_SAMPLE
+ },
+ &oauth_validator_command,
+ "",
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, NULL, NULL, NULL, NULL
diff --git a/src/include/libpq/auth.h b/src/include/libpq/auth.h
index 227b41daf6..22f6ab9f1d 100644
--- a/src/include/libpq/auth.h
+++ b/src/include/libpq/auth.h
@@ -16,6 +16,22 @@
#include "libpq/libpq-be.h"
+/*
+ * Maximum accepted size of GSS and SSPI authentication tokens.
+ * We also use this as a limit on ordinary password packet lengths.
+ *
+ * Kerberos tickets are usually quite small, but the TGTs issued by Windows
+ * domain controllers include an authorization field known as the Privilege
+ * Attribute Certificate (PAC), which contains the user's Windows permissions
+ * (group memberships etc.). The PAC is copied into all tickets obtained on
+ * the basis of this TGT (even those issued by Unix realms which the Windows
+ * realm trusts), and can be several kB in size. The maximum token size
+ * accepted by Windows systems is determined by the MaxAuthToken Windows
+ * registry setting. Microsoft recommends that it is not set higher than
+ * 65535 bytes, so that seems like a reasonable limit for us as well.
+ */
+#define PG_MAX_AUTH_TOKEN_LENGTH 65535
+
extern PGDLLIMPORT char *pg_krb_server_keyfile;
extern PGDLLIMPORT bool pg_krb_caseins_users;
extern PGDLLIMPORT bool pg_gss_accept_delegation;
@@ -23,6 +39,7 @@ extern PGDLLIMPORT bool pg_gss_accept_delegation;
extern void ClientAuthentication(Port *port);
extern void sendAuthRequest(Port *port, AuthRequest areq, const char *extradata,
int extralen);
+extern void set_authn_id(Port *port, const char *id);
/* Hook for plugins to get control in ClientAuthentication() */
typedef void (*ClientAuthentication_hook_type) (Port *, int);
diff --git a/src/include/libpq/hba.h b/src/include/libpq/hba.h
index 8ea837ae82..53c999a69f 100644
--- a/src/include/libpq/hba.h
+++ b/src/include/libpq/hba.h
@@ -39,7 +39,8 @@ typedef enum UserAuth
uaCert,
uaRADIUS,
uaPeer,
-#define USER_AUTH_LAST uaPeer /* Must be last value of this enum */
+ uaOAuth,
+#define USER_AUTH_LAST uaOAuth /* Must be last value of this enum */
} UserAuth;
/*
@@ -135,6 +136,9 @@ typedef struct HbaLine
char *radiusidentifiers_s;
List *radiusports;
char *radiusports_s;
+ char *oauth_issuer;
+ char *oauth_scope;
+ bool oauth_skip_usermap;
} HbaLine;
typedef struct IdentLine
diff --git a/src/include/libpq/oauth.h b/src/include/libpq/oauth.h
new file mode 100644
index 0000000000..5edab3b25a
--- /dev/null
+++ b/src/include/libpq/oauth.h
@@ -0,0 +1,24 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth.h
+ * Interface to libpq/auth-oauth.c
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/libpq/oauth.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_OAUTH_H
+#define PG_OAUTH_H
+
+#include "libpq/libpq-be.h"
+#include "libpq/sasl.h"
+
+extern char *oauth_validator_command;
+
+/* Implementation */
+extern const pg_be_sasl_mech pg_be_oauth_mech;
+
+#endif /* PG_OAUTH_H */
diff --git a/src/include/libpq/sasl.h b/src/include/libpq/sasl.h
index 7a1f970cca..3f2c02b8f2 100644
--- a/src/include/libpq/sasl.h
+++ b/src/include/libpq/sasl.h
@@ -26,6 +26,14 @@
#define PG_SASL_EXCHANGE_SUCCESS 1
#define PG_SASL_EXCHANGE_FAILURE 2
+/*
+ * Maximum accepted size of SASL messages.
+ *
+ * The messages that the server or libpq generate are much smaller than this,
+ * but have some headroom.
+ */
+#define PG_MAX_SASL_MESSAGE_LENGTH 1024
+
/*
* Backend SASL mechanism callbacks.
*
@@ -127,6 +135,9 @@ typedef struct pg_be_sasl_mech
const char *input, int inputlen,
char **output, int *outputlen,
const char **logdetail);
+
+ /* The maximum size allowed for client SASLResponses. */
+ int max_message_length;
} pg_be_sasl_mech;
/* Common implementation for auth.c */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 6234fe66f1..fe65a4222b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3543,6 +3543,7 @@ normal_rand_fctx
nsphash_hash
ntile_context
numeric
+oauth_state
object_access_hook_type
object_access_hook_type_str
off_t
--
2.34.1
v18-0003-Explicitly-require-password-for-SCRAM-exchange.patch (application/octet-stream, 3.2K) download
From e4ad0260d54a5be3964c06621cf3d2bb35fa60d5 Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <dgustafsson@postgresql.org>
Date: Fri, 23 Feb 2024 11:19:55 +0100
Subject: [PATCH v18 3/9] Explicitly require password for SCRAM exchange
This refactors the SASL init flow to set password_needed on the two
SCRAM exchanges currently supported. The code already required this
but was set up in such a way that all SASL exchanges required using
a password, a restriction which may not hold for all exchanges (the
example at hand being the proposed OAuthbearer exchange).
This was extracted from a larger patchset to introduce OAuthBearer
authentication and authorization.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://postgr.es/m/d1b467a78e0e36ed85a09adf979d04cf124a9d4b.camel@vmware.com
---
src/interfaces/libpq/fe-auth.c | 28 +++++++++++++++-------------
1 file changed, 15 insertions(+), 13 deletions(-)
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index cf8af4c62e..81ec08485d 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -425,7 +425,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
int initialresponselen;
const char *selected_mechanism;
PQExpBufferData mechanism_buf;
- char *password;
+ char *password = NULL;
SASLStatus status;
initPQExpBuffer(&mechanism_buf);
@@ -446,8 +446,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
/*
* Parse the list of SASL authentication mechanisms in the
* AuthenticationSASL message, and select the best mechanism that we
- * support. SCRAM-SHA-256-PLUS and SCRAM-SHA-256 are the only ones
- * supported at the moment, listed by order of decreasing importance.
+ * support. Mechanisms are listed by order of decreasing importance.
*/
selected_mechanism = NULL;
for (;;)
@@ -487,6 +486,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
selected_mechanism = SCRAM_SHA_256_PLUS_NAME;
conn->sasl = &pg_scram_mech;
+ conn->password_needed = true;
}
#else
/*
@@ -522,6 +522,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
selected_mechanism = SCRAM_SHA_256_NAME;
conn->sasl = &pg_scram_mech;
+ conn->password_needed = true;
}
}
@@ -545,18 +546,19 @@ pg_SASL_init(PGconn *conn, int payloadlen)
/*
* First, select the password to use for the exchange, complaining if
- * there isn't one. Currently, all supported SASL mechanisms require a
- * password, so we can just go ahead here without further distinction.
+ * there isn't one and the selected SASL mechanism needs it.
*/
- conn->password_needed = true;
- password = conn->connhost[conn->whichhost].password;
- if (password == NULL)
- password = conn->pgpass;
- if (password == NULL || password[0] == '\0')
+ if (conn->password_needed)
{
- appendPQExpBufferStr(&conn->errorMessage,
- PQnoPasswordSupplied);
- goto error;
+ password = conn->connhost[conn->whichhost].password;
+ if (password == NULL)
+ password = conn->pgpass;
+ if (password == NULL || password[0] == '\0')
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ PQnoPasswordSupplied);
+ goto error;
+ }
}
Assert(conn->sasl);
--
2.34.1
v18-0002-Refactor-SASL-exchange-to-return-tri-state-statu.patch (application/octet-stream, 9.9K) download
From db625e1d01c3de1156fc7d68b119892e7f147f49 Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <dgustafsson@postgresql.org>
Date: Fri, 23 Feb 2024 11:09:54 +0100
Subject: [PATCH v18 2/9] Refactor SASL exchange to return tri-state status
The SASL exchange callback returned state in to output variables:
done and success. This refactors that logic by introducing a new
return variable of type SASLStatus which makes the code easier to
read and understand, and prepares for future SASL exchanges which
operate asynchronously.
This was extracted from a larger patchset to introduce OAuthBearer
authentication and authorization.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://postgr.es/m/d1b467a78e0e36ed85a09adf979d04cf124a9d4b.camel@vmware.com
---
src/interfaces/libpq/fe-auth-sasl.h | 31 +++++++----
src/interfaces/libpq/fe-auth-scram.c | 78 +++++++++++++---------------
src/interfaces/libpq/fe-auth.c | 28 +++++-----
src/tools/pgindent/typedefs.list | 1 +
4 files changed, 71 insertions(+), 67 deletions(-)
diff --git a/src/interfaces/libpq/fe-auth-sasl.h b/src/interfaces/libpq/fe-auth-sasl.h
index ee5d1525b5..4eecf53a15 100644
--- a/src/interfaces/libpq/fe-auth-sasl.h
+++ b/src/interfaces/libpq/fe-auth-sasl.h
@@ -21,6 +21,17 @@
#include "libpq-fe.h"
+/*
+ * Possible states for the SASL exchange, see the comment on exchange for an
+ * explanation of these.
+ */
+typedef enum
+{
+ SASL_COMPLETE = 0,
+ SASL_FAILED,
+ SASL_CONTINUE,
+} SASLStatus;
+
/*
* Frontend SASL mechanism callbacks.
*
@@ -59,7 +70,8 @@ typedef struct pg_fe_sasl_mech
* Produces a client response to a server challenge. As a special case
* for client-first SASL mechanisms, exchange() is called with a NULL
* server response once at the start of the authentication exchange to
- * generate an initial response.
+ * generate an initial response. Returns a SASLStatus indicating the
+ * state and status of the exchange.
*
* Input parameters:
*
@@ -79,22 +91,23 @@ typedef struct pg_fe_sasl_mech
*
* output: A malloc'd buffer containing the client's response to
* the server (can be empty), or NULL if the exchange should
- * be aborted. (*success should be set to false in the
+ * be aborted. (The callback should return SASL_FAILED in the
* latter case.)
*
* outputlen: The length (0 or higher) of the client response buffer,
* ignored if output is NULL.
*
- * done: Set to true if the SASL exchange should not continue,
- * because the exchange is either complete or failed
+ * Return value:
*
- * success: Set to true if the SASL exchange completed successfully.
- * Ignored if *done is false.
+ * SASL_CONTINUE: The output buffer is filled with a client response.
+ * Additional server challenge is expected
+ * SASL_COMPLETE: The SASL exchange has completed successfully.
+ * SASL_FAILED: The exchange has failed and the connection should be
+ * dropped.
*--------
*/
- void (*exchange) (void *state, char *input, int inputlen,
- char **output, int *outputlen,
- bool *done, bool *success);
+ SASLStatus (*exchange) (void *state, char *input, int inputlen,
+ char **output, int *outputlen);
/*--------
* channel_bound()
diff --git a/src/interfaces/libpq/fe-auth-scram.c b/src/interfaces/libpq/fe-auth-scram.c
index 04f0e5713d..0bb820e0d9 100644
--- a/src/interfaces/libpq/fe-auth-scram.c
+++ b/src/interfaces/libpq/fe-auth-scram.c
@@ -24,9 +24,8 @@
/* The exported SCRAM callback mechanism. */
static void *scram_init(PGconn *conn, const char *password,
const char *sasl_mechanism);
-static void scram_exchange(void *opaq, char *input, int inputlen,
- char **output, int *outputlen,
- bool *done, bool *success);
+static SASLStatus scram_exchange(void *opaq, char *input, int inputlen,
+ char **output, int *outputlen);
static bool scram_channel_bound(void *opaq);
static void scram_free(void *opaq);
@@ -202,17 +201,14 @@ scram_free(void *opaq)
/*
* Exchange a SCRAM message with backend.
*/
-static void
+static SASLStatus
scram_exchange(void *opaq, char *input, int inputlen,
- char **output, int *outputlen,
- bool *done, bool *success)
+ char **output, int *outputlen)
{
fe_scram_state *state = (fe_scram_state *) opaq;
PGconn *conn = state->conn;
const char *errstr = NULL;
- *done = false;
- *success = false;
*output = NULL;
*outputlen = 0;
@@ -225,12 +221,12 @@ scram_exchange(void *opaq, char *input, int inputlen,
if (inputlen == 0)
{
libpq_append_conn_error(conn, "malformed SCRAM message (empty message)");
- goto error;
+ return SASL_FAILED;
}
if (inputlen != strlen(input))
{
libpq_append_conn_error(conn, "malformed SCRAM message (length mismatch)");
- goto error;
+ return SASL_FAILED;
}
}
@@ -240,61 +236,59 @@ scram_exchange(void *opaq, char *input, int inputlen,
/* Begin the SCRAM handshake, by sending client nonce */
*output = build_client_first_message(state);
if (*output == NULL)
- goto error;
+ return SASL_FAILED;
*outputlen = strlen(*output);
- *done = false;
state->state = FE_SCRAM_NONCE_SENT;
- break;
+ return SASL_CONTINUE;
case FE_SCRAM_NONCE_SENT:
/* Receive salt and server nonce, send response. */
if (!read_server_first_message(state, input))
- goto error;
+ return SASL_FAILED;
*output = build_client_final_message(state);
if (*output == NULL)
- goto error;
+ return SASL_FAILED;
*outputlen = strlen(*output);
- *done = false;
state->state = FE_SCRAM_PROOF_SENT;
- break;
+ return SASL_CONTINUE;
case FE_SCRAM_PROOF_SENT:
- /* Receive server signature */
- if (!read_server_final_message(state, input))
- goto error;
-
- /*
- * Verify server signature, to make sure we're talking to the
- * genuine server.
- */
- if (!verify_server_signature(state, success, &errstr))
- {
- libpq_append_conn_error(conn, "could not verify server signature: %s", errstr);
- goto error;
- }
-
- if (!*success)
{
- libpq_append_conn_error(conn, "incorrect server signature");
+ bool match;
+
+ /* Receive server signature */
+ if (!read_server_final_message(state, input))
+ return SASL_FAILED;
+
+ /*
+ * Verify server signature, to make sure we're talking to the
+ * genuine server.
+ */
+ if (!verify_server_signature(state, &match, &errstr))
+ {
+ libpq_append_conn_error(conn, "could not verify server signature: %s", errstr);
+ return SASL_FAILED;
+ }
+
+ if (!match)
+ {
+ libpq_append_conn_error(conn, "incorrect server signature");
+ }
+ state->state = FE_SCRAM_FINISHED;
+ state->conn->client_finished_auth = true;
+ return match ? SASL_COMPLETE : SASL_FAILED;
}
- *done = true;
- state->state = FE_SCRAM_FINISHED;
- state->conn->client_finished_auth = true;
- break;
default:
/* shouldn't happen */
libpq_append_conn_error(conn, "invalid SCRAM exchange state");
- goto error;
+ break;
}
- return;
-error:
- *done = true;
- *success = false;
+ return SASL_FAILED;
}
/*
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index 1a8e4f6fbf..cf8af4c62e 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -423,11 +423,10 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
char *initialresponse = NULL;
int initialresponselen;
- bool done;
- bool success;
const char *selected_mechanism;
PQExpBufferData mechanism_buf;
char *password;
+ SASLStatus status;
initPQExpBuffer(&mechanism_buf);
@@ -575,12 +574,11 @@ pg_SASL_init(PGconn *conn, int payloadlen)
goto oom_error;
/* Get the mechanism-specific Initial Client Response, if any */
- conn->sasl->exchange(conn->sasl_state,
- NULL, -1,
- &initialresponse, &initialresponselen,
- &done, &success);
+ status = conn->sasl->exchange(conn->sasl_state,
+ NULL, -1,
+ &initialresponse, &initialresponselen);
- if (done && !success)
+ if (status == SASL_FAILED)
goto error;
/*
@@ -629,10 +627,9 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
{
char *output;
int outputlen;
- bool done;
- bool success;
int res;
char *challenge;
+ SASLStatus status;
/* Read the SASL challenge from the AuthenticationSASLContinue message. */
challenge = malloc(payloadlen + 1);
@@ -651,13 +648,12 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
/* For safety and convenience, ensure the buffer is NULL-terminated. */
challenge[payloadlen] = '\0';
- conn->sasl->exchange(conn->sasl_state,
- challenge, payloadlen,
- &output, &outputlen,
- &done, &success);
+ status = conn->sasl->exchange(conn->sasl_state,
+ challenge, payloadlen,
+ &output, &outputlen);
free(challenge); /* don't need the input anymore */
- if (final && !done)
+ if (final && status == SASL_CONTINUE)
{
if (outputlen != 0)
free(output);
@@ -670,7 +666,7 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
* If the exchange is not completed yet, we need to make sure that the
* SASL mechanism has generated a message to send back.
*/
- if (output == NULL && !done)
+ if (output == NULL && status == SASL_CONTINUE)
{
libpq_append_conn_error(conn, "no client response found after SASL exchange success");
return STATUS_ERROR;
@@ -692,7 +688,7 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
return STATUS_ERROR;
}
- if (done && !success)
+ if (status == SASL_FAILED)
return STATUS_ERROR;
return STATUS_OK;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index fc8b15d0cf..2461567026 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2423,6 +2423,7 @@ RuleLock
RuleStmt
RunningTransactions
RunningTransactionsData
+SASLStatus
SC_HANDLE
SECURITY_ATTRIBUTES
SECURITY_STATUS
--
2.34.1
v18-0001-common-jsonapi-support-FRONTEND-clients.patch (application/octet-stream, 20.4K) download
From e2a0b485617f0f6f3479c784a90f8f9579c35fea Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Mon, 3 May 2021 15:38:26 -0700
Subject: [PATCH v18 1/9] common/jsonapi: support FRONTEND clients
Based on a patch by Michael Paquier.
For frontend code, use PQExpBuffer instead of StringInfo. This requires
us to track allocation failures so that we can return JSON_OUT_OF_MEMORY
as needed. json_errdetail() now allocates its error message inside
memory owned by the JsonLexContext, so clients don't need to worry about
freeing it.
We can now partially revert b44669b2ca, now that json_errdetail() works
correctly.
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
---
src/bin/pg_verifybackup/t/005_bad_manifest.pl | 3 +-
src/common/Makefile | 2 +-
src/common/jsonapi.c | 268 +++++++++++++-----
src/common/meson.build | 8 +-
src/common/parse_manifest.c | 2 +-
src/common/stringinfo.c | 7 +
src/include/common/jsonapi.h | 18 +-
src/include/lib/stringinfo.h | 2 +
8 files changed, 225 insertions(+), 85 deletions(-)
diff --git a/src/bin/pg_verifybackup/t/005_bad_manifest.pl b/src/bin/pg_verifybackup/t/005_bad_manifest.pl
index e278ccea5a..e2a297930e 100644
--- a/src/bin/pg_verifybackup/t/005_bad_manifest.pl
+++ b/src/bin/pg_verifybackup/t/005_bad_manifest.pl
@@ -13,7 +13,8 @@ use Test::More;
my $tempdir = PostgreSQL::Test::Utils::tempdir;
test_bad_manifest('input string ended unexpectedly',
- qr/could not parse backup manifest: parsing failed/, <<EOM);
+ qr/could not parse backup manifest: The input string ended unexpectedly/,
+ <<EOM);
{
EOM
diff --git a/src/common/Makefile b/src/common/Makefile
index 2ba5069dca..bbb5c3ab11 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -40,7 +40,7 @@ override CPPFLAGS += -DVAL_LDFLAGS_EX="\"$(LDFLAGS_EX)\""
override CPPFLAGS += -DVAL_LDFLAGS_SL="\"$(LDFLAGS_SL)\""
override CPPFLAGS += -DVAL_LIBS="\"$(LIBS)\""
-override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common $(CPPFLAGS)
+override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common -I$(libpq_srcdir) $(CPPFLAGS)
LIBS += $(PTHREAD_LIBS)
OBJS_COMMON = \
diff --git a/src/common/jsonapi.c b/src/common/jsonapi.c
index 32931ded82..2d1f30353a 100644
--- a/src/common/jsonapi.c
+++ b/src/common/jsonapi.c
@@ -21,10 +21,43 @@
#include "mb/pg_wchar.h"
#include "port/pg_lfind.h"
-#ifndef FRONTEND
+#ifdef FRONTEND
+#include "pqexpbuffer.h"
+#else
+#include "lib/stringinfo.h"
#include "miscadmin.h"
#endif
+/*
+ * In backend, we will use palloc/pfree along with StringInfo. In frontend, use
+ * malloc and PQExpBuffer, and return JSON_OUT_OF_MEMORY on out-of-memory.
+ */
+#ifdef FRONTEND
+
+#define STRDUP(s) strdup(s)
+#define ALLOC(size) malloc(size)
+
+#define appendStrVal appendPQExpBuffer
+#define appendBinaryStrVal appendBinaryPQExpBuffer
+#define appendStrValChar appendPQExpBufferChar
+#define createStrVal createPQExpBuffer
+#define resetStrVal resetPQExpBuffer
+#define destroyStrVal destroyPQExpBuffer
+
+#else /* !FRONTEND */
+
+#define STRDUP(s) pstrdup(s)
+#define ALLOC(size) palloc(size)
+
+#define appendStrVal appendStringInfo
+#define appendBinaryStrVal appendBinaryStringInfo
+#define appendStrValChar appendStringInfoChar
+#define createStrVal makeStringInfo
+#define resetStrVal resetStringInfo
+#define destroyStrVal destroyStringInfo
+
+#endif
+
/*
* The context of the parser is maintained by the recursive descent
* mechanism, but is passed explicitly to the error reporting routine
@@ -167,9 +200,16 @@ makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
lex->input_encoding = encoding;
if (need_escapes)
{
- lex->strval = makeStringInfo();
+ /*
+ * This call can fail in FRONTEND code. We defer error handling to
+ * time of use (json_lex_string()) since we might not need to parse
+ * any strings anyway.
+ */
+ lex->strval = createStrVal();
lex->flags |= JSONLEX_FREE_STRVAL;
+ lex->parse_strval = true;
}
+ lex->errormsg = NULL;
return lex;
}
@@ -182,13 +222,18 @@ makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
void
freeJsonLexContext(JsonLexContext *lex)
{
+ static const JsonLexContext empty = {0};
+
if (lex->flags & JSONLEX_FREE_STRVAL)
- {
- pfree(lex->strval->data);
- pfree(lex->strval);
- }
+ destroyStrVal(lex->strval);
+
+ if (lex->errormsg)
+ destroyStrVal(lex->errormsg);
+
if (lex->flags & JSONLEX_FREE_STRUCT)
pfree(lex);
+ else
+ *lex = empty;
}
/*
@@ -254,7 +299,7 @@ json_count_array_elements(JsonLexContext *lex, int *elements)
* etc, so doing this with a copy makes that safe.
*/
memcpy(©lex, lex, sizeof(JsonLexContext));
- copylex.strval = NULL; /* not interested in values here */
+ copylex.parse_strval = false; /* not interested in values here */
copylex.lex_level++;
count = 0;
@@ -316,14 +361,21 @@ parse_scalar(JsonLexContext *lex, JsonSemAction *sem)
/* extract the de-escaped string value, or the raw lexeme */
if (lex_peek(lex) == JSON_TOKEN_STRING)
{
- if (lex->strval != NULL)
- val = pstrdup(lex->strval->data);
+ if (lex->parse_strval)
+ {
+ val = STRDUP(lex->strval->data);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
}
else
{
int len = (lex->token_terminator - lex->token_start);
- val = palloc(len + 1);
+ val = ALLOC(len + 1);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+
memcpy(val, lex->token_start, len);
val[len] = '\0';
}
@@ -357,8 +409,12 @@ parse_object_field(JsonLexContext *lex, JsonSemAction *sem)
if (lex_peek(lex) != JSON_TOKEN_STRING)
return report_parse_error(JSON_PARSE_STRING, lex);
- if ((ostart != NULL || oend != NULL) && lex->strval != NULL)
- fname = pstrdup(lex->strval->data);
+ if ((ostart != NULL || oend != NULL) && lex->parse_strval)
+ {
+ fname = STRDUP(lex->strval->data);
+ if (fname == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
result = json_lex(lex);
if (result != JSON_SUCCESS)
return result;
@@ -414,6 +470,11 @@ parse_object(JsonLexContext *lex, JsonSemAction *sem)
JsonParseErrorType result;
#ifndef FRONTEND
+
+ /*
+ * TODO: clients need some way to put a bound on stack growth. Parse level
+ * limits maybe?
+ */
check_stack_depth();
#endif
@@ -762,8 +823,15 @@ json_lex_string(JsonLexContext *lex)
return code; \
} while (0)
- if (lex->strval != NULL)
- resetStringInfo(lex->strval);
+ if (lex->parse_strval)
+ {
+#ifdef FRONTEND
+ /* make sure initialization succeeded */
+ if (lex->strval == NULL)
+ return JSON_OUT_OF_MEMORY;
+#endif
+ resetStrVal(lex->strval);
+ }
Assert(lex->input_length > 0);
s = lex->token_start;
@@ -800,7 +868,7 @@ json_lex_string(JsonLexContext *lex)
else
FAIL_AT_CHAR_END(JSON_UNICODE_ESCAPE_FORMAT);
}
- if (lex->strval != NULL)
+ if (lex->parse_strval)
{
/*
* Combine surrogate pairs.
@@ -857,19 +925,19 @@ json_lex_string(JsonLexContext *lex)
unicode_to_utf8(ch, (unsigned char *) utf8str);
utf8len = pg_utf_mblen((unsigned char *) utf8str);
- appendBinaryStringInfo(lex->strval, utf8str, utf8len);
+ appendBinaryPQExpBuffer(lex->strval, utf8str, utf8len);
}
else if (ch <= 0x007f)
{
/* The ASCII range is the same in all encodings */
- appendStringInfoChar(lex->strval, (char) ch);
+ appendPQExpBufferChar(lex->strval, (char) ch);
}
else
FAIL_AT_CHAR_END(JSON_UNICODE_HIGH_ESCAPE);
#endif /* FRONTEND */
}
}
- else if (lex->strval != NULL)
+ else if (lex->parse_strval)
{
if (hi_surrogate != -1)
FAIL_AT_CHAR_END(JSON_UNICODE_LOW_SURROGATE);
@@ -879,22 +947,22 @@ json_lex_string(JsonLexContext *lex)
case '"':
case '\\':
case '/':
- appendStringInfoChar(lex->strval, *s);
+ appendStrValChar(lex->strval, *s);
break;
case 'b':
- appendStringInfoChar(lex->strval, '\b');
+ appendStrValChar(lex->strval, '\b');
break;
case 'f':
- appendStringInfoChar(lex->strval, '\f');
+ appendStrValChar(lex->strval, '\f');
break;
case 'n':
- appendStringInfoChar(lex->strval, '\n');
+ appendStrValChar(lex->strval, '\n');
break;
case 'r':
- appendStringInfoChar(lex->strval, '\r');
+ appendStrValChar(lex->strval, '\r');
break;
case 't':
- appendStringInfoChar(lex->strval, '\t');
+ appendStrValChar(lex->strval, '\t');
break;
default:
@@ -929,7 +997,7 @@ json_lex_string(JsonLexContext *lex)
/*
* Skip to the first byte that requires special handling, so we
- * can batch calls to appendBinaryStringInfo.
+ * can batch calls to appendBinaryStrVal.
*/
while (p < end - sizeof(Vector8) &&
!pg_lfind8('\\', (uint8 *) p, sizeof(Vector8)) &&
@@ -953,8 +1021,8 @@ json_lex_string(JsonLexContext *lex)
}
}
- if (lex->strval != NULL)
- appendBinaryStringInfo(lex->strval, s, p - s);
+ if (lex->parse_strval)
+ appendBinaryStrVal(lex->strval, s, p - s);
/*
* s will be incremented at the top of the loop, so set it to just
@@ -970,6 +1038,11 @@ json_lex_string(JsonLexContext *lex)
return JSON_UNICODE_LOW_SURROGATE;
}
+#ifdef FRONTEND
+ if (lex->parse_strval && PQExpBufferBroken(lex->strval))
+ return JSON_OUT_OF_MEMORY;
+#endif
+
/* Hooray, we found the end of the string! */
lex->prev_token_terminator = lex->token_terminator;
lex->token_terminator = s + 1;
@@ -1145,72 +1218,93 @@ report_parse_error(JsonParseContext ctx, JsonLexContext *lex)
return JSON_SUCCESS; /* silence stupider compilers */
}
-
-#ifndef FRONTEND
-/*
- * Extract the current token from a lexing context, for error reporting.
- */
-static char *
-extract_token(JsonLexContext *lex)
-{
- int toklen = lex->token_terminator - lex->token_start;
- char *token = palloc(toklen + 1);
-
- memcpy(token, lex->token_start, toklen);
- token[toklen] = '\0';
- return token;
-}
-
/*
* Construct an (already translated) detail message for a JSON error.
*
- * Note that the error message generated by this routine may not be
- * palloc'd, making it unsafe for frontend code as there is no way to
- * know if this can be safely pfree'd or not.
+ * The returned allocation is either static or owned by the JsonLexContext and
+ * should not be freed.
*/
char *
json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
{
+ int toklen = lex->token_terminator - lex->token_start;
+
+ if (error == JSON_OUT_OF_MEMORY)
+ {
+ /* Short circuit. Allocating anything for this case is unhelpful. */
+ return _("out of memory");
+ }
+
+ if (lex->errormsg)
+ resetStrVal(lex->errormsg);
+ else
+ lex->errormsg = createStrVal();
+
switch (error)
{
case JSON_SUCCESS:
/* fall through to the error code after switch */
break;
case JSON_ESCAPING_INVALID:
- return psprintf(_("Escape sequence \"\\%s\" is invalid."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Escape sequence \"\\%.*s\" is invalid."),
+ toklen, lex->token_start);
+ break;
case JSON_ESCAPING_REQUIRED:
- return psprintf(_("Character with value 0x%02x must be escaped."),
- (unsigned char) *(lex->token_terminator));
+ appendStrVal(lex->errormsg,
+ _("Character with value 0x%02x must be escaped."),
+ (unsigned char) *(lex->token_terminator));
+ break;
case JSON_EXPECTED_END:
- return psprintf(_("Expected end of input, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected end of input, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_ARRAY_FIRST:
- return psprintf(_("Expected array element or \"]\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected array element or \"]\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_ARRAY_NEXT:
- return psprintf(_("Expected \",\" or \"]\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \",\" or \"]\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_COLON:
- return psprintf(_("Expected \":\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \":\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_JSON:
- return psprintf(_("Expected JSON value, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected JSON value, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_MORE:
return _("The input string ended unexpectedly.");
case JSON_EXPECTED_OBJECT_FIRST:
- return psprintf(_("Expected string or \"}\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected string or \"}\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_OBJECT_NEXT:
- return psprintf(_("Expected \",\" or \"}\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \",\" or \"}\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_STRING:
- return psprintf(_("Expected string, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected string, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_INVALID_TOKEN:
- return psprintf(_("Token \"%s\" is invalid."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Token \"%.*s\" is invalid."),
+ toklen, lex->token_start);
+ break;
+ case JSON_OUT_OF_MEMORY:
+ /* should have been handled above; use the error path */
+ break;
case JSON_UNICODE_CODE_POINT_ZERO:
return _("\\u0000 cannot be converted to text.");
case JSON_UNICODE_ESCAPE_FORMAT:
@@ -1219,9 +1313,19 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
/* note: this case is only reachable in frontend not backend */
return _("Unicode escape values cannot be used for code point values above 007F when the encoding is not UTF8.");
case JSON_UNICODE_UNTRANSLATABLE:
- /* note: this case is only reachable in backend not frontend */
+
+ /*
+ * note: this case is only reachable in backend not frontend.
+ * #ifdef it away so the frontend doesn't try to link against
+ * backend functionality.
+ */
+#ifndef FRONTEND
return psprintf(_("Unicode escape value could not be translated to the server's encoding %s."),
GetDatabaseEncodingName());
+#else
+ Assert(false);
+ break;
+#endif
case JSON_UNICODE_HIGH_SURROGATE:
return _("Unicode high surrogate must not follow a high surrogate.");
case JSON_UNICODE_LOW_SURROGATE:
@@ -1231,12 +1335,22 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
break;
}
- /*
- * We don't use a default: case, so that the compiler will warn about
- * unhandled enum values. But this needs to be here anyway to cover the
- * possibility of an incorrect input.
- */
- elog(ERROR, "unexpected json parse error type: %d", (int) error);
- return NULL;
-}
+ /* Note that lex->errormsg can be NULL in FRONTEND code. */
+ if (lex->errormsg && !lex->errormsg->data[0])
+ {
+ /*
+ * We don't use a default: case, so that the compiler will warn about
+ * unhandled enum values. But this needs to be here anyway to cover
+ * the possibility of an incorrect input.
+ */
+ appendStrVal(lex->errormsg,
+ "unexpected json parse error type: %d", (int) error);
+ }
+
+#ifdef FRONTEND
+ if (PQExpBufferBroken(lex->errormsg))
+ return _("out of memory while constructing error description");
#endif
+
+ return lex->errormsg->data;
+}
diff --git a/src/common/meson.build b/src/common/meson.build
index 4eb16024cb..5d2c7abaa6 100644
--- a/src/common/meson.build
+++ b/src/common/meson.build
@@ -124,13 +124,18 @@ common_sources_frontend_static += files(
# least cryptohash_openssl.c, hmac_openssl.c depend on it.
# controldata_utils.c depends on wait_event_types_h. That's arguably a
# layering violation, but ...
+#
+# XXX Frontend builds need libpq's pqexpbuffer.h, so adjust the include paths
+# appropriately. This seems completely broken.
pgcommon = {}
pgcommon_variants = {
'_srv': internal_lib_args + {
+ 'include_directories': include_directories('.'),
'sources': common_sources + [lwlocknames_h] + [wait_event_types_h],
'dependencies': [backend_common_code],
},
'': default_lib_args + {
+ 'include_directories': include_directories('../interfaces/libpq', '.'),
'sources': common_sources_frontend_static,
'dependencies': [frontend_common_code],
# Files in libpgcommon.a should use/export the "xxx_private" versions
@@ -139,6 +144,7 @@ pgcommon_variants = {
},
'_shlib': default_lib_args + {
'pic': true,
+ 'include_directories': include_directories('../interfaces/libpq', '.'),
'sources': common_sources_frontend_shlib,
'dependencies': [frontend_common_code],
},
@@ -156,7 +162,6 @@ foreach name, opts : pgcommon_variants
c_args = opts.get('c_args', []) + common_cflags[cflagname]
cflag_libs += static_library('libpgcommon@0@_@1@'.format(name, cflagname),
c_pch: pch_c_h,
- include_directories: include_directories('.'),
kwargs: opts + {
'sources': sources,
'c_args': c_args,
@@ -169,7 +174,6 @@ foreach name, opts : pgcommon_variants
lib = static_library('libpgcommon@0@'.format(name),
link_with: cflag_libs,
c_pch: pch_c_h,
- include_directories: include_directories('.'),
kwargs: opts + {
'dependencies': opts['dependencies'] + [ssl],
}
diff --git a/src/common/parse_manifest.c b/src/common/parse_manifest.c
index 92a97714f3..62d93989be 100644
--- a/src/common/parse_manifest.c
+++ b/src/common/parse_manifest.c
@@ -147,7 +147,7 @@ json_parse_manifest(JsonManifestParseContext *context, char *buffer,
/* Run the actual JSON parser. */
json_error = pg_parse_json(lex, &sem);
if (json_error != JSON_SUCCESS)
- json_manifest_parse_failure(context, "parsing failed");
+ json_manifest_parse_failure(context, json_errdetail(json_error, lex));
if (parse.state != JM_EXPECT_EOF)
json_manifest_parse_failure(context, "manifest ended unexpectedly");
diff --git a/src/common/stringinfo.c b/src/common/stringinfo.c
index c61d5c58f3..09419f6042 100644
--- a/src/common/stringinfo.c
+++ b/src/common/stringinfo.c
@@ -350,3 +350,10 @@ enlargeStringInfo(StringInfo str, int needed)
str->maxlen = newlen;
}
+
+void
+destroyStringInfo(StringInfo str)
+{
+ pfree(str->data);
+ pfree(str);
+}
diff --git a/src/include/common/jsonapi.h b/src/include/common/jsonapi.h
index 02943cdad8..75d444c17a 100644
--- a/src/include/common/jsonapi.h
+++ b/src/include/common/jsonapi.h
@@ -14,8 +14,6 @@
#ifndef JSONAPI_H
#define JSONAPI_H
-#include "lib/stringinfo.h"
-
typedef enum JsonTokenType
{
JSON_TOKEN_INVALID,
@@ -48,6 +46,7 @@ typedef enum JsonParseErrorType
JSON_EXPECTED_OBJECT_NEXT,
JSON_EXPECTED_STRING,
JSON_INVALID_TOKEN,
+ JSON_OUT_OF_MEMORY,
JSON_UNICODE_CODE_POINT_ZERO,
JSON_UNICODE_ESCAPE_FORMAT,
JSON_UNICODE_HIGH_ESCAPE,
@@ -57,6 +56,17 @@ typedef enum JsonParseErrorType
JSON_SEM_ACTION_FAILED, /* error should already be reported */
} JsonParseErrorType;
+/*
+ * Don't depend on the internal type header for strval; if callers need access
+ * then they can include the appropriate header themselves.
+ */
+#ifdef FRONTEND
+#define StrValType PQExpBufferData
+#else
+#define StrValType StringInfoData
+#endif
+
+typedef struct StrValType StrValType;
/*
* All the fields in this structure should be treated as read-only.
@@ -88,7 +98,9 @@ typedef struct JsonLexContext
bits32 flags;
int line_number; /* line number, starting from 1 */
char *line_start; /* where that line starts within input */
- StringInfo strval;
+ bool parse_strval;
+ StrValType *strval; /* only used if parse_strval == true */
+ StrValType *errormsg;
} JsonLexContext;
typedef JsonParseErrorType (*json_struct_action) (void *state);
diff --git a/src/include/lib/stringinfo.h b/src/include/lib/stringinfo.h
index 2cd636b01c..64ec6419af 100644
--- a/src/include/lib/stringinfo.h
+++ b/src/include/lib/stringinfo.h
@@ -233,4 +233,6 @@ extern void appendBinaryStringInfoNT(StringInfo str,
*/
extern void enlargeStringInfo(StringInfo str, int needed);
+
+extern void destroyStringInfo(StringInfo str);
#endif /* STRINGINFO_H */
--
2.34.1
v18-0004-libpq-add-OAUTHBEARER-SASL-mechanism.patch (application/octet-stream, 118.9K) download
From 229f602d5c801cd602fe61b23c87f4f769497f64 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 13 Apr 2021 10:27:27 -0700
Subject: [PATCH v18 4/9] libpq: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) and OAuth 2.0 Device Authorization
Grants (RFC 8628) on the client side. When speaking to a OAuth-enabled
server, it looks a bit like this:
$ psql 'host=example.org oauth_client_id=f02c6361-0635-...'
Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG
The OAuth issuer must support device authorization. No other OAuth flows
are currently implemented (but clients may provide their own flows; see
below).
The client implementation requires either libcurl or libiddawc and their
development headers. Pass `curl` or `iddawc` to --with-oauth/-Doauth
during configuration.
Thomas Munro wrote the kqueue() implementation for oauth-curl; thanks!
= PQauthDataHook =
Clients may override two pieces of OAuth handling using the new
PQsetAuthDataHook():
- PQAUTHDATA_PROMPT_OAUTH_DEVICE: replaces the default user prompt to
standard error when using the builtin device authorization flow
- PQAUTHDATA_OAUTH_BEARER_TOKEN: replaces the entire OAuth flow with a
custom asynchronous implementation
In general, a hook implementation should examine the incoming `type` to
decide whether or not to handle a specific piece of authdata; if not, it
should delegate to the previous hook in the chain (retrievable via
PQgetAuthDataHook()). Otherwise, it should return an integer > 0 and
follow the authdata-specific instructions. Returning an integer < 0
signals an error condition and abandons the connection attempt.
== PQAUTHDATA_PROMPT_OAUTH_DEVICE ==
The hook should display the device prompt (URL + code) using whatever
method it prefers.
== PQAUTHDATA_OAUTH_BEARER_TOKEN ==
The hook should either directly return a Bearer token for the current
user/issuer/scope combination, if one is available without blocking, or
else set up an asynchronous callback to retrieve one. See the
documentation for PQoauthBearerRequest.
Several TODOs:
- don't retry forever if the server won't accept our token
- perform several sanity checks on the OAuth issuer's responses
- handle cases where the client has been set up with an issuer and
scope, but the Postgres server wants to use something different
- improve error debuggability during the OAuth handshake
- fix libcurl initialization thread-safety
- harden the libcurl flow implementation
- figure out pgsocket/int difference on Windows
- fix intermittent failure in the cleanup callback tests (race
condition?)
- support require_auth
- ...and more.
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
---
configure | 186 ++
configure.ac | 37 +
meson.build | 45 +
meson_options.txt | 4 +
src/Makefile.global.in | 1 +
src/include/common/oauth-common.h | 19 +
src/include/pg_config.h.in | 18 +
src/interfaces/libpq/Makefile | 12 +-
src/interfaces/libpq/exports.txt | 3 +
src/interfaces/libpq/fe-auth-oauth-curl.c | 1982 +++++++++++++++++++
src/interfaces/libpq/fe-auth-oauth-iddawc.c | 319 +++
src/interfaces/libpq/fe-auth-oauth.c | 659 ++++++
src/interfaces/libpq/fe-auth-oauth.h | 42 +
src/interfaces/libpq/fe-auth-sasl.h | 10 +-
src/interfaces/libpq/fe-auth-scram.c | 6 +-
src/interfaces/libpq/fe-auth.c | 105 +-
src/interfaces/libpq/fe-auth.h | 9 +-
src/interfaces/libpq/fe-connect.c | 85 +-
src/interfaces/libpq/fe-misc.c | 7 +-
src/interfaces/libpq/libpq-fe.h | 77 +-
src/interfaces/libpq/libpq-int.h | 14 +
src/interfaces/libpq/meson.build | 9 +
src/makefiles/meson.build | 1 +
src/tools/pgindent/typedefs.list | 10 +
24 files changed, 3633 insertions(+), 27 deletions(-)
create mode 100644 src/include/common/oauth-common.h
create mode 100644 src/interfaces/libpq/fe-auth-oauth-curl.c
create mode 100644 src/interfaces/libpq/fe-auth-oauth-iddawc.c
create mode 100644 src/interfaces/libpq/fe-auth-oauth.c
create mode 100644 src/interfaces/libpq/fe-auth-oauth.h
diff --git a/configure b/configure
index 46859a4244..142d49d7b6 100755
--- a/configure
+++ b/configure
@@ -712,6 +712,7 @@ with_uuid
with_readline
with_systemd
with_selinux
+with_oauth
with_ldap
with_krb_srvnam
krb_srvtab
@@ -858,6 +859,7 @@ with_krb_srvnam
with_pam
with_bsd_auth
with_ldap
+with_oauth
with_bonjour
with_selinux
with_systemd
@@ -1568,6 +1570,7 @@ Optional Packages:
--with-pam build with PAM support
--with-bsd-auth build with BSD Authentication support
--with-ldap build with LDAP support
+ --with-oauth=LIB use LIB for OAuth 2.0 support (curl, iddawc)
--with-bonjour build with Bonjour support
--with-selinux build with SELinux support
--with-systemd build with systemd support
@@ -8485,6 +8488,59 @@ $as_echo "$with_ldap" >&6; }
+#
+# OAuth 2.0
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with OAuth support" >&5
+$as_echo_n "checking whether to build with OAuth support... " >&6; }
+
+
+
+# Check whether --with-oauth was given.
+if test "${with_oauth+set}" = set; then :
+ withval=$with_oauth;
+ case $withval in
+ yes)
+ as_fn_error $? "argument required for --with-oauth option" "$LINENO" 5
+ ;;
+ no)
+ as_fn_error $? "argument required for --with-oauth option" "$LINENO" 5
+ ;;
+ *)
+
+ ;;
+ esac
+
+fi
+
+
+if test x"$with_oauth" = x"" ; then
+ with_oauth=no
+fi
+
+if test x"$with_oauth" = x"curl"; then
+
+$as_echo "#define USE_OAUTH 1" >>confdefs.h
+
+
+$as_echo "#define USE_OAUTH_CURL 1" >>confdefs.h
+
+elif test x"$with_oauth" = x"iddawc"; then
+
+$as_echo "#define USE_OAUTH 1" >>confdefs.h
+
+
+$as_echo "#define USE_OAUTH_IDDAWC 1" >>confdefs.h
+
+elif test x"$with_oauth" != x"no"; then
+ as_fn_error $? "--with-oauth must specify curl or iddawc" "$LINENO" 5
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_oauth" >&5
+$as_echo "$with_oauth" >&6; }
+
+
+
#
# Bonjour
#
@@ -13037,6 +13093,116 @@ fi
+if test "$with_oauth" = curl ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for curl_multi_init in -lcurl" >&5
+$as_echo_n "checking for curl_multi_init in -lcurl... " >&6; }
+if ${ac_cv_lib_curl_curl_multi_init+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lcurl $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char curl_multi_init ();
+int
+main ()
+{
+return curl_multi_init ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_curl_curl_multi_init=yes
+else
+ ac_cv_lib_curl_curl_multi_init=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_curl_curl_multi_init" >&5
+$as_echo "$ac_cv_lib_curl_curl_multi_init" >&6; }
+if test "x$ac_cv_lib_curl_curl_multi_init" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBCURL 1
+_ACEOF
+
+ LIBS="-lcurl $LIBS"
+
+else
+ as_fn_error $? "library 'curl' is required for --with-oauth=curl" "$LINENO" 5
+fi
+
+elif test "$with_oauth" = iddawc ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for i_init_session in -liddawc" >&5
+$as_echo_n "checking for i_init_session in -liddawc... " >&6; }
+if ${ac_cv_lib_iddawc_i_init_session+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-liddawc $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char i_init_session ();
+int
+main ()
+{
+return i_init_session ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_iddawc_i_init_session=yes
+else
+ ac_cv_lib_iddawc_i_init_session=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_iddawc_i_init_session" >&5
+$as_echo "$ac_cv_lib_iddawc_i_init_session" >&6; }
+if test "x$ac_cv_lib_iddawc_i_init_session" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBIDDAWC 1
+_ACEOF
+
+ LIBS="-liddawc $LIBS"
+
+else
+ as_fn_error $? "library 'iddawc' is required for --with-oauth=iddawc" "$LINENO" 5
+fi
+
+ # Check for an older spelling of i_get_openid_config
+ for ac_func in i_load_openid_config
+do :
+ ac_fn_c_check_func "$LINENO" "i_load_openid_config" "ac_cv_func_i_load_openid_config"
+if test "x$ac_cv_func_i_load_openid_config" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_I_LOAD_OPENID_CONFIG 1
+_ACEOF
+
+fi
+done
+
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for security_compute_create_name in -lselinux" >&5
@@ -14062,6 +14228,26 @@ fi
done
+fi
+
+if test "$with_oauth" = curl; then
+ ac_fn_c_check_header_mongrel "$LINENO" "curl/curl.h" "ac_cv_header_curl_curl_h" "$ac_includes_default"
+if test "x$ac_cv_header_curl_curl_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <curl/curl.h> is required for OAuth" "$LINENO" 5
+fi
+
+
+elif test "$with_oauth" = iddawc; then
+ ac_fn_c_check_header_mongrel "$LINENO" "iddawc.h" "ac_cv_header_iddawc_h" "$ac_includes_default"
+if test "x$ac_cv_header_iddawc_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <iddawc.h> is required for OAuth" "$LINENO" 5
+fi
+
+
fi
if test "$PORTNAME" = "win32" ; then
diff --git a/configure.ac b/configure.ac
index 88b75a7696..a4c2e558f9 100644
--- a/configure.ac
+++ b/configure.ac
@@ -927,6 +927,29 @@ AC_MSG_RESULT([$with_ldap])
AC_SUBST(with_ldap)
+#
+# OAuth 2.0
+#
+AC_MSG_CHECKING([whether to build with OAuth support])
+PGAC_ARG_REQ(with, oauth, [LIB], [use LIB for OAuth 2.0 support (curl, iddawc)])
+if test x"$with_oauth" = x"" ; then
+ with_oauth=no
+fi
+
+if test x"$with_oauth" = x"curl"; then
+ AC_DEFINE([USE_OAUTH], 1, [Define to 1 to build with OAuth 2.0 support. (--with-oauth)])
+ AC_DEFINE([USE_OAUTH_CURL], 1, [Define to 1 to use libcurl for OAuth support.])
+elif test x"$with_oauth" = x"iddawc"; then
+ AC_DEFINE([USE_OAUTH], 1, [Define to 1 to build with OAuth 2.0 support. (--with-oauth)])
+ AC_DEFINE([USE_OAUTH_IDDAWC], 1, [Define to 1 to use libiddawc for OAuth support.])
+elif test x"$with_oauth" != x"no"; then
+ AC_MSG_ERROR([--with-oauth must specify curl or iddawc])
+fi
+
+AC_MSG_RESULT([$with_oauth])
+AC_SUBST(with_oauth)
+
+
#
# Bonjour
#
@@ -1423,6 +1446,14 @@ fi
AC_SUBST(LDAP_LIBS_FE)
AC_SUBST(LDAP_LIBS_BE)
+if test "$with_oauth" = curl ; then
+ AC_CHECK_LIB(curl, curl_multi_init, [], [AC_MSG_ERROR([library 'curl' is required for --with-oauth=curl])])
+elif test "$with_oauth" = iddawc ; then
+ AC_CHECK_LIB(iddawc, i_init_session, [], [AC_MSG_ERROR([library 'iddawc' is required for --with-oauth=iddawc])])
+ # Check for an older spelling of i_get_openid_config
+ AC_CHECK_FUNCS([i_load_openid_config])
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
AC_CHECK_LIB(selinux, security_compute_create_name, [],
@@ -1614,6 +1645,12 @@ elif test "$with_uuid" = ossp ; then
[AC_MSG_ERROR([header file <ossp/uuid.h> or <uuid.h> is required for OSSP UUID])])])
fi
+if test "$with_oauth" = curl; then
+ AC_CHECK_HEADER(curl/curl.h, [], [AC_MSG_ERROR([header file <curl/curl.h> is required for OAuth])])
+elif test "$with_oauth" = iddawc; then
+ AC_CHECK_HEADER(iddawc.h, [], [AC_MSG_ERROR([header file <iddawc.h> is required for OAuth])])
+fi
+
if test "$PORTNAME" = "win32" ; then
AC_CHECK_HEADERS(crtdefs.h)
fi
diff --git a/meson.build b/meson.build
index a198eca25d..43066a017f 100644
--- a/meson.build
+++ b/meson.build
@@ -830,6 +830,49 @@ endif
+###############################################################
+# Library: oauth
+###############################################################
+
+oauth = not_found_dep
+oauth_library = 'none'
+oauthopt = get_option('oauth')
+
+if oauthopt == 'auto' and auto_features.disabled()
+ oauthopt = 'none'
+endif
+
+if oauthopt in ['auto', 'curl']
+ oauth = dependency('libcurl', required: (oauthopt == 'curl'))
+
+ if oauth.found()
+ oauth_library = 'curl'
+ cdata.set('USE_OAUTH', 1)
+ cdata.set('USE_OAUTH_CURL', 1)
+ endif
+endif
+
+if not oauth.found() and oauthopt in ['auto', 'iddawc']
+ oauth = dependency('libiddawc', required: (oauthopt == 'iddawc'))
+
+ if oauth.found()
+ oauth_library = 'iddawc'
+ cdata.set('USE_OAUTH', 1)
+ cdata.set('USE_OAUTH_IDDAWC', 1)
+
+ # Check for an older spelling of i_get_openid_config
+ if cc.has_function('i_load_openid_config',
+ dependencies: oauth, args: test_c_args)
+ cdata.set('HAVE_I_LOAD_OPENID_CONFIG', 1)
+ endif
+ endif
+endif
+
+if oauthopt == 'auto' and auto_features.enabled() and not oauth.found()
+ error('no OAuth implementation library found')
+endif
+
+
###############################################################
# Library: Tcl (for pltcl)
#
@@ -2834,6 +2877,7 @@ libpq_deps += [
gssapi,
ldap_r,
libintl,
+ oauth,
ssl,
]
@@ -3435,6 +3479,7 @@ if meson.version().version_compare('>=0.57')
'llvm': llvm,
'lz4': lz4,
'nls': libintl,
+ 'oauth': oauth,
'openssl': ssl,
'pam': pam,
'plperl': perl_dep,
diff --git a/meson_options.txt b/meson_options.txt
index 249ecc5ffd..f54f7fd717 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -121,6 +121,10 @@ option('lz4', type: 'feature', value: 'auto',
option('nls', type: 'feature', value: 'auto',
description: 'Native language support')
+option('oauth', type : 'combo', choices : ['auto', 'none', 'curl', 'iddawc'],
+ value: 'auto',
+ description: 'use LIB for OAuth 2.0 support (curl, iddawc)')
+
option('pam', type: 'feature', value: 'auto',
description: 'PAM support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 8b3f8c24e0..79b3647834 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -193,6 +193,7 @@ with_ldap = @with_ldap@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
+with_oauth = @with_oauth@
with_system_tzdata = @with_system_tzdata@
with_uuid = @with_uuid@
with_zlib = @with_zlib@
diff --git a/src/include/common/oauth-common.h b/src/include/common/oauth-common.h
new file mode 100644
index 0000000000..5ff3488bfb
--- /dev/null
+++ b/src/include/common/oauth-common.h
@@ -0,0 +1,19 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth-common.h
+ * Declarations for helper functions used for OAuth/OIDC authentication
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/common/oauth-common.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef OAUTH_COMMON_H
+#define OAUTH_COMMON_H
+
+/* Name of SASL mechanism per IANA */
+#define OAUTHBEARER_NAME "OAUTHBEARER"
+
+#endif /* OAUTH_COMMON_H */
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 07e73567dc..f470c77669 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -231,6 +231,9 @@
/* Define to 1 if __builtin_constant_p(x) implies "i"(x) acceptance. */
#undef HAVE_I_CONSTRAINT__BUILTIN_CONSTANT_P
+/* Define to 1 if you have the `i_load_openid_config' function. */
+#undef HAVE_I_LOAD_OPENID_CONFIG
+
/* Define to 1 if you have the `kqueue' function. */
#undef HAVE_KQUEUE
@@ -243,6 +246,12 @@
/* Define to 1 if you have the `crypto' library (-lcrypto). */
#undef HAVE_LIBCRYPTO
+/* Define to 1 if you have the `curl' library (-lcurl). */
+#undef HAVE_LIBCURL
+
+/* Define to 1 if you have the `iddawc' library (-liddawc). */
+#undef HAVE_LIBIDDAWC
+
/* Define to 1 if you have the `ldap' library (-lldap). */
#undef HAVE_LIBLDAP
@@ -711,6 +720,15 @@
/* Define to select named POSIX semaphores. */
#undef USE_NAMED_POSIX_SEMAPHORES
+/* Define to 1 to build with OAuth 2.0 support. (--with-oauth) */
+#undef USE_OAUTH
+
+/* Define to 1 to use libcurl for OAuth support. */
+#undef USE_OAUTH_CURL
+
+/* Define to 1 to use libiddawc for OAuth support. */
+#undef USE_OAUTH_IDDAWC
+
/* Define to 1 to build with OpenSSL support. (--with-ssl=openssl) */
#undef USE_OPENSSL
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index fe2af575c5..86aada810f 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -61,6 +61,16 @@ OBJS += \
fe-secure-gssapi.o
endif
+ifneq ($(with_oauth),no)
+OBJS += fe-auth-oauth.o
+
+ifeq ($(with_oauth),iddawc)
+OBJS += fe-auth-oauth-iddawc.o
+else
+OBJS += fe-auth-oauth-curl.o
+endif
+endif
+
ifeq ($(PORTNAME), cygwin)
override shlib = cyg$(NAME)$(DLSUFFIX)
endif
@@ -79,7 +89,7 @@ endif
# that are built correctly for use in a shlib.
SHLIB_LINK_INTERNAL = -lpgcommon_shlib -lpgport_shlib
ifneq ($(PORTNAME), win32)
-SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
+SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lcurl -liddawc -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
else
SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi32 -lssl -lsocket -lnsl -lresolv -lintl -lm $(PTHREAD_LIBS), $(LIBS)) $(LDAP_LIBS_FE)
endif
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 088592deb1..0f8f5e3125 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -193,3 +193,6 @@ PQsendClosePrepared 190
PQsendClosePortal 191
PQchangePassword 192
PQsendPipelineSync 193
+PQsetAuthDataHook 194
+PQgetAuthDataHook 195
+PQdefaultAuthDataHook 196
diff --git a/src/interfaces/libpq/fe-auth-oauth-curl.c b/src/interfaces/libpq/fe-auth-oauth-curl.c
new file mode 100644
index 0000000000..0504f96e4e
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth-curl.c
@@ -0,0 +1,1982 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth-curl.c
+ * The libcurl implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth-curl.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <curl/curl.h>
+#include <math.h>
+#ifdef HAVE_SYS_EPOLL_H
+#include <sys/epoll.h>
+#include <sys/timerfd.h>
+#endif
+#ifdef HAVE_SYS_EVENT_H
+#include <sys/event.h>
+#endif
+#include <unistd.h>
+
+#include "common/jsonapi.h"
+#include "fe-auth.h"
+#include "fe-auth-oauth.h"
+#include "libpq-int.h"
+#include "mb/pg_wchar.h"
+
+/*
+ * Parsed JSON Representations
+ *
+ * As a general rule, we parse and cache only the fields we're currently using.
+ * When adding new fields, ensure the corresponding free_*() function is updated
+ * too.
+ */
+
+/*
+ * The OpenID Provider configuration (alternatively named "authorization server
+ * metadata") jointly described by OpenID Connect Discovery 1.0 and RFC 8414:
+ *
+ * https://openid.net/specs/openid-connect-discovery-1_0.html
+ * https://www.rfc-editor.org/rfc/rfc8414#section-3.2
+ */
+struct provider
+{
+ char *issuer;
+ char *token_endpoint;
+ char *device_authorization_endpoint;
+ struct curl_slist *grant_types_supported;
+};
+
+static void
+free_provider(struct provider *provider)
+{
+ free(provider->issuer);
+ free(provider->token_endpoint);
+ free(provider->device_authorization_endpoint);
+ curl_slist_free_all(provider->grant_types_supported);
+}
+
+/*
+ * The Device Authorization response, described by RFC 8628:
+ *
+ * https://www.rfc-editor.org/rfc/rfc8628#section-3.2
+ */
+struct device_authz
+{
+ char *device_code;
+ char *user_code;
+ char *verification_uri;
+ char *interval_str;
+
+ /* Fields below are parsed from the corresponding string above. */
+ int interval;
+};
+
+static void
+free_device_authz(struct device_authz *authz)
+{
+ free(authz->device_code);
+ free(authz->user_code);
+ free(authz->verification_uri);
+ free(authz->interval_str);
+}
+
+/*
+ * The Token Endpoint error response, as described by RFC 6749:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-5.2
+ *
+ * Note that this response type can also be returned from the Device
+ * Authorization Endpoint.
+ */
+struct token_error
+{
+ char *error;
+ char *error_description;
+};
+
+static void
+free_token_error(struct token_error *err)
+{
+ free(err->error);
+ free(err->error_description);
+}
+
+/*
+ * The Access Token response, as described by RFC 6749:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-4.1.4
+ *
+ * During the Device Authorization flow, several temporary errors are expected
+ * as part of normal operation. To make it easy to handle these in the happy
+ * path, this contains an embedded token_error that is filled in if needed.
+ */
+struct token
+{
+ /* for successful responses */
+ char *access_token;
+ char *token_type;
+
+ /* for error responses */
+ struct token_error err;
+};
+
+static void
+free_token(struct token *tok)
+{
+ free(tok->access_token);
+ free(tok->token_type);
+ free_token_error(&tok->err);
+}
+
+/*
+ * Asynchronous State
+ */
+
+/* States for the overall async machine. */
+typedef enum
+{
+ OAUTH_STEP_INIT,
+ OAUTH_STEP_DISCOVERY,
+ OAUTH_STEP_DEVICE_AUTHORIZATION,
+ OAUTH_STEP_TOKEN_REQUEST,
+ OAUTH_STEP_WAIT_INTERVAL,
+} OAuthStep;
+
+/*
+ * The async_ctx holds onto state that needs to persist across multiple calls to
+ * pg_fe_run_oauth_flow(). Almost everything interacts with this in some way.
+ */
+struct async_ctx
+{
+ OAuthStep step; /* where are we in the flow? */
+
+#ifdef HAVE_SYS_EPOLL_H
+ int timerfd; /* a timerfd for signaling async timeouts */
+#endif
+ pgsocket mux; /* the multiplexer socket containing all
+ * descriptors tracked by cURL, plus the
+ * timerfd */
+ CURLM *curlm; /* top-level multi handle for cURL operations */
+ CURL *curl; /* the (single) easy handle for serial
+ * requests */
+
+ struct curl_slist *headers; /* common headers for all requests */
+ PQExpBufferData work_data; /* scratch buffer for general use (remember to
+ * clear out prior contents first!) */
+
+ /*------
+ * Since a single logical operation may stretch across multiple calls to
+ * our entry point, errors have three parts:
+ *
+ * - errctx: an optional static string, describing the global operation
+ * currently in progress. It'll be translated for you.
+ *
+ * - errbuf: contains the actual error message. Generally speaking, use
+ * actx_error[_str] to manipulate this. This must be filled
+ * with something useful on an error.
+ *
+ * - curl_err: an optional static error buffer used by cURL to put
+ * detailed information about failures. Unfortunately
+ * untranslatable.
+ *
+ * These pieces will be combined into a single error message looking
+ * something like the following, with errctx and/or curl_err omitted when
+ * absent:
+ *
+ * connection to server ... failed: errctx: errbuf (curl_err)
+ */
+ const char *errctx; /* not freed; must point to static allocation */
+ PQExpBufferData errbuf;
+ char curl_err[CURL_ERROR_SIZE];
+
+ /*
+ * These documents need to survive over multiple calls, and are therefore
+ * cached directly in the async_ctx.
+ */
+ struct provider provider;
+ struct device_authz authz;
+
+ bool user_prompted; /* have we already sent the authz prompt? */
+};
+
+/*
+ * Frees the async_ctx, which is stored directly on the PGconn. This is called
+ * during pqDropConnection() so that we don't leak resources even if
+ * PQconnectPoll() never calls us back.
+ *
+ * TODO: we should probably call this at the end of a successful authentication,
+ * too, to proactively free up resources.
+ */
+static void
+free_curl_async_ctx(PGconn *conn, void *ctx)
+{
+ struct async_ctx *actx = ctx;
+
+ Assert(actx); /* oauth_free() shouldn't call us otherwise */
+
+ /*
+ * TODO: in general, none of the error cases below should ever happen if
+ * we have no bugs above. But if we do hit them, surfacing those errors
+ * somehow might be the only way to have a chance to debug them. What's
+ * the best way to do that? Assertions? Spraying messages on stderr?
+ * Bubbling an error code to the top? Appending to the connection's error
+ * message only helps if the bug caused a connection failure; otherwise
+ * it'll be buried...
+ */
+
+ if (actx->curlm && actx->curl)
+ {
+ CURLMcode err = curl_multi_remove_handle(actx->curlm, actx->curl);
+
+ if (err)
+ libpq_append_conn_error(conn,
+ "cURL easy handle removal failed: %s",
+ curl_multi_strerror(err));
+ }
+
+ if (actx->curl)
+ {
+ /*
+ * curl_multi_cleanup() doesn't free any associated easy handles; we
+ * need to do that separately. We only ever have one easy handle per
+ * multi handle.
+ */
+ curl_easy_cleanup(actx->curl);
+ }
+
+ if (actx->curlm)
+ {
+ CURLMcode err = curl_multi_cleanup(actx->curlm);
+
+ if (err)
+ libpq_append_conn_error(conn,
+ "cURL multi handle cleanup failed: %s",
+ curl_multi_strerror(err));
+ }
+
+ free_provider(&actx->provider);
+ free_device_authz(&actx->authz);
+
+ curl_slist_free_all(actx->headers);
+ termPQExpBuffer(&actx->work_data);
+ termPQExpBuffer(&actx->errbuf);
+
+ if (actx->mux != PGINVALID_SOCKET)
+ close(actx->mux);
+#ifdef HAVE_SYS_EPOLL_H
+ if (actx->timerfd >= 0)
+ close(actx->timerfd);
+#endif
+
+ free(actx);
+}
+
+/*
+ * Macros for manipulating actx->errbuf. actx_error() translates and formats a
+ * string for you; actx_error_str() appends a string directly without
+ * translation.
+ */
+
+#define actx_error(ACTX, FMT, ...) \
+ appendPQExpBuffer(&(ACTX)->errbuf, libpq_gettext(FMT), ##__VA_ARGS__)
+
+#define actx_error_str(ACTX, S) \
+ appendPQExpBufferStr(&(ACTX)->errbuf, S)
+
+/*
+ * Macros for getting and setting state for the connection's two cURL handles,
+ * so you don't have to write out the error handling every time.
+ */
+
+#define CHECK_MSETOPT(ACTX, OPT, VAL, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLMcode _setopterr = curl_multi_setopt(_actx->curlm, OPT, VAL); \
+ if (_setopterr) { \
+ actx_error(_actx, "failed to set %s on OAuth connection: %s",\
+ #OPT, curl_multi_strerror(_setopterr)); \
+ FAILACTION; \
+ } \
+ } while (0)
+
+#define CHECK_SETOPT(ACTX, OPT, VAL, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLcode _setopterr = curl_easy_setopt(_actx->curl, OPT, VAL); \
+ if (_setopterr) { \
+ actx_error(_actx, "failed to set %s on OAuth connection: %s",\
+ #OPT, curl_easy_strerror(_setopterr)); \
+ FAILACTION; \
+ } \
+ } while (0)
+
+#define CHECK_GETINFO(ACTX, INFO, OUT, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLcode _getinfoerr = curl_easy_getinfo(_actx->curl, INFO, OUT); \
+ if (_getinfoerr) { \
+ actx_error(_actx, "failed to get %s from OAuth response: %s",\
+ #INFO, curl_easy_strerror(_getinfoerr)); \
+ FAILACTION; \
+ } \
+ } while (0)
+
+/*
+ * General JSON Parsing for OAuth Responses
+ */
+
+/*
+ * Represents a single name/value pair in a JSON object. This is the primary
+ * interface to parse_oauth_json().
+ *
+ * All fields are stored internally as strings or lists of strings, so clients
+ * have to explicitly parse other scalar types (though they will have gone
+ * through basic lexical validation). Storing nested objects is not currently
+ * supported, nor is parsing arrays of anything other than strings.
+ */
+struct json_field
+{
+ const char *name; /* name (key) of the member */
+
+ JsonTokenType type; /* currently supports JSON_TOKEN_STRING,
+ * JSON_TOKEN_NUMBER, and
+ * JSON_TOKEN_ARRAY_START */
+ union
+ {
+ char **scalar; /* for all scalar types */
+ struct curl_slist **array; /* for type == JSON_TOKEN_ARRAY_START */
+ };
+
+ bool required; /* REQUIRED field, or just OPTIONAL? */
+};
+
+/* Documentation macros for json_field.required. */
+#define REQUIRED true
+#define OPTIONAL false
+
+/* Parse state for parse_oauth_json(). */
+struct oauth_parse
+{
+ PQExpBuffer errbuf; /* detail message for JSON_SEM_ACTION_FAILED */
+ int nested; /* nesting level (zero is the top) */
+
+ const struct json_field *fields; /* field definition array */
+ const struct json_field *active; /* points inside the fields array */
+};
+
+#define oauth_parse_set_error(ctx, fmt, ...) \
+ appendPQExpBuffer((ctx)->errbuf, libpq_gettext(fmt), ##__VA_ARGS__)
+
+static void
+report_type_mismatch(struct oauth_parse *ctx)
+{
+ char *msgfmt;
+
+ Assert(ctx->active);
+
+ /*
+ * At the moment, the only fields we're interested in are strings,
+ * numbers, and arrays of strings.
+ */
+ switch (ctx->active->type)
+ {
+ case JSON_TOKEN_STRING:
+ msgfmt = "field \"%s\" must be a string";
+ break;
+
+ case JSON_TOKEN_NUMBER:
+ msgfmt = "field \"%s\" must be a number";
+ break;
+
+ case JSON_TOKEN_ARRAY_START:
+ msgfmt = "field \"%s\" must be an array of strings";
+ break;
+
+ default:
+ Assert(false);
+ msgfmt = "field \"%s\" has unexpected type";
+ }
+
+ oauth_parse_set_error(ctx, msgfmt, ctx->active->name);
+}
+
+static JsonParseErrorType
+oauth_json_object_start(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (ctx->active)
+ {
+ /*
+ * Currently, none of the fields we're interested in can be or contain
+ * objects, so we can reject this case outright.
+ */
+ report_type_mismatch(ctx);
+ return JSON_SEM_ACTION_FAILED;
+ }
+
+ ++ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_field_start(void *state, char *name, bool isnull)
+{
+ struct oauth_parse *ctx = state;
+
+ /* We care only about the top-level fields. */
+ if (ctx->nested == 1)
+ {
+ const struct json_field *field = ctx->fields;
+
+ /*
+ * We should never start parsing a new field while a previous one is
+ * still active.
+ *
+ * TODO: this code relies on assertions too much. We need to exit
+ * sanely on internal logic errors, to avoid turning bugs into
+ * vulnerabilities.
+ */
+ Assert(!ctx->active);
+
+ while (field->name)
+ {
+ if (strcmp(name, field->name) == 0)
+ {
+ ctx->active = field;
+ break;
+ }
+
+ ++field;
+ }
+ }
+
+ free(name);
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_end(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_start(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (!ctx->nested)
+ {
+ oauth_parse_set_error(ctx, "top-level element must be an object");
+ return JSON_SEM_ACTION_FAILED;
+ }
+
+ if (ctx->active)
+ {
+ if (ctx->active->type != JSON_TOKEN_ARRAY_START
+ /* The arrays we care about must not have arrays as values. */
+ || ctx->nested > 1)
+ {
+ report_type_mismatch(ctx);
+ return JSON_SEM_ACTION_FAILED;
+ }
+ }
+
+ ++ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_end(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (ctx->active)
+ {
+ /*
+ * This assumes that no target arrays can contain other arrays, which
+ * we check in the array_start callback.
+ */
+ Assert(ctx->nested == 2);
+ Assert(ctx->active->type == JSON_TOKEN_ARRAY_START);
+
+ ctx->active = NULL;
+ }
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_scalar(void *state, char *token, JsonTokenType type)
+{
+ struct oauth_parse *ctx = state;
+ JsonParseErrorType result = JSON_SUCCESS;
+
+ if (!ctx->nested)
+ {
+ oauth_parse_set_error(ctx, "top-level element must be an object");
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ if (ctx->active)
+ {
+ JsonTokenType expected;
+
+ /*
+ * Make sure this matches what the active field expects. Arrays must
+ * contain only strings with the current implementation.
+ */
+ if (ctx->active->type == JSON_TOKEN_ARRAY_START)
+ expected = JSON_TOKEN_STRING;
+ else
+ expected = ctx->active->type;
+
+ if (type != expected)
+ {
+ report_type_mismatch(ctx);
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ /*
+ * FIXME if the JSON field is duplicated, we'll leak the prior value.
+ * Error out in that case instead.
+ */
+ if (ctx->active->type != JSON_TOKEN_ARRAY_START)
+ {
+ Assert(ctx->nested == 1);
+
+ *ctx->active->scalar = token;
+ ctx->active = NULL;
+
+ return JSON_SUCCESS; /* don't free the token */
+ }
+ else /* ctx->target_array */
+ {
+ struct curl_slist *temp;
+
+ Assert(ctx->nested == 2);
+
+ temp = curl_slist_append(*ctx->active->array, token);
+ if (!temp)
+ {
+ oauth_parse_set_error(ctx, "out of memory");
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ *ctx->active->array = temp;
+
+ /*
+ * Note that curl_slist_append() makes a copy of the token, so we
+ * can free it below.
+ */
+ }
+ }
+ else
+ {
+ /* otherwise we just ignore it */
+ }
+
+cleanup:
+ free(token);
+ return result;
+}
+
+/*
+ * A helper function for general JSON parsing. fields is the array of field
+ * definitions with their backing pointers. The response will be parsed from
+ * actx->curl and actx->work_data (as set up by start_request()), and any
+ * parsing errors will be placed into actx->errbuf.
+ */
+static bool
+parse_oauth_json(struct async_ctx *actx, const struct json_field *fields)
+{
+ PQExpBuffer resp = &actx->work_data;
+ char *content_type;
+ JsonLexContext lex = {0};
+ JsonSemAction sem = {0};
+ JsonParseErrorType err;
+ struct oauth_parse ctx = {0};
+ bool success = false;
+
+ /* Make sure the server thinks it's given us JSON. */
+ CHECK_GETINFO(actx, CURLINFO_CONTENT_TYPE, &content_type, return false);
+
+ if (!content_type)
+ {
+ actx_error(actx, "no content type was provided");
+ goto cleanup;
+ }
+ else if (strcasecmp(content_type, "application/json") != 0)
+ {
+ actx_error(actx, "unexpected content type \"%s\"", content_type);
+ goto cleanup;
+ }
+
+ if (strlen(resp->data) != resp->len)
+ {
+ actx_error(actx, "response contains embedded NULLs");
+ goto cleanup;
+ }
+
+ makeJsonLexContextCstringLen(&lex, resp->data, resp->len, PG_UTF8, true);
+
+ ctx.errbuf = &actx->errbuf;
+ ctx.fields = fields;
+ sem.semstate = &ctx;
+
+ sem.object_start = oauth_json_object_start;
+ sem.object_field_start = oauth_json_object_field_start;
+ sem.object_end = oauth_json_object_end;
+ sem.array_start = oauth_json_array_start;
+ sem.array_end = oauth_json_array_end;
+ sem.scalar = oauth_json_scalar;
+
+ err = pg_parse_json(&lex, &sem);
+
+ if (err != JSON_SUCCESS)
+ {
+ /*
+ * For JSON_SEM_ACTION_FAILED, we've already written the error
+ * message. Other errors come directly from pg_parse_json(), already
+ * translated.
+ */
+ if (err != JSON_SEM_ACTION_FAILED)
+ actx_error_str(actx, json_errdetail(err, &lex));
+
+ goto cleanup;
+ }
+
+ /* Check all required fields. */
+ while (fields->name)
+ {
+ if (fields->required && !*fields->scalar && !*fields->array)
+ {
+ actx_error(actx, "field \"%s\" is missing", fields->name);
+ goto cleanup;
+ }
+
+ fields++;
+ }
+
+ success = true;
+
+cleanup:
+ freeJsonLexContext(&lex);
+ return success;
+}
+
+/*
+ * JSON Parser Definitions
+ */
+
+static bool
+parse_provider(struct async_ctx *actx, struct provider *provider)
+{
+ struct json_field fields[] = {
+ {"issuer", JSON_TOKEN_STRING, {&provider->issuer}, REQUIRED},
+ {"token_endpoint", JSON_TOKEN_STRING, {&provider->token_endpoint}, REQUIRED},
+
+ /*----
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - jwks_uri
+ * - response_types_supported
+ * - subject_types_supported
+ * - id_token_signing_alg_values_supported
+ */
+
+ {"device_authorization_endpoint", JSON_TOKEN_STRING, {&provider->device_authorization_endpoint}, OPTIONAL},
+ {"grant_types_supported", JSON_TOKEN_ARRAY_START, {.array = &provider->grant_types_supported}, OPTIONAL},
+
+ {0},
+ };
+
+ return parse_oauth_json(actx, fields);
+}
+
+/*
+ * Parses the "interval" JSON number, corresponding to the number of seconds to
+ * wait between token endpoint requests.
+ *
+ * RFC 8628 is pretty silent on sanity checks for the interval. As a matter of
+ * practicality, round any fractional intervals up to the next second, and clamp
+ * the result at a minimum of one. (Zero-second intervals would result in an
+ * expensive network polling loop.)
+ *
+ * TODO: maybe clamp the upper bound too, based on the libpq timeout and/or the
+ * code expiration time?
+ */
+static int
+parse_interval(const char *interval_str)
+{
+ double parsed;
+ int cnt;
+
+ /*
+ * The JSON lexer has already validated the number, which is stricter than
+ * the %f format, so we should be good to use sscanf().
+ */
+ cnt = sscanf(interval_str, "%lf", &parsed);
+
+ if (cnt != 1)
+ {
+ /*
+ * Either the lexer screwed up or our assumption above isn't true, and
+ * either way a developer needs to take a look.
+ */
+ Assert(cnt == 1);
+ return 1; /* don't fall through in release builds */
+ }
+
+ parsed = ceil(parsed);
+
+ if (parsed < 1)
+ return 1; /* TODO this slows down the tests
+ * considerably... */
+ else if (INT_MAX <= parsed)
+ return INT_MAX;
+
+ return parsed;
+}
+
+static bool
+parse_device_authz(struct async_ctx *actx, struct device_authz *authz)
+{
+ struct json_field fields[] = {
+ {"device_code", JSON_TOKEN_STRING, {&authz->device_code}, REQUIRED},
+ {"user_code", JSON_TOKEN_STRING, {&authz->user_code}, REQUIRED},
+ {"verification_uri", JSON_TOKEN_STRING, {&authz->verification_uri}, REQUIRED},
+
+ /*
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - expires_in
+ */
+
+ {"interval", JSON_TOKEN_NUMBER, {&authz->interval_str}, OPTIONAL},
+
+ {0},
+ };
+
+ if (!parse_oauth_json(actx, fields))
+ return false;
+
+ /*
+ * Parse our numeric fields. Lexing has already completed by this time, so
+ * we at least know they're valid JSON numbers.
+ */
+ if (authz->interval_str)
+ authz->interval = parse_interval(authz->interval_str);
+ else
+ {
+ /* TODO: handle default interval of 5 seconds */
+ }
+
+ return true;
+}
+
+static bool
+parse_token_error(struct async_ctx *actx, struct token_error *err)
+{
+ bool result;
+ struct json_field fields[] = {
+ {"error", JSON_TOKEN_STRING, {&err->error}, REQUIRED},
+
+ {"error_description", JSON_TOKEN_STRING, {&err->error_description}, OPTIONAL},
+
+ {0},
+ };
+
+ result = parse_oauth_json(actx, fields);
+
+ /*
+ * Since token errors are parsed during other active error paths, only
+ * override the errctx if parsing explicitly fails.
+ */
+ if (!result)
+ actx->errctx = "failed to parse token error response";
+
+ return result;
+}
+
+static bool
+parse_access_token(struct async_ctx *actx, struct token *tok)
+{
+ struct json_field fields[] = {
+ {"access_token", JSON_TOKEN_STRING, {&tok->access_token}, REQUIRED},
+ {"token_type", JSON_TOKEN_STRING, {&tok->token_type}, REQUIRED},
+
+ /*
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - scope (only required if different than requested -- TODO check)
+ */
+
+ {0},
+ };
+
+ return parse_oauth_json(actx, fields);
+}
+
+/*
+ * cURL Multi Setup/Callbacks
+ */
+
+/*
+ * Sets up the actx->mux, which is the altsock that PQconnectPoll clients will
+ * select() on instead of the Postgres socket during OAuth negotiation.
+ *
+ * This is just an epoll set or kqueue abstracting multiple other descriptors.
+ * A timerfd is always part of the set when using epoll; it's just disabled
+ * when we're not using it.
+ */
+static bool
+setup_multiplexer(struct async_ctx *actx)
+{
+#ifdef HAVE_SYS_EPOLL_H
+ struct epoll_event ev = {.events = EPOLLIN};
+
+ actx->mux = epoll_create1(EPOLL_CLOEXEC);
+ if (actx->mux < 0)
+ {
+ actx_error(actx, "failed to create epoll set: %m");
+ return false;
+ }
+
+ actx->timerfd = timerfd_create(CLOCK_MONOTONIC, TFD_CLOEXEC);
+ if (actx->timerfd < 0)
+ {
+ actx_error(actx, "failed to create timerfd: %m");
+ return false;
+ }
+
+ if (epoll_ctl(actx->mux, EPOLL_CTL_ADD, actx->timerfd, &ev) < 0)
+ {
+ actx_error(actx, "failed to add timerfd to epoll set: %m");
+ return false;
+ }
+
+ return true;
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ actx->mux = kqueue();
+ if (actx->mux < 0)
+ {
+ actx_error(actx, "failed to create kqueue: %m");
+ return false;
+ }
+
+ return true;
+#endif
+
+ actx_error(actx, "here's a nickel kid, get yourself a better computer");
+ return false;
+}
+
+/*
+ * Adds and removes sockets from the multiplexer set, as directed by the
+ * cURL multi handle.
+ */
+static int
+register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
+ void *socketp)
+{
+#ifdef HAVE_SYS_EPOLL_H
+ struct async_ctx *actx = ctx;
+ struct epoll_event ev = {0};
+ int res;
+ int op = EPOLL_CTL_ADD;
+
+ switch (what)
+ {
+ case CURL_POLL_IN:
+ ev.events = EPOLLIN;
+ break;
+
+ case CURL_POLL_OUT:
+ ev.events = EPOLLOUT;
+ break;
+
+ case CURL_POLL_INOUT:
+ ev.events = EPOLLIN | EPOLLOUT;
+ break;
+
+ case CURL_POLL_REMOVE:
+ op = EPOLL_CTL_DEL;
+ break;
+
+ default:
+ actx_error(actx, "unknown cURL socket operation (%d)", what);
+ return -1;
+ }
+
+ res = epoll_ctl(actx->mux, op, socket, &ev);
+ if (res < 0 && errno == EEXIST)
+ {
+ /* We already had this socket in the pollset. */
+ op = EPOLL_CTL_MOD;
+ res = epoll_ctl(actx->mux, op, socket, &ev);
+ }
+
+ if (res < 0)
+ {
+ switch (op)
+ {
+ case EPOLL_CTL_ADD:
+ actx_error(actx, "could not add to epoll set: %m");
+ break;
+
+ case EPOLL_CTL_DEL:
+ actx_error(actx, "could not delete from epoll set: %m");
+ break;
+
+ default:
+ actx_error(actx, "could not update epoll set: %m");
+ }
+
+ return -1;
+ }
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct async_ctx *actx = ctx;
+ struct kevent ev[2] = {{0}};
+ struct kevent ev_out[2];
+ struct timespec timeout = {0};
+ int nev = 0;
+ int res;
+
+ switch (what)
+ {
+ case CURL_POLL_IN:
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_OUT:
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_INOUT:
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_REMOVE:
+
+ /*
+ * We don't know which of these is currently registered, perhaps
+ * both, so we try to remove both. This means we need to tolerate
+ * ENOENT below.
+ */
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_DELETE | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_DELETE | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ default:
+ actx_error(actx, "unknown cURL socket operation (%d)", what);
+ return -1;
+ }
+
+ res = kevent(actx->mux, ev, nev, ev_out, lengthof(ev_out), &timeout);
+ if (res < 0)
+ {
+ actx_error(actx, "could not modify kqueue: %m");
+ return -1;
+ }
+
+ /*
+ * We can't use the simple errno version of kevent, because we need to
+ * skip over ENOENT while still allowing a second change to be processed.
+ * So we need a longer-form error checking loop.
+ */
+ for (int i = 0; i < res; ++i)
+ {
+ /*
+ * EV_RECEIPT should guarantee one EV_ERROR result for every change,
+ * whether successful or not. Failed entries contain a non-zero errno
+ * in the `data` field.
+ */
+ Assert(ev_out[i].flags & EV_ERROR);
+
+ errno = ev_out[i].data;
+ if (errno && errno != ENOENT)
+ {
+ switch (what)
+ {
+ case CURL_POLL_REMOVE:
+ actx_error(actx, "could not delete from kqueue: %m");
+ break;
+ default:
+ actx_error(actx, "could not add to kqueue: %m");
+ }
+ return -1;
+ }
+ }
+#endif
+
+ return 0;
+}
+
+/*
+ * Adds or removes timeouts from the multiplexer set, as directed by the
+ * cURL multi handle. Rather than continually adding and removing the timer,
+ * we keep it in the set at all times and just disarm it when it's not
+ * needed.
+ */
+static int
+register_timer(CURLM *curlm, long timeout, void *ctx)
+{
+#if HAVE_SYS_EPOLL_H
+ struct async_ctx *actx = ctx;
+ struct itimerspec spec = {0};
+
+ if (timeout < 0)
+ {
+ /* the zero itimerspec will disarm the timer below */
+ }
+ else if (timeout == 0)
+ {
+ /*
+ * A zero timeout means cURL wants us to call back immediately. That's
+ * not technically an option for timerfd, but we can make the timeout
+ * ridiculously short.
+ *
+ * TODO: maybe just signal drive_request() to immediately call back in
+ * this case?
+ */
+ spec.it_value.tv_nsec = 1;
+ }
+ else
+ {
+ spec.it_value.tv_sec = timeout / 1000;
+ spec.it_value.tv_nsec = (timeout % 1000) * 1000000;
+ }
+
+ if (timerfd_settime(actx->timerfd, 0 /* no flags */ , &spec, NULL) < 0)
+ {
+ actx_error(actx, "setting timerfd to %ld: %m", timeout);
+ return -1;
+ }
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct async_ctx *actx = ctx;
+ struct kevent ev;
+
+ EV_SET(&ev, 1, EVFILT_TIMER, timeout < 0 ? EV_DELETE : EV_ADD,
+ 0, timeout, 0);
+ if (kevent(actx->mux, &ev, 1, NULL, 0, NULL) < 0 && errno != ENOENT)
+ {
+ actx_error(actx, "setting kqueue timer to %ld: %m", timeout);
+ return -1;
+ }
+#endif
+
+ return 0;
+}
+
+/*
+ * Initializes the two cURL handles in the async_ctx. The multi handle,
+ * actx->curlm, is what drives the asynchronous engine and tells us what to do
+ * next. The easy handle, actx->curl, encapsulates the state for a single
+ * request/response. It's added to the multi handle as needed, during
+ * start_request().
+ */
+static bool
+setup_curl_handles(struct async_ctx *actx)
+{
+ curl_version_info_data *curl_info;
+
+ /*
+ * Create our multi handle. This encapsulates the entire conversation with
+ * cURL for this connection.
+ */
+ actx->curlm = curl_multi_init();
+ if (!actx->curlm)
+ {
+ /* We don't get a lot of feedback on the failure reason. */
+ actx_error(actx, "failed to create cURL multi handle");
+ return false;
+ }
+
+ /*
+ * Extract information about the libcurl we are linked against.
+ */
+ curl_info = curl_version_info(CURLVERSION_NOW);
+
+ /*
+ * The multi handle tells us what to wait on using two callbacks. These
+ * will manipulate actx->mux as needed.
+ */
+ CHECK_MSETOPT(actx, CURLMOPT_SOCKETFUNCTION, register_socket, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_SOCKETDATA, actx, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_TIMERFUNCTION, register_timer, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_TIMERDATA, actx, return false);
+
+ /*
+ * Set up an easy handle. All of our requests are made serially, so we
+ * only ever need to keep track of one.
+ */
+ actx->curl = curl_easy_init();
+ if (!actx->curl)
+ {
+ actx_error(actx, "failed to create cURL handle");
+ return false;
+ }
+
+ /*
+ * Multi-threaded applications must set CURLOPT_NOSIGNAL. This requires us
+ * to handle the possibility of SIGPIPE ourselves.
+ *
+ * TODO: handle SIGPIPE via pq_block_sigpipe(), or via a
+ * CURLOPT_SOCKOPTFUNCTION maybe...
+ */
+ CHECK_SETOPT(actx, CURLOPT_NOSIGNAL, 1L, return false);
+ if (!curl_info->ares_num)
+ {
+ /* No alternative resolver, TODO: warn about timeouts */
+ }
+
+ /* TODO investigate using conn->Pfdebug and CURLOPT_DEBUGFUNCTION here */
+ CHECK_SETOPT(actx, CURLOPT_VERBOSE, 1L, return false);
+ CHECK_SETOPT(actx, CURLOPT_ERRORBUFFER, actx->curl_err, return false);
+
+ /*
+ * Only HTTP[S] is allowed. TODO: disallow HTTP without user opt-in
+ */
+ CHECK_SETOPT(actx, CURLOPT_PROTOCOLS, CURLPROTO_HTTP | CURLPROTO_HTTPS, return false);
+
+ /*
+ * Suppress the Accept header to make our request as minimal as possible.
+ * (Ideally we would set it to "application/json" instead, but OpenID is
+ * pretty strict when it comes to provider behavior, so we have to check
+ * what comes back anyway.)
+ */
+ actx->headers = curl_slist_append(actx->headers, "Accept:"); /* TODO: check result */
+ CHECK_SETOPT(actx, CURLOPT_HTTPHEADER, actx->headers, return false);
+
+ return true;
+}
+
+/*
+ * Generic HTTP Request Handlers
+ */
+
+/*
+ * Response callback from cURL; appends the response body into actx->work_data.
+ * See start_request().
+ */
+static size_t
+append_data(char *buf, size_t size, size_t nmemb, void *userdata)
+{
+ PQExpBuffer resp = userdata;
+ size_t len = size * nmemb;
+
+ /* TODO: cap the maximum size */
+ appendBinaryPQExpBuffer(resp, buf, len);
+ /* TODO: check for broken buffer */
+
+ return len;
+}
+
+/*
+ * Begins an HTTP request on the multi handle. The caller should have set up all
+ * request-specific options on actx->curl first. The server's response body will
+ * be accumulated in actx->work_data (which will be reset, so don't store
+ * anything important there across this call).
+ *
+ * Once a request is queued, it can be driven to completion via drive_request().
+ */
+static bool
+start_request(struct async_ctx *actx)
+{
+ CURLMcode err;
+ int running;
+
+ resetPQExpBuffer(&actx->work_data);
+ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data, return false);
+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, &actx->work_data, return false);
+
+ err = curl_multi_add_handle(actx->curlm, actx->curl);
+ if (err)
+ {
+ actx_error(actx, "failed to queue HTTP request: %s",
+ curl_multi_strerror(err));
+ return false;
+ }
+
+ err = curl_multi_socket_action(actx->curlm, CURL_SOCKET_TIMEOUT, 0, &running);
+ if (err)
+ {
+ actx_error(actx, "asynchronous HTTP request failed: %s",
+ curl_multi_strerror(err));
+ return false;
+ }
+
+ /*
+ * Sanity check.
+ *
+ * TODO: even though this is nominally an asynchronous process, there are
+ * apparently operations that can synchronously fail by this point, such
+ * as connections to closed local ports. Maybe we need to let this case
+ * fall through to drive_request instead, or else perform a
+ * curl_multi_info_read immediately.
+ */
+ if (running != 1)
+ {
+ actx_error(actx, "failed to queue HTTP request");
+ return false;
+ }
+
+ return true;
+}
+
+/*
+ * Drives the multi handle towards completion. The caller should have already
+ * set up an asynchronous request via start_request().
+ */
+static PostgresPollingStatusType
+drive_request(struct async_ctx *actx)
+{
+ CURLMcode err;
+ int running;
+ CURLMsg *msg;
+ int msgs_left;
+ bool done;
+
+ err = curl_multi_socket_all(actx->curlm, &running);
+ if (err)
+ {
+ actx_error(actx, "asynchronous HTTP request failed: %s",
+ curl_multi_strerror(err));
+ return PGRES_POLLING_FAILED;
+ }
+
+ if (running)
+ {
+ /* We'll come back again. */
+ return PGRES_POLLING_READING;
+ }
+
+ done = false;
+ while ((msg = curl_multi_info_read(actx->curlm, &msgs_left)) != NULL)
+ {
+ if (msg->msg != CURLMSG_DONE)
+ {
+ /*
+ * Future cURL versions may define new message types; we don't
+ * know how to handle them, so we'll ignore them.
+ */
+ continue;
+ }
+
+ /* First check the status of the request itself. */
+ if (msg->data.result != CURLE_OK)
+ {
+ actx_error_str(actx, curl_easy_strerror(msg->data.result));
+ return PGRES_POLLING_FAILED;
+ }
+
+ /* Now remove the finished handle; we'll add it back later if needed. */
+ err = curl_multi_remove_handle(actx->curlm, msg->easy_handle);
+ if (err)
+ {
+ actx_error(actx, "cURL easy handle removal failed: %s",
+ curl_multi_strerror(err));
+ return PGRES_POLLING_FAILED;
+ }
+
+ done = true;
+ }
+
+ /* Sanity check. */
+ if (!done)
+ {
+ actx_error(actx, "no result was retrieved for the finished handle");
+ return PGRES_POLLING_FAILED;
+ }
+
+ return PGRES_POLLING_OK;
+}
+
+/*
+ * Specific HTTP Request Handlers
+ *
+ * This is finally the beginning of the actual application logic. Generally
+ * speaking, a single request consists of a start_* and a finish_* step, with
+ * drive_request() pumping the machine in between.
+ */
+
+/*
+ * Queue an OpenID Provider Configuration Request:
+ *
+ * https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderConfigurationRequest
+ * https://www.rfc-editor.org/rfc/rfc8414#section-3.1
+ *
+ * This is done first to get the endpoint URIs we need to contact and to make
+ * sure the provider provides a device authorization flow. finish_discovery()
+ * will fill in actx->provider.
+ */
+static bool
+start_discovery(struct async_ctx *actx, const char *discovery_uri)
+{
+ CHECK_SETOPT(actx, CURLOPT_HTTPGET, 1L, return false);
+ CHECK_SETOPT(actx, CURLOPT_URL, discovery_uri, return false);
+
+ return start_request(actx);
+}
+
+static bool
+finish_discovery(struct async_ctx *actx)
+{
+ long response_code;
+
+ /*----
+ * Now check the response. OIDC Discovery 1.0 is pretty strict:
+ *
+ * A successful response MUST use the 200 OK HTTP status code and
+ * return a JSON object using the application/json content type that
+ * contains a set of Claims as its members that are a subset of the
+ * Metadata values defined in Section 3.
+ *
+ * Compared to standard HTTP semantics, this makes life easy -- we don't
+ * need to worry about redirections (which would call the Issuer host
+ * validation into question), or non-authoritative responses, or any other
+ * complications.
+ */
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ if (response_code != 200)
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ actx->errctx = "failed to parse OpenID discovery document";
+ if (!parse_provider(actx, &actx->provider))
+ return false; /* error message already set */
+
+ /*
+ * Fill in any defaults for OPTIONAL/RECOMMENDED fields we care about.
+ */
+ if (!actx->provider.grant_types_supported)
+ {
+ /*
+ * Per Section 3, the default is ["authorization_code", "implicit"].
+ */
+ struct curl_slist *temp = actx->provider.grant_types_supported;
+
+ temp = curl_slist_append(temp, "authorization_code");
+ if (temp)
+ {
+ temp = curl_slist_append(temp, "implicit");
+ }
+
+ if (!temp)
+ {
+ actx_error(actx, "out of memory");
+ return false;
+ }
+
+ actx->provider.grant_types_supported = temp;
+ }
+
+ return true;
+}
+
+#define OAUTH_GRANT_TYPE_DEVICE_CODE "urn:ietf:params:oauth:grant-type:device_code"
+
+/*
+ * Ensure that the provider supports the Device Authorization flow (i.e. it
+ * accepts the device_code grant type and provides an authorization endpoint).
+ */
+static bool
+check_for_device_flow(struct async_ctx *actx)
+{
+ const struct provider *provider = &actx->provider;
+ const struct curl_slist *grant;
+ bool device_grant_found = false;
+
+ Assert(provider->issuer); /* ensured by get_discovery_document() */
+
+ /*------
+ * First, sanity checks for discovery contents that are OPTIONAL in the
+ * spec but required for our flow:
+ * - the issuer must support the device_code grant
+ * - the issuer must have actually given us a
+ * device_authorization_endpoint
+ */
+
+ grant = provider->grant_types_supported;
+ while (grant)
+ {
+ if (strcmp(grant->data, OAUTH_GRANT_TYPE_DEVICE_CODE) == 0)
+ {
+ device_grant_found = true;
+ break;
+ }
+
+ grant = grant->next;
+ }
+
+ if (!device_grant_found)
+ {
+ actx_error(actx, "issuer \"%s\" does not support device code grants",
+ provider->issuer);
+ return false;
+ }
+
+ if (!provider->device_authorization_endpoint)
+ {
+ actx_error(actx,
+ "issuer \"%s\" does not provide a device authorization endpoint",
+ provider->issuer);
+ return false;
+ }
+
+ /* TODO: check that the endpoint uses HTTPS */
+
+ return true;
+}
+
+/*
+ * Queue a Device Authorization Request:
+ *
+ * https://www.rfc-editor.org/rfc/rfc8628#section-3.1
+ *
+ * This is the second step. We ask the provider to verify the end user out of
+ * band and authorize us to act on their behalf; it will give us the required
+ * nonces for us to later poll the request status, which we'll grab in
+ * finish_device_authz().
+ */
+static bool
+start_device_authz(struct async_ctx *actx, PGconn *conn)
+{
+ const char *device_authz_uri = actx->provider.device_authorization_endpoint;
+ PQExpBuffer work_buffer = &actx->work_data;
+
+ Assert(conn->oauth_client_id); /* ensured by get_auth_token() */
+ Assert(device_authz_uri); /* ensured by check_for_device_flow() */
+
+ /* Construct our request body. TODO: url-encode */
+ resetPQExpBuffer(work_buffer);
+ appendPQExpBuffer(work_buffer, "client_id=%s", conn->oauth_client_id);
+ if (conn->oauth_scope)
+ appendPQExpBuffer(work_buffer, "&scope=%s", conn->oauth_scope);
+ /* TODO check for broken buffer */
+
+ /* Make our request. */
+ CHECK_SETOPT(actx, CURLOPT_URL, device_authz_uri, return false);
+ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data, return false);
+
+ if (conn->oauth_client_secret)
+ {
+ /*----
+ * Use HTTP Basic auth to send the password. Per RFC 6749, Sec. 2.3.1,
+ *
+ * Including the client credentials in the request-body using the
+ * two parameters is NOT RECOMMENDED and SHOULD be limited to
+ * clients unable to directly utilize the HTTP Basic authentication
+ * scheme (or other password-based HTTP authentication schemes).
+ *
+ * TODO: should we omit client_id from the body in this case?
+ */
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC, return false);
+ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id, return false);
+ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret, return false);
+ }
+ else
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE, return false);
+
+ return start_request(actx);
+}
+
+static bool
+finish_device_authz(struct async_ctx *actx)
+{
+ long response_code;
+
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ /*
+ * The device authorization endpoint uses the same error response as the
+ * token endpoint, so we have to handle 400/401 here too.
+ */
+ if (response_code != 200
+ && response_code != 400
+ /* && response_code != 401 TODO */ )
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ if (response_code != 200)
+ {
+ struct token_error err = {0};
+
+ if (!parse_token_error(actx, &err))
+ {
+ free_token_error(&err);
+ return false;
+ }
+
+ if (err.error_description)
+ appendPQExpBuffer(&actx->errbuf, "%s ", err.error_description);
+
+ appendPQExpBuffer(&actx->errbuf, "(%s)", err.error);
+
+ free_token_error(&err);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ actx->errctx = "failed to parse device authorization";
+ if (!parse_device_authz(actx, &actx->authz))
+ return false; /* error message already set */
+
+ return true;
+}
+
+/*
+ * Queue an Access Token Request:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-4.1.3
+ *
+ * This is the final step. We continually poll the token endpoint to see if the
+ * user has authorized us yet. finish_token_request() will pull either the token
+ * or a (ideally temporary) error status from the provider.
+ */
+static bool
+start_token_request(struct async_ctx *actx, PGconn *conn)
+{
+ const char *token_uri = actx->provider.token_endpoint;
+ const char *device_code = actx->authz.device_code;
+ PQExpBuffer work_buffer = &actx->work_data;
+
+ Assert(conn->oauth_client_id); /* ensured by get_auth_token() */
+ Assert(token_uri); /* ensured by get_discovery_document() */
+ Assert(device_code); /* ensured by run_device_authz() */
+
+ /* Construct our request body. TODO: url-encode */
+ resetPQExpBuffer(work_buffer);
+ appendPQExpBuffer(work_buffer, "client_id=%s", conn->oauth_client_id);
+ appendPQExpBuffer(work_buffer, "&device_code=%s", device_code);
+ appendPQExpBuffer(work_buffer, "&grant_type=%s",
+ OAUTH_GRANT_TYPE_DEVICE_CODE);
+ /* TODO check for broken buffer */
+
+ /* Make our request. */
+ CHECK_SETOPT(actx, CURLOPT_URL, token_uri, return false);
+ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data, return false);
+
+ if (conn->oauth_client_secret)
+ {
+ /*----
+ * Use HTTP Basic auth to send the password. Per RFC 6749, Sec. 2.3.1,
+ *
+ * Including the client credentials in the request-body using the
+ * two parameters is NOT RECOMMENDED and SHOULD be limited to
+ * clients unable to directly utilize the HTTP Basic authentication
+ * scheme (or other password-based HTTP authentication schemes).
+ *
+ * TODO: should we omit client_id from the body in this case?
+ */
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC, return false);
+ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id, return false);
+ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret, return false);
+ }
+ else
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE, return false);
+
+ resetPQExpBuffer(work_buffer);
+ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data, return false);
+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, work_buffer, return false);
+
+ return start_request(actx);
+}
+
+static bool
+finish_token_request(struct async_ctx *actx, struct token *tok)
+{
+ long response_code;
+
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ /*
+ * Per RFC 6749, Section 5, a successful response uses 200 OK. An error
+ * response uses either 400 Bad Request or 401 Unauthorized.
+ *
+ * TODO: there are references online to 403 appearing in the wild...
+ */
+ if (response_code != 200
+ && response_code != 400
+ /* && response_code != 401 TODO */ )
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ if (response_code == 200)
+ {
+ actx->errctx = "failed to parse access token response";
+ if (!parse_access_token(actx, tok))
+ return false; /* error message already set */
+ }
+ else if (!parse_token_error(actx, &tok->err))
+ return false;
+
+ return true;
+}
+
+/*
+ * The top-level, nonblocking entry point for the cURL implementation. This will
+ * be called several times to pump the async engine.
+ *
+ * The architecture is based on PQconnectPoll(). The first half drives the
+ * connection state forward as necessary, returning if we're not ready to
+ * proceed to the next step yet. The second half performs the actual transition
+ * between states.
+ *
+ * You can trace the overall OAuth flow through the second half. It's linear
+ * until we get to the end, where we flip back and forth between
+ * OAUTH_STEP_TOKEN_REQUEST and OAUTH_STEP_WAIT_INTERVAL to regularly ping the
+ * provider.
+ */
+PostgresPollingStatusType
+pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
+{
+ fe_oauth_state *state = conn->sasl_state;
+ struct async_ctx *actx;
+
+ struct token tok = {0};
+
+ /*
+ * XXX This is not safe. cURL has stringent requirements for the thread
+ * context in which you call curl_global_init(), because it's going to try
+ * initializing a bunch of other libraries (OpenSSL, Winsock...). And we
+ * probably need to consider both the TLS backend libcurl is compiled
+ * against and what the user has asked us to do via PQinit[Open]SSL.
+ *
+ * Recent versions of libcurl have improved the thread-safety situation,
+ * but you apparently can't check at compile time whether the
+ * implementation is thread-safe, and there's a chicken-and-egg problem
+ * where you can't check the thread safety until you've initialized cURL,
+ * which you can't do before you've made sure it's thread-safe...
+ *
+ * We know we've already initialized Winsock by this point, so we should
+ * be able to safely skip that bit. But we have to tell cURL to initialize
+ * everything else, because other pieces of our client executable may
+ * already be using cURL for their own purposes. If we initialize libcurl
+ * first, with only a subset of its features, we could break those other
+ * clients nondeterministically, and that would probably be a nightmare to
+ * debug.
+ */
+ curl_global_init(CURL_GLOBAL_ALL
+ & ~CURL_GLOBAL_WIN32); /* we already initialized Winsock */
+
+ if (!state->async_ctx)
+ {
+ /*
+ * Create our asynchronous state, and hook it into the upper-level
+ * OAuth state immediately, so any failures below won't leak the
+ * context allocation.
+ */
+ actx = calloc(1, sizeof(*actx));
+ if (!actx)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return PGRES_POLLING_FAILED;
+ }
+
+ actx->mux = PGINVALID_SOCKET;
+#ifdef HAVE_SYS_EPOLL_H
+ actx->timerfd = -1;
+#endif
+
+ state->async_ctx = actx;
+ state->free_async_ctx = free_curl_async_ctx;
+
+ initPQExpBuffer(&actx->work_data);
+ initPQExpBuffer(&actx->errbuf);
+
+ if (!setup_multiplexer(actx))
+ goto error_return;
+
+ if (!setup_curl_handles(actx))
+ goto error_return;
+ }
+
+ actx = state->async_ctx;
+
+ /* By default, the multiplexer is the altsock. Reassign as desired. */
+ *altsock = actx->mux;
+
+ switch (actx->step)
+ {
+ case OAUTH_STEP_INIT:
+ break;
+
+ case OAUTH_STEP_DISCOVERY:
+ case OAUTH_STEP_DEVICE_AUTHORIZATION:
+ case OAUTH_STEP_TOKEN_REQUEST:
+ {
+ PostgresPollingStatusType status;
+
+ status = drive_request(actx);
+
+ if (status == PGRES_POLLING_FAILED)
+ goto error_return;
+ else if (status != PGRES_POLLING_OK)
+ {
+ /* not done yet */
+ free_token(&tok);
+ return status;
+ }
+ }
+
+ case OAUTH_STEP_WAIT_INTERVAL:
+ /* TODO check that the timer has expired */
+ break;
+ }
+
+ switch (actx->step)
+ {
+ case OAUTH_STEP_INIT:
+ actx->errctx = "failed to fetch OpenID discovery document";
+ if (!start_discovery(actx, conn->oauth_discovery_uri))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_DISCOVERY;
+ break;
+
+ case OAUTH_STEP_DISCOVERY:
+ if (!finish_discovery(actx))
+ goto error_return;
+
+ /* TODO: check issuer */
+
+ actx->errctx = "cannot run OAuth device authorization";
+ if (!check_for_device_flow(actx))
+ goto error_return;
+
+ actx->errctx = "failed to obtain device authorization";
+ if (!start_device_authz(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_DEVICE_AUTHORIZATION;
+ break;
+
+ case OAUTH_STEP_DEVICE_AUTHORIZATION:
+ if (!finish_device_authz(actx))
+ goto error_return;
+
+ actx->errctx = "failed to obtain access token";
+ if (!start_token_request(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_TOKEN_REQUEST;
+ break;
+
+ case OAUTH_STEP_TOKEN_REQUEST:
+ {
+ const struct token_error *err;
+#ifdef HAVE_SYS_EPOLL_H
+ struct itimerspec spec = {0};
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct kevent ev = {0};
+#endif
+
+ if (!finish_token_request(actx, &tok))
+ goto error_return;
+
+ if (!actx->user_prompted)
+ {
+ int res;
+ PQpromptOAuthDevice prompt = {
+ .verification_uri = actx->authz.verification_uri,
+ .user_code = actx->authz.user_code,
+ /* TODO: optional fields */
+ };
+
+ /*
+ * Now that we know the token endpoint isn't broken, give
+ * the user the login instructions.
+ */
+ res = PQauthDataHook(PQAUTHDATA_PROMPT_OAUTH_DEVICE, conn,
+ &prompt);
+
+ if (!res)
+ {
+ fprintf(stderr, "Visit %s and enter the code: %s",
+ prompt.verification_uri, prompt.user_code);
+ }
+ else if (res < 0)
+ {
+ actx_error(actx, "device prompt failed");
+ goto error_return;
+ }
+
+ actx->user_prompted = true;
+ }
+
+ if (tok.access_token)
+ {
+ /* Construct our Bearer token. */
+ resetPQExpBuffer(&actx->work_data);
+ appendPQExpBuffer(&actx->work_data, "Bearer %s",
+ tok.access_token);
+
+ if (PQExpBufferDataBroken(actx->work_data))
+ {
+ actx_error(actx, "out of memory");
+ goto error_return;
+ }
+
+ state->token = strdup(actx->work_data.data);
+ break;
+ }
+
+ /*
+ * authorization_pending and slow_down are the only acceptable
+ * errors; anything else and we bail.
+ */
+ err = &tok.err;
+ if (!err->error || (strcmp(err->error, "authorization_pending")
+ && strcmp(err->error, "slow_down")))
+ {
+ /* TODO handle !err->error */
+ if (err->error_description)
+ appendPQExpBuffer(&actx->errbuf, "%s ",
+ err->error_description);
+
+ appendPQExpBuffer(&actx->errbuf, "(%s)", err->error);
+
+ goto error_return;
+ }
+
+ /*
+ * A slow_down error requires us to permanently increase our
+ * retry interval by five seconds. RFC 8628, Sec. 3.5.
+ */
+ if (strcmp(err->error, "slow_down") == 0)
+ {
+ actx->authz.interval += 5; /* TODO check for overflow? */
+ }
+
+ /*
+ * Wait for the required interval before issuing the next
+ * request.
+ */
+ Assert(actx->authz.interval > 0);
+#ifdef HAVE_SYS_EPOLL_H
+ spec.it_value.tv_sec = actx->authz.interval;
+
+ if (timerfd_settime(actx->timerfd, 0 /* no flags */ , &spec, NULL) < 0)
+ {
+ actx_error(actx, "failed to set timerfd: %m");
+ goto error_return;
+ }
+
+ *altsock = actx->timerfd;
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ /* XXX: I guess this wants to be hidden in a routine */
+ EV_SET(&ev, 1, EVFILT_TIMER, EV_ADD, 0,
+ actx->authz.interval * 1000, 0);
+ if (kevent(actx->mux, &ev, 1, NULL, 0, NULL) < 0)
+ {
+ actx_error(actx, "failed to set kqueue timer: %m");
+ goto error_return;
+ }
+ /* XXX: why did we change the altsock in the epoll version? */
+#endif
+ actx->step = OAUTH_STEP_WAIT_INTERVAL;
+ break;
+ }
+
+ case OAUTH_STEP_WAIT_INTERVAL:
+ actx->errctx = "failed to obtain access token";
+ if (!start_token_request(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_TOKEN_REQUEST;
+ break;
+ }
+
+ free_token(&tok);
+
+ /* If we've stored a token, we're done. Otherwise come back later. */
+ return state->token ? PGRES_POLLING_OK : PGRES_POLLING_READING;
+
+error_return:
+
+ /*
+ * Assemble the three parts of our error: context, body, and detail. See
+ * also the documentation for struct async_ctx.
+ */
+ if (actx->errctx)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext(actx->errctx));
+ appendPQExpBufferStr(&conn->errorMessage, ": ");
+ }
+
+ if (PQExpBufferDataBroken(actx->errbuf))
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("out of memory"));
+ else
+ appendPQExpBufferStr(&conn->errorMessage, actx->errbuf.data);
+
+ if (actx->curl_err[0])
+ {
+ size_t len;
+
+ appendPQExpBuffer(&conn->errorMessage, " (%s)", actx->curl_err);
+
+ /* Sometimes libcurl adds a newline to the error buffer. :( */
+ len = conn->errorMessage.len;
+ if (len >= 2 && conn->errorMessage.data[len - 2] == '\n')
+ {
+ conn->errorMessage.data[len - 2] = ')';
+ conn->errorMessage.data[len - 1] = '\0';
+ conn->errorMessage.len--;
+ }
+ }
+
+ appendPQExpBufferStr(&conn->errorMessage, "\n");
+
+ free_token(&tok);
+ return PGRES_POLLING_FAILED;
+}
diff --git a/src/interfaces/libpq/fe-auth-oauth-iddawc.c b/src/interfaces/libpq/fe-auth-oauth-iddawc.c
new file mode 100644
index 0000000000..e78d4304d3
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth-iddawc.c
@@ -0,0 +1,319 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth-iddawc.c
+ * The libiddawc implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth-iddawc.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <iddawc.h>
+
+#include "fe-auth.h"
+#include "fe-auth-oauth.h"
+#include "libpq-int.h"
+
+#ifdef HAVE_I_LOAD_OPENID_CONFIG
+/* Older versions of iddawc used 'load' instead of 'get' for some APIs. */
+#define i_get_openid_config i_load_openid_config
+#endif
+
+static const char *
+iddawc_error_string(int errcode)
+{
+ switch (errcode)
+ {
+ case I_OK:
+ return "I_OK";
+
+ case I_ERROR:
+ return "I_ERROR";
+
+ case I_ERROR_PARAM:
+ return "I_ERROR_PARAM";
+
+ case I_ERROR_MEMORY:
+ return "I_ERROR_MEMORY";
+
+ case I_ERROR_UNAUTHORIZED:
+ return "I_ERROR_UNAUTHORIZED";
+
+ case I_ERROR_SERVER:
+ return "I_ERROR_SERVER";
+ }
+
+ return "<unknown>";
+}
+
+static void
+iddawc_error(PGconn *conn, int errcode, const char *msg)
+{
+ appendPQExpBufferStr(&conn->errorMessage, libpq_gettext(msg));
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext(" (iddawc error %s)\n"),
+ iddawc_error_string(errcode));
+}
+
+static void
+iddawc_request_error(PGconn *conn, struct _i_session *i, int err, const char *msg)
+{
+ const char *error_code;
+ const char *desc;
+
+ appendPQExpBuffer(&conn->errorMessage, "%s: ", libpq_gettext(msg));
+
+ error_code = i_get_str_parameter(i, I_OPT_ERROR);
+ if (!error_code)
+ {
+ /*
+ * The server didn't give us any useful information, so just print the
+ * error code.
+ */
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("(iddawc error %s)\n"),
+ iddawc_error_string(err));
+ return;
+ }
+
+ /* If the server gave a string description, print that too. */
+ desc = i_get_str_parameter(i, I_OPT_ERROR_DESCRIPTION);
+ if (desc)
+ appendPQExpBuffer(&conn->errorMessage, "%s ", desc);
+
+ appendPQExpBuffer(&conn->errorMessage, "(%s)\n", error_code);
+}
+
+/*
+ * Runs the device authorization flow using libiddawc. If successful, a malloc'd
+ * token string in "Bearer xxxx..." format, suitable for sending to an
+ * OAUTHBEARER server, is returned. NULL is returned on error.
+ */
+static char *
+run_iddawc_auth_flow(PGconn *conn, const char *discovery_uri)
+{
+ struct _i_session session;
+ PQExpBuffer token_buf = NULL;
+ int err;
+ int auth_method;
+ bool user_prompted = false;
+ const char *verification_uri;
+ const char *user_code;
+ const char *access_token;
+ const char *token_type;
+ char *token = NULL;
+
+ i_init_session(&session);
+
+ token_buf = createPQExpBuffer();
+ if (!token_buf)
+ goto cleanup;
+
+ err = i_set_str_parameter(&session, I_OPT_OPENID_CONFIG_ENDPOINT, discovery_uri);
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set OpenID config endpoint");
+ goto cleanup;
+ }
+
+ err = i_get_openid_config(&session);
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to fetch OpenID discovery document");
+ goto cleanup;
+ }
+
+ if (!i_get_str_parameter(&session, I_OPT_TOKEN_ENDPOINT))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer has no token endpoint"));
+ goto cleanup;
+ }
+
+ if (!i_get_str_parameter(&session, I_OPT_DEVICE_AUTHORIZATION_ENDPOINT))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer does not support device authorization"));
+ goto cleanup;
+ }
+
+ err = i_set_response_type(&session, I_RESPONSE_TYPE_DEVICE_CODE);
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set device code response type");
+ goto cleanup;
+ }
+
+ auth_method = I_TOKEN_AUTH_METHOD_NONE;
+ if (conn->oauth_client_secret && *conn->oauth_client_secret)
+ auth_method = I_TOKEN_AUTH_METHOD_SECRET_BASIC;
+
+ err = i_set_parameter_list(&session,
+ I_OPT_CLIENT_ID, conn->oauth_client_id,
+ I_OPT_CLIENT_SECRET, conn->oauth_client_secret,
+ I_OPT_TOKEN_METHOD, auth_method,
+ I_OPT_SCOPE, conn->oauth_scope,
+ I_OPT_NONE
+ );
+ if (err)
+ {
+ iddawc_error(conn, err, "failed to set client identifier");
+ goto cleanup;
+ }
+
+ err = i_run_device_auth_request(&session);
+ if (err)
+ {
+ iddawc_request_error(conn, &session, err,
+ "failed to obtain device authorization");
+ goto cleanup;
+ }
+
+ verification_uri = i_get_str_parameter(&session, I_OPT_DEVICE_AUTH_VERIFICATION_URI);
+ if (!verification_uri)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer did not provide a verification URI"));
+ goto cleanup;
+ }
+
+ user_code = i_get_str_parameter(&session, I_OPT_DEVICE_AUTH_USER_CODE);
+ if (!user_code)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer did not provide a user code"));
+ goto cleanup;
+ }
+
+ /*
+ * Poll the token endpoint until either the user logs in and authorizes
+ * the use of a token, or a hard failure occurs. We perform one ping
+ * _before_ prompting the user, so that we don't make them do the work of
+ * logging in only to find that the token endpoint is completely
+ * unreachable.
+ */
+ err = i_run_token_request(&session);
+ while (err)
+ {
+ const char *error_code;
+ uint interval;
+
+ error_code = i_get_str_parameter(&session, I_OPT_ERROR);
+
+ /*
+ * authorization_pending and slow_down are the only acceptable errors;
+ * anything else and we bail.
+ */
+ if (!error_code || (strcmp(error_code, "authorization_pending")
+ && strcmp(error_code, "slow_down")))
+ {
+ iddawc_request_error(conn, &session, err,
+ "failed to obtain access token");
+ goto cleanup;
+ }
+
+ if (!user_prompted)
+ {
+ int res;
+ PQpromptOAuthDevice prompt = {
+ .verification_uri = verification_uri,
+ .user_code = user_code,
+ /* TODO: optional fields */
+ };
+
+ /*
+ * Now that we know the token endpoint isn't broken, give the user
+ * the login instructions.
+ */
+ res = PQauthDataHook(PQAUTHDATA_PROMPT_OAUTH_DEVICE, conn,
+ &prompt);
+
+ if (!res)
+ {
+ fprintf(stderr, "Visit %s and enter the code: %s",
+ prompt.verification_uri, prompt.user_code);
+ }
+ else if (res < 0)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("device prompt failed\n"));
+ goto cleanup;
+ }
+
+ user_prompted = true;
+ }
+
+ /*---
+ * We are required to wait between polls; the server tells us how
+ * long.
+ * TODO: if interval's not set, we need to default to five seconds
+ * TODO: sanity check the interval
+ */
+ interval = i_get_int_parameter(&session, I_OPT_DEVICE_AUTH_INTERVAL);
+
+ /*
+ * A slow_down error requires us to permanently increase our retry
+ * interval by five seconds. RFC 8628, Sec. 3.5.
+ */
+ if (!strcmp(error_code, "slow_down"))
+ {
+ interval += 5;
+ i_set_int_parameter(&session, I_OPT_DEVICE_AUTH_INTERVAL, interval);
+ }
+
+ sleep(interval);
+
+ /*
+ * XXX Reset the error code before every call, because iddawc won't do
+ * that for us. This matters if the server first sends a "pending"
+ * error code, then later hard-fails without sending an error code to
+ * overwrite the first one.
+ *
+ * That we have to do this at all seems like a bug in iddawc.
+ */
+ i_set_str_parameter(&session, I_OPT_ERROR, NULL);
+
+ err = i_run_token_request(&session);
+ }
+
+ access_token = i_get_str_parameter(&session, I_OPT_ACCESS_TOKEN);
+ token_type = i_get_str_parameter(&session, I_OPT_TOKEN_TYPE);
+
+ if (!access_token || !token_type || strcasecmp(token_type, "Bearer"))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("issuer did not provide a bearer token"));
+ goto cleanup;
+ }
+
+ appendPQExpBufferStr(token_buf, "Bearer ");
+ appendPQExpBufferStr(token_buf, access_token);
+
+ if (PQExpBufferBroken(token_buf))
+ goto cleanup;
+
+ token = strdup(token_buf->data);
+
+cleanup:
+ if (token_buf)
+ destroyPQExpBuffer(token_buf);
+ i_clean_session(&session);
+
+ return token;
+}
+
+PostgresPollingStatusType
+pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
+{
+ fe_oauth_state *state = conn->sasl_state;
+
+ /* TODO: actually make this asynchronous */
+ state->token = run_iddawc_auth_flow(conn, conn->oauth_discovery_uri);
+ return state->token ? PGRES_POLLING_OK : PGRES_POLLING_FAILED;
+}
diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
new file mode 100644
index 0000000000..66ee8ff076
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth.c
@@ -0,0 +1,659 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth.c
+ * The front-end (client) implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "common/base64.h"
+#include "common/hmac.h"
+#include "common/jsonapi.h"
+#include "common/oauth-common.h"
+#include "fe-auth.h"
+#include "fe-auth-oauth.h"
+#include "mb/pg_wchar.h"
+
+/* The exported OAuth callback mechanism. */
+static void *oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism);
+static SASLStatus oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen);
+static bool oauth_channel_bound(void *opaq);
+static void oauth_free(void *opaq);
+
+const pg_fe_sasl_mech pg_oauth_mech = {
+ oauth_init,
+ oauth_exchange,
+ oauth_channel_bound,
+ oauth_free,
+};
+
+static void *
+oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism)
+{
+ fe_oauth_state *state;
+
+ /*
+ * We only support one SASL mechanism here; anything else is programmer
+ * error.
+ */
+ Assert(sasl_mechanism != NULL);
+ Assert(strcmp(sasl_mechanism, OAUTHBEARER_NAME) == 0);
+
+ state = calloc(1, sizeof(*state));
+ if (!state)
+ return NULL;
+
+ state->state = FE_OAUTH_INIT;
+ state->conn = conn;
+
+ return state;
+}
+
+#define kvsep "\x01"
+
+static char *
+client_initial_response(PGconn *conn, const char *token)
+{
+ static const char *const resp_format = "n,," kvsep "auth=%s" kvsep kvsep;
+
+ PQExpBufferData buf;
+ char *response = NULL;
+
+ if (!token)
+ {
+ /*
+ * Either programmer error, or something went badly wrong during the
+ * asynchronous fetch.
+ *
+ * TODO: users shouldn't see this; what action should they take if
+ * they do?
+ */
+ libpq_append_conn_error(conn, "no OAuth token was set for the connection");
+ return NULL;
+ }
+
+ initPQExpBuffer(&buf);
+ appendPQExpBuffer(&buf, resp_format, token);
+
+ if (!PQExpBufferDataBroken(buf))
+ response = strdup(buf.data);
+
+ termPQExpBuffer(&buf);
+ return response;
+}
+
+#define ERROR_STATUS_FIELD "status"
+#define ERROR_SCOPE_FIELD "scope"
+#define ERROR_OPENID_CONFIGURATION_FIELD "openid-configuration"
+
+struct json_ctx
+{
+ char *errmsg; /* any non-NULL value stops all processing */
+ PQExpBufferData errbuf; /* backing memory for errmsg */
+ int nested; /* nesting level (zero is the top) */
+
+ const char *target_field_name; /* points to a static allocation */
+ char **target_field; /* see below */
+
+ /* target_field, if set, points to one of the following: */
+ char *status;
+ char *scope;
+ char *discovery_uri;
+};
+
+#define oauth_json_has_error(ctx) \
+ (PQExpBufferDataBroken((ctx)->errbuf) || (ctx)->errmsg)
+
+#define oauth_json_set_error(ctx, ...) \
+ do { \
+ appendPQExpBuffer(&(ctx)->errbuf, __VA_ARGS__); \
+ (ctx)->errmsg = (ctx)->errbuf.data; \
+ } while (0)
+
+static JsonParseErrorType
+oauth_json_object_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ ++ctx->nested;
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_end(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_field_start(void *state, char *name, bool isnull)
+{
+ struct json_ctx *ctx = state;
+
+ if (ctx->nested == 1)
+ {
+ if (strcmp(name, ERROR_STATUS_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_STATUS_FIELD;
+ ctx->target_field = &ctx->status;
+ }
+ else if (strcmp(name, ERROR_SCOPE_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_SCOPE_FIELD;
+ ctx->target_field = &ctx->scope;
+ }
+ else if (strcmp(name, ERROR_OPENID_CONFIGURATION_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_OPENID_CONFIGURATION_FIELD;
+ ctx->target_field = &ctx->discovery_uri;
+ }
+ }
+
+ free(name);
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_scalar(void *state, char *token, JsonTokenType type)
+{
+ struct json_ctx *ctx = state;
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ if (type == JSON_TOKEN_STRING)
+ {
+ *ctx->target_field = token;
+
+ ctx->target_field = NULL;
+ ctx->target_field_name = NULL;
+
+ return JSON_SUCCESS; /* don't free the token we're using */
+ }
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ free(token);
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static bool
+handle_oauth_sasl_error(PGconn *conn, char *msg, int msglen)
+{
+ JsonLexContext lex = {0};
+ JsonSemAction sem = {0};
+ JsonParseErrorType err;
+ struct json_ctx ctx = {0};
+ char *errmsg = NULL;
+
+ /* Sanity check. */
+ if (strlen(msg) != msglen)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server's error message contained an embedded NULL, and was discarded"));
+ return false;
+ }
+
+ makeJsonLexContextCstringLen(&lex, msg, msglen, PG_UTF8, true);
+
+ initPQExpBuffer(&ctx.errbuf);
+ sem.semstate = &ctx;
+
+ sem.object_start = oauth_json_object_start;
+ sem.object_end = oauth_json_object_end;
+ sem.object_field_start = oauth_json_object_field_start;
+ sem.array_start = oauth_json_array_start;
+ sem.scalar = oauth_json_scalar;
+
+ err = pg_parse_json(&lex, &sem);
+
+ if (err == JSON_SEM_ACTION_FAILED)
+ {
+ if (PQExpBufferDataBroken(ctx.errbuf))
+ errmsg = libpq_gettext("out of memory");
+ else if (ctx.errmsg)
+ errmsg = ctx.errmsg;
+ else
+ {
+ /*
+ * Developer error: one of the action callbacks didn't call
+ * oauth_json_set_error() before erroring out.
+ */
+ Assert(oauth_json_has_error(&ctx));
+ errmsg = "<unexpected empty error>";
+ }
+ }
+ else if (err != JSON_SUCCESS)
+ errmsg = json_errdetail(err, &lex);
+
+ if (errmsg)
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("failed to parse server's error response: %s"),
+ errmsg);
+
+ /* Don't need the error buffer or the JSON lexer anymore. */
+ termPQExpBuffer(&ctx.errbuf);
+ freeJsonLexContext(&lex);
+
+ if (errmsg)
+ return false;
+
+ /* TODO: what if these override what the user already specified? */
+ if (ctx.discovery_uri)
+ {
+ if (conn->oauth_discovery_uri)
+ free(conn->oauth_discovery_uri);
+
+ conn->oauth_discovery_uri = ctx.discovery_uri;
+ }
+
+ if (ctx.scope)
+ {
+ if (conn->oauth_scope)
+ free(conn->oauth_scope);
+
+ conn->oauth_scope = ctx.scope;
+ }
+ /* TODO: missing error scope should clear any existing connection scope */
+
+ if (!ctx.status)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("server sent error response without a status"));
+ return false;
+ }
+
+ if (strcmp(ctx.status, "invalid_token") == 0)
+ {
+ /*
+ * invalid_token is the only error code we'll automatically retry for,
+ * but only if we have enough information to do so.
+ */
+ if (conn->oauth_discovery_uri)
+ conn->oauth_want_retry = true;
+ }
+ /* TODO: include status in hard failure message */
+
+ return true;
+}
+
+static void
+free_request(PGconn *conn, void *vreq)
+{
+ PQoauthBearerRequest *request = vreq;
+
+ if (request->cleanup)
+ request->cleanup(conn, request);
+
+ free(request);
+}
+
+static PostgresPollingStatusType
+run_user_oauth_flow(PGconn *conn, pgsocket *altsock)
+{
+ fe_oauth_state *state = conn->sasl_state;
+ PQoauthBearerRequest *request = state->async_ctx;
+ PostgresPollingStatusType status;
+
+ if (!request->async)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow provided neither a token nor an async callback");
+ return PGRES_POLLING_FAILED;
+ }
+
+ status = request->async(conn, request, altsock);
+ if (status == PGRES_POLLING_FAILED)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow failed");
+ return status;
+ }
+ else if (status == PGRES_POLLING_OK)
+ {
+ /*
+ * We already have a token, so copy it into the state. (We can't hold
+ * onto the original string, since it may not be safe for us to free()
+ * it.)
+ */
+ PQExpBufferData token;
+
+ if (!request->token)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow did not provide a token");
+ return PGRES_POLLING_FAILED;
+ }
+
+ initPQExpBuffer(&token);
+ appendPQExpBuffer(&token, "Bearer %s", request->token);
+
+ if (PQExpBufferDataBroken(token))
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return PGRES_POLLING_FAILED;
+ }
+
+ state->token = token.data;
+ return PGRES_POLLING_OK;
+ }
+
+ /* TODO: what if no altsock was set? */
+ return status;
+}
+
+static bool
+setup_token_request(PGconn *conn, fe_oauth_state *state)
+{
+ int res;
+ PQoauthBearerRequest request = {
+ .openid_configuration = conn->oauth_discovery_uri,
+ .scope = conn->oauth_scope,
+ };
+
+ Assert(request.openid_configuration);
+
+ /* The client may have overridden the OAuth flow. */
+ res = PQauthDataHook(PQAUTHDATA_OAUTH_BEARER_TOKEN, conn, &request);
+ if (res > 0)
+ {
+ PQoauthBearerRequest *request_copy;
+
+ if (request.token)
+ {
+ /*
+ * We already have a token, so copy it into the state. (We can't
+ * hold onto the original string, since it may not be safe for us
+ * to free() it.)
+ */
+ PQExpBufferData token;
+
+ initPQExpBuffer(&token);
+ appendPQExpBuffer(&token, "Bearer %s", request.token);
+
+ if (PQExpBufferDataBroken(token))
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ goto fail;
+ }
+
+ state->token = token.data;
+
+ /* short-circuit */
+ if (request.cleanup)
+ request.cleanup(conn, &request);
+ return true;
+ }
+
+ request_copy = malloc(sizeof(*request_copy));
+ if (!request_copy)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ goto fail;
+ }
+
+ memcpy(request_copy, &request, sizeof(request));
+
+ conn->async_auth = run_user_oauth_flow;
+ state->async_ctx = request_copy;
+ state->free_async_ctx = free_request;
+ }
+ else if (res < 0)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow failed");
+ goto fail;
+ }
+ else
+ {
+ /* Use our built-in OAuth flow. */
+ conn->async_auth = pg_fe_run_oauth_flow;
+ }
+
+ return true;
+
+fail:
+ if (request.cleanup)
+ request.cleanup(conn, &request);
+ return false;
+}
+
+static bool
+derive_discovery_uri(PGconn *conn)
+{
+ PQExpBufferData discovery_buf;
+
+ if (conn->oauth_discovery_uri || !conn->oauth_issuer)
+ {
+ /*
+ * Either we already have one, or we aren't able to derive one
+ * ourselves. The latter case is not an error condition; we'll just
+ * ask the server to provide one for us.
+ */
+ return true;
+ }
+
+ initPQExpBuffer(&discovery_buf);
+
+ Assert(!conn->oauth_discovery_uri);
+ Assert(conn->oauth_issuer);
+
+ /*
+ * If we don't yet have a discovery URI, but the user gave us an explicit
+ * issuer, use the .well-known discovery URI for that issuer.
+ */
+ appendPQExpBufferStr(&discovery_buf, conn->oauth_issuer);
+ appendPQExpBufferStr(&discovery_buf, "/.well-known/openid-configuration");
+
+ if (PQExpBufferDataBroken(discovery_buf))
+ goto cleanup;
+
+ conn->oauth_discovery_uri = strdup(discovery_buf.data);
+
+cleanup:
+ termPQExpBuffer(&discovery_buf);
+ return (conn->oauth_discovery_uri != NULL);
+}
+
+static SASLStatus
+oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen)
+{
+ fe_oauth_state *state = opaq;
+ PGconn *conn = state->conn;
+
+ *output = NULL;
+ *outputlen = 0;
+
+ switch (state->state)
+ {
+ case FE_OAUTH_INIT:
+ Assert(inputlen == -1);
+
+ if (!derive_discovery_uri(conn))
+ return SASL_FAILED;
+
+ if (conn->oauth_discovery_uri)
+ {
+ if (!conn->oauth_client_id)
+ {
+ /* We can't talk to a server without a client identifier. */
+ libpq_append_conn_error(conn, "no oauth_client_id is set for the connection");
+ return SASL_FAILED;
+ }
+
+ /*
+ * Decide whether we're using a user-provided OAuth flow, or
+ * the one we have built in.
+ */
+ if (!setup_token_request(conn, state))
+ return SASL_FAILED;
+
+ if (state->token)
+ {
+ /*
+ * A really smart user implementation may have already
+ * given us the token (e.g. if there was an unexpired copy
+ * already cached). In that case, we can just fall
+ * through.
+ */
+ }
+ else
+ {
+ /*
+ * Otherwise, we have to hand the connection over to our
+ * OAuth implementation. This involves a number of HTTP
+ * connections and timed waits, so we escape the
+ * synchronous auth processing and tell PQconnectPoll to
+ * transfer control to our async implementation.
+ */
+ Assert(conn->async_auth); /* should have been set
+ * already */
+ state->state = FE_OAUTH_REQUESTING_TOKEN;
+ return SASL_ASYNC;
+ }
+ }
+ else
+ {
+ /*
+ * If we don't have a discovery URI to be able to request a
+ * token, we ask the server for one explicitly with an empty
+ * token. This doesn't require any asynchronous work.
+ */
+ state->token = strdup("");
+ if (!state->token)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return SASL_FAILED;
+ }
+ }
+
+ /* fall through */
+
+ case FE_OAUTH_REQUESTING_TOKEN:
+ /* We should still be in the initial response phase. */
+ Assert(inputlen == -1);
+
+ *output = client_initial_response(conn, state->token);
+ if (!*output)
+ return SASL_FAILED;
+
+ *outputlen = strlen(*output);
+ state->state = FE_OAUTH_BEARER_SENT;
+
+ return SASL_CONTINUE;
+
+ case FE_OAUTH_BEARER_SENT:
+ if (final)
+ {
+ /* TODO: ensure there is no message content here. */
+ return SASL_COMPLETE;
+ }
+
+ /*
+ * Error message sent by the server.
+ */
+ if (!handle_oauth_sasl_error(conn, input, inputlen))
+ return SASL_FAILED;
+
+ /*
+ * Respond with the required dummy message (RFC 7628, sec. 3.2.3).
+ */
+ *output = strdup(kvsep);
+ *outputlen = strlen(*output); /* == 1 */
+
+ state->state = FE_OAUTH_SERVER_ERROR;
+ return SASL_CONTINUE;
+
+ case FE_OAUTH_SERVER_ERROR:
+
+ /*
+ * After an error, the server should send an error response to
+ * fail the SASL handshake, which is handled in higher layers.
+ *
+ * If we get here, the server either sent *another* challenge
+ * which isn't defined in the RFC, or completed the handshake
+ * successfully after telling us it was going to fail. Neither is
+ * acceptable.
+ */
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server sent additional OAuth data after error\n"));
+ return SASL_FAILED;
+
+ default:
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("invalid OAuth exchange state\n"));
+ break;
+ }
+
+ return SASL_FAILED;
+}
+
+static bool
+oauth_channel_bound(void *opaq)
+{
+ /* This mechanism does not support channel binding. */
+ return false;
+}
+
+static void
+oauth_free(void *opaq)
+{
+ fe_oauth_state *state = opaq;
+
+ free(state->token);
+ if (state->async_ctx)
+ state->free_async_ctx(state->conn, state->async_ctx);
+
+ free(state);
+}
diff --git a/src/interfaces/libpq/fe-auth-oauth.h b/src/interfaces/libpq/fe-auth-oauth.h
new file mode 100644
index 0000000000..8d4ea45aa8
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth.h
@@ -0,0 +1,42 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth.h
+ *
+ * Definitions for OAuth authentication implementations
+ *
+ * Portions Copyright (c) 2023, PostgreSQL Global Development Group
+ *
+ * src/interfaces/libpq/fe-auth-oauth.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FE_AUTH_OAUTH_H
+#define FE_AUTH_OAUTH_H
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+
+
+typedef enum
+{
+ FE_OAUTH_INIT,
+ FE_OAUTH_REQUESTING_TOKEN,
+ FE_OAUTH_BEARER_SENT,
+ FE_OAUTH_SERVER_ERROR,
+} fe_oauth_state_enum;
+
+typedef struct
+{
+ fe_oauth_state_enum state;
+
+ PGconn *conn;
+ char *token;
+
+ void *async_ctx;
+ void (*free_async_ctx) (PGconn *conn, void *ctx);
+} fe_oauth_state;
+
+extern PostgresPollingStatusType pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock);
+
+#endif /* FE_AUTH_OAUTH_H */
diff --git a/src/interfaces/libpq/fe-auth-sasl.h b/src/interfaces/libpq/fe-auth-sasl.h
index 4eecf53a15..83bdab1f40 100644
--- a/src/interfaces/libpq/fe-auth-sasl.h
+++ b/src/interfaces/libpq/fe-auth-sasl.h
@@ -30,6 +30,7 @@ typedef enum
SASL_COMPLETE = 0,
SASL_FAILED,
SASL_CONTINUE,
+ SASL_ASYNC,
} SASLStatus;
/*
@@ -77,6 +78,8 @@ typedef struct pg_fe_sasl_mech
*
* state: The opaque mechanism state returned by init()
*
+ * final: true if the server has sent a final exchange outcome
+ *
* input: The challenge data sent by the server, or NULL when
* generating a client-first initial response (that is, when
* the server expects the client to send a message to start
@@ -101,12 +104,17 @@ typedef struct pg_fe_sasl_mech
*
* SASL_CONTINUE: The output buffer is filled with a client response.
* Additional server challenge is expected
+ * SASL_ASYNC: Some asynchronous processing external to the
+ * connection needs to be done before a response can be
+ * generated. The mechanism is responsible for setting up
+ * conn->async_auth appropriately before returning.
* SASL_COMPLETE: The SASL exchange has completed successfully.
* SASL_FAILED: The exchange has failed and the connection should be
* dropped.
*--------
*/
- SASLStatus (*exchange) (void *state, char *input, int inputlen,
+ SASLStatus (*exchange) (void *state, bool final,
+ char *input, int inputlen,
char **output, int *outputlen);
/*--------
diff --git a/src/interfaces/libpq/fe-auth-scram.c b/src/interfaces/libpq/fe-auth-scram.c
index 0bb820e0d9..da168eb2f5 100644
--- a/src/interfaces/libpq/fe-auth-scram.c
+++ b/src/interfaces/libpq/fe-auth-scram.c
@@ -24,7 +24,8 @@
/* The exported SCRAM callback mechanism. */
static void *scram_init(PGconn *conn, const char *password,
const char *sasl_mechanism);
-static SASLStatus scram_exchange(void *opaq, char *input, int inputlen,
+static SASLStatus scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen);
static bool scram_channel_bound(void *opaq);
static void scram_free(void *opaq);
@@ -202,7 +203,8 @@ scram_free(void *opaq)
* Exchange a SCRAM message with backend.
*/
static SASLStatus
-scram_exchange(void *opaq, char *input, int inputlen,
+scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen)
{
fe_scram_state *state = (fe_scram_state *) opaq;
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index 81ec08485d..9cd5c8cfb1 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -39,6 +39,7 @@
#endif
#include "common/md5.h"
+#include "common/oauth-common.h"
#include "common/scram-common.h"
#include "fe-auth.h"
#include "fe-auth-sasl.h"
@@ -419,7 +420,7 @@ pg_SSPI_startup(PGconn *conn, int use_negotiate, int payloadlen)
* Initialize SASL authentication exchange.
*/
static int
-pg_SASL_init(PGconn *conn, int payloadlen)
+pg_SASL_init(PGconn *conn, int payloadlen, bool *async)
{
char *initialresponse = NULL;
int initialresponselen;
@@ -437,7 +438,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
goto error;
}
- if (conn->sasl_state)
+ if (conn->sasl_state && !conn->async_auth)
{
libpq_append_conn_error(conn, "duplicate SASL authentication request");
goto error;
@@ -524,6 +525,15 @@ pg_SASL_init(PGconn *conn, int payloadlen)
conn->sasl = &pg_scram_mech;
conn->password_needed = true;
}
+#ifdef USE_OAUTH
+ else if (strcmp(mechanism_buf.data, OAUTHBEARER_NAME) == 0 &&
+ !selected_mechanism)
+ {
+ selected_mechanism = OAUTHBEARER_NAME;
+ conn->sasl = &pg_oauth_mech;
+ conn->password_needed = false;
+ }
+#endif
}
if (!selected_mechanism)
@@ -563,26 +573,48 @@ pg_SASL_init(PGconn *conn, int payloadlen)
Assert(conn->sasl);
- /*
- * Initialize the SASL state information with all the information gathered
- * during the initial exchange.
- *
- * Note: Only tls-unique is supported for the moment.
- */
- conn->sasl_state = conn->sasl->init(conn,
- password,
- selected_mechanism);
if (!conn->sasl_state)
- goto oom_error;
+ {
+ /*
+ * Initialize the SASL state information with all the information
+ * gathered during the initial exchange.
+ *
+ * Note: Only tls-unique is supported for the moment.
+ */
+ conn->sasl_state = conn->sasl->init(conn,
+ password,
+ selected_mechanism);
+ if (!conn->sasl_state)
+ goto oom_error;
+ }
+ else
+ {
+ /*
+ * This is only possible if we're returning from an async loop.
+ * Disconnect it now.
+ */
+ Assert(conn->async_auth);
+ conn->async_auth = NULL;
+ }
/* Get the mechanism-specific Initial Client Response, if any */
- status = conn->sasl->exchange(conn->sasl_state,
+ status = conn->sasl->exchange(conn->sasl_state, false,
NULL, -1,
&initialresponse, &initialresponselen);
if (status == SASL_FAILED)
goto error;
+ if (status == SASL_ASYNC)
+ {
+ /*
+ * The mechanism should have set up the necessary callbacks; all we
+ * need to do is signal the caller.
+ */
+ *async = true;
+ return STATUS_OK;
+ }
+
/*
* Build a SASLInitialResponse message, and send it.
*/
@@ -625,7 +657,7 @@ oom_error:
* the protocol.
*/
static int
-pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
+pg_SASL_continue(PGconn *conn, int payloadlen, bool final, bool *async)
{
char *output;
int outputlen;
@@ -650,11 +682,21 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
/* For safety and convenience, ensure the buffer is NULL-terminated. */
challenge[payloadlen] = '\0';
- status = conn->sasl->exchange(conn->sasl_state,
+ status = conn->sasl->exchange(conn->sasl_state, final,
challenge, payloadlen,
&output, &outputlen);
free(challenge); /* don't need the input anymore */
+ if (status == SASL_ASYNC)
+ {
+ /*
+ * The mechanism should have set up the necessary callbacks; all we
+ * need to do is signal the caller.
+ */
+ *async = true;
+ return STATUS_OK;
+ }
+
if (final && status == SASL_CONTINUE)
{
if (outputlen != 0)
@@ -955,12 +997,18 @@ check_expected_areq(AuthRequest areq, PGconn *conn)
* it. We are responsible for reading any remaining extra data, specific
* to the authentication method. 'payloadlen' is the remaining length in
* the message.
+ *
+ * If *async is set to true on return, the client doesn't yet have enough
+ * information to respond, and the caller must temporarily switch to
+ * conn->async_auth() to continue driving the exchange.
*/
int
-pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
+pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn, bool *async)
{
int oldmsglen;
+ *async = false;
+
if (!check_expected_areq(areq, conn))
return STATUS_ERROR;
@@ -1118,7 +1166,7 @@ pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
* The request contains the name (as assigned by IANA) of the
* authentication mechanism.
*/
- if (pg_SASL_init(conn, payloadlen) != STATUS_OK)
+ if (pg_SASL_init(conn, payloadlen, async) != STATUS_OK)
{
/* pg_SASL_init already set the error message */
return STATUS_ERROR;
@@ -1135,7 +1183,8 @@ pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
}
oldmsglen = conn->errorMessage.len;
if (pg_SASL_continue(conn, payloadlen,
- (areq == AUTH_REQ_SASL_FIN)) != STATUS_OK)
+ (areq == AUTH_REQ_SASL_FIN),
+ async) != STATUS_OK)
{
/* Use this message if pg_SASL_continue didn't supply one */
if (conn->errorMessage.len == oldmsglen)
@@ -1451,3 +1500,23 @@ PQchangePassword(PGconn *conn, const char *user, const char *passwd)
}
}
}
+
+PQauthDataHook_type PQauthDataHook = PQdefaultAuthDataHook;
+
+PQauthDataHook_type
+PQgetAuthDataHook(void)
+{
+ return PQauthDataHook;
+}
+
+void
+PQsetAuthDataHook(PQauthDataHook_type hook)
+{
+ PQauthDataHook = hook ? hook : PQdefaultAuthDataHook;
+}
+
+int
+PQdefaultAuthDataHook(PGAuthData type, PGconn *conn, void *data)
+{
+ return 0; /* handle nothing */
+}
diff --git a/src/interfaces/libpq/fe-auth.h b/src/interfaces/libpq/fe-auth.h
index a18c508688..3e2bc1333f 100644
--- a/src/interfaces/libpq/fe-auth.h
+++ b/src/interfaces/libpq/fe-auth.h
@@ -18,8 +18,12 @@
#include "libpq-int.h"
+extern PQauthDataHook_type PQauthDataHook;
+
+
/* Prototypes for functions in fe-auth.c */
-extern int pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn);
+extern int pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn,
+ bool *async);
extern char *pg_fe_getusername(uid_t user_id, PQExpBuffer errorMessage);
extern char *pg_fe_getauthname(PQExpBuffer errorMessage);
@@ -29,4 +33,7 @@ extern char *pg_fe_scram_build_secret(const char *password,
int iterations,
const char **errstr);
+/* Mechanisms in fe-auth-oauth.c */
+extern const pg_fe_sasl_mech pg_oauth_mech;
+
#endif /* FE_AUTH_H */
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index d4e10a0c4f..15ceb73d01 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -359,6 +359,23 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"Load-Balance-Hosts", "", 8, /* sizeof("disable") = 8 */
offsetof(struct pg_conn, load_balance_hosts)},
+ /* OAuth v2 */
+ {"oauth_issuer", NULL, NULL, NULL,
+ "OAuth-Issuer", "", 40,
+ offsetof(struct pg_conn, oauth_issuer)},
+
+ {"oauth_client_id", NULL, NULL, NULL,
+ "OAuth-Client-ID", "", 40,
+ offsetof(struct pg_conn, oauth_client_id)},
+
+ {"oauth_client_secret", NULL, NULL, NULL,
+ "OAuth-Client-Secret", "", 40,
+ offsetof(struct pg_conn, oauth_client_secret)},
+
+ {"oauth_scope", NULL, NULL, NULL,
+ "OAuth-Scope", "", 15,
+ offsetof(struct pg_conn, oauth_scope)},
+
/* Terminating entry --- MUST BE LAST */
{NULL, NULL, NULL, NULL,
NULL, NULL, 0}
@@ -618,6 +635,7 @@ pqDropServerData(PGconn *conn)
conn->write_err_msg = NULL;
conn->be_pid = 0;
conn->be_key = 0;
+ /* conn->oauth_want_retry = false; TODO */
}
@@ -2536,6 +2554,7 @@ PQconnectPoll(PGconn *conn)
case CONNECTION_NEEDED:
case CONNECTION_GSS_STARTUP:
case CONNECTION_CHECK_TARGET:
+ case CONNECTION_AUTHENTICATING:
break;
default:
@@ -3517,6 +3536,7 @@ keep_going: /* We will come back to here until there is
int avail;
AuthRequest areq;
int res;
+ bool async;
/*
* Scan the message from current point (note that if we find
@@ -3672,6 +3692,16 @@ keep_going: /* We will come back to here until there is
/* Check to see if we should mention pgpassfile */
pgpassfileWarning(conn);
+#ifdef USE_OAUTH
+ if (conn->sasl == &pg_oauth_mech
+ && conn->oauth_want_retry)
+ {
+ /* TODO: only allow retry once */
+ need_new_connection = true;
+ goto keep_going;
+ }
+#endif
+
#ifdef ENABLE_GSS
/*
@@ -3753,7 +3783,17 @@ keep_going: /* We will come back to here until there is
* Note that conn->pghost must be non-NULL if we are going to
* avoid the Kerberos code doing a hostname look-up.
*/
- res = pg_fe_sendauth(areq, msgLength, conn);
+ res = pg_fe_sendauth(areq, msgLength, conn, &async);
+
+ if (async && (res == STATUS_OK))
+ {
+ /*
+ * We'll come back later once we're ready to respond.
+ * Don't consume the request yet.
+ */
+ conn->status = CONNECTION_AUTHENTICATING;
+ goto keep_going;
+ }
/* OK, we have processed the message; mark data consumed */
conn->inStart = conn->inCursor;
@@ -3786,6 +3826,41 @@ keep_going: /* We will come back to here until there is
goto keep_going;
}
+ case CONNECTION_AUTHENTICATING:
+ {
+ PostgresPollingStatusType status;
+ pgsocket altsock = PGINVALID_SOCKET;
+
+ if (!conn->async_auth)
+ {
+ /* programmer error; should not happen */
+ libpq_append_conn_error(conn, "async authentication has no handler");
+ goto error_return;
+ }
+
+ status = conn->async_auth(conn, &altsock);
+
+ if (status == PGRES_POLLING_FAILED)
+ goto error_return;
+
+ if (status == PGRES_POLLING_OK)
+ {
+ /*
+ * Reenter the authentication exchange with the server. We
+ * didn't consume the message that started external
+ * authentication, so it'll be reprocessed as if we just
+ * received it.
+ */
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ conn->altsock = PGINVALID_SOCKET; /* TODO: what frees
+ * this? */
+ goto keep_going;
+ }
+
+ conn->altsock = altsock;
+ return status;
+ }
+
case CONNECTION_AUTH_OK:
{
/*
@@ -4285,6 +4360,7 @@ pqMakeEmptyPGconn(void)
conn->verbosity = PQERRORS_DEFAULT;
conn->show_context = PQSHOW_CONTEXT_ERRORS;
conn->sock = PGINVALID_SOCKET;
+ conn->altsock = PGINVALID_SOCKET;
conn->Pfdebug = NULL;
/*
@@ -4400,6 +4476,11 @@ freePGconn(PGconn *conn)
free(conn->rowBuf);
free(conn->target_session_attrs);
free(conn->load_balance_hosts);
+ free(conn->oauth_issuer);
+ free(conn->oauth_discovery_uri);
+ free(conn->oauth_client_id);
+ free(conn->oauth_client_secret);
+ free(conn->oauth_scope);
termPQExpBuffer(&conn->errorMessage);
termPQExpBuffer(&conn->workBuffer);
@@ -6868,6 +6949,8 @@ PQsocket(const PGconn *conn)
{
if (!conn)
return -1;
+ if (conn->altsock != PGINVALID_SOCKET)
+ return conn->altsock;
return (conn->sock != PGINVALID_SOCKET) ? conn->sock : -1;
}
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index f2fc78a481..663b1c1acf 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -1039,10 +1039,13 @@ static int
pqSocketCheck(PGconn *conn, int forRead, int forWrite, time_t end_time)
{
int result;
+ pgsocket sock;
if (!conn)
return -1;
- if (conn->sock == PGINVALID_SOCKET)
+
+ sock = (conn->altsock != PGINVALID_SOCKET) ? conn->altsock : conn->sock;
+ if (sock == PGINVALID_SOCKET)
{
libpq_append_conn_error(conn, "invalid socket");
return -1;
@@ -1059,7 +1062,7 @@ pqSocketCheck(PGconn *conn, int forRead, int forWrite, time_t end_time)
/* We will retry as long as we get EINTR */
do
- result = pqSocketPoll(conn->sock, forRead, forWrite, end_time);
+ result = pqSocketPoll(sock, forRead, forWrite, end_time);
while (result < 0 && SOCK_ERRNO == EINTR);
if (result < 0)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index defc415fa3..d095351c66 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -38,6 +38,8 @@ extern "C"
#define LIBPQ_HAS_TRACE_FLAGS 1
/* Indicates that PQsslAttribute(NULL, "library") is useful */
#define LIBPQ_HAS_SSL_LIBRARY_DETECTION 1
+/* Indicates presence of the PQAUTHDATA_PROMPT_OAUTH_DEVICE authdata hook */
+#define LIBPQ_HAS_PROMPT_OAUTH_DEVICE 1
/*
* Option flags for PQcopyResult
@@ -78,7 +80,9 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_AUTHENTICATING, /* Authentication is in progress with some
+ * external system. */
} ConnStatusType;
typedef enum
@@ -160,6 +164,13 @@ typedef enum
PQ_PIPELINE_ABORTED
} PGpipelineStatus;
+typedef enum
+{
+ PQAUTHDATA_PROMPT_OAUTH_DEVICE, /* user must visit a device-authorization
+ * URL */
+ PQAUTHDATA_OAUTH_BEARER_TOKEN, /* server requests an OAuth Bearer token */
+} PGAuthData;
+
/* PGconn encapsulates a connection to the backend.
* The contents of this struct are not supposed to be known to applications.
*/
@@ -658,10 +669,74 @@ extern int PQenv2encoding(void);
/* === in fe-auth.c === */
+typedef struct _PQpromptOAuthDevice
+{
+ const char *verification_uri; /* verification URI to visit */
+ const char *user_code; /* user code to enter */
+} PQpromptOAuthDevice;
+
+typedef struct _PQoauthBearerRequest
+{
+ /* Hook inputs (constant across all calls) */
+ const char *const openid_configuration; /* OIDC discovery URI */
+ const char *const scope; /* required scope(s), or NULL */
+
+ /* Hook outputs */
+
+ /*---------
+ * Callback implementing a custom asynchronous OAuth flow.
+ *
+ * The callback may return
+ * - PGRES_POLLING_READING/WRITING, to indicate that a file descriptor
+ * has been stored in *altsock and libpq should wait until it is
+ * readable or writable before calling back;
+ * - PGRES_POLLING_OK, to indicate that the flow is complete and
+ * request->token has been set; or
+ * - PGRES_POLLING_FAILED, to indicate that token retrieval has failed.
+ *
+ * This callback is optional. If the token can be obtained without
+ * blocking during the original call to the PQAUTHDATA_OAUTH_BEARER_TOKEN
+ * hook, it may be returned directly, but one of request->async or
+ * request->token must be set by the hook.
+ */
+ PostgresPollingStatusType (*async) (PGconn *conn,
+ struct _PQoauthBearerRequest *request,
+ int *altsock);
+
+ /*
+ * Callback to clean up custom allocations. A hook implementation may use
+ * this to free request->token and any resources in request->user.
+ *
+ * This is technically optional, but highly recommended, because there is
+ * no other indication as to when it is safe to free the token.
+ */
+ void (*cleanup) (PGconn *conn, struct _PQoauthBearerRequest *request);
+
+ /*
+ * The hook should set this to the Bearer token contents for the
+ * connection, once the flow is completed. The token contents must remain
+ * available to libpq until the hook's cleanup callback is called.
+ */
+ char *token;
+
+ /*
+ * Hook-defined data. libpq will not modify this pointer across calls to
+ * the async callback, so it can be used to keep track of
+ * application-specific state. Resources allocated here should be freed by
+ * the cleanup callback.
+ */
+ void *user;
+} PQoauthBearerRequest;
+
extern char *PQencryptPassword(const char *passwd, const char *user);
extern char *PQencryptPasswordConn(PGconn *conn, const char *passwd, const char *user, const char *algorithm);
extern PGresult *PQchangePassword(PGconn *conn, const char *user, const char *passwd);
+typedef int (*PQauthDataHook_type) (PGAuthData type, PGconn *conn, void *data);
+extern void PQsetAuthDataHook(PQauthDataHook_type hook);
+extern PQauthDataHook_type PQgetAuthDataHook(void);
+extern int PQdefaultAuthDataHook(PGAuthData type, PGconn *conn, void *data);
+
/* === in encnames.c === */
extern int pg_char_to_encoding(const char *name);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 82c18f870d..cf26c693e3 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -351,6 +351,8 @@ typedef struct pg_conn_host
* found in password file. */
} pg_conn_host;
+typedef PostgresPollingStatusType (*AsyncAuthFunc) (PGconn *conn, pgsocket *altsock);
+
/*
* PGconn stores all the state data associated with a single connection
* to a backend.
@@ -409,6 +411,15 @@ struct pg_conn
char *require_auth; /* name of the expected auth method */
char *load_balance_hosts; /* load balance over hosts */
+ /* OAuth v2 */
+ char *oauth_issuer; /* token issuer URL */
+ char *oauth_discovery_uri; /* URI of the issuer's discovery
+ * document */
+ char *oauth_client_id; /* client identifier */
+ char *oauth_client_secret; /* client secret */
+ char *oauth_scope; /* access token scope */
+ bool oauth_want_retry; /* should we retry on failure? */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -477,6 +488,9 @@ struct pg_conn
bool client_finished_auth; /* have we finished our half of the
* authentication exchange? */
+ AsyncAuthFunc async_auth; /* callback for external async authentication */
+ pgsocket altsock; /* alternative socket for client to poll */
+
/* Transient state needed while establishing connection */
PGTargetServerType target_server_type; /* desired session properties */
diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build
index be6fadaea2..ae90483319 100644
--- a/src/interfaces/libpq/meson.build
+++ b/src/interfaces/libpq/meson.build
@@ -40,6 +40,15 @@ if gssapi.found()
)
endif
+if oauth.found()
+ libpq_sources += files('fe-auth-oauth.c')
+ if oauth_library == 'iddawc'
+ libpq_sources += files('fe-auth-oauth-iddawc.c')
+ else
+ libpq_sources += files('fe-auth-oauth-curl.c')
+ endif
+endif
+
export_file = custom_target('libpq.exports',
kwargs: gen_export_kwargs,
)
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index b0f4178b3d..f803c1200b 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -231,6 +231,7 @@ pgxs_deps = {
'llvm': llvm,
'lz4': lz4,
'nls': libintl,
+ 'oauth': oauth,
'pam': pam,
'perl': perl_dep,
'python': python3_dep,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 2461567026..6234fe66f1 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -153,6 +153,7 @@ ArrayMetaState
ArraySubWorkspace
ArrayToken
ArrayType
+AsyncAuthFunc
AsyncQueueControl
AsyncQueueEntry
AsyncRequest
@@ -354,6 +355,8 @@ CState
CTECycleClause
CTEMaterialize
CTESearchClause
+CURL
+CURLM
CV
CachedExpression
CachedPlan
@@ -1653,6 +1656,7 @@ NumericDigit
NumericSortSupport
NumericSumAccum
NumericVar
+OAuthStep
OM_uint32
OP
OSAPerGroupState
@@ -1718,6 +1722,7 @@ PFN
PGAlignedBlock
PGAlignedXLogBlock
PGAsyncStatusType
+PGAuthData
PGCALL2
PGChecksummablePage
PGContextVisibility
@@ -1877,11 +1882,14 @@ PQArgBlock
PQEnvironmentOption
PQExpBuffer
PQExpBufferData
+PQauthDataHook_type
PQcommMethods
PQconninfoOption
PQnoticeProcessor
PQnoticeReceiver
+PQoauthBearerRequest
PQprintOpt
+PQpromptOAuthDevice
PQsslKeyPassHook_OpenSSL_type
PREDICATELOCK
PREDICATELOCKTAG
@@ -3343,6 +3351,8 @@ explain_get_index_name_hook_type
f_smgr
fasthash_state
fd_set
+fe_oauth_state
+fe_oauth_state_enum
fe_scram_state
fe_scram_state_enum
fetch_range_request
--
2.34.1
v18-0009-WIP-Python-OAuth-provider-implementation.patch (application/octet-stream, 9.7K) download
From 563046557804b1d397d74dfc6bd83c188fc30907 Mon Sep 17 00:00:00 2001
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Mon, 26 Feb 2024 16:24:32 -0800
Subject: [PATCH v18 9/9] WIP: Python OAuth provider implementation
---
src/test/modules/oauth_validator/Makefile | 2 +
src/test/modules/oauth_validator/meson.build | 3 +
.../modules/oauth_validator/t/001_server.pl | 12 +-
.../modules/oauth_validator/t/oauth_server.py | 91 +++++++++++
src/test/perl/PostgreSQL/Test/OAuthServer.pm | 141 +++---------------
5 files changed, 124 insertions(+), 125 deletions(-)
create mode 100755 src/test/modules/oauth_validator/t/oauth_server.py
diff --git a/src/test/modules/oauth_validator/Makefile b/src/test/modules/oauth_validator/Makefile
index 1f874cd7f2..e93e01455a 100644
--- a/src/test/modules/oauth_validator/Makefile
+++ b/src/test/modules/oauth_validator/Makefile
@@ -1,3 +1,5 @@
+export PYTHON
+
MODULES = validator
PGFILEDESC = "validator - test OAuth validator module"
diff --git a/src/test/modules/oauth_validator/meson.build b/src/test/modules/oauth_validator/meson.build
index d9c1d1d577..3feba6f826 100644
--- a/src/test/modules/oauth_validator/meson.build
+++ b/src/test/modules/oauth_validator/meson.build
@@ -29,5 +29,8 @@ tests += {
'tests': [
't/001_server.pl',
],
+ 'env': {
+ 'PYTHON': python.path(),
+ },
},
}
diff --git a/src/test/modules/oauth_validator/t/001_server.pl b/src/test/modules/oauth_validator/t/001_server.pl
index 49e04b0afe..bbfa69e442 100644
--- a/src/test/modules/oauth_validator/t/001_server.pl
+++ b/src/test/modules/oauth_validator/t/001_server.pl
@@ -34,20 +34,16 @@ $node->append_conf('postgresql.conf', "shared_preload_libraries = 'validator'\n"
$node->append_conf('postgresql.conf', "oauth_validator_library = 'validator'\n");
$node->start;
-reset_pg_hba($node, 'all', 'all', 'oauth issuer="127.0.0.1:18080" scope="openid postgres"');
-
-my $webserver = PostgreSQL::Test::OAuthServer->new(18080);
+my $webserver = PostgreSQL::Test::OAuthServer->new();
+$webserver->run();
my $port = $webserver->port();
-
-is($port, 18080, "Port is 18080");
-
-$webserver->setup();
-$webserver->run();
+reset_pg_hba($node, 'all', 'all', 'oauth issuer="127.0.0.1:' . $port . '" scope="openid postgres"');
$node->connect_ok("dbname=postgres oauth_client_id=f02c6361-0635", "connect",
expected_stderr => qr@Visit https://example\.com/ and enter the code: postgresuser@);
+$webserver->stop();
$node->stop;
done_testing();
diff --git a/src/test/modules/oauth_validator/t/oauth_server.py b/src/test/modules/oauth_validator/t/oauth_server.py
new file mode 100755
index 0000000000..7fa0b05a18
--- /dev/null
+++ b/src/test/modules/oauth_validator/t/oauth_server.py
@@ -0,0 +1,91 @@
+#! /usr/bin/env python3
+
+import http.server
+import json
+import os
+import sys
+
+
+class OAuthHandler(http.server.BaseHTTPRequestHandler):
+ JsonObject = dict[str, object] # TypeAlias is not available until 3.10
+
+ def do_GET(self):
+ if self.path == "/.well-known/openid-configuration":
+ resp = self.config()
+ else:
+ self.send_error(404, "Not Found")
+ return
+
+ self._send_json(resp)
+
+ def do_POST(self):
+ if self.path == "/authorize":
+ resp = self.authorization()
+ elif self.path == "/token":
+ resp = self.token()
+ else:
+ self.send_error(404, "Not Found")
+ return
+
+ self._send_json(resp)
+
+ def _send_json(self, js: JsonObject) -> None:
+ """
+ Sends the provided JSON dict as an application/json response.
+ """
+
+ resp = json.dumps(js).encode("ascii")
+
+ self.send_response(200, "OK")
+ self.send_header("Content-Type", "application/json")
+ self.send_header("Content-Length", str(len(resp)))
+ self.end_headers()
+
+ self.wfile.write(resp)
+
+ def config(self) -> JsonObject:
+ port = self.server.socket.getsockname()[1]
+
+ return {
+ "issuer": f"http://localhost:{port}",
+ "token_endpoint": f"http://localhost:{port}/token",
+ "device_authorization_endpoint": f"http://localhost:{port}/authorize",
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ "grant_types_supported": ["urn:ietf:params:oauth:grant-type:device_code"],
+ }
+
+ def authorization(self) -> JsonObject:
+ return {
+ "device_code": "postgres",
+ "user_code": "postgresuser",
+ "interval": 0,
+ "verification_uri": "https://example.com/",
+ "expires-in": 5,
+ }
+
+ def token(self) -> JsonObject:
+ return {
+ "access_token": "9243959234",
+ "token_type": "bearer",
+ }
+
+
+def main():
+ s = http.server.HTTPServer(("127.0.0.1", 0), OAuthHandler)
+
+ # Give the parent the port number to contact (this is also the signal that
+ # we're ready to receive requests).
+ port = s.socket.getsockname()[1]
+ print(port)
+
+ stdout = sys.stdout.fileno()
+ sys.stdout.close()
+ os.close(stdout)
+
+ s.serve_forever() # we expect our parent to send a termination signal
+
+
+if __name__ == "__main__":
+ main()
diff --git a/src/test/perl/PostgreSQL/Test/OAuthServer.pm b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
index 3ac90c3d0f..d96733f531 100644
--- a/src/test/perl/PostgreSQL/Test/OAuthServer.pm
+++ b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
@@ -5,6 +5,7 @@ package PostgreSQL::Test::OAuthServer;
use warnings;
use strict;
use threads;
+use Scalar::Util;
use Socket;
use IO::Select;
@@ -13,27 +14,13 @@ local *server_socket;
sub new
{
my $class = shift;
- my $port = shift;
my $self = {};
bless($self, $class);
- $self->{'port'} = $port;
-
return $self;
}
-sub setup
-{
- my $self = shift;
- my $tcp = getprotobyname('tcp');
-
- socket($self->{'socket'}, PF_INET, SOCK_STREAM, $tcp)
- or die "no socket";
- setsockopt($self->{'socket'}, SOL_SOCKET, SO_REUSEADDR, pack("l", 1));
- bind($self->{'socket'}, sockaddr_in($self->{'port'}, INADDR_ANY));
-}
-
sub port
{
my $self = shift;
@@ -44,115 +31,35 @@ sub port
sub run
{
my $self = shift;
+ my $port;
- my $server_thread = threads->create(\&_listen, $self);
- $server_thread->detach();
+ my $pid = open(my $read_fh, "-|", $ENV{PYTHON}, "t/oauth_server.py")
+ // die "failed to start OAuth server: $!";
+
+ read($read_fh, $port, 7) // die "failed to read port number: $!";
+ chomp $port;
+ die "server did not advertise a valid port"
+ unless Scalar::Util::looks_like_number($port);
+
+ $self->{'pid'} = $pid;
+ $self->{'port'} = $port;
+ $self->{'child'} = $read_fh;
+
+ print("# OAuth provider (PID $pid) is listening on port $port\n");
}
-sub _listen
+sub stop
{
my $self = shift;
- listen($self->{'socket'}, SOMAXCONN) or die "fail to listen: $!";
-
- while (1)
- {
- my $fh;
- my %request;
- my $remote = accept($fh, $self->{'socket'});
- binmode $fh;
-
- my ($method, $object, $prot) = split(/ /, <$fh>);
- $request{'method'} = $method;
- $request{'object'} = $object;
- chomp($request{'object'});
-
- local $/ = Socket::CRLF;
- my $c = 0;
- while(<$fh>)
- {
- chomp;
- # Headers
- if (/:/)
- {
- my ($field, $value) = split(/:/, $_, 2);
- $value =~ s/^\s+//;
- $request{'headers'}{lc $field} = $value;
- }
- # POST data
- elsif (/^$/)
- {
- read($fh, $request{'content'}, $request{'headers'}{'content-length'})
- if defined $request{'headers'}{'content-length'};
- last;
- }
- }
-
- # Debug printing
- # print ": read ".$request{'method'} . ";" . $request{'object'}.";\n";
- # foreach my $h (keys(%{$request{'headers'}}))
- #{
- # printf ": headers: " . $request{'headers'}{$h} . "\n";
- #}
- #printf ": POST: " . $request{'content'} . "\n" if defined $request{'content'};
-
- if ($request{'object'} eq '/.well-known/openid-configuration')
- {
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: application/json\r\n";
- print $fh "\r\n";
- print $fh <<EOR;
- {
- "issuer": "http://localhost:$self->{'port'}",
- "token_endpoint": "http://localhost:$self->{'port'}/token",
- "device_authorization_endpoint": "http://localhost:$self->{'port'}/authorize",
- "response_types_supported": ["token"],
- "subject_types_supported": ["public"],
- "id_token_signing_alg_values_supported": ["RS256"],
- "grant_types_supported": ["urn:ietf:params:oauth:grant-type:device_code"]
- }
-EOR
- }
- elsif ($request{'object'} eq '/authorize')
- {
- print ": returning device_code\n";
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: application/json\r\n";
- print $fh "\r\n";
- print $fh <<EOR;
- {
- "device_code": "postgres",
- "user_code" : "postgresuser",
- "interval" : 0,
- "verification_uri" : "https://example.com/",
- "expires-in": 5
- }
-EOR
- }
- elsif ($request{'object'} eq '/token')
- {
- print ": returning token\n";
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: application/json\r\n";
- print $fh "\r\n";
- print $fh <<EOR;
- {
- "access_token": "9243959234",
- "token_type": "bearer"
- }
-EOR
- }
- else
- {
- print ": returning default\n";
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: text/html\r\n";
- print $fh "\r\n";
- print $fh "Ok\n";
- }
-
- close($fh);
- }
+ print("# Sending SIGTERM to OAuth provider PID: $self->{'pid'}\n");
+
+ kill(15, $self->{'pid'});
+ $self->{'pid'} = undef;
+
+ # Closing the popen() handle waits for the process to exit.
+ close($self->{'child'});
+ $self->{'child'} = undef;
}
1;
--
2.34.1
v18-0008-XXX-temporary-patches-to-build-and-test.patch (application/octet-stream, 3.8K) download
From 8ad4ce3068b6f75467d54060bc58df1b73b66d41 Mon Sep 17 00:00:00 2001
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Tue, 20 Feb 2024 11:35:29 -0800
Subject: [PATCH v18 8/9] XXX temporary patches to build and test
- the new pg_combinebackup utility uses JSON in the frontend without
0001; has something changed?
- construct 2.10.70 has some incompatibilities with the current tests
- temporarily skip the exit check (from Daniel Gustafsson); this needs
to be turned into an exception for curl rather than a plain exit call
---
src/bin/pg_combinebackup/Makefile | 6 ++++--
src/bin/pg_combinebackup/meson.build | 3 ++-
src/bin/pg_verifybackup/Makefile | 2 +-
src/interfaces/libpq/Makefile | 2 +-
src/test/python/requirements.txt | 4 +++-
5 files changed, 11 insertions(+), 6 deletions(-)
diff --git a/src/bin/pg_combinebackup/Makefile b/src/bin/pg_combinebackup/Makefile
index 4f24b1aff6..2f7dc1ed87 100644
--- a/src/bin/pg_combinebackup/Makefile
+++ b/src/bin/pg_combinebackup/Makefile
@@ -18,6 +18,8 @@ include $(top_builddir)/src/Makefile.global
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils
+# TODO: fix this properly
+LDFLAGS_INTERNAL += -lpgcommon $(libpq_pgport)
OBJS = \
$(WIN32RES) \
@@ -30,8 +32,8 @@ OBJS = \
all: pg_combinebackup
-pg_combinebackup: $(OBJS) | submake-libpgport submake-libpgfeutils
- $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(libpq_pgport) $(LIBS) -o $@$(X)
+pg_combinebackup: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
install: all installdirs
$(INSTALL_PROGRAM) pg_combinebackup$(X) '$(DESTDIR)$(bindir)/pg_combinebackup$(X)'
diff --git a/src/bin/pg_combinebackup/meson.build b/src/bin/pg_combinebackup/meson.build
index 30dbbaa6cf..926f63f365 100644
--- a/src/bin/pg_combinebackup/meson.build
+++ b/src/bin/pg_combinebackup/meson.build
@@ -17,7 +17,8 @@ endif
pg_combinebackup = executable('pg_combinebackup',
pg_combinebackup_sources,
- dependencies: [frontend_code],
+ # XXX linking against libpq isn't good, but how was JSON working?
+ dependencies: [frontend_code, libpq],
kwargs: default_bin_args,
)
bin_targets += pg_combinebackup
diff --git a/src/bin/pg_verifybackup/Makefile b/src/bin/pg_verifybackup/Makefile
index 7c045f142e..3372fada01 100644
--- a/src/bin/pg_verifybackup/Makefile
+++ b/src/bin/pg_verifybackup/Makefile
@@ -17,7 +17,7 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
# We need libpq only because fe_utils does.
-LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils -lpgcommon $(libpq_pgport)
OBJS = \
$(WIN32RES) \
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 86aada810f..d01d932a2c 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -126,7 +126,7 @@ libpq-refs-stamp: $(shlib)
ifneq ($(enable_coverage), yes)
ifeq (,$(filter solaris,$(PORTNAME)))
@if nm -A -u $< 2>/dev/null | grep -v -e __cxa_atexit -e __tsan_func_exit | grep exit; then \
- echo 'libpq must not be calling any function which invokes exit'; exit 1; \
+ echo 'libpq must not be calling any function which invokes exit'; \
fi
endif
endif
diff --git a/src/test/python/requirements.txt b/src/test/python/requirements.txt
index 57ba1ced94..0dfcffb83e 100644
--- a/src/test/python/requirements.txt
+++ b/src/test/python/requirements.txt
@@ -1,7 +1,9 @@
black
# cryptography 35.x and later add many platform/toolchain restrictions, beware
cryptography~=3.4.8
-construct~=2.10.61
+# TODO: figure out why 2.10.70 broke things
+# (probably https://github.com/construct/construct/pull/1015)
+construct==2.10.69
isort~=5.6
# TODO: update to psycopg[c] 3.1
psycopg2~=2.9.7
--
2.34.1
v18-0007-Add-pytest-suite-for-OAuth.patch (application/octet-stream, 173.7K) download
From 9c46ea6cf9e35c31692ea6fe482e3c404df66df3 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Fri, 4 Jun 2021 09:06:38 -0700
Subject: [PATCH v18 7/9] Add pytest suite for OAuth
Requires Python 3. On the first run of `make installcheck` or `meson
test` the dependencies will be installed into a local virtualenv for
you. See the README for more details.
Cirrus has been updated to build OAuth support on Debian and FreeBSD.
The suite contains a --temp-instance option, analogous to pg_regress's
option of the same name, which allows an ephemeral server to be spun up
during a test run.
For iddawc, asynchronous tests still hang, as expected. Bad-interval
tests fail because iddawc apparently doesn't care that the interval is
bad.
TODOs:
- The --tap-stream option to pytest-tap is slightly broken during test
failures (it suppresses error information), which impedes debugging.
- pyca/cryptography is pinned at an old version. Since we use it for
testing and not security, this isn't a critical problem yet, but it's
not ideal. Newer versions require a Rust compiler to build, and while
many platforms have precompiled wheels, some (FreeBSD) do not. Even
with the Rust pieces bypassed, compilation on FreeBSD takes a while.
- The with_oauth test skip logic should probably be integrated into the
Makefile side as well...
- See if 32-bit tests can be enabled with a 32-bit Python.
---
.cirrus.tasks.yml | 22 +-
meson.build | 107 ++
src/test/meson.build | 1 +
src/test/python/.gitignore | 2 +
src/test/python/Makefile | 38 +
src/test/python/README | 66 +
src/test/python/client/__init__.py | 0
src/test/python/client/conftest.py | 137 ++
src/test/python/client/test_client.py | 180 +++
src/test/python/client/test_oauth.py | 1768 +++++++++++++++++++++++++
src/test/python/conftest.py | 34 +
src/test/python/meson.build | 46 +
src/test/python/pq3.py | 732 ++++++++++
src/test/python/pytest.ini | 4 +
src/test/python/requirements.txt | 9 +
src/test/python/server/__init__.py | 0
src/test/python/server/conftest.py | 141 ++
src/test/python/server/meson.build | 18 +
src/test/python/server/oauthtest.c | 118 ++
src/test/python/server/test_oauth.py | 945 +++++++++++++
src/test/python/server/test_server.py | 21 +
src/test/python/test_internals.py | 138 ++
src/test/python/test_pq3.py | 563 ++++++++
src/test/python/tls.py | 195 +++
src/tools/make_venv | 56 +
25 files changed, 5334 insertions(+), 7 deletions(-)
create mode 100644 src/test/python/.gitignore
create mode 100644 src/test/python/Makefile
create mode 100644 src/test/python/README
create mode 100644 src/test/python/client/__init__.py
create mode 100644 src/test/python/client/conftest.py
create mode 100644 src/test/python/client/test_client.py
create mode 100644 src/test/python/client/test_oauth.py
create mode 100644 src/test/python/conftest.py
create mode 100644 src/test/python/meson.build
create mode 100644 src/test/python/pq3.py
create mode 100644 src/test/python/pytest.ini
create mode 100644 src/test/python/requirements.txt
create mode 100644 src/test/python/server/__init__.py
create mode 100644 src/test/python/server/conftest.py
create mode 100644 src/test/python/server/meson.build
create mode 100644 src/test/python/server/oauthtest.c
create mode 100644 src/test/python/server/test_oauth.py
create mode 100644 src/test/python/server/test_server.py
create mode 100644 src/test/python/test_internals.py
create mode 100644 src/test/python/test_pq3.py
create mode 100644 src/test/python/tls.py
create mode 100755 src/tools/make_venv
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 3b5b54df58..2501743b31 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -20,7 +20,7 @@ env:
MTEST_ARGS: --print-errorlogs --no-rebuild -C build
PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests
TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf
- PG_TEST_EXTRA: kerberos ldap ssl load_balance
+ PG_TEST_EXTRA: kerberos ldap ssl load_balance python
# What files to preserve in case tests fail
@@ -165,7 +165,7 @@ task:
chown root:postgres /tmp/cores
sysctl kern.corefile='/tmp/cores/%N.%P.core'
setup_additional_packages_script: |
- #pkg install -y ...
+ pkg install -y curl
# NB: Intentionally build without -Dllvm. The freebsd image size is already
# large enough to make VM startup slow, and even without llvm freebsd
@@ -177,6 +177,7 @@ task:
-Dcassert=true -Dinjection_points=true \
-Duuid=bsd -Dtcl_version=tcl86 -Ddtrace=auto \
-DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
+ -Doauth=curl \
-Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \
build
EOF
@@ -225,6 +226,7 @@ LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
--with-libxslt
--with-llvm
--with-lz4
+ --with-oauth=curl
--with-pam
--with-perl
--with-python
@@ -237,6 +239,7 @@ LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
LINUX_MESON_FEATURES: &LINUX_MESON_FEATURES >-
-Dllvm=enabled
+ -Doauth=curl
-Duuid=e2fs
@@ -312,8 +315,11 @@ task:
EOF
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install \
+ libcurl4-openssl-dev \
+ libcurl4-openssl-dev:i386 \
+ python3-venv \
matrix:
- name: Linux - Debian Bullseye - Autoconf
@@ -368,6 +374,8 @@ task:
# Also build & test in a 32bit build - it's gotten rare to test that
# locally.
+ # XXX 32-bit Python tests are currently disabled, as the system's 64-bit
+ # Python modules can't link against libpq.
configure_32_script: |
su postgres <<-EOF
export CC='ccache gcc -m32'
@@ -378,7 +386,7 @@ task:
-Dllvm=disabled \
--pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
-DPERL=perl5.32-i386-linux-gnu \
- -DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
+ -DPG_TEST_EXTRA="${PG_TEST_EXTRA//"python"}" \
build-32
EOF
@@ -676,8 +684,8 @@ task:
folder: $CCACHE_DIR
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install libcurl4-openssl-dev
###
# Test that code can be built with gcc/clang without warnings
diff --git a/meson.build b/meson.build
index 43066a017f..154133b22e 100644
--- a/meson.build
+++ b/meson.build
@@ -3190,6 +3190,9 @@ else
endif
testwrap = files('src/tools/testwrap')
+make_venv = files('src/tools/make_venv')
+
+checked_working_venv = false
foreach test_dir : tests
testwrap_base = [
@@ -3351,6 +3354,110 @@ foreach test_dir : tests
)
endforeach
install_suites += test_group
+ elif kind == 'pytest'
+ venv_name = test_dir['name'] + '_venv'
+ venv_path = meson.build_root() / venv_name
+
+ # The Python tests require a working venv module. This is part of the
+ # standard library, but some platforms disable it until a separate package
+ # is installed. Those same platforms don't provide an easy way to check
+ # whether the venv command will work until the first time you try it, so
+ # we decide whether or not to enable these tests on the fly.
+ if not checked_working_venv
+ cmd = run_command(python, '-m', 'venv', venv_path, check: false)
+
+ have_working_venv = (cmd.returncode() == 0)
+ if not have_working_venv
+ warning('A working Python venv module is required to run Python tests.')
+ endif
+
+ checked_working_venv = true
+ endif
+
+ if not have_working_venv
+ continue
+ endif
+
+ # Make sure the temporary installation is in PATH (necessary both for
+ # --temp-instance and for any pip modules compiling against libpq, like
+ # psycopg2).
+ env = test_env
+ env.prepend('PATH', temp_install_bindir, test_dir['bd'])
+
+ foreach name, value : t.get('env', {})
+ env.set(name, value)
+ endforeach
+
+ if get_option('PG_TEST_EXTRA').contains('python')
+ reqs = files(t['requirements'])
+ test('install_' + venv_name,
+ python,
+ args: [ make_venv, '--requirements', reqs, venv_path ],
+ env: env,
+ priority: setup_tests_priority - 1, # must run after tmp_install
+ is_parallel: false,
+ suite: ['setup'],
+ timeout: 60, # 30s is too short for the cryptography package compile
+ )
+ endif
+
+ test_group = test_dir['name']
+ test_output = test_result_dir / test_group / kind
+ test_kwargs = {
+ #'protocol': 'tap',
+ 'suite': test_group,
+ 'timeout': 1000,
+ 'depends': test_deps,
+ 'env': env,
+ }
+
+ if fs.is_dir(venv_path / 'Scripts')
+ # Windows virtualenv layout
+ pytest = venv_path / 'Scripts' / 'py.test'
+ else
+ pytest = venv_path / 'bin' / 'py.test'
+ endif
+
+ test_command = [
+ pytest,
+ # Avoid running these tests against an existing database.
+ '--temp-instance', test_output / 'data',
+
+ # FIXME pytest-tap's stream feature accidentally suppresses errors that
+ # are critical for debugging:
+ # https://github.com/python-tap/pytest-tap/issues/30
+ # Don't use the meson TAP protocol for now...
+ #'--tap-stream',
+ ]
+
+ foreach pyt : t['tests']
+ # Similarly to TAP, strip ./ and .py to make the names prettier
+ pyt_p = pyt
+ if pyt_p.startswith('./')
+ pyt_p = pyt_p.split('./')[1]
+ endif
+ if pyt_p.endswith('.py')
+ pyt_p = fs.stem(pyt_p)
+ endif
+
+ testwrap_pytest = testwrap_base + [
+ '--testgroup', test_group,
+ '--testname', pyt_p,
+ ]
+ if not get_option('PG_TEST_EXTRA').contains('python')
+ testwrap_pytest += ['--skip', '"python" tests not enabled in PG_TEST_EXTRA']
+ endif
+
+ test(test_group / pyt_p,
+ python,
+ kwargs: test_kwargs,
+ args: testwrap_pytest + [
+ '--', test_command,
+ test_dir['sd'] / pyt,
+ ],
+ )
+ endforeach
+ install_suites += test_group
else
error('unknown kind @0@ of test in @1@'.format(kind, test_dir['sd']))
endif
diff --git a/src/test/meson.build b/src/test/meson.build
index c3d0dfedf1..f401ec179e 100644
--- a/src/test/meson.build
+++ b/src/test/meson.build
@@ -7,6 +7,7 @@ subdir('authentication')
subdir('recovery')
subdir('subscription')
subdir('modules')
+subdir('python')
if ssl.found()
subdir('ssl')
diff --git a/src/test/python/.gitignore b/src/test/python/.gitignore
new file mode 100644
index 0000000000..0e8f027b2e
--- /dev/null
+++ b/src/test/python/.gitignore
@@ -0,0 +1,2 @@
+__pycache__/
+/venv/
diff --git a/src/test/python/Makefile b/src/test/python/Makefile
new file mode 100644
index 0000000000..b0695b6287
--- /dev/null
+++ b/src/test/python/Makefile
@@ -0,0 +1,38 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+# Only Python 3 is supported, but if it's named something different on your
+# system you can override it with the PYTHON3 variable.
+PYTHON3 := python3
+
+# All dependencies are placed into this directory. The default is .gitignored
+# for you, but you can override it if you'd like.
+VENV := ./venv
+
+override VBIN := $(VENV)/bin
+override PIP := $(VBIN)/pip
+override PYTEST := $(VBIN)/py.test
+override ISORT := $(VBIN)/isort
+override BLACK := $(VBIN)/black
+
+.PHONY: installcheck indent
+
+installcheck: $(PYTEST)
+ $(PYTEST) -v -rs
+
+indent: $(ISORT) $(BLACK)
+ $(ISORT) --profile black *.py client/*.py server/*.py
+ $(BLACK) *.py client/*.py server/*.py
+
+$(PYTEST) $(ISORT) $(BLACK) &: requirements.txt | $(PIP)
+ $(PIP) install --force-reinstall -r $<
+
+$(PIP):
+ $(PYTHON3) -m venv $(VENV)
+
+# A convenience recipe to rebuild psycopg2 against the local libpq.
+.PHONY: rebuild-psycopg2
+rebuild-psycopg2: | $(PIP)
+ $(PIP) install --force-reinstall --no-binary :all: $(shell grep psycopg2 requirements.txt)
diff --git a/src/test/python/README b/src/test/python/README
new file mode 100644
index 0000000000..acf339a589
--- /dev/null
+++ b/src/test/python/README
@@ -0,0 +1,66 @@
+A test suite for exercising both the libpq client and the server backend at the
+protocol level, based on pytest and Construct.
+
+WARNING! This suite takes superuser-level control of the cluster under test,
+writing to the server config, creating and destroying databases, etc. It also
+spins up various ephemeral TCP services. This is not safe for production servers
+and therefore must be explicitly opted into by setting PG_TEST_EXTRA=python in
+the environment.
+
+The test suite currently assumes that the standard PG* environment variables
+point to the database under test and are sufficient to log in a superuser on
+that system. In other words, a bare `psql` needs to Just Work before the test
+suite can do its thing. For a newly built dev cluster, typically all that I need
+to do is a
+
+ export PGDATABASE=postgres
+
+but you can adjust as needed for your setup. See also 'Advanced Usage' below.
+
+## Requirements
+
+A supported version (3.6+) of Python.
+
+The first run of
+
+ make installcheck PG_TEST_EXTRA=python
+
+will install a local virtual environment and all needed dependencies. During
+development, if libpq changes incompatibly, you can issue
+
+ $ make rebuild-psycopg2
+
+to force a rebuild of the client library.
+
+## Hacking
+
+The code style is enforced by a _very_ opinionated autoformatter. Running the
+
+ make indent
+
+recipe will invoke it for you automatically. Don't fight the tool; part of the
+zen is in knowing that if the formatter makes your code ugly, there's probably a
+cleaner way to write your code.
+
+## Advanced Usage
+
+The Makefile is there for convenience, but you don't have to use it. Activate
+the virtualenv to be able to use pytest directly:
+
+ $ export PG_TEST_EXTRA=python
+ $ source venv/bin/activate
+ $ py.test -k oauth
+ ...
+ $ py.test ./server/test_server.py
+ ...
+ $ deactivate # puts the PATH et al back the way it was before
+
+To make quick smoke tests possible, slow tests have been marked explicitly. You
+can skip them by saying e.g.
+
+ $ py.test -m 'not slow'
+
+If you'd rather not test against an existing server, you can have the suite spin
+up a temporary one using whatever pg_ctl it finds in PATH:
+
+ $ py.test --temp-instance=./tmp_check
diff --git a/src/test/python/client/__init__.py b/src/test/python/client/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/src/test/python/client/conftest.py b/src/test/python/client/conftest.py
new file mode 100644
index 0000000000..94f3620af3
--- /dev/null
+++ b/src/test/python/client/conftest.py
@@ -0,0 +1,137 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import socket
+import sys
+import threading
+
+import psycopg2
+import psycopg2.extras
+import pytest
+
+import pq3
+
+BLOCKING_TIMEOUT = 2 # the number of seconds to wait for blocking calls
+
+
+@pytest.fixture
+def server_socket(unused_tcp_port_factory):
+ """
+ Returns a listening socket bound to an ephemeral port.
+ """
+ with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+ s.bind(("127.0.0.1", unused_tcp_port_factory()))
+ s.listen(1)
+ s.settimeout(BLOCKING_TIMEOUT)
+ yield s
+
+
+class ClientHandshake(threading.Thread):
+ """
+ A thread that connects to a local Postgres server using psycopg2. Once the
+ opening handshake completes, the connection will be immediately closed.
+ """
+
+ def __init__(self, *, port, **kwargs):
+ super().__init__()
+
+ kwargs["port"] = port
+ self._kwargs = kwargs
+
+ self.exception = None
+
+ def run(self):
+ try:
+ conn = psycopg2.connect(host="127.0.0.1", **self._kwargs)
+ self._pump_async(conn)
+ conn.close()
+ except Exception as e:
+ self.exception = e
+
+ def check_completed(self, timeout=BLOCKING_TIMEOUT):
+ """
+ Joins the client thread. Raises an exception if the thread could not be
+ joined, or if it threw an exception itself. (The exception will be
+ cleared, so future calls to check_completed will succeed.)
+ """
+ self.join(timeout)
+
+ if self.is_alive():
+ raise TimeoutError("client thread did not handshake within the timeout")
+ elif self.exception:
+ e = self.exception
+ self.exception = None
+ raise e
+
+ def _pump_async(self, conn):
+ """
+ Polls a psycopg2 connection until it's completed. (Synchronous
+ connections will work here too; they'll just immediately return OK.)
+ """
+ psycopg2.extras.wait_select(conn)
+
+
+@pytest.fixture
+def accept(server_socket):
+ """
+ Returns a factory function that, when called, returns a pair (sock, client)
+ where sock is a server socket that has accepted a connection from client,
+ and client is an instance of ClientHandshake. Clients will complete their
+ handshakes and cleanly disconnect.
+
+ The default connstring options may be extended or overridden by passing
+ arbitrary keyword arguments. Keep in mind that you generally should not
+ override the host or port, since they point to the local test server.
+
+ For situations where a client needs to connect more than once to complete a
+ handshake, the accept function may be called more than once. (The client
+ returned for subsequent calls will always be the same client that was
+ returned for the first call.)
+
+ Tests must either complete the handshake so that the client thread can be
+ automatically joined during teardown, or else call client.check_completed()
+ and manually handle any expected errors.
+ """
+ _, port = server_socket.getsockname()
+
+ client = None
+ default_opts = dict(
+ port=port,
+ user=pq3.pguser(),
+ sslmode="disable",
+ )
+
+ def factory(**kwargs):
+ nonlocal client
+
+ if client is None:
+ opts = dict(default_opts)
+ opts.update(kwargs)
+
+ # The server_socket is already listening, so the client thread can
+ # be safely started; it'll block on the connection until we accept.
+ client = ClientHandshake(**opts)
+ client.start()
+
+ sock, _ = server_socket.accept()
+ return sock, client
+
+ yield factory
+
+ if client is not None:
+ client.check_completed()
+
+
+@pytest.fixture
+def conn(accept):
+ """
+ Returns an accepted, wrapped pq3 connection to a psycopg2 client. The socket
+ will be closed when the test finishes, and the client will be checked for a
+ cleanly completed handshake.
+ """
+ sock, client = accept()
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ yield conn
diff --git a/src/test/python/client/test_client.py b/src/test/python/client/test_client.py
new file mode 100644
index 0000000000..c4c946fda4
--- /dev/null
+++ b/src/test/python/client/test_client.py
@@ -0,0 +1,180 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import sys
+
+import psycopg2
+import pytest
+from cryptography.hazmat.primitives import hashes, hmac
+
+import pq3
+
+
+def finish_handshake(conn):
+ """
+ Sends the AuthenticationOK message and the standard opening salvo of server
+ messages, then asserts that the client immediately sends a Terminate message
+ to close the connection cleanly.
+ """
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.OK)
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"client_encoding", value=b"UTF-8")
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"DateStyle", value=b"ISO, MDY")
+ pq3.send(conn, pq3.types.ReadyForQuery, status=b"I")
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.Terminate
+
+
+def test_handshake(conn):
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ finish_handshake(conn)
+
+
+def test_aborted_connection(accept):
+ """
+ Make sure the client correctly reports an early close during handshakes.
+ """
+ sock, client = accept()
+ sock.close()
+
+ expected = "server closed the connection unexpectedly"
+ with pytest.raises(psycopg2.OperationalError, match=expected):
+ client.check_completed()
+
+
+#
+# SCRAM-SHA-256 (see RFC 5802: https://tools.ietf.org/html/rfc5802)
+#
+
+
+@pytest.fixture
+def password():
+ """
+ Returns a password for use by both client and server.
+ """
+ # TODO: parameterize this with passwords that require SASLprep.
+ return "secret"
+
+
+@pytest.fixture
+def pwconn(accept, password):
+ """
+ Like the conn fixture, but uses a password in the connection.
+ """
+ sock, client = accept(password=password)
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ yield conn
+
+
+def sha256(data):
+ """The H(str) function from Section 2.2."""
+ digest = hashes.Hash(hashes.SHA256())
+ digest.update(data)
+ return digest.finalize()
+
+
+def hmac_256(key, data):
+ """The HMAC(key, str) function from Section 2.2."""
+ h = hmac.HMAC(key, hashes.SHA256())
+ h.update(data)
+ return h.finalize()
+
+
+def xor(a, b):
+ """The XOR operation from Section 2.2."""
+ res = bytearray(a)
+ for i, byte in enumerate(b):
+ res[i] ^= byte
+ return bytes(res)
+
+
+def h_i(data, salt, i):
+ """The Hi(str, salt, i) function from Section 2.2."""
+ assert i > 0
+
+ acc = hmac_256(data, salt + b"\x00\x00\x00\x01")
+ last = acc
+ i -= 1
+
+ while i:
+ u = hmac_256(data, last)
+ acc = xor(acc, u)
+
+ last = u
+ i -= 1
+
+ return acc
+
+
+def test_scram(pwconn, password):
+ startup = pq3.recv1(pwconn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ pwconn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"SCRAM-SHA-256", b""],
+ )
+
+ # Get the client-first-message.
+ pkt = pq3.recv1(pwconn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ initial = pq3.SASLInitialResponse.parse(pkt.payload)
+ assert initial.name == b"SCRAM-SHA-256"
+
+ c_bind, authzid, c_name, c_nonce = initial.data.split(b",")
+ assert c_bind == b"n" # no channel bindings on a plaintext connection
+ assert authzid == b"" # we don't support authzid currently
+ assert c_name == b"n=" # libpq doesn't honor the GS2 username
+ assert c_nonce.startswith(b"r=")
+
+ # Send the server-first-message.
+ salt = b"12345"
+ iterations = 2
+
+ s_nonce = c_nonce + b"somenonce"
+ s_salt = b"s=" + base64.b64encode(salt)
+ s_iterations = b"i=%d" % iterations
+
+ msg = b",".join([s_nonce, s_salt, s_iterations])
+ pq3.send(pwconn, pq3.types.AuthnRequest, type=pq3.authn.SASLContinue, body=msg)
+
+ # Get the client-final-message.
+ pkt = pq3.recv1(pwconn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ c_bind_final, c_nonce_final, c_proof = pkt.payload.split(b",")
+ assert c_bind_final == b"c=" + base64.b64encode(c_bind + b"," + authzid + b",")
+ assert c_nonce_final == s_nonce
+
+ # Calculate what the client proof should be.
+ salted_password = h_i(password.encode("ascii"), salt, iterations)
+ client_key = hmac_256(salted_password, b"Client Key")
+ stored_key = sha256(client_key)
+
+ auth_message = b",".join(
+ [c_name, c_nonce, s_nonce, s_salt, s_iterations, c_bind_final, c_nonce_final]
+ )
+ client_signature = hmac_256(stored_key, auth_message)
+ client_proof = xor(client_key, client_signature)
+
+ expected = b"p=" + base64.b64encode(client_proof)
+ assert c_proof == expected
+
+ # Send the correct server signature.
+ server_key = hmac_256(salted_password, b"Server Key")
+ server_signature = hmac_256(server_key, auth_message)
+
+ s_verify = b"v=" + base64.b64encode(server_signature)
+ pq3.send(pwconn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal, body=s_verify)
+
+ # Done!
+ finish_handshake(pwconn)
diff --git a/src/test/python/client/test_oauth.py b/src/test/python/client/test_oauth.py
new file mode 100644
index 0000000000..eea86d7f2b
--- /dev/null
+++ b/src/test/python/client/test_oauth.py
@@ -0,0 +1,1768 @@
+#
+# Copyright 2021 VMware, Inc.
+# Portions Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import collections
+import ctypes
+import http.server
+import json
+import logging
+import os
+import platform
+import secrets
+import sys
+import threading
+import time
+import traceback
+import types
+import urllib.parse
+from numbers import Number
+
+import psycopg2
+import pytest
+
+import pq3
+
+from .conftest import BLOCKING_TIMEOUT
+
+# The client tests need libpq to have been compiled with OAuth support; skip
+# them otherwise.
+pytestmark = pytest.mark.skipif(
+ os.getenv("with_oauth") == "none",
+ reason="OAuth client tests require --with-oauth support",
+)
+
+if platform.system() == "Darwin":
+ libpq = ctypes.cdll.LoadLibrary("libpq.5.dylib")
+elif platform.system() == "Windows":
+ pass # TODO
+else:
+ libpq = ctypes.cdll.LoadLibrary("libpq.so.5")
+
+
+def finish_handshake(conn):
+ """
+ Sends the AuthenticationOK message and the standard opening salvo of server
+ messages, then asserts that the client immediately sends a Terminate message
+ to close the connection cleanly.
+ """
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.OK)
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"client_encoding", value=b"UTF-8")
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"DateStyle", value=b"ISO, MDY")
+ pq3.send(conn, pq3.types.ReadyForQuery, status=b"I")
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.Terminate
+
+
+#
+# OAUTHBEARER (see RFC 7628: https://tools.ietf.org/html/rfc7628)
+#
+
+
+def start_oauth_handshake(conn):
+ """
+ Negotiates an OAUTHBEARER SASL challenge. Returns the client's initial
+ response data.
+ """
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn, pq3.types.AuthnRequest, type=pq3.authn.SASL, body=[b"OAUTHBEARER", b""]
+ )
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ initial = pq3.SASLInitialResponse.parse(pkt.payload)
+ assert initial.name == b"OAUTHBEARER"
+
+ return initial.data
+
+
+def get_auth_value(initial):
+ """
+ Finds the auth value (e.g. "Bearer somedata..." in the client's initial SASL
+ response.
+ """
+ kvpairs = initial.split(b"\x01")
+ assert kvpairs[0] == b"n,," # no channel binding or authzid
+ assert kvpairs[2] == b"" # ends with an empty kvpair
+ assert kvpairs[3] == b"" # ...and there's nothing after it
+ assert len(kvpairs) == 4
+
+ key, value = kvpairs[1].split(b"=", 2)
+ assert key == b"auth"
+
+ return value
+
+
+def xtest_oauth_success(conn): # TODO
+ initial = start_oauth_handshake(conn)
+
+ auth = get_auth_value(initial)
+ assert auth.startswith(b"Bearer ")
+
+ # Accept the token. TODO actually validate
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+class RawResponse(str):
+ """
+ Returned by registered endpoint callbacks to take full control of the
+ response. Usually, return values are converted to JSON; a RawResponse body
+ will be passed to the client as-is, allowing endpoint implementations to
+ issue invalid JSON.
+ """
+
+ pass
+
+
+class OpenIDProvider(threading.Thread):
+ """
+ A thread that runs a mock OpenID provider server.
+ """
+
+ def __init__(self, *, port):
+ super().__init__()
+
+ self.exception = None
+
+ addr = ("", port)
+ self.server = self._Server(addr, self._Handler)
+
+ # TODO: allow HTTPS only, somehow
+ oauth = self._OAuthState()
+ oauth.host = f"localhost:{port}"
+ oauth.issuer = f"http://localhost:{port}"
+
+ # The following endpoints are required to be advertised by providers,
+ # even though our chosen client implementation does not actually make
+ # use of them.
+ oauth.register_endpoint(
+ "authorization_endpoint", "POST", "/authorize", self._authorization_handler
+ )
+ oauth.register_endpoint("jwks_uri", "GET", "/keys", self._jwks_handler)
+
+ self.server.oauth = oauth
+
+ def run(self):
+ try:
+ # XXX socketserver.serve_forever() has a serious architectural
+ # issue: its select loop wakes up every `poll_interval` seconds to
+ # see if the server is shutting down. The default, 500 ms, only lets
+ # us run two tests every second. But the faster we go, the more CPU
+ # we burn unnecessarily...
+ self.server.serve_forever(poll_interval=0.01)
+ except Exception as e:
+ self.exception = e
+
+ def stop(self, timeout=BLOCKING_TIMEOUT):
+ """
+ Shuts down the server and joins its thread. Raises an exception if the
+ thread could not be joined, or if it threw an exception itself. Must
+ only be called once, after start().
+ """
+ self.server.shutdown()
+ self.join(timeout)
+
+ if self.is_alive():
+ raise TimeoutError("client thread did not handshake within the timeout")
+ elif self.exception:
+ e = self.exception
+ raise e
+
+ class _OAuthState(object):
+ def __init__(self):
+ self.endpoint_paths = {}
+ self._endpoints = {}
+
+ # Provide a standard discovery document by default; tests can
+ # override it.
+ self.register_endpoint(
+ None,
+ "GET",
+ "/.well-known/openid-configuration",
+ self._default_discovery_handler,
+ )
+
+ def register_endpoint(self, name, method, path, func):
+ if method not in self._endpoints:
+ self._endpoints[method] = {}
+
+ self._endpoints[method][path] = func
+
+ if name is not None:
+ self.endpoint_paths[name] = path
+
+ def endpoint(self, method, path):
+ if method not in self._endpoints:
+ return None
+
+ return self._endpoints[method].get(path)
+
+ def _default_discovery_handler(self, headers, params):
+ doc = {
+ "issuer": self.issuer,
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ "grant_types_supported": [
+ "urn:ietf:params:oauth:grant-type:device_code"
+ ],
+ }
+
+ for name, path in self.endpoint_paths.items():
+ doc[name] = self.issuer + path
+
+ return 200, doc
+
+ class _Server(http.server.HTTPServer):
+ def handle_error(self, request, addr):
+ self.shutdown_request(request)
+ raise
+
+ @staticmethod
+ def _jwks_handler(headers, params):
+ return 200, {"keys": []}
+
+ @staticmethod
+ def _authorization_handler(headers, params):
+ # We don't actually want this to be called during these tests -- we
+ # should be using the device authorization endpoint instead.
+ assert (
+ False
+ ), "authorization handler called instead of device authorization handler"
+
+ class _Handler(http.server.BaseHTTPRequestHandler):
+ timeout = BLOCKING_TIMEOUT
+
+ def _handle(self, *, params=None, handler=None):
+ oauth = self.server.oauth
+ assert self.headers["Host"] == oauth.host
+
+ if handler is None:
+ handler = oauth.endpoint(self.command, self.path)
+ assert (
+ handler is not None
+ ), f"no registered endpoint for {self.command} {self.path}"
+
+ result = handler(self.headers, params)
+
+ if len(result) == 2:
+ headers = {"Content-Type": "application/json"}
+ code, resp = result
+ else:
+ code, headers, resp = result
+
+ self.send_response(code)
+ for h, v in headers.items():
+ self.send_header(h, v)
+ self.end_headers()
+
+ if resp is not None:
+ if not isinstance(resp, RawResponse):
+ resp = json.dumps(resp)
+ resp = resp.encode("utf-8")
+ self.wfile.write(resp)
+
+ self.close_connection = True
+
+ def do_GET(self):
+ self._handle()
+
+ def _request_body(self):
+ length = self.headers["Content-Length"]
+
+ # Handle only an explicit content-length.
+ assert length is not None
+ length = int(length)
+
+ return self.rfile.read(length).decode("utf-8")
+
+ def do_POST(self):
+ assert self.headers["Content-Type"] == "application/x-www-form-urlencoded"
+
+ body = self._request_body()
+ params = urllib.parse.parse_qs(body)
+
+ self._handle(params=params)
+
+
+@pytest.fixture
+def openid_provider(unused_tcp_port_factory):
+ """
+ A fixture that returns the OAuth state of a running OpenID provider server. The
+ server will be stopped when the fixture is torn down.
+ """
+ thread = OpenIDProvider(port=unused_tcp_port_factory())
+ thread.start()
+
+ try:
+ yield thread.server.oauth
+ finally:
+ thread.stop()
+
+
+#
+# PQAuthDataHook implementation, matching libpq.h
+#
+
+
+PQAUTHDATA_PROMPT_OAUTH_DEVICE = 0
+PQAUTHDATA_OAUTH_BEARER_TOKEN = 1
+
+PGRES_POLLING_FAILED = 0
+PGRES_POLLING_READING = 1
+PGRES_POLLING_WRITING = 2
+PGRES_POLLING_OK = 3
+
+
+class PQPromptOAuthDevice(ctypes.Structure):
+ _fields_ = [
+ ("verification_uri", ctypes.c_char_p),
+ ("user_code", ctypes.c_char_p),
+ ]
+
+
+class PQOAuthBearerRequest(ctypes.Structure):
+ pass
+
+
+PQOAuthBearerRequest._fields_ = [
+ ("openid_configuration", ctypes.c_char_p),
+ ("scope", ctypes.c_char_p),
+ (
+ "async_",
+ ctypes.CFUNCTYPE(
+ ctypes.c_int,
+ ctypes.c_void_p,
+ ctypes.POINTER(PQOAuthBearerRequest),
+ ctypes.POINTER(ctypes.c_int),
+ ),
+ ),
+ (
+ "cleanup",
+ ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(PQOAuthBearerRequest)),
+ ),
+ ("token", ctypes.c_char_p),
+ ("user", ctypes.c_void_p),
+]
+
+
+@pytest.fixture
+def auth_data_cb():
+ """
+ Tracks calls to the libpq authdata hook. The yielded object contains a calls
+ member that records the data sent to the hook. If a test needs to perform
+ custom actions during a call, it can set the yielded object's impl callback;
+ beware that the callback takes place on a different thread.
+
+ This is done differently from the other callback implementations on purpose.
+ For the others, we can declare test-specific callbacks and have them perform
+ direct assertions on the data they receive. But that won't work for a C
+ callback, because there's no way for us to bubble up the assertion through
+ libpq. Instead, this mock-style approach is taken, where we just record the
+ calls and let the test examine them later.
+ """
+
+ class _Call:
+ pass
+
+ class _cb(object):
+ def __init__(self):
+ self.calls = []
+
+ cb = _cb()
+ cb.impl = None
+
+ # The callback will occur on a different thread, so protect the cb object.
+ cb_lock = threading.Lock()
+
+ @ctypes.CFUNCTYPE(ctypes.c_int, ctypes.c_byte, ctypes.c_void_p, ctypes.c_void_p)
+ def auth_data_cb(typ, pgconn, data):
+ handle_by_default = 0 # does an implementation have to be provided?
+
+ if typ == PQAUTHDATA_PROMPT_OAUTH_DEVICE:
+ cls = PQPromptOAuthDevice
+ handle_by_default = 1
+ elif typ == PQAUTHDATA_OAUTH_BEARER_TOKEN:
+ cls = PQOAuthBearerRequest
+ else:
+ return 0
+
+ call = _Call()
+ call.type = typ
+
+ # The lifetime of the underlying data being pointed to doesn't
+ # necessarily match the lifetime of the Python object, so we can't
+ # reference a Structure's fields after returning. Explicitly copy the
+ # contents over, field by field.
+ data = ctypes.cast(data, ctypes.POINTER(cls))
+ for name, _ in cls._fields_:
+ setattr(call, name, getattr(data.contents, name))
+
+ with cb_lock:
+ cb.calls.append(call)
+
+ if cb.impl:
+ # Pass control back to the test.
+ try:
+ return cb.impl(typ, pgconn, data.contents)
+ except Exception:
+ # This can't escape into the C stack, but we can fail the flow
+ # and hope the traceback gives us enough detail.
+ logging.error(
+ "Exception during authdata hook callback:\n"
+ + traceback.format_exc()
+ )
+ return -1
+
+ return handle_by_default
+
+ libpq.PQsetAuthDataHook(auth_data_cb)
+ try:
+ yield cb
+ finally:
+ # The callback is about to go out of scope, so make sure libpq is
+ # disconnected from it. (We wouldn't want to accidentally influence
+ # later tests anyway.)
+ libpq.PQsetAuthDataHook(None)
+
+
+@pytest.mark.parametrize("secret", [None, "", "hunter2"])
+@pytest.mark.parametrize("scope", [None, "", "openid email"])
+@pytest.mark.parametrize("retries", [0, 1])
+@pytest.mark.parametrize(
+ "asynchronous",
+ [
+ pytest.param(False, id="synchronous"),
+ pytest.param(True, id="asynchronous"),
+ ],
+)
+def test_oauth_with_explicit_issuer(
+ accept, openid_provider, asynchronous, retries, scope, secret, auth_data_cb
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ oauth_client_secret=secret,
+ oauth_scope=scope,
+ async_=asynchronous,
+ )
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+ verification_url = "https://example.com/device"
+
+ access_token = secrets.token_urlsafe()
+
+ def check_client_authn(headers, params):
+ if not secret:
+ assert params["client_id"] == [client_id]
+ return
+
+ # Require the client to use Basic authn; request-body credentials are
+ # NOT RECOMMENDED (RFC 6749, Sec. 2.3.1).
+ assert "Authorization" in headers
+
+ method, creds = headers["Authorization"].split()
+ assert method == "Basic"
+
+ expected = f"{client_id}:{secret}"
+ assert base64.b64decode(creds) == expected.encode("ascii")
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ check_client_authn(headers, params)
+
+ if scope:
+ assert params["scope"] == [scope]
+ else:
+ assert "scope" not in params
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": verification_url,
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ attempts = 0
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ check_client_authn(headers, params)
+
+ assert params["grant_type"] == ["urn:ietf:params:oauth:grant-type:device_code"]
+ assert params["device_code"] == [device_code]
+
+ now = time.monotonic()
+
+ with retry_lock:
+ nonlocal attempts
+
+ # If the test wants to force the client to retry, return an
+ # authorization_pending response and decrement the retry count.
+ if attempts < retries:
+ attempts += 1
+ return 400, {"error": "authorization_pending"}
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+ if retries:
+ # Finally, make sure that the client prompted the user once with the
+ # expected authorization URL and user code.
+ assert len(auth_data_cb.calls) == 2
+
+ # First call should have been for a custom flow, which we ignored.
+ assert auth_data_cb.calls[0].type == PQAUTHDATA_OAUTH_BEARER_TOKEN
+
+ # Second call is for our user prompt.
+ call = auth_data_cb.calls[1]
+ assert call.type == PQAUTHDATA_PROMPT_OAUTH_DEVICE
+ assert call.verification_uri.decode() == verification_url
+ assert call.user_code.decode() == user_code
+
+
+def expect_disconnected_handshake(sock):
+ """
+ Helper for any tests that expect the client to disconnect immediately after
+ being sent the OAUTHBEARER SASL method. Generally speaking, this requires
+ the client to have an oauth_issuer set so that it doesn't try to go through
+ discovery.
+ """
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake.
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"OAUTHBEARER", b""],
+ )
+
+ # The client should disconnect at this point.
+ assert not conn.read()
+
+
+def test_oauth_requires_client_id(accept, openid_provider):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ # Do not set a client ID; this should cause a client error after the
+ # server asks for OAUTHBEARER and the client tries to contact the
+ # issuer.
+ )
+
+ expect_disconnected_handshake(sock)
+
+ expected_error = "no oauth_client_id is set"
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.slow
+@pytest.mark.parametrize("error_code", ["authorization_pending", "slow_down"])
+@pytest.mark.parametrize("retries", [1, 2])
+def test_oauth_retry_interval(accept, openid_provider, retries, error_code):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id="some-id",
+ )
+
+ expected_retry_interval = 1
+ access_token = secrets.token_urlsafe()
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ resp = {
+ "device_code": "my-device-code",
+ "user_code": "my-user-code",
+ "interval": expected_retry_interval,
+ "verification_uri": "https://example.com",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ attempts = 0
+ last_retry = None
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ now = time.monotonic()
+
+ with retry_lock:
+ nonlocal attempts, last_retry, expected_retry_interval
+
+ # Make sure the retry interval is being respected by the client.
+ if last_retry is not None:
+ interval = now - last_retry
+ assert interval >= expected_retry_interval
+
+ last_retry = now
+
+ # If the test wants to force the client to retry, return the desired
+ # error response and decrement the retry count.
+ if attempts < retries:
+ attempts += 1
+
+ # A slow_down code requires the client to additionally increase
+ # its interval by five seconds.
+ if error_code == "slow_down":
+ expected_retry_interval += 5
+
+ return 400, {"error": error_code}
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+@pytest.fixture
+def self_pipe():
+ """
+ Yields a pipe fd pair.
+ """
+
+ class _Pipe:
+ pass
+
+ p = _Pipe()
+ p.readfd, p.writefd = os.pipe()
+
+ try:
+ yield p
+ finally:
+ os.close(p.readfd)
+ os.close(p.writefd)
+
+
+@pytest.mark.parametrize("scope", [None, "", "openid email"])
+@pytest.mark.parametrize(
+ "retries",
+ [
+ -1, # no async callback
+ 0, # async callback immediately returns token
+ 1, # async callback waits on altsock once
+ 2, # async callback waits on altsock twice
+ ],
+)
+@pytest.mark.parametrize(
+ "asynchronous",
+ [
+ pytest.param(False, id="synchronous"),
+ pytest.param(True, id="asynchronous"),
+ ],
+)
+def test_user_defined_flow(
+ accept, auth_data_cb, self_pipe, scope, retries, asynchronous
+):
+ issuer = "http://localhost"
+ discovery_uri = issuer + "/.well-known/openid-configuration"
+ access_token = secrets.token_urlsafe()
+
+ sock, _ = accept(
+ oauth_issuer=issuer,
+ oauth_client_id="some-id",
+ oauth_scope=scope,
+ async_=asynchronous,
+ )
+
+ # Track callbacks.
+ attempts = 0
+ wakeup_called = False
+ cleanup_calls = 0
+ lock = threading.Lock()
+
+ def wakeup():
+ """Writes a byte to the wakeup pipe."""
+ nonlocal wakeup_called
+ with lock:
+ wakeup_called = True
+ os.write(self_pipe.writefd, b"\0")
+
+ def get_token(pgconn, request, p_altsock):
+ """
+ Async token callback. While attempts < retries, libpq will be instructed
+ to wait on the self_pipe. When attempts == retries, the token will be
+ set.
+
+ Note that assertions and exceptions raised here are allowed but not very
+ helpful, since they can't bubble through the libpq stack to be collected
+ by the test suite. Try not to rely too heavily on them.
+ """
+ # Make sure libpq passed our user data through.
+ assert request.user == 42
+
+ with lock:
+ nonlocal attempts, wakeup_called
+
+ if attempts:
+ # If we've already started the timer, we shouldn't get a
+ # call back before it trips.
+ assert wakeup_called, "authdata hook was called before the timer"
+
+ # Drain the wakeup byte.
+ os.read(self_pipe.readfd, 1)
+
+ if attempts < retries:
+ attempts += 1
+
+ # Wake up the client in a little bit of time.
+ wakeup_called = False
+ threading.Timer(0.1, wakeup).start()
+
+ # Tell libpq to wait on the other end of the wakeup pipe.
+ p_altsock[0] = self_pipe.readfd
+ return PGRES_POLLING_READING
+
+ # Done!
+ request.token = access_token.encode()
+ return PGRES_POLLING_OK
+
+ @ctypes.CFUNCTYPE(
+ ctypes.c_int,
+ ctypes.c_void_p,
+ ctypes.POINTER(PQOAuthBearerRequest),
+ ctypes.POINTER(ctypes.c_int),
+ )
+ def get_token_wrapper(pgconn, p_request, p_altsock):
+ """
+ Translation layer between C and Python for the async callback.
+ Assertions and exceptions will be swallowed at the boundary, so make
+ sure they don't escape here.
+ """
+ try:
+ return get_token(pgconn, p_request.contents, p_altsock)
+ except Exception:
+ logging.error("Exception during async callback:\n" + traceback.format_exc())
+ return PGRES_POLLING_FAILED
+
+ @ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(PQOAuthBearerRequest))
+ def cleanup(pgconn, p_request):
+ """
+ Should be called exactly once per connection.
+ """
+ nonlocal cleanup_calls
+ with lock:
+ cleanup_calls += 1
+
+ def bearer_hook(typ, pgconn, request):
+ """
+ Implementation of the PQAuthDataHook, which either sets up an async
+ callback or returns the token directly, depending on the value of
+ retries.
+
+ As above, try not to rely too much on assertions/exceptions here.
+ """
+ assert typ == PQAUTHDATA_OAUTH_BEARER_TOKEN
+ request.cleanup = cleanup
+
+ if retries < 0:
+ # Special case: return a token immediately without a callback.
+ request.token = access_token.encode()
+ return 1
+
+ # Tell libpq to call us back.
+ request.async_ = get_token_wrapper
+ request.user = ctypes.c_void_p(42) # will be checked in the callback
+ return 1
+
+ auth_data_cb.impl = bearer_hook
+
+ # Now drive the server side.
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in our custom callback
+ # being invoked to fetch the token.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+ # Check the data provided to the hook.
+ assert len(auth_data_cb.calls) == 1
+
+ call = auth_data_cb.calls[0]
+ assert call.type == PQAUTHDATA_OAUTH_BEARER_TOKEN
+ assert call.openid_configuration.decode() == discovery_uri
+ assert call.scope == (None if scope is None else scope.encode())
+
+ # Make sure we cleaned up after ourselves.
+ assert cleanup_calls == 1
+
+
+def alt_patterns(*patterns):
+ """
+ Just combines multiple alternative regexes into one. It's not very efficient
+ but IMO it's easier to read and maintain.
+ """
+ pat = ""
+
+ for p in patterns:
+ if pat:
+ pat += "|"
+ pat += f"({p})"
+
+ return pat
+
+
+@pytest.mark.parametrize(
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
+ (
+ 400,
+ {
+ "error": "invalid_client",
+ "error_description": "client authentication failed",
+ },
+ ),
+ r"failed to obtain device authorization: client authentication failed \(invalid_client\)",
+ id="authentication failure with description",
+ ),
+ pytest.param(
+ (400, {"error": "invalid_request"}),
+ r"failed to obtain device authorization: \(invalid_request\)",
+ id="invalid request without description",
+ ),
+ pytest.param(
+ (400, {}),
+ alt_patterns(
+ r'failed to parse token error response: field "error" is missing',
+ r"failed to obtain device authorization: \(iddawc error I_ERROR_PARAM\)",
+ ),
+ id="broken error response",
+ ),
+ pytest.param(
+ (200, RawResponse(r'{ "interval": 3.5.8 }')),
+ alt_patterns(
+ r"failed to parse device authorization: Token .* is invalid",
+ r"failed to obtain device authorization: \(iddawc error I_ERROR\)",
+ ),
+ id="non-numeric interval",
+ ),
+ pytest.param(
+ (200, RawResponse(r'{ "interval": 08 }')),
+ alt_patterns(
+ r"failed to parse device authorization: Token .* is invalid",
+ r"failed to obtain device authorization: \(iddawc error I_ERROR\)",
+ ),
+ id="invalid numeric interval",
+ ),
+ ],
+)
+def test_oauth_device_authorization_failures(
+ accept, openid_provider, failure_mode, error_pattern
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ )
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ return failure_mode
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert False, "token endpoint was invoked unexpectedly"
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+Missing = object() # sentinel for test_oauth_device_authorization_bad_json()
+
+
+@pytest.mark.parametrize(
+ "bad_value",
+ [
+ pytest.param({"device_code": 3}, id="object"),
+ pytest.param([1, 2, 3], id="array"),
+ pytest.param("some string", id="string"),
+ pytest.param(4, id="numeric"),
+ pytest.param(False, id="boolean"),
+ pytest.param(None, id="null"),
+ pytest.param(Missing, id="missing"),
+ ],
+)
+@pytest.mark.parametrize(
+ "field_name,ok_type,required",
+ [
+ ("device_code", str, True),
+ ("user_code", str, True),
+ ("verification_uri", str, True),
+ ("interval", int, False),
+ ],
+)
+def test_oauth_device_authorization_bad_json_schema(
+ accept, openid_provider, field_name, ok_type, required, bad_value
+):
+ # To make the test matrix easy, just skip the tests that aren't actually
+ # interesting (field of the correct type, missing optional field).
+ if bad_value is Missing and not required:
+ pytest.skip("not interesting: optional field")
+ elif type(bad_value) == ok_type: # not isinstance(), because bool is an int
+ pytest.skip("not interesting: correct type")
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=secrets.token_hex(),
+ )
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ # Begin with an acceptable base response...
+ resp = {
+ "device_code": "my-device-code",
+ "user_code": "my-user-code",
+ "interval": 0,
+ "verification_uri": "https://example.com",
+ "expires_in": 5,
+ }
+
+ # ...then tweak it so the client fails.
+ if bad_value is Missing:
+ del resp[field_name]
+ else:
+ resp[field_name] = bad_value
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert False, "token endpoint was invoked unexpectedly"
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ if bad_value is Missing:
+ error_pattern = f'field "{field_name}" is missing'
+ elif ok_type == str:
+ error_pattern = f'field "{field_name}" must be a string'
+ elif ok_type == int:
+ error_pattern = f'field "{field_name}" must be a number'
+ else:
+ assert False, "update error_pattern for new failure mode"
+
+ # XXX iddawc doesn't really check for problems in the device authorization
+ # response, leading to this patchwork:
+ if field_name == "verification_uri":
+ error_pattern = alt_patterns(
+ error_pattern,
+ "issuer did not provide a verification URI",
+ )
+ elif field_name == "user_code":
+ error_pattern = alt_patterns(
+ error_pattern,
+ "issuer did not provide a user code",
+ )
+ else:
+ error_pattern = alt_patterns(
+ error_pattern,
+ r"failed to obtain access token: \(iddawc error I_ERROR_PARAM\)",
+ )
+
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
+ (
+ 400,
+ {
+ "error": "expired_token",
+ "error_description": "the device code has expired",
+ },
+ ),
+ r"failed to obtain access token: the device code has expired \(expired_token\)",
+ id="expired token with description",
+ ),
+ pytest.param(
+ (400, {"error": "access_denied"}),
+ r"failed to obtain access token: \(access_denied\)",
+ id="access denied without description",
+ ),
+ pytest.param(
+ (400, {}),
+ alt_patterns(
+ r'failed to parse token error response: field "error" is missing',
+ r"failed to obtain access token: \(iddawc error I_ERROR_PARAM\)",
+ ),
+ id="empty error response",
+ ),
+ pytest.param(
+ (200, {}, {}),
+ alt_patterns(
+ r"failed to parse access token response: no content type was provided",
+ r"failed to obtain access token: \(iddawc error I_ERROR\)",
+ ),
+ id="missing content type",
+ ),
+ pytest.param(
+ (200, {"Content-Type": "text/plain"}, {}),
+ alt_patterns(
+ r"failed to parse access token response: unexpected content type",
+ r"failed to obtain access token: \(iddawc error I_ERROR\)",
+ ),
+ id="wrong content type",
+ ),
+ ],
+)
+@pytest.mark.parametrize("retries", [0, 1])
+def test_oauth_token_failures(
+ accept, openid_provider, retries, failure_mode, error_pattern
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ )
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ assert params["client_id"] == [client_id]
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": "https://example.com/device",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ retry_lock = threading.Lock()
+ final_sent = False
+
+ def token_endpoint(headers, params):
+ with retry_lock:
+ nonlocal retries, final_sent
+
+ # If the test wants to force the client to retry, return an
+ # authorization_pending response and decrement the retry count.
+ if retries > 0:
+ retries -= 1
+ return 400, {"error": "authorization_pending"}
+
+ # We should only return our failure_mode response once; any further
+ # requests indicate that the client isn't correctly bailing out.
+ assert not final_sent, "client continued after token error"
+
+ final_sent = True
+
+ return failure_mode
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "bad_value",
+ [
+ pytest.param({"device_code": 3}, id="object"),
+ pytest.param([1, 2, 3], id="array"),
+ pytest.param("some string", id="string"),
+ pytest.param(4, id="numeric"),
+ pytest.param(False, id="boolean"),
+ pytest.param(None, id="null"),
+ pytest.param(Missing, id="missing"),
+ ],
+)
+@pytest.mark.parametrize(
+ "field_name,ok_type,required",
+ [
+ ("access_token", str, True),
+ ("token_type", str, True),
+ ],
+)
+def test_oauth_token_bad_json_schema(
+ accept, openid_provider, field_name, ok_type, required, bad_value
+):
+ # To make the test matrix easy, just skip the tests that aren't actually
+ # interesting (field of the correct type, missing optional field).
+ if bad_value is Missing and not required:
+ pytest.skip("not interesting: optional field")
+ elif type(bad_value) == ok_type: # not isinstance(), because bool is an int
+ pytest.skip("not interesting: correct type")
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=secrets.token_hex(),
+ )
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ resp = {
+ "device_code": "my-device-code",
+ "user_code": "my-user-code",
+ "interval": 0,
+ "verification_uri": "https://example.com",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ # Begin with an acceptable base response...
+ resp = {
+ "access_token": secrets.token_urlsafe(),
+ "token_type": "bearer",
+ }
+
+ # ...then tweak it so the client fails.
+ if bad_value is Missing:
+ del resp[field_name]
+ else:
+ resp[field_name] = bad_value
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ error_pattern = "failed to parse access token response: "
+ if bad_value is Missing:
+ error_pattern += f'field "{field_name}" is missing'
+ elif ok_type == str:
+ error_pattern += f'field "{field_name}" must be a string'
+ elif ok_type == int:
+ error_pattern += f'field "{field_name}" must be a number'
+ else:
+ assert False, "update error_pattern for new failure mode"
+
+ # XXX iddawc is fairly silent on the topic.
+ error_pattern = alt_patterns(
+ error_pattern,
+ r"failed to obtain access token: \(iddawc error I_ERROR_PARAM\)",
+ )
+
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize("scope", [None, "openid email"])
+@pytest.mark.parametrize(
+ "base_response",
+ [
+ {"status": "invalid_token"},
+ {"extra_object": {"key": "value"}, "status": "invalid_token"},
+ {"extra_object": {"status": 1}, "status": "invalid_token"},
+ ],
+)
+def test_oauth_discovery(accept, openid_provider, base_response, scope):
+ sock, client = accept(oauth_client_id=secrets.token_hex())
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+ verification_url = "https://example.com/device"
+
+ access_token = secrets.token_urlsafe()
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ if scope:
+ assert params["scope"] == [scope]
+ else:
+ assert "scope" not in params
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": verification_url,
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert params["grant_type"] == ["urn:ietf:params:oauth:grant-type:device_code"]
+ assert params["device_code"] == [device_code]
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # For discovery, the client should send an empty auth header. See
+ # RFC 7628, Sec. 4.3.
+ auth = get_auth_value(initial)
+ assert auth == b""
+
+ # We will fail the first SASL exchange. First return a link to the
+ # discovery document, pointing to the test provider server.
+ resp = dict(base_response)
+
+ discovery_uri = f"{openid_provider.issuer}/.well-known/openid-configuration"
+ resp["openid-configuration"] = discovery_uri
+
+ if scope:
+ resp["scope"] = scope
+
+ resp = json.dumps(resp)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASLContinue,
+ body=resp.encode("ascii"),
+ )
+
+ # Per RFC, the client is required to send a dummy ^A response.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+ assert pkt.payload == b"\x01"
+
+ # Now fail the SASL exchange.
+ pq3.send(
+ conn,
+ pq3.types.ErrorResponse,
+ fields=[
+ b"SFATAL",
+ b"C28000",
+ b"Mdoesn't matter",
+ b"",
+ ],
+ )
+
+ # The client will connect to us a second time, using the parameters we sent
+ # it.
+ sock, _ = accept()
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+@pytest.mark.parametrize(
+ "response,expected_error",
+ [
+ pytest.param(
+ "abcde",
+ 'Token "abcde" is invalid',
+ id="bad JSON: invalid syntax",
+ ),
+ pytest.param(
+ '"abcde"',
+ "top-level element must be an object",
+ id="bad JSON: top-level element is a string",
+ ),
+ pytest.param(
+ "[]",
+ "top-level element must be an object",
+ id="bad JSON: top-level element is an array",
+ ),
+ pytest.param(
+ "{}",
+ "server sent error response without a status",
+ id="bad JSON: no status member",
+ ),
+ pytest.param(
+ '{ "status": null }',
+ 'field "status" must be a string',
+ id="bad JSON: null status member",
+ ),
+ pytest.param(
+ '{ "status": 0 }',
+ 'field "status" must be a string',
+ id="bad JSON: int status member",
+ ),
+ pytest.param(
+ '{ "status": [ "bad" ] }',
+ 'field "status" must be a string',
+ id="bad JSON: array status member",
+ ),
+ pytest.param(
+ '{ "status": { "bad": "bad" } }',
+ 'field "status" must be a string',
+ id="bad JSON: object status member",
+ ),
+ pytest.param(
+ '{ "nested": { "status": "bad" } }',
+ "server sent error response without a status",
+ id="bad JSON: nested status",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token" ',
+ "The input string ended unexpectedly",
+ id="bad JSON: unterminated object",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token" } { }',
+ 'Expected end of input, but found "{"',
+ id="bad JSON: trailing data",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "openid-configuration": 1 }',
+ 'field "openid-configuration" must be a string',
+ id="bad JSON: int openid-configuration member",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "openid-configuration": 1 }',
+ 'field "openid-configuration" must be a string',
+ id="bad JSON: int openid-configuration member",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "scope": 1 }',
+ 'field "scope" must be a string',
+ id="bad JSON: int scope member",
+ ),
+ ],
+)
+def test_oauth_discovery_server_error(accept, response, expected_error):
+ sock, client = accept(oauth_client_id=secrets.token_hex())
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # Fail the SASL exchange with an invalid JSON response.
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASLContinue,
+ body=response.encode("utf-8"),
+ )
+
+ # The client should disconnect, so the socket is closed here. (If
+ # the client doesn't disconnect, it will report a different error
+ # below and the test will fail.)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "bad_response,expected_error",
+ [
+ pytest.param(
+ (200, {"Content-Type": "text/plain"}, {}),
+ r'failed to parse OpenID discovery document: unexpected content type "text/plain"',
+ id="not JSON",
+ ),
+ pytest.param(
+ (200, {}, {}),
+ r"failed to parse OpenID discovery document: no content type was provided",
+ id="no Content-Type",
+ ),
+ pytest.param(
+ (204, {}, None),
+ r"failed to fetch OpenID discovery document: unexpected response code 204",
+ id="no content",
+ ),
+ pytest.param(
+ (301, {"Location": "https://localhost/"}, None),
+ r"failed to fetch OpenID discovery document: unexpected response code 301",
+ id="redirection",
+ ),
+ pytest.param(
+ (404, {}),
+ r"failed to fetch OpenID discovery document: unexpected response code 404",
+ id="not found",
+ ),
+ pytest.param(
+ (200, RawResponse("blah\x00blah")),
+ r"failed to parse OpenID discovery document: response contains embedded NULLs",
+ id="NULL bytes in document",
+ ),
+ pytest.param(
+ (200, 123),
+ r"failed to parse OpenID discovery document: top-level element must be an object",
+ id="scalar at top level",
+ ),
+ pytest.param(
+ (200, []),
+ r"failed to parse OpenID discovery document: top-level element must be an object",
+ id="array at top level",
+ ),
+ pytest.param(
+ (200, RawResponse("{")),
+ r"failed to parse OpenID discovery document.* input string ended unexpectedly",
+ id="unclosed object",
+ ),
+ pytest.param(
+ (200, RawResponse(r'{ "hello": ] }')),
+ r"failed to parse OpenID discovery document.* Expected JSON value",
+ id="bad array",
+ ),
+ pytest.param(
+ (200, {"issuer": 123}),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="non-string issuer",
+ ),
+ pytest.param(
+ (200, {"issuer": ["something"]}),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="issuer array",
+ ),
+ pytest.param(
+ (200, {"issuer": {}}),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="issuer object",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": 123}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="scalar grant types field",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": {}}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="object grant types field",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": [123]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="non-string grant types",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": ["something", 123]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="non-string grant types later in the list",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": ["something", {}]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="object grant types later in the list",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": ["something", ["something"]]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="embedded array grant types later in the list",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "grant_types_supported": ["something"],
+ "token_endpoint": "https://example.com/",
+ "issuer": 123,
+ },
+ ),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="non-string issuer after other valid fields",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "ignored": {"grant_types_supported": 123, "token_endpoint": 123},
+ "issuer": 123,
+ },
+ ),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="non-string issuer after other ignored fields",
+ ),
+ pytest.param(
+ (200, {"token_endpoint": "https://example.com/"}),
+ r'failed to parse OpenID discovery document: field "issuer" is missing',
+ id="missing issuer",
+ ),
+ pytest.param(
+ (200, {"issuer": "https://example.com/"}),
+ r'failed to parse OpenID discovery document: field "token_endpoint" is missing',
+ id="missing token endpoint",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "issuer": "https://example.com",
+ "token_endpoint": "https://example.com/token",
+ "device_authorization_endpoint": "https://example.com/dev",
+ },
+ ),
+ r'cannot run OAuth device authorization: issuer "https://example.com" does not support device code grants',
+ id="missing device code grants",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "issuer": "https://example.com",
+ "token_endpoint": "https://example.com/token",
+ "grant_types_supported": [
+ "urn:ietf:params:oauth:grant-type:device_code"
+ ],
+ },
+ ),
+ r'cannot run OAuth device authorization: issuer "https://example.com" does not provide a device authorization endpoint',
+ id="missing device_authorization_endpoint",
+ ),
+ #
+ # Exercise HTTP-level failures by breaking the protocol. Note that the
+ # error messages here are implementation-dependent.
+ #
+ pytest.param(
+ (1000, {}),
+ r"failed to fetch OpenID discovery document: Unsupported protocol \(.*\)",
+ id="invalid HTTP response code",
+ ),
+ pytest.param(
+ (200, {"Content-Length": -1}, {}),
+ r"failed to fetch OpenID discovery document: Weird server reply \(.*Content-Length.*\)",
+ id="bad HTTP Content-Length",
+ ),
+ ],
+)
+def test_oauth_discovery_provider_failure(
+ accept, openid_provider, bad_response, expected_error
+):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=secrets.token_hex(),
+ )
+
+ def failing_discovery_handler(headers, params):
+ return bad_response
+
+ openid_provider.register_endpoint(
+ None,
+ "GET",
+ "/.well-known/openid-configuration",
+ failing_discovery_handler,
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # XXX iddawc doesn't differentiate...
+ expected_error = alt_patterns(
+ expected_error,
+ r"failed to fetch OpenID discovery document \(iddawc error I_ERROR(_PARAM)?\)",
+ )
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "sasl_err,resp_type,resp_payload,expected_error",
+ [
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.ErrorResponse,
+ dict(
+ fields=[b"SFATAL", b"C28000", b"Mexpected error message", b""],
+ ),
+ "expected error message",
+ id="standard server error: invalid_request",
+ ),
+ pytest.param(
+ {"status": "invalid_token"},
+ pq3.types.ErrorResponse,
+ dict(
+ fields=[b"SFATAL", b"C28000", b"Mexpected error message", b""],
+ ),
+ "expected error message",
+ id="standard server error: invalid_token without discovery URI",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASLContinue, body=b""),
+ "server sent additional OAuth data",
+ id="broken server: additional challenge after error",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASLFinal),
+ "server sent additional OAuth data",
+ id="broken server: SASL success after error",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASL, body=[b"OAUTHBEARER", b""]),
+ "duplicate SASL authentication request",
+ id="broken server: SASL reinitialization after error",
+ ),
+ ],
+)
+def test_oauth_server_error(accept, sasl_err, resp_type, resp_payload, expected_error):
+ sock, client = accept()
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ start_oauth_handshake(conn)
+
+ # Ignore the client data. Return an error "challenge".
+ resp = json.dumps(sasl_err)
+ resp = resp.encode("utf-8")
+
+ pq3.send(
+ conn, pq3.types.AuthnRequest, type=pq3.authn.SASLContinue, body=resp
+ )
+
+ # Per RFC, the client is required to send a dummy ^A response.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+ assert pkt.payload == b"\x01"
+
+ # Now fail the SASL exchange (in either a valid way, or an invalid
+ # one, depending on the test).
+ pq3.send(conn, resp_type, **resp_payload)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
diff --git a/src/test/python/conftest.py b/src/test/python/conftest.py
new file mode 100644
index 0000000000..1a73865ee4
--- /dev/null
+++ b/src/test/python/conftest.py
@@ -0,0 +1,34 @@
+#
+# Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import os
+
+import pytest
+
+
+def pytest_addoption(parser):
+ """
+ Adds custom command line options to py.test. We add one to signal temporary
+ Postgres instance creation for the server tests.
+
+ Per pytest documentation, this must live in the top level test directory.
+ """
+ parser.addoption(
+ "--temp-instance",
+ metavar="DIR",
+ help="create a temporary Postgres instance in DIR",
+ )
+
+
+@pytest.fixture(scope="session", autouse=True)
+def _check_PG_TEST_EXTRA(request):
+ """
+ Automatically skips the whole suite if PG_TEST_EXTRA doesn't contain
+ 'python'. pytestmark doesn't seem to work in a top-level conftest.py, so
+ I've made this an autoused fixture instead.
+ """
+ extra_tests = os.getenv("PG_TEST_EXTRA", "").split()
+ if "python" not in extra_tests:
+ pytest.skip("Potentially unsafe test 'python' not enabled in PG_TEST_EXTRA")
diff --git a/src/test/python/meson.build b/src/test/python/meson.build
new file mode 100644
index 0000000000..045526a43a
--- /dev/null
+++ b/src/test/python/meson.build
@@ -0,0 +1,46 @@
+# Copyright (c) 2023, PostgreSQL Global Development Group
+
+subdir('server')
+
+pytest_env = {
+ 'with_oauth': oauth_library,
+
+ # Point to the default database; the tests will create their own databases as
+ # needed.
+ 'PGDATABASE': 'postgres',
+
+ # Avoid the need for a Rust compiler on platforms without prebuilt wheels for
+ # pyca/cryptography.
+ 'CRYPTOGRAPHY_DONT_BUILD_RUST': '1',
+}
+
+# Some modules (psycopg2) need OpenSSL at compile time; for platforms where we
+# might have multiple implementations installed (macOS+brew), try to use the
+# same one that libpq is using.
+if ssl.found()
+ pytest_incdir = ssl.get_variable(pkgconfig: 'includedir', default_value: '')
+ if pytest_incdir != ''
+ pytest_env += { 'CPPFLAGS': '-I@0@'.format(pytest_incdir) }
+ endif
+
+ pytest_libdir = ssl.get_variable(pkgconfig: 'libdir', default_value: '')
+ if pytest_libdir != ''
+ pytest_env += { 'LDFLAGS': '-L@0@'.format(pytest_libdir) }
+ endif
+endif
+
+tests += {
+ 'name': 'python',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'pytest': {
+ 'requirements': meson.current_source_dir() / 'requirements.txt',
+ 'tests': [
+ './client',
+ './server',
+ './test_internals.py',
+ './test_pq3.py',
+ ],
+ 'env': pytest_env,
+ },
+}
diff --git a/src/test/python/pq3.py b/src/test/python/pq3.py
new file mode 100644
index 0000000000..a2d2812f0e
--- /dev/null
+++ b/src/test/python/pq3.py
@@ -0,0 +1,732 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import getpass
+import io
+import os
+import platform
+import ssl
+import sys
+import textwrap
+
+from construct import *
+
+import tls
+
+
+def protocol(major, minor):
+ """
+ Returns the protocol version, in integer format, corresponding to the given
+ major and minor version numbers.
+ """
+ return (major << 16) | minor
+
+
+# Startup
+
+StringList = GreedyRange(NullTerminated(GreedyBytes))
+
+
+class KeyValueAdapter(Adapter):
+ """
+ Turns a key-value store into a null-terminated list of null-terminated
+ strings, as presented on the wire in the startup packet.
+ """
+
+ def _encode(self, obj, context, path):
+ if isinstance(obj, list):
+ return obj
+
+ l = []
+
+ for k, v in obj.items():
+ if isinstance(k, str):
+ k = k.encode("utf-8")
+ l.append(k)
+
+ if isinstance(v, str):
+ v = v.encode("utf-8")
+ l.append(v)
+
+ l.append(b"")
+ return l
+
+ def _decode(self, obj, context, path):
+ # TODO: turn a list back into a dict
+ return obj
+
+
+KeyValues = KeyValueAdapter(StringList)
+
+_startup_payload = Switch(
+ this.proto,
+ {
+ protocol(3, 0): KeyValues,
+ },
+ default=GreedyBytes,
+)
+
+
+def _default_protocol(this):
+ try:
+ if isinstance(this.payload, (list, dict)):
+ return protocol(3, 0)
+ except AttributeError:
+ pass # no payload passed during build
+
+ return 0
+
+
+def _startup_payload_len(this):
+ """
+ The payload field has a fixed size based on the length of the packet. But
+ if the caller hasn't supplied an explicit length at build time, we have to
+ build the payload to figure out how long it is, which requires us to know
+ the length first... This function exists solely to break the cycle.
+ """
+ assert this._building, "_startup_payload_len() cannot be called during parsing"
+
+ try:
+ payload = this.payload
+ except AttributeError:
+ return 0 # no payload
+
+ if isinstance(payload, bytes):
+ # already serialized; just use the given length
+ return len(payload)
+
+ try:
+ proto = this.proto
+ except AttributeError:
+ proto = _default_protocol(this)
+
+ data = _startup_payload.build(payload, proto=proto)
+ return len(data)
+
+
+Startup = Struct(
+ "len" / Default(Int32sb, lambda this: _startup_payload_len(this) + 8),
+ "proto" / Default(Hex(Int32sb), _default_protocol),
+ "payload" / FixedSized(this.len - 8, Default(_startup_payload, b"")),
+)
+
+# Pq3
+
+
+# Adapted from construct.core.EnumIntegerString
+class EnumNamedByte:
+ def __init__(self, val, name):
+ self._val = val
+ self._name = name
+
+ def __int__(self):
+ return ord(self._val)
+
+ def __str__(self):
+ return "(enum) %s %r" % (self._name, self._val)
+
+ def __repr__(self):
+ return "EnumNamedByte(%r)" % self._val
+
+ def __eq__(self, other):
+ if isinstance(other, EnumNamedByte):
+ other = other._val
+ if not isinstance(other, bytes):
+ return NotImplemented
+
+ return self._val == other
+
+ def __hash__(self):
+ return hash(self._val)
+
+
+# Adapted from construct.core.Enum
+class ByteEnum(Adapter):
+ def __init__(self, **mapping):
+ super(ByteEnum, self).__init__(Byte)
+ self.namemapping = {k: EnumNamedByte(v, k) for k, v in mapping.items()}
+ self.decmapping = {v: EnumNamedByte(v, k) for k, v in mapping.items()}
+
+ def __getattr__(self, name):
+ if name in self.namemapping:
+ return self.decmapping[self.namemapping[name]]
+ raise AttributeError
+
+ def _decode(self, obj, context, path):
+ b = bytes([obj])
+ try:
+ return self.decmapping[b]
+ except KeyError:
+ return EnumNamedByte(b, "(unknown)")
+
+ def _encode(self, obj, context, path):
+ if isinstance(obj, int):
+ return obj
+ elif isinstance(obj, bytes):
+ return ord(obj)
+ return int(obj)
+
+
+types = ByteEnum(
+ ErrorResponse=b"E",
+ ReadyForQuery=b"Z",
+ Query=b"Q",
+ EmptyQueryResponse=b"I",
+ AuthnRequest=b"R",
+ PasswordMessage=b"p",
+ BackendKeyData=b"K",
+ CommandComplete=b"C",
+ ParameterStatus=b"S",
+ DataRow=b"D",
+ Terminate=b"X",
+)
+
+
+authn = Enum(
+ Int32ub,
+ OK=0,
+ SASL=10,
+ SASLContinue=11,
+ SASLFinal=12,
+)
+
+
+_authn_body = Switch(
+ this.type,
+ {
+ authn.OK: Terminated,
+ authn.SASL: StringList,
+ },
+ default=GreedyBytes,
+)
+
+
+def _data_len(this):
+ assert this._building, "_data_len() cannot be called during parsing"
+
+ if not hasattr(this, "data") or this.data is None:
+ return -1
+
+ return len(this.data)
+
+
+# The protocol reuses the PasswordMessage for several authentication response
+# types, and there's no good way to figure out which is which without keeping
+# state for the entire stream. So this is a separate Construct that can be
+# explicitly parsed/built by code that knows it's needed.
+SASLInitialResponse = Struct(
+ "name" / NullTerminated(GreedyBytes),
+ "len" / Default(Int32sb, lambda this: _data_len(this)),
+ "data"
+ / IfThenElse(
+ # Allow tests to explicitly pass an incorrect length during testing, by
+ # not enforcing a FixedSized during build. (The len calculation above
+ # defaults to the correct size.)
+ this._building,
+ Optional(GreedyBytes),
+ If(this.len != -1, Default(FixedSized(this.len, GreedyBytes), b"")),
+ ),
+ Terminated, # make sure the entire response is consumed
+)
+
+
+_column = FocusedSeq(
+ "data",
+ "len" / Default(Int32sb, lambda this: _data_len(this)),
+ "data" / If(this.len != -1, FixedSized(this.len, GreedyBytes)),
+)
+
+
+_payload_map = {
+ types.ErrorResponse: Struct("fields" / StringList),
+ types.ReadyForQuery: Struct("status" / Bytes(1)),
+ types.Query: Struct("query" / NullTerminated(GreedyBytes)),
+ types.EmptyQueryResponse: Terminated,
+ types.AuthnRequest: Struct("type" / authn, "body" / Default(_authn_body, b"")),
+ types.BackendKeyData: Struct("pid" / Int32ub, "key" / Hex(Int32ub)),
+ types.CommandComplete: Struct("tag" / NullTerminated(GreedyBytes)),
+ types.ParameterStatus: Struct(
+ "name" / NullTerminated(GreedyBytes), "value" / NullTerminated(GreedyBytes)
+ ),
+ types.DataRow: Struct("columns" / Default(PrefixedArray(Int16sb, _column), b"")),
+ types.Terminate: Terminated,
+}
+
+
+_payload = FocusedSeq(
+ "_payload",
+ "_payload"
+ / Switch(
+ this._.type,
+ _payload_map,
+ default=GreedyBytes,
+ ),
+ Terminated, # make sure every payload consumes the entire packet
+)
+
+
+def _payload_len(this):
+ """
+ See _startup_payload_len() for an explanation.
+ """
+ assert this._building, "_payload_len() cannot be called during parsing"
+
+ try:
+ payload = this.payload
+ except AttributeError:
+ return 0 # no payload
+
+ if isinstance(payload, bytes):
+ # already serialized; just use the given length
+ return len(payload)
+
+ data = _payload.build(payload, type=this.type)
+ return len(data)
+
+
+Pq3 = Struct(
+ "type" / types,
+ "len" / Default(Int32ub, lambda this: _payload_len(this) + 4),
+ "payload" / FixedSized(this.len - 4, Default(_payload, b"")),
+)
+
+
+# Environment
+
+
+def pghost():
+ return os.environ.get("PGHOST", default="localhost")
+
+
+def pgport():
+ return int(os.environ.get("PGPORT", default=5432))
+
+
+def pguser():
+ try:
+ return os.environ["PGUSER"]
+ except KeyError:
+ if platform.system() == "Windows":
+ # libpq defaults to GetUserName() on Windows.
+ return os.getlogin()
+ return getpass.getuser()
+
+
+def pgdatabase():
+ return os.environ.get("PGDATABASE", default="postgres")
+
+
+# Connections
+
+
+def _hexdump_translation_map():
+ """
+ For hexdumps. Translates any unprintable or non-ASCII bytes into '.'.
+ """
+ input = bytearray()
+
+ for i in range(128):
+ c = chr(i)
+
+ if not c.isprintable():
+ input += bytes([i])
+
+ input += bytes(range(128, 256))
+
+ return bytes.maketrans(input, b"." * len(input))
+
+
+class _DebugStream(object):
+ """
+ Wraps a file-like object and adds hexdumps of the read and write data. Call
+ end_packet() on a _DebugStream to write the accumulated hexdumps to the
+ output stream, along with the packet that was sent.
+ """
+
+ _translation_map = _hexdump_translation_map()
+
+ def __init__(self, stream, out=sys.stdout):
+ """
+ Creates a new _DebugStream wrapping the given stream (which must have
+ been created by wrap()). All attributes not provided by the _DebugStream
+ are delegated to the wrapped stream. out is the text stream to which
+ hexdumps are written.
+ """
+ self.raw = stream
+ self._out = out
+ self._rbuf = io.BytesIO()
+ self._wbuf = io.BytesIO()
+
+ def __getattr__(self, name):
+ return getattr(self.raw, name)
+
+ def __setattr__(self, name, value):
+ if name in ("raw", "_out", "_rbuf", "_wbuf"):
+ return object.__setattr__(self, name, value)
+
+ setattr(self.raw, name, value)
+
+ def read(self, *args, **kwargs):
+ buf = self.raw.read(*args, **kwargs)
+
+ self._rbuf.write(buf)
+ return buf
+
+ def write(self, b):
+ self._wbuf.write(b)
+ return self.raw.write(b)
+
+ def recv(self, *args):
+ buf = self.raw.recv(*args)
+
+ self._rbuf.write(buf)
+ return buf
+
+ def _flush(self, buf, prefix):
+ width = 16
+ hexwidth = width * 3 - 1
+
+ count = 0
+ buf.seek(0)
+
+ while True:
+ line = buf.read(16)
+
+ if not line:
+ if count:
+ self._out.write("\n") # separate the output block with a newline
+ return
+
+ self._out.write("%s %04X:\t" % (prefix, count))
+ self._out.write("%*s\t" % (-hexwidth, line.hex(" ")))
+ self._out.write(line.translate(self._translation_map).decode("ascii"))
+ self._out.write("\n")
+
+ count += 16
+
+ def print_debug(self, obj, *, prefix=""):
+ contents = ""
+ if obj is not None:
+ contents = str(obj)
+
+ for line in contents.splitlines():
+ self._out.write("%s%s\n" % (prefix, line))
+
+ self._out.write("\n")
+
+ def flush_debug(self, *, prefix=""):
+ self._flush(self._rbuf, prefix + "<")
+ self._rbuf = io.BytesIO()
+
+ self._flush(self._wbuf, prefix + ">")
+ self._wbuf = io.BytesIO()
+
+ def end_packet(self, pkt, *, read=False, prefix="", indent=" "):
+ """
+ Marks the end of a logical "packet" of data. A string representation of
+ pkt will be printed, and the debug buffers will be flushed with an
+ indent. All lines can be optionally prefixed.
+
+ If read is True, the packet representation is written after the debug
+ buffers; otherwise the default of False (meaning write) causes the
+ packet representation to be dumped first. This is meant to capture the
+ logical flow of layer translation.
+ """
+ write = not read
+
+ if write:
+ self.print_debug(pkt, prefix=prefix + "> ")
+
+ self.flush_debug(prefix=prefix + indent)
+
+ if read:
+ self.print_debug(pkt, prefix=prefix + "< ")
+
+
+@contextlib.contextmanager
+def wrap(socket, *, debug_stream=None):
+ """
+ Transforms a raw socket into a connection that can be used for Construct
+ building and parsing. The return value is a context manager and can be used
+ in a with statement.
+ """
+ # It is critical that buffering be disabled here, so that we can still
+ # manipulate the raw socket without desyncing the stream.
+ with socket.makefile("rwb", buffering=0) as sfile:
+ # Expose the original socket's recv() on the SocketIO object we return.
+ def recv(self, *args):
+ return socket.recv(*args)
+
+ sfile.recv = recv.__get__(sfile)
+
+ conn = sfile
+ if debug_stream:
+ conn = _DebugStream(conn, debug_stream)
+
+ try:
+ yield conn
+ finally:
+ if debug_stream:
+ conn.flush_debug(prefix="? ")
+
+
+def _send(stream, cls, obj):
+ debugging = hasattr(stream, "flush_debug")
+ out = io.BytesIO()
+
+ # Ideally we would build directly to the passed stream, but because we need
+ # to reparse the generated output for the debugging case, build to an
+ # intermediate BytesIO and send it instead.
+ cls.build_stream(obj, out)
+ buf = out.getvalue()
+
+ stream.write(buf)
+ if debugging:
+ pkt = cls.parse(buf)
+ stream.end_packet(pkt)
+
+ stream.flush()
+
+
+def send(stream, packet_type, payload_data=None, **payloadkw):
+ """
+ Sends a packet on the given pq3 connection. type is the pq3.types member
+ that should be assigned to the packet. If payload_data is given, it will be
+ used as the packet payload; otherwise the key/value pairs in payloadkw will
+ be the payload contents.
+ """
+ data = payloadkw
+
+ if payload_data is not None:
+ if payloadkw:
+ raise ValueError(
+ "payload_data and payload keywords may not be used simultaneously"
+ )
+
+ data = payload_data
+
+ _send(stream, Pq3, dict(type=packet_type, payload=data))
+
+
+def send_startup(stream, proto=None, **kwargs):
+ """
+ Sends a startup packet on the given pq3 connection. In most cases you should
+ use the handshake functions instead, which will do this for you.
+
+ By default, a protocol version 3 packet will be sent. This can be overridden
+ with the proto parameter.
+ """
+ pkt = {}
+
+ if proto is not None:
+ pkt["proto"] = proto
+ if kwargs:
+ pkt["payload"] = kwargs
+
+ _send(stream, Startup, pkt)
+
+
+def recv1(stream, *, cls=Pq3):
+ """
+ Receives a single pq3 packet from the given stream and returns it.
+ """
+ resp = cls.parse_stream(stream)
+
+ debugging = hasattr(stream, "flush_debug")
+ if debugging:
+ stream.end_packet(resp, read=True)
+
+ return resp
+
+
+def handshake(stream, **kwargs):
+ """
+ Performs a libpq v3 startup handshake. kwargs should contain the key/value
+ parameters to send to the server in the startup packet.
+ """
+ # Send our startup parameters.
+ send_startup(stream, **kwargs)
+
+ # Receive and dump packets until the server indicates it's ready for our
+ # first query.
+ while True:
+ resp = recv1(stream)
+ if resp is None:
+ raise RuntimeError("server closed connection during handshake")
+
+ if resp.type == types.ReadyForQuery:
+ return
+ elif resp.type == types.ErrorResponse:
+ raise RuntimeError(
+ f"received error response from peer: {resp.payload.fields!r}"
+ )
+
+
+# TLS
+
+
+class _TLSStream(object):
+ """
+ A file-like object that performs TLS encryption/decryption on a wrapped
+ stream. Differs from ssl.SSLSocket in that we have full visibility and
+ control over the TLS layer.
+ """
+
+ def __init__(self, stream, context):
+ self._stream = stream
+ self._debugging = hasattr(stream, "flush_debug")
+
+ self._in = ssl.MemoryBIO()
+ self._out = ssl.MemoryBIO()
+ self._ssl = context.wrap_bio(self._in, self._out)
+
+ def handshake(self):
+ try:
+ self._pump(lambda: self._ssl.do_handshake())
+ finally:
+ self._flush_debug(prefix="? ")
+
+ def read(self, *args):
+ return self._pump(lambda: self._ssl.read(*args))
+
+ def write(self, *args):
+ return self._pump(lambda: self._ssl.write(*args))
+
+ def _decode(self, buf):
+ """
+ Attempts to decode a buffer of TLS data into a packet representation
+ that can be printed.
+
+ TODO: handle buffers (and record fragments) that don't align with packet
+ boundaries.
+ """
+ end = len(buf)
+ bio = io.BytesIO(buf)
+
+ ret = io.StringIO()
+
+ while bio.tell() < end:
+ record = tls.Plaintext.parse_stream(bio)
+
+ if ret.tell() > 0:
+ ret.write("\n")
+ ret.write("[Record] ")
+ ret.write(str(record))
+ ret.write("\n")
+
+ if record.type == tls.ContentType.handshake:
+ record_cls = tls.Handshake
+ else:
+ continue
+
+ innerlen = len(record.fragment)
+ inner = io.BytesIO(record.fragment)
+
+ while inner.tell() < innerlen:
+ msg = record_cls.parse_stream(inner)
+
+ indented = "[Message] " + str(msg)
+ indented = textwrap.indent(indented, " ")
+
+ ret.write("\n")
+ ret.write(indented)
+ ret.write("\n")
+
+ return ret.getvalue()
+
+ def flush(self):
+ if not self._out.pending:
+ self._stream.flush()
+ return
+
+ buf = self._out.read()
+ self._stream.write(buf)
+
+ if self._debugging:
+ pkt = self._decode(buf)
+ self._stream.end_packet(pkt, prefix=" ")
+
+ self._stream.flush()
+
+ def _pump(self, operation):
+ while True:
+ try:
+ return operation()
+ except (ssl.SSLWantReadError, ssl.SSLWantWriteError) as e:
+ want = e
+ self._read_write(want)
+
+ def _recv(self, maxsize):
+ buf = self._stream.recv(4096)
+ if not buf:
+ self._in.write_eof()
+ return
+
+ self._in.write(buf)
+
+ if not self._debugging:
+ return
+
+ pkt = self._decode(buf)
+ self._stream.end_packet(pkt, read=True, prefix=" ")
+
+ def _read_write(self, want):
+ # XXX This needs work. So many corner cases yet to handle. For one,
+ # doing blocking writes in flush may lead to distributed deadlock if the
+ # peer is already blocking on its writes.
+
+ if isinstance(want, ssl.SSLWantWriteError):
+ assert self._out.pending, "SSL backend wants write without data"
+
+ self.flush()
+
+ if isinstance(want, ssl.SSLWantReadError):
+ self._recv(4096)
+
+ def _flush_debug(self, prefix):
+ if not self._debugging:
+ return
+
+ self._stream.flush_debug(prefix=prefix)
+
+
+@contextlib.contextmanager
+def tls_handshake(stream, context):
+ """
+ Performs a TLS handshake over the given stream (which must have been created
+ via a call to wrap()), and returns a new stream which transparently tunnels
+ data over the TLS connection.
+
+ If the passed stream has debugging enabled, the returned stream will also
+ have debugging, using the same output IO.
+ """
+ debugging = hasattr(stream, "flush_debug")
+
+ # Send our startup parameters.
+ send_startup(stream, proto=protocol(1234, 5679))
+
+ # Look at the SSL response.
+ resp = stream.read(1)
+ if debugging:
+ stream.flush_debug(prefix=" ")
+
+ if resp == b"N":
+ raise RuntimeError("server does not support SSLRequest")
+ if resp != b"S":
+ raise RuntimeError(f"unexpected response of type {resp!r} during TLS startup")
+
+ tls = _TLSStream(stream, context)
+ tls.handshake()
+
+ if debugging:
+ tls = _DebugStream(tls, stream._out)
+
+ try:
+ yield tls
+ # TODO: teardown/unwrap the connection?
+ finally:
+ if debugging:
+ tls.flush_debug(prefix="? ")
diff --git a/src/test/python/pytest.ini b/src/test/python/pytest.ini
new file mode 100644
index 0000000000..ab7a6e7fb9
--- /dev/null
+++ b/src/test/python/pytest.ini
@@ -0,0 +1,4 @@
+[pytest]
+
+markers =
+ slow: mark test as slow
diff --git a/src/test/python/requirements.txt b/src/test/python/requirements.txt
new file mode 100644
index 0000000000..57ba1ced94
--- /dev/null
+++ b/src/test/python/requirements.txt
@@ -0,0 +1,9 @@
+black
+# cryptography 35.x and later add many platform/toolchain restrictions, beware
+cryptography~=3.4.8
+construct~=2.10.61
+isort~=5.6
+# TODO: update to psycopg[c] 3.1
+psycopg2~=2.9.7
+pytest~=7.3
+pytest-asyncio~=0.21.0
diff --git a/src/test/python/server/__init__.py b/src/test/python/server/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/src/test/python/server/conftest.py b/src/test/python/server/conftest.py
new file mode 100644
index 0000000000..c89f0756ae
--- /dev/null
+++ b/src/test/python/server/conftest.py
@@ -0,0 +1,141 @@
+#
+# Portions Copyright 2021 VMware, Inc.
+# Portions Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import collections
+import contextlib
+import os
+import shutil
+import socket
+import subprocess
+import sys
+
+import pytest
+
+import pq3
+
+BLOCKING_TIMEOUT = 2 # the number of seconds to wait for blocking calls
+
+
+def cleanup_prior_instance(datadir):
+ """
+ Clean up an existing data directory, but make sure it actually looks like a
+ data directory first. (Empty folders will remain untouched, since initdb can
+ populate them.)
+ """
+ required_entries = set(["base", "PG_VERSION", "postgresql.conf"])
+ empty = True
+
+ try:
+ with os.scandir(datadir) as entries:
+ for e in entries:
+ empty = False
+ required_entries.discard(e.name)
+
+ except FileNotFoundError:
+ return # nothing to clean up
+
+ if empty:
+ return # initdb can handle an empty datadir
+
+ if required_entries:
+ pytest.fail(
+ f"--temp-instance directory \"{datadir}\" is not empty and doesn't look like a data directory (missing {', '.join(required_entries)})"
+ )
+
+ # Okay, seems safe enough now.
+ shutil.rmtree(datadir)
+
+
+@pytest.fixture(scope="session")
+def postgres_instance(pytestconfig, unused_tcp_port_factory):
+ """
+ If --temp-instance has been passed to pytest, this fixture runs a temporary
+ Postgres instance on an available port. Otherwise, the fixture will attempt
+ to contact a running Postgres server on (PGHOST, PGPORT); dependent tests
+ will be skipped if the connection fails.
+
+ Yields a (host, port) tuple for connecting to the server.
+ """
+ PGInstance = collections.namedtuple("PGInstance", ["addr", "temporary"])
+
+ datadir = pytestconfig.getoption("temp_instance")
+ if datadir:
+ # We were told to create a temporary instance. Use pg_ctl to set it up
+ # on an unused port.
+ cleanup_prior_instance(datadir)
+ subprocess.run(["pg_ctl", "-D", datadir, "init"], check=True)
+
+ # The CI looks for *.log files to upload, so the file name here isn't
+ # completely arbitrary.
+ log = os.path.join(datadir, "postmaster.log")
+ port = unused_tcp_port_factory()
+
+ subprocess.run(
+ [
+ "pg_ctl",
+ "-D",
+ datadir,
+ "-l",
+ log,
+ "-o",
+ " ".join(
+ [
+ f"-c port={port}",
+ "-c listen_addresses=localhost",
+ "-c log_connections=on",
+ "-c shared_preload_libraries=oauthtest",
+ "-c oauth_validator_library=oauthtest",
+ ]
+ ),
+ "start",
+ ],
+ check=True,
+ )
+
+ yield ("localhost", port)
+
+ subprocess.run(["pg_ctl", "-D", datadir, "stop"], check=True)
+
+ else:
+ # Try to contact an already running server; skip the suite if we can't
+ # find one.
+ addr = (pq3.pghost(), pq3.pgport())
+
+ try:
+ with socket.create_connection(addr, timeout=BLOCKING_TIMEOUT):
+ pass
+ except ConnectionError as e:
+ pytest.skip(f"unable to connect to Postgres server at {addr}: {e}")
+
+ yield addr
+
+
+@pytest.fixture
+def connect(postgres_instance):
+ """
+ A factory fixture that, when called, returns a socket connected to a
+ Postgres server, wrapped in a pq3 connection. Dependent tests will be
+ skipped if no server is available.
+ """
+ addr = postgres_instance
+
+ # Set up an ExitStack to handle safe cleanup of all of the moving pieces.
+ with contextlib.ExitStack() as stack:
+
+ def conn_factory():
+ sock = socket.create_connection(addr, timeout=BLOCKING_TIMEOUT)
+
+ # Have ExitStack close our socket.
+ stack.enter_context(sock)
+
+ # Wrap the connection in a pq3 layer and have ExitStack clean it up
+ # too.
+ wrap_ctx = pq3.wrap(sock, debug_stream=sys.stdout)
+ conn = stack.enter_context(wrap_ctx)
+
+ return conn
+
+ yield conn_factory
diff --git a/src/test/python/server/meson.build b/src/test/python/server/meson.build
new file mode 100644
index 0000000000..85534b9cc9
--- /dev/null
+++ b/src/test/python/server/meson.build
@@ -0,0 +1,18 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+oauthtest_sources = files(
+ 'oauthtest.c',
+)
+
+if host_system == 'windows'
+ oauthtest_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'oauthtest',
+ '--FILEDESC', 'passthrough module to validate OAuth tests',
+ ])
+endif
+
+oauthtest = shared_module('oauthtest',
+ oauthtest_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += oauthtest
diff --git a/src/test/python/server/oauthtest.c b/src/test/python/server/oauthtest.c
new file mode 100644
index 0000000000..f8e6c1651b
--- /dev/null
+++ b/src/test/python/server/oauthtest.c
@@ -0,0 +1,118 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauthtest.c
+ * Test module for serverside OAuth token validation callbacks
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/test/python/server/oauthtest.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "libpq/oauth.h"
+#include "utils/guc.h"
+#include "utils/memutils.h"
+
+PG_MODULE_MAGIC;
+
+static void test_startup(ValidatorModuleState *state);
+static void test_shutdown(ValidatorModuleState *state);
+static ValidatorModuleResult * test_validate(ValidatorModuleState *state,
+ const char *token,
+ const char *role);
+
+static const OAuthValidatorCallbacks callbacks = {
+ .startup_cb = test_startup,
+ .shutdown_cb = test_shutdown,
+ .validate_cb = test_validate,
+};
+
+static char *expected_bearer = "";
+static bool set_authn_id = false;
+static char *authn_id = "";
+static bool reflect_role = false;
+
+void
+_PG_init(void)
+{
+ DefineCustomStringVariable("oauthtest.expected_bearer",
+ "Expected Bearer token for future connections",
+ NULL,
+ &expected_bearer,
+ "",
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("oauthtest.set_authn_id",
+ "Whether to set an authenticated identity",
+ NULL,
+ &set_authn_id,
+ false,
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+ DefineCustomStringVariable("oauthtest.authn_id",
+ "Authenticated identity to use for future connections",
+ NULL,
+ &authn_id,
+ "",
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("oauthtest.reflect_role",
+ "Ignore the bearer token; use the requested role as the authn_id",
+ NULL,
+ &reflect_role,
+ false,
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+
+ MarkGUCPrefixReserved("oauthtest");
+}
+
+const OAuthValidatorCallbacks *
+_PG_oauth_validator_module_init(void)
+{
+ return &callbacks;
+}
+
+static void
+test_startup(ValidatorModuleState *state)
+{
+}
+
+static void
+test_shutdown(ValidatorModuleState *state)
+{
+}
+
+static ValidatorModuleResult *
+test_validate(ValidatorModuleState *state, const char *token, const char *role)
+{
+ ValidatorModuleResult *res;
+
+ res = palloc0(sizeof(ValidatorModuleResult)); /* TODO: palloc context? */
+
+ if (reflect_role)
+ {
+ res->authenticated = true;
+ res->authn_id = pstrdup(role); /* TODO: constify? */
+ }
+ else
+ {
+ if (*expected_bearer && !strcmp(token, expected_bearer))
+ res->authenticated = true;
+ if (set_authn_id)
+ res->authn_id = authn_id;
+ }
+
+ return res;
+}
diff --git a/src/test/python/server/test_oauth.py b/src/test/python/server/test_oauth.py
new file mode 100644
index 0000000000..2a2ca59e94
--- /dev/null
+++ b/src/test/python/server/test_oauth.py
@@ -0,0 +1,945 @@
+#
+# Copyright 2021 VMware, Inc.
+# Portions Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import contextlib
+import json
+import os
+import pathlib
+import platform
+import secrets
+import shlex
+import shutil
+import socket
+import struct
+from multiprocessing import shared_memory
+
+import psycopg2
+import pytest
+from psycopg2 import sql
+
+import pq3
+
+from .conftest import BLOCKING_TIMEOUT
+
+MAX_SASL_MESSAGE_LENGTH = 65535
+
+INVALID_AUTHORIZATION_ERRCODE = b"28000"
+PROTOCOL_VIOLATION_ERRCODE = b"08P01"
+FEATURE_NOT_SUPPORTED_ERRCODE = b"0A000"
+
+SHARED_MEM_NAME = "oauth-pytest"
+MAX_UINT16 = 2**16 - 1
+
+
+@contextlib.contextmanager
+def prepend_file(path, lines):
+ """
+ A context manager that prepends a file on disk with the desired lines of
+ text. When the context manager is exited, the file will be restored to its
+ original contents.
+ """
+ # First make a backup of the original file.
+ bak = path + ".bak"
+ shutil.copy2(path, bak)
+
+ try:
+ # Write the new lines, followed by the original file content.
+ with open(path, "w") as new, open(bak, "r") as orig:
+ new.writelines(lines)
+ shutil.copyfileobj(orig, new)
+
+ # Return control to the calling code.
+ yield
+
+ finally:
+ # Put the backup back into place.
+ os.replace(bak, path)
+
+
+@pytest.fixture(scope="module")
+def oauth_ctx(postgres_instance):
+ """
+ Creates a database and user that use the oauth auth method. The context
+ object contains the dbname and user attributes as strings to be used during
+ connection, as well as the issuer and scope that have been set in the HBA
+ configuration.
+
+ This fixture assumes that the standard PG* environment variables point to a
+ server running on a local machine, and that the PGUSER has rights to create
+ databases and roles.
+ """
+ id = secrets.token_hex(4)
+
+ class Context:
+ dbname = "oauth_test_" + id
+
+ user = "oauth_user_" + id
+ punct_user = "oauth_\"'? ;&!_user_" + id # username w/ punctuation
+ map_user = "oauth_map_user_" + id
+ authz_user = "oauth_authz_user_" + id
+
+ issuer = "https://example.com/" + id
+ scope = "openid " + id
+
+ ctx = Context()
+ hba_lines = [
+ f'host {ctx.dbname} {ctx.map_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" map=oauth\n',
+ f'host {ctx.dbname} {ctx.authz_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" trust_validator_authz=1\n',
+ f'host {ctx.dbname} all samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}"\n',
+ ]
+ ident_lines = [r"oauth /^(.*)@example\.com$ \1"]
+
+ if platform.system() == "Windows":
+ # XXX why is 'samehost' not behaving as expected on Windows?
+ for l in list(hba_lines):
+ hba_lines.append(l.replace("samehost", "::1/128"))
+
+ host, port = postgres_instance
+ conn = psycopg2.connect(host=host, port=port)
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+
+ # Create our roles and database.
+ user = sql.Identifier(ctx.user)
+ punct_user = sql.Identifier(ctx.punct_user)
+ map_user = sql.Identifier(ctx.map_user)
+ authz_user = sql.Identifier(ctx.authz_user)
+ dbname = sql.Identifier(ctx.dbname)
+
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(punct_user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(map_user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(authz_user))
+ c.execute(sql.SQL("CREATE DATABASE {};").format(dbname))
+
+ # Replace pg_hba and pg_ident.
+ c.execute("SHOW hba_file;")
+ hba = c.fetchone()[0]
+
+ c.execute("SHOW ident_file;")
+ ident = c.fetchone()[0]
+
+ with prepend_file(hba, hba_lines), prepend_file(ident, ident_lines):
+ c.execute("SELECT pg_reload_conf();")
+
+ # Use the new database and user.
+ yield ctx
+
+ # Put things back the way they were.
+ c.execute("SELECT pg_reload_conf();")
+
+ c.execute(sql.SQL("DROP DATABASE {};").format(dbname))
+ c.execute(sql.SQL("DROP ROLE {};").format(authz_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(map_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(punct_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(user))
+
+
+@pytest.fixture()
+def conn(oauth_ctx, connect):
+ """
+ A convenience wrapper for connect(). The main purpose of this fixture is to
+ make sure oauth_ctx runs its setup code before the connection is made.
+ """
+ return connect()
+
+
+def bearer_token(*, size=16):
+ """
+ Generates a Bearer token using secrets.token_urlsafe(). The generated token
+ size in bytes may be specified; if unset, a small 16-byte token will be
+ generated.
+ """
+
+ if size % 4:
+ raise ValueError(f"requested token size {size} is not a multiple of 4")
+
+ token = secrets.token_urlsafe(size // 4 * 3)
+ assert len(token) == size
+
+ return token
+
+
+def begin_oauth_handshake(conn, oauth_ctx, *, user=None):
+ if user is None:
+ user = oauth_ctx.authz_user
+
+ pq3.send_startup(conn, user=user, database=oauth_ctx.dbname)
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.AuthnRequest
+
+ # The server should advertise exactly one mechanism.
+ assert resp.payload.type == pq3.authn.SASL
+ assert resp.payload.body == [b"OAUTHBEARER", b""]
+
+
+def send_initial_response(conn, *, auth=None, bearer=None):
+ """
+ Sends the OAUTHBEARER initial response on the connection, using the given
+ bearer token. Alternatively to a bearer token, the initial response's auth
+ field may be explicitly specified to test corner cases.
+ """
+ if bearer is not None and auth is not None:
+ raise ValueError("exactly one of the auth and bearer kwargs must be set")
+
+ if bearer is not None:
+ auth = b"Bearer " + bearer
+
+ if auth is None:
+ raise ValueError("exactly one of the auth and bearer kwargs must be set")
+
+ initial = pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"n,,\x01auth=" + auth + b"\x01\x01",
+ )
+ )
+ pq3.send(conn, pq3.types.PasswordMessage, initial)
+
+
+def expect_handshake_success(conn):
+ """
+ Validates that the server responds with an AuthnOK message, and then drains
+ the connection until a ReadyForQuery message is received.
+ """
+ resp = pq3.recv1(conn)
+
+ assert resp.type == pq3.types.AuthnRequest
+ assert resp.payload.type == pq3.authn.OK
+ assert not resp.payload.body
+
+ receive_until(conn, pq3.types.ReadyForQuery)
+
+
+def expect_handshake_failure(conn, oauth_ctx):
+ """
+ Performs the OAUTHBEARER SASL failure "handshake" and validates the server's
+ side of the conversation, including the final ErrorResponse.
+ """
+
+ # We expect a discovery "challenge" back from the server before the authn
+ # failure message.
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.AuthnRequest
+
+ req = resp.payload
+ assert req.type == pq3.authn.SASLContinue
+
+ body = json.loads(req.body)
+ assert body["status"] == "invalid_token"
+ assert body["scope"] == oauth_ctx.scope
+
+ expected_config = oauth_ctx.issuer + "/.well-known/openid-configuration"
+ assert body["openid-configuration"] == expected_config
+
+ # Send the dummy response to complete the failed handshake.
+ pq3.send(conn, pq3.types.PasswordMessage, b"\x01")
+ resp = pq3.recv1(conn)
+
+ err = ExpectedError(INVALID_AUTHORIZATION_ERRCODE, "bearer authentication failed")
+ err.match(resp)
+
+
+def receive_until(conn, type):
+ """
+ receive_until pulls packets off the pq3 connection until a packet with the
+ desired type is found, or an error response is received.
+ """
+ while True:
+ pkt = pq3.recv1(conn)
+
+ if pkt.type == type:
+ return pkt
+ elif pkt.type == pq3.types.ErrorResponse:
+ raise RuntimeError(
+ f"received error response from peer: {pkt.payload.fields!r}"
+ )
+
+
+@pytest.fixture()
+def setup_validator(postgres_instance):
+ """
+ A per-test fixture that sets up the test validator with expected behavior.
+ The setting will be reverted during teardown.
+ """
+ host, port = postgres_instance
+ conn = psycopg2.connect(host=host, port=port)
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+ prev = dict()
+
+ def setter(**gucs):
+ for guc, val in gucs.items():
+ # Save the previous value.
+ c.execute(sql.SQL("SHOW oauthtest.{};").format(sql.Identifier(guc)))
+ prev[guc] = c.fetchone()[0]
+
+ c.execute(
+ sql.SQL("ALTER SYSTEM SET oauthtest.{} TO %s;").format(
+ sql.Identifier(guc)
+ ),
+ (val,),
+ )
+ c.execute("SELECT pg_reload_conf();")
+
+ yield setter
+
+ # Restore the previous values.
+ for guc, val in prev.items():
+ c.execute(
+ sql.SQL("ALTER SYSTEM SET oauthtest.{} TO %s;").format(
+ sql.Identifier(guc)
+ ),
+ (val,),
+ )
+ c.execute("SELECT pg_reload_conf();")
+
+
+@pytest.mark.parametrize("token_len", [16, 1024, 4096])
+@pytest.mark.parametrize(
+ "auth_prefix",
+ [
+ b"Bearer ",
+ b"bearer ",
+ b"Bearer ",
+ ],
+)
+def test_oauth(setup_validator, connect, oauth_ctx, auth_prefix, token_len):
+ # Generate our bearer token with the desired length.
+ token = bearer_token(size=token_len)
+ setup_validator(expected_bearer=token)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ auth = auth_prefix + token.encode("ascii")
+ send_initial_response(conn, auth=auth)
+ expect_handshake_success(conn)
+
+ # Make sure that the server has not set an authenticated ID.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT system_user;")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ assert row.columns == [None]
+
+
+@pytest.mark.parametrize(
+ "token_value",
+ [
+ "abcdzA==",
+ "123456M=",
+ "x-._~+/x",
+ ],
+)
+def test_oauth_bearer_corner_cases(setup_validator, connect, oauth_ctx, token_value):
+ setup_validator(expected_bearer=token_value)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ send_initial_response(conn, bearer=token_value.encode("ascii"))
+
+ expect_handshake_success(conn)
+
+
+@pytest.mark.parametrize(
+ "user,authn_id,should_succeed",
+ [
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: ctx.user,
+ True,
+ id="validator authn: succeeds when authn_id == username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: None,
+ False,
+ id="validator authn: fails when authn_id is not set",
+ ),
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: ctx.authz_user,
+ False,
+ id="validator authn: fails when authn_id != username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: ctx.map_user + "@example.com",
+ True,
+ id="validator with map: succeeds when authn_id matches map",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: None,
+ False,
+ id="validator with map: fails when authn_id is not set",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: ctx.map_user + "@example.net",
+ False,
+ id="validator with map: fails when authn_id doesn't match map",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: None,
+ True,
+ id="validator authz: succeeds with no authn_id",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "",
+ True,
+ id="validator authz: succeeds with empty authn_id",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "postgres",
+ True,
+ id="validator authz: succeeds with basic username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "me@example.com",
+ True,
+ id="validator authz: succeeds with email address",
+ ),
+ ],
+)
+def test_oauth_authn_id(
+ setup_validator, connect, oauth_ctx, user, authn_id, should_succeed
+):
+ token = bearer_token()
+ authn_id = authn_id(oauth_ctx)
+
+ # Set up the validator appropriately.
+ gucs = dict(expected_bearer=token)
+ if authn_id is not None:
+ gucs["set_authn_id"] = True
+ gucs["authn_id"] = authn_id
+ setup_validator(**gucs)
+
+ conn = connect()
+ username = user(oauth_ctx)
+ begin_oauth_handshake(conn, oauth_ctx, user=username)
+ send_initial_response(conn, bearer=token.encode("ascii"))
+
+ if not should_succeed:
+ expect_handshake_failure(conn, oauth_ctx)
+ return
+
+ expect_handshake_success(conn)
+
+ # Check the reported authn_id.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT system_user;")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ expected = authn_id
+ if expected is not None:
+ expected = b"oauth:" + expected.encode("ascii")
+
+ row = resp.payload
+ assert row.columns == [expected]
+
+
+class ExpectedError(object):
+ def __init__(self, code, msg=None, detail=None):
+ self.code = code
+ self.msg = msg
+ self.detail = detail
+
+ # Protect against the footgun of an accidental empty string, which will
+ # "match" anything. If you don't want to match message or detail, just
+ # don't pass them.
+ if self.msg == "":
+ raise ValueError("msg must be non-empty or None")
+ if self.detail == "":
+ raise ValueError("detail must be non-empty or None")
+
+ def _getfield(self, resp, type):
+ """
+ Searches an ErrorResponse for a single field of the given type (e.g.
+ "M", "C", "D") and returns its value. Asserts if it doesn't find exactly
+ one field.
+ """
+ prefix = type.encode("ascii")
+ fields = [f for f in resp.payload.fields if f.startswith(prefix)]
+
+ assert len(fields) == 1
+ return fields[0][1:] # strip off the type byte
+
+ def match(self, resp):
+ """
+ Checks that the given response matches the expected code, message, and
+ detail (if given). The error code must match exactly. The expected
+ message and detail must be contained within the actual strings.
+ """
+ assert resp.type == pq3.types.ErrorResponse
+
+ code = self._getfield(resp, "C")
+ assert code == self.code
+
+ if self.msg:
+ msg = self._getfield(resp, "M")
+ expected = self.msg.encode("utf-8")
+ assert expected in msg
+
+ if self.detail:
+ detail = self._getfield(resp, "D")
+ expected = self.detail.encode("utf-8")
+ assert expected in detail
+
+
+def test_oauth_rejected_bearer(conn, oauth_ctx):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send a bearer token that doesn't match what the validator expects. It
+ # should fail the connection.
+ send_initial_response(conn, bearer=b"xxxxxx")
+
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.parametrize(
+ "bad_bearer",
+ [
+ b"Bearer ",
+ b"Bearer a===b",
+ b"Bearer hello!",
+ b"Bearer trailingspace ",
+ b"Bearer trailingtab\t",
+ b"Bearer me@example.com",
+ b"Beare abcd",
+ b'OAuth realm="Example"',
+ b"",
+ ],
+)
+def test_oauth_invalid_bearer(setup_validator, connect, oauth_ctx, bad_bearer):
+ # Tell the validator to accept any token. This ensures that the invalid
+ # bearer tokens are rejected before the validation step.
+ setup_validator(reflect_role=True)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+ send_initial_response(conn, auth=bad_bearer)
+
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.slow
+@pytest.mark.parametrize(
+ "resp_type,resp,err",
+ [
+ pytest.param(
+ None,
+ None,
+ None,
+ marks=pytest.mark.slow,
+ id="no response (expect timeout)",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"hello",
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not send a kvsep response",
+ ),
+ id="bad dummy response",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"\x01\x01",
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not send a kvsep response",
+ ),
+ id="multiple kvseps",
+ ),
+ pytest.param(
+ pq3.types.Query,
+ dict(query=b""),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "expected SASL response"),
+ id="bad response message type",
+ ),
+ ],
+)
+def test_oauth_bad_response_to_error_challenge(conn, oauth_ctx, resp_type, resp, err):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send an empty auth initial response, which will force an authn failure.
+ send_initial_response(conn, auth=b"")
+
+ # We expect a discovery "challenge" back from the server before the authn
+ # failure message.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.AuthnRequest
+
+ req = pkt.payload
+ assert req.type == pq3.authn.SASLContinue
+
+ body = json.loads(req.body)
+ assert body["status"] == "invalid_token"
+
+ if resp_type is None:
+ # Do not send the dummy response. We should time out and not get a
+ # response from the server.
+ with pytest.raises(socket.timeout):
+ conn.read(1)
+
+ # Done with the test.
+ return
+
+ # Send the bad response.
+ pq3.send(conn, resp_type, resp)
+
+ # Make sure the server fails the connection correctly.
+ pkt = pq3.recv1(conn)
+ err.match(pkt)
+
+
+@pytest.mark.parametrize(
+ "type,payload,err",
+ [
+ pytest.param(
+ pq3.types.ErrorResponse,
+ dict(fields=[b""]),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "expected SASL response"),
+ id="error response in initial message",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"x" * (MAX_SASL_MESSAGE_LENGTH + 1),
+ ExpectedError(
+ INVALID_AUTHORIZATION_ERRCODE, "bearer authentication failed"
+ ),
+ id="overlong initial response data",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"SCRAM-SHA-256")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE, "invalid SASL authentication mechanism"
+ ),
+ id="bad SASL mechanism selection",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", len=2, data=b"x")),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "insufficient data"),
+ id="SASL data underflow",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", len=0, data=b"x")),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "invalid message format"),
+ id="SASL data overflow",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "message is empty",
+ ),
+ id="empty",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"n,,\x01auth=\x01\x01\0")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "length does not match input length",
+ ),
+ id="contains null byte",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected channel-binding flag", # XXX this is a bit strange
+ ),
+ id="initial error response",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"p=tls-server-end-point,,\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "server does not support channel binding",
+ ),
+ id="uses channel binding",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"x,,\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected channel-binding flag",
+ ),
+ id="invalid channel binding specifier",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Comma expected",
+ ),
+ id="bad GS2 header: missing channel binding terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,a")),
+ ExpectedError(
+ FEATURE_NOT_SUPPORTED_ERRCODE,
+ "client uses authorization identity",
+ ),
+ id="bad GS2 header: authzid in use",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,b,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected attribute",
+ ),
+ id="bad GS2 header: extra attribute",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected attribute 0x00", # XXX this is a bit strange
+ ),
+ id="bad GS2 header: missing authzid terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Key-value separator expected",
+ ),
+ id="missing initial kvsep",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Key-value separator expected",
+ ),
+ id="missing initial kvsep",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "does not contain an auth value",
+ ),
+ id="missing auth value: empty key-value list",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01host=example.com\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "does not contain an auth value",
+ ),
+ id="missing auth value: other keys present",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01host=example.com")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "unterminated key/value pair",
+ ),
+ id="missing value terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not contain a final terminator",
+ ),
+ id="missing list terminator: empty list",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01auth=Bearer 0\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not contain a final terminator",
+ ),
+ id="missing list terminator: with auth value",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01auth=Bearer 0\x01\x01blah")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "additional data after the final terminator",
+ ),
+ id="additional key after terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01key\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "key without a value",
+ ),
+ id="key without value",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"y,,\x01auth=Bearer 0\x01auth=Bearer 1\x01\x01",
+ )
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "contains multiple auth values",
+ ),
+ id="multiple auth values",
+ ),
+ ],
+)
+def test_oauth_bad_initial_response(conn, oauth_ctx, type, payload, err):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # The server expects a SASL response; give it something else instead.
+ if not isinstance(payload, dict):
+ payload = dict(payload_data=payload)
+ pq3.send(conn, type, **payload)
+
+ resp = pq3.recv1(conn)
+ err.match(resp)
+
+
+def test_oauth_empty_initial_response(setup_validator, connect, oauth_ctx):
+ token = bearer_token()
+ setup_validator(expected_bearer=token)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send an initial response without data.
+ initial = pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER"))
+ pq3.send(conn, pq3.types.PasswordMessage, initial)
+
+ # The server should respond with an empty challenge so we can send the data
+ # it wants.
+ pkt = pq3.recv1(conn)
+
+ assert pkt.type == pq3.types.AuthnRequest
+ assert pkt.payload.type == pq3.authn.SASLContinue
+ assert not pkt.payload.body
+
+ # Now send the initial data.
+ data = b"n,,\x01auth=Bearer " + token.encode("ascii") + b"\x01\x01"
+ pq3.send(conn, pq3.types.PasswordMessage, data)
+
+ # Server should now complete the handshake.
+ expect_handshake_success(conn)
+
+
+# TODO: see if there's a way to test this easily after the API switch
+def xtest_oauth_no_validator(setup_validator, oauth_ctx, connect):
+ # Clear out our validator command, then establish a new connection.
+ set_validator("")
+ conn = connect()
+
+ begin_oauth_handshake(conn, oauth_ctx)
+ send_initial_response(conn, bearer=bearer_token())
+
+ # The server should fail the connection.
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.parametrize(
+ "user",
+ [
+ pytest.param(
+ lambda ctx: ctx.user,
+ id="basic username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.punct_user,
+ id="'unsafe' characters are passed through correctly",
+ ),
+ ],
+)
+def test_oauth_validator_role(setup_validator, oauth_ctx, connect, user):
+ username = user(oauth_ctx)
+
+ # Tell the validator to reflect the PGUSER as the authenticated identity.
+ setup_validator(reflect_role=True)
+ conn = connect()
+
+ # Log in. Note that reflection ignores the bearer token.
+ begin_oauth_handshake(conn, oauth_ctx, user=username)
+ send_initial_response(conn, bearer=b"dontcare")
+ expect_handshake_success(conn)
+
+ # Check the user identity.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT system_user;")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ expected = b"oauth:" + username.encode("utf-8")
+ assert row.columns == [expected]
diff --git a/src/test/python/server/test_server.py b/src/test/python/server/test_server.py
new file mode 100644
index 0000000000..02126dba79
--- /dev/null
+++ b/src/test/python/server/test_server.py
@@ -0,0 +1,21 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import pq3
+
+
+def test_handshake(connect):
+ """Basic sanity check."""
+ conn = connect()
+
+ pq3.handshake(conn, user=pq3.pguser(), database=pq3.pgdatabase())
+
+ pq3.send(conn, pq3.types.Query, query=b"")
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.EmptyQueryResponse
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.ReadyForQuery
diff --git a/src/test/python/test_internals.py b/src/test/python/test_internals.py
new file mode 100644
index 0000000000..dee4855fc0
--- /dev/null
+++ b/src/test/python/test_internals.py
@@ -0,0 +1,138 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import io
+
+from pq3 import _DebugStream
+
+
+def test_DebugStream_read():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+
+ stream = _DebugStream(under, out)
+
+ res = stream.read(5)
+ assert res == b"abcde"
+
+ res = stream.read(16)
+ assert res == b"fghijklmnopqrstu"
+
+ stream.flush_debug()
+
+ res = stream.read()
+ assert res == b"vwxyz"
+
+ stream.flush_debug()
+
+ expected = (
+ "< 0000:\t61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 70\tabcdefghijklmnop\n"
+ "< 0010:\t71 72 73 74 75 \tqrstu\n"
+ "\n"
+ "< 0000:\t76 77 78 79 7a \tvwxyz\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_write():
+ under = io.BytesIO()
+ out = io.StringIO()
+
+ stream = _DebugStream(under, out)
+
+ stream.write(b"\x00\x01\x02")
+ stream.flush()
+
+ assert under.getvalue() == b"\x00\x01\x02"
+
+ stream.write(b"\xc0\xc1\xc2")
+ stream.flush()
+
+ assert under.getvalue() == b"\x00\x01\x02\xc0\xc1\xc2"
+
+ stream.flush_debug()
+
+ expected = "> 0000:\t00 01 02 c0 c1 c2 \t......\n\n"
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_read_write():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+ stream = _DebugStream(under, out)
+
+ res = stream.read(5)
+ assert res == b"abcde"
+
+ stream.write(b"xxxxx")
+ stream.flush()
+
+ assert under.getvalue() == b"abcdexxxxxklmnopqrstuvwxyz"
+
+ res = stream.read(5)
+ assert res == b"klmno"
+
+ stream.write(b"xxxxx")
+ stream.flush()
+
+ assert under.getvalue() == b"abcdexxxxxklmnoxxxxxuvwxyz"
+
+ stream.flush_debug()
+
+ expected = (
+ "< 0000:\t61 62 63 64 65 6b 6c 6d 6e 6f \tabcdeklmno\n"
+ "\n"
+ "> 0000:\t78 78 78 78 78 78 78 78 78 78 \txxxxxxxxxx\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_end_packet():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+ stream = _DebugStream(under, out)
+
+ stream.read(5)
+ stream.end_packet("read description", read=True, indent=" ")
+
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("write description", indent=" ")
+
+ stream.read(5)
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("read/write combo for read", read=True, indent=" ")
+
+ stream.read(5)
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("read/write combo for write", indent=" ")
+
+ expected = (
+ " < 0000:\t61 62 63 64 65 \tabcde\n"
+ "\n"
+ "< read description\n"
+ "\n"
+ "> write description\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ " < 0000:\t6b 6c 6d 6e 6f \tklmno\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ "< read/write combo for read\n"
+ "\n"
+ "> read/write combo for write\n"
+ "\n"
+ " < 0000:\t75 76 77 78 79 \tuvwxy\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
diff --git a/src/test/python/test_pq3.py b/src/test/python/test_pq3.py
new file mode 100644
index 0000000000..638dd337a6
--- /dev/null
+++ b/src/test/python/test_pq3.py
@@ -0,0 +1,563 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import getpass
+import io
+import platform
+import struct
+import sys
+
+import pytest
+from construct import Container, PaddingError, StreamError, TerminatedError
+
+import pq3
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"\x00\x00\x00\x10\x00\x04\x00\x00abcdefgh",
+ Container(len=16, proto=0x40000, payload=b"abcdefgh"),
+ b"",
+ id="8-byte payload",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x08\x00\x04\x00\x00",
+ Container(len=8, proto=0x40000, payload=b""),
+ b"",
+ id="no payload",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x09\x00\x04\x00\x00abcde",
+ Container(len=9, proto=0x40000, payload=b"a"),
+ b"bcde",
+ id="1-byte payload and extra padding",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x0B\x00\x03\x00\x00hi\x00",
+ Container(len=11, proto=pq3.protocol(3, 0), payload=[b"hi"]),
+ b"",
+ id="implied parameter list when using proto version 3.0",
+ ),
+ ],
+)
+def test_Startup_parse(raw, expected, extra):
+ with io.BytesIO(raw) as stream:
+ actual = pq3.Startup.parse_stream(stream)
+
+ assert actual == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "packet,expected_bytes",
+ [
+ pytest.param(
+ dict(),
+ b"\x00\x00\x00\x08\x00\x00\x00\x00",
+ id="nothing set",
+ ),
+ pytest.param(
+ dict(len=10, proto=0x12345678),
+ b"\x00\x00\x00\x0A\x12\x34\x56\x78\x00\x00",
+ id="len and proto set explicitly",
+ ),
+ pytest.param(
+ dict(proto=0x12345678),
+ b"\x00\x00\x00\x08\x12\x34\x56\x78",
+ id="implied len with no payload",
+ ),
+ pytest.param(
+ dict(proto=0x12345678, payload=b"abcd"),
+ b"\x00\x00\x00\x0C\x12\x34\x56\x78abcd",
+ id="implied len with payload",
+ ),
+ pytest.param(
+ dict(payload=[b""]),
+ b"\x00\x00\x00\x09\x00\x03\x00\x00\x00",
+ id="implied proto version 3 when sending parameters",
+ ),
+ pytest.param(
+ dict(payload=[b"hi", b""]),
+ b"\x00\x00\x00\x0C\x00\x03\x00\x00hi\x00\x00",
+ id="implied proto version 3 and len when sending more than one parameter",
+ ),
+ pytest.param(
+ dict(payload=dict(user="jsmith", database="postgres")),
+ b"\x00\x00\x00\x27\x00\x03\x00\x00user\x00jsmith\x00database\x00postgres\x00\x00",
+ id="auto-serialization of dict parameters",
+ ),
+ ],
+)
+def test_Startup_build(packet, expected_bytes):
+ actual = pq3.Startup.build(packet)
+ assert actual == expected_bytes
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"*\x00\x00\x00\x08abcd",
+ dict(type=b"*", len=8, payload=b"abcd"),
+ b"",
+ id="4-byte payload",
+ ),
+ pytest.param(
+ b"*\x00\x00\x00\x04",
+ dict(type=b"*", len=4, payload=b""),
+ b"",
+ id="no payload",
+ ),
+ pytest.param(
+ b"*\x00\x00\x00\x05xabcd",
+ dict(type=b"*", len=5, payload=b"x"),
+ b"abcd",
+ id="1-byte payload with extra padding",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x08\x00\x00\x00\x00",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=8,
+ payload=dict(type=pq3.authn.OK, body=None),
+ ),
+ b"",
+ id="AuthenticationOk",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x12\x00\x00\x00\x0AEXTERNAL\x00\x00",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=18,
+ payload=dict(type=pq3.authn.SASL, body=[b"EXTERNAL", b""]),
+ ),
+ b"",
+ id="AuthenticationSASL",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0B12345",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=13,
+ payload=dict(type=pq3.authn.SASLContinue, body=b"12345"),
+ ),
+ b"",
+ id="AuthenticationSASLContinue",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0C12345",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=13,
+ payload=dict(type=pq3.authn.SASLFinal, body=b"12345"),
+ ),
+ b"",
+ id="AuthenticationSASLFinal",
+ ),
+ pytest.param(
+ b"p\x00\x00\x00\x0Bhunter2",
+ dict(
+ type=pq3.types.PasswordMessage,
+ len=11,
+ payload=b"hunter2",
+ ),
+ b"",
+ id="PasswordMessage",
+ ),
+ pytest.param(
+ b"K\x00\x00\x00\x0C\x00\x00\x00\x00\x12\x34\x56\x78",
+ dict(
+ type=pq3.types.BackendKeyData,
+ len=12,
+ payload=dict(pid=0, key=0x12345678),
+ ),
+ b"",
+ id="BackendKeyData",
+ ),
+ pytest.param(
+ b"C\x00\x00\x00\x08SET\x00",
+ dict(
+ type=pq3.types.CommandComplete,
+ len=8,
+ payload=dict(tag=b"SET"),
+ ),
+ b"",
+ id="CommandComplete",
+ ),
+ pytest.param(
+ b"E\x00\x00\x00\x11Mbad!\x00Mdog!\x00\x00",
+ dict(type=b"E", len=17, payload=dict(fields=[b"Mbad!", b"Mdog!", b""])),
+ b"",
+ id="ErrorResponse",
+ ),
+ pytest.param(
+ b"S\x00\x00\x00\x08a\x00b\x00",
+ dict(
+ type=pq3.types.ParameterStatus,
+ len=8,
+ payload=dict(name=b"a", value=b"b"),
+ ),
+ b"",
+ id="ParameterStatus",
+ ),
+ pytest.param(
+ b"Z\x00\x00\x00\x05x",
+ dict(type=b"Z", len=5, payload=dict(status=b"x")),
+ b"",
+ id="ReadyForQuery",
+ ),
+ pytest.param(
+ b"Q\x00\x00\x00\x06!\x00",
+ dict(type=pq3.types.Query, len=6, payload=dict(query=b"!")),
+ b"",
+ id="Query",
+ ),
+ pytest.param(
+ b"D\x00\x00\x00\x0B\x00\x01\x00\x00\x00\x01!",
+ dict(type=pq3.types.DataRow, len=11, payload=dict(columns=[b"!"])),
+ b"",
+ id="DataRow",
+ ),
+ pytest.param(
+ b"D\x00\x00\x00\x06\x00\x00extra",
+ dict(type=pq3.types.DataRow, len=6, payload=dict(columns=[])),
+ b"extra",
+ id="DataRow with extra data",
+ ),
+ pytest.param(
+ b"I\x00\x00\x00\x04",
+ dict(type=pq3.types.EmptyQueryResponse, len=4, payload=None),
+ b"",
+ id="EmptyQueryResponse",
+ ),
+ pytest.param(
+ b"I\x00\x00\x00\x04\xFF",
+ dict(type=b"I", len=4, payload=None),
+ b"\xFF",
+ id="EmptyQueryResponse with extra bytes",
+ ),
+ pytest.param(
+ b"X\x00\x00\x00\x04",
+ dict(type=pq3.types.Terminate, len=4, payload=None),
+ b"",
+ id="Terminate",
+ ),
+ ],
+)
+def test_Pq3_parse(raw, expected, extra):
+ with io.BytesIO(raw) as stream:
+ actual = pq3.Pq3.parse_stream(stream)
+
+ assert actual == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(type=b"*", len=5),
+ b"*\x00\x00\x00\x05\x00",
+ id="type and len set explicitly",
+ ),
+ pytest.param(
+ dict(type=b"*"),
+ b"*\x00\x00\x00\x04",
+ id="implied len with no payload",
+ ),
+ pytest.param(
+ dict(type=b"*", payload=b"1234"),
+ b"*\x00\x00\x00\x081234",
+ id="implied len with payload",
+ ),
+ pytest.param(
+ dict(type=pq3.types.AuthnRequest, payload=dict(type=pq3.authn.OK)),
+ b"R\x00\x00\x00\x08\x00\x00\x00\x00",
+ id="implied len/type for AuthenticationOK",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(
+ type=pq3.authn.SASL,
+ body=[b"SCRAM-SHA-256-PLUS", b"SCRAM-SHA-256", b""],
+ ),
+ ),
+ b"R\x00\x00\x00\x2A\x00\x00\x00\x0ASCRAM-SHA-256-PLUS\x00SCRAM-SHA-256\x00\x00",
+ id="implied len/type for AuthenticationSASL",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(type=pq3.authn.SASLContinue, body=b"12345"),
+ ),
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0B12345",
+ id="implied len/type for AuthenticationSASLContinue",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(type=pq3.authn.SASLFinal, body=b"12345"),
+ ),
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0C12345",
+ id="implied len/type for AuthenticationSASLFinal",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.PasswordMessage,
+ payload=b"hunter2",
+ ),
+ b"p\x00\x00\x00\x0Bhunter2",
+ id="implied len/type for PasswordMessage",
+ ),
+ pytest.param(
+ dict(type=pq3.types.BackendKeyData, payload=dict(pid=1, key=7)),
+ b"K\x00\x00\x00\x0C\x00\x00\x00\x01\x00\x00\x00\x07",
+ id="implied len/type for BackendKeyData",
+ ),
+ pytest.param(
+ dict(type=pq3.types.CommandComplete, payload=dict(tag=b"SET")),
+ b"C\x00\x00\x00\x08SET\x00",
+ id="implied len/type for CommandComplete",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ErrorResponse, payload=dict(fields=[b"error", b""])),
+ b"E\x00\x00\x00\x0Berror\x00\x00",
+ id="implied len/type for ErrorResponse",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ParameterStatus, payload=dict(name=b"a", value=b"b")),
+ b"S\x00\x00\x00\x08a\x00b\x00",
+ id="implied len/type for ParameterStatus",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ReadyForQuery, payload=dict(status=b"I")),
+ b"Z\x00\x00\x00\x05I",
+ id="implied len/type for ReadyForQuery",
+ ),
+ pytest.param(
+ dict(type=pq3.types.Query, payload=dict(query=b"SELECT 1;")),
+ b"Q\x00\x00\x00\x0eSELECT 1;\x00",
+ id="implied len/type for Query",
+ ),
+ pytest.param(
+ dict(type=pq3.types.DataRow, payload=dict(columns=[b"abcd"])),
+ b"D\x00\x00\x00\x0E\x00\x01\x00\x00\x00\x04abcd",
+ id="implied len/type for DataRow",
+ ),
+ pytest.param(
+ dict(type=pq3.types.EmptyQueryResponse),
+ b"I\x00\x00\x00\x04",
+ id="implied len for EmptyQueryResponse",
+ ),
+ pytest.param(
+ dict(type=pq3.types.Terminate),
+ b"X\x00\x00\x00\x04",
+ id="implied len for Terminate",
+ ),
+ ],
+)
+def test_Pq3_build(fields, expected):
+ actual = pq3.Pq3.build(fields)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"\x00\x00",
+ dict(columns=[]),
+ b"",
+ id="no columns",
+ ),
+ pytest.param(
+ b"\x00\x01\x00\x00\x00\x04abcd",
+ dict(columns=[b"abcd"]),
+ b"",
+ id="one column",
+ ),
+ pytest.param(
+ b"\x00\x02\x00\x00\x00\x04abcd\x00\x00\x00\x01x",
+ dict(columns=[b"abcd", b"x"]),
+ b"",
+ id="multiple columns",
+ ),
+ pytest.param(
+ b"\x00\x02\x00\x00\x00\x00\x00\x00\x00\x01x",
+ dict(columns=[b"", b"x"]),
+ b"",
+ id="empty column value",
+ ),
+ pytest.param(
+ b"\x00\x02\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF",
+ dict(columns=[None, None]),
+ b"",
+ id="null columns",
+ ),
+ ],
+)
+def test_DataRow_parse(raw, expected, extra):
+ pkt = b"D" + struct.pack("!i", len(raw) + 4) + raw
+ with io.BytesIO(pkt) as stream:
+ actual = pq3.Pq3.parse_stream(stream)
+
+ assert actual.type == pq3.types.DataRow
+ assert actual.payload == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(),
+ b"\x00\x00",
+ id="no columns",
+ ),
+ pytest.param(
+ dict(columns=[None, None]),
+ b"\x00\x02\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF",
+ id="null columns",
+ ),
+ ],
+)
+def test_DataRow_build(fields, expected):
+ actual = pq3.Pq3.build(dict(type=pq3.types.DataRow, payload=fields))
+
+ expected = b"D" + struct.pack("!i", len(expected) + 4) + expected
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "raw,expected,exception",
+ [
+ pytest.param(
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ dict(name=b"EXTERNAL", len=-1, data=None),
+ None,
+ id="no initial response",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\x02me",
+ dict(name=b"EXTERNAL", len=2, data=b"me"),
+ None,
+ id="initial response",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\x02meextra",
+ None,
+ TerminatedError,
+ id="extra data",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\xFFme",
+ None,
+ StreamError,
+ id="underflow",
+ ),
+ ],
+)
+def test_SASLInitialResponse_parse(raw, expected, exception):
+ ctx = contextlib.nullcontext()
+ if exception:
+ ctx = pytest.raises(exception)
+
+ with ctx:
+ actual = pq3.SASLInitialResponse.parse(raw)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(name=b"EXTERNAL"),
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ id="no initial response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=None),
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ id="no initial response (explicit None)",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=b""),
+ b"EXTERNAL\x00\x00\x00\x00\x00",
+ id="empty response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=b"me@example.com"),
+ b"EXTERNAL\x00\x00\x00\x00\x0Eme@example.com",
+ id="initial response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", len=2, data=b"me@example.com"),
+ b"EXTERNAL\x00\x00\x00\x00\x02me@example.com",
+ id="data overflow",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", len=14, data=b"me"),
+ b"EXTERNAL\x00\x00\x00\x00\x0Eme",
+ id="data underflow",
+ ),
+ ],
+)
+def test_SASLInitialResponse_build(fields, expected):
+ actual = pq3.SASLInitialResponse.build(fields)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "version,expected_bytes",
+ [
+ pytest.param((3, 0), b"\x00\x03\x00\x00", id="version 3"),
+ pytest.param((1234, 5679), b"\x04\xd2\x16\x2f", id="SSLRequest"),
+ ],
+)
+def test_protocol(version, expected_bytes):
+ # Make sure the integer returned by protocol is correctly serialized on the
+ # wire.
+ assert struct.pack("!i", pq3.protocol(*version)) == expected_bytes
+
+
+@pytest.mark.parametrize(
+ "envvar,func,expected",
+ [
+ ("PGHOST", pq3.pghost, "localhost"),
+ ("PGPORT", pq3.pgport, 5432),
+ (
+ "PGUSER",
+ pq3.pguser,
+ os.getlogin() if platform.system() == "Windows" else getpass.getuser(),
+ ),
+ ("PGDATABASE", pq3.pgdatabase, "postgres"),
+ ],
+)
+def test_env_defaults(monkeypatch, envvar, func, expected):
+ monkeypatch.delenv(envvar, raising=False)
+
+ actual = func()
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "envvars,func,expected",
+ [
+ (dict(PGHOST="otherhost"), pq3.pghost, "otherhost"),
+ (dict(PGPORT="6789"), pq3.pgport, 6789),
+ (dict(PGUSER="postgres"), pq3.pguser, "postgres"),
+ (dict(PGDATABASE="template1"), pq3.pgdatabase, "template1"),
+ ],
+)
+def test_env(monkeypatch, envvars, func, expected):
+ for k, v in envvars.items():
+ monkeypatch.setenv(k, v)
+
+ actual = func()
+ assert actual == expected
diff --git a/src/test/python/tls.py b/src/test/python/tls.py
new file mode 100644
index 0000000000..075c02c1ca
--- /dev/null
+++ b/src/test/python/tls.py
@@ -0,0 +1,195 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+from construct import *
+
+#
+# TLS 1.3
+#
+# Most of the types below are transcribed from RFC 8446:
+#
+# https://tools.ietf.org/html/rfc8446
+#
+
+
+def _Vector(size_field, element):
+ return Prefixed(size_field, GreedyRange(element))
+
+
+# Alerts
+
+AlertLevel = Enum(
+ Byte,
+ warning=1,
+ fatal=2,
+)
+
+AlertDescription = Enum(
+ Byte,
+ close_notify=0,
+ unexpected_message=10,
+ bad_record_mac=20,
+ decryption_failed_RESERVED=21,
+ record_overflow=22,
+ decompression_failure=30,
+ handshake_failure=40,
+ no_certificate_RESERVED=41,
+ bad_certificate=42,
+ unsupported_certificate=43,
+ certificate_revoked=44,
+ certificate_expired=45,
+ certificate_unknown=46,
+ illegal_parameter=47,
+ unknown_ca=48,
+ access_denied=49,
+ decode_error=50,
+ decrypt_error=51,
+ export_restriction_RESERVED=60,
+ protocol_version=70,
+ insufficient_security=71,
+ internal_error=80,
+ user_canceled=90,
+ no_renegotiation=100,
+ unsupported_extension=110,
+)
+
+Alert = Struct(
+ "level" / AlertLevel,
+ "description" / AlertDescription,
+)
+
+
+# Extensions
+
+ExtensionType = Enum(
+ Int16ub,
+ server_name=0,
+ max_fragment_length=1,
+ status_request=5,
+ supported_groups=10,
+ signature_algorithms=13,
+ use_srtp=14,
+ heartbeat=15,
+ application_layer_protocol_negotiation=16,
+ signed_certificate_timestamp=18,
+ client_certificate_type=19,
+ server_certificate_type=20,
+ padding=21,
+ pre_shared_key=41,
+ early_data=42,
+ supported_versions=43,
+ cookie=44,
+ psk_key_exchange_modes=45,
+ certificate_authorities=47,
+ oid_filters=48,
+ post_handshake_auth=49,
+ signature_algorithms_cert=50,
+ key_share=51,
+)
+
+Extension = Struct(
+ "extension_type" / ExtensionType,
+ "extension_data" / Prefixed(Int16ub, GreedyBytes),
+)
+
+
+# ClientHello
+
+
+class _CipherSuiteAdapter(Adapter):
+ class _hextuple(tuple):
+ def __repr__(self):
+ return f"(0x{self[0]:02X}, 0x{self[1]:02X})"
+
+ def _encode(self, obj, context, path):
+ return bytes(obj)
+
+ def _decode(self, obj, context, path):
+ assert len(obj) == 2
+ return self._hextuple(obj)
+
+
+ProtocolVersion = Hex(Int16ub)
+
+Random = Hex(Bytes(32))
+
+CipherSuite = _CipherSuiteAdapter(Byte[2])
+
+ClientHello = Struct(
+ "legacy_version" / ProtocolVersion,
+ "random" / Random,
+ "legacy_session_id" / Prefixed(Byte, Hex(GreedyBytes)),
+ "cipher_suites" / _Vector(Int16ub, CipherSuite),
+ "legacy_compression_methods" / Prefixed(Byte, GreedyBytes),
+ "extensions" / _Vector(Int16ub, Extension),
+)
+
+# ServerHello
+
+ServerHello = Struct(
+ "legacy_version" / ProtocolVersion,
+ "random" / Random,
+ "legacy_session_id_echo" / Prefixed(Byte, Hex(GreedyBytes)),
+ "cipher_suite" / CipherSuite,
+ "legacy_compression_method" / Hex(Byte),
+ "extensions" / _Vector(Int16ub, Extension),
+)
+
+# Handshake
+
+HandshakeType = Enum(
+ Byte,
+ client_hello=1,
+ server_hello=2,
+ new_session_ticket=4,
+ end_of_early_data=5,
+ encrypted_extensions=8,
+ certificate=11,
+ certificate_request=13,
+ certificate_verify=15,
+ finished=20,
+ key_update=24,
+ message_hash=254,
+)
+
+Handshake = Struct(
+ "msg_type" / HandshakeType,
+ "length" / Int24ub,
+ "payload"
+ / Switch(
+ this.msg_type,
+ {
+ HandshakeType.client_hello: ClientHello,
+ HandshakeType.server_hello: ServerHello,
+ # HandshakeType.end_of_early_data: EndOfEarlyData,
+ # HandshakeType.encrypted_extensions: EncryptedExtensions,
+ # HandshakeType.certificate_request: CertificateRequest,
+ # HandshakeType.certificate: Certificate,
+ # HandshakeType.certificate_verify: CertificateVerify,
+ # HandshakeType.finished: Finished,
+ # HandshakeType.new_session_ticket: NewSessionTicket,
+ # HandshakeType.key_update: KeyUpdate,
+ },
+ default=FixedSized(this.length, GreedyBytes),
+ ),
+)
+
+# Records
+
+ContentType = Enum(
+ Byte,
+ invalid=0,
+ change_cipher_spec=20,
+ alert=21,
+ handshake=22,
+ application_data=23,
+)
+
+Plaintext = Struct(
+ "type" / ContentType,
+ "legacy_record_version" / ProtocolVersion,
+ "length" / Int16ub,
+ "fragment" / FixedSized(this.length, GreedyBytes),
+)
diff --git a/src/tools/make_venv b/src/tools/make_venv
new file mode 100755
index 0000000000..804307ee12
--- /dev/null
+++ b/src/tools/make_venv
@@ -0,0 +1,56 @@
+#!/usr/bin/env python3
+
+import argparse
+import subprocess
+import os
+import platform
+import sys
+
+parser = argparse.ArgumentParser()
+
+parser.add_argument('--requirements', help='path to pip requirements file', type=str)
+parser.add_argument('--privatedir', help='private directory for target', type=str)
+parser.add_argument('venv_path', help='desired venv location')
+
+args = parser.parse_args()
+
+# Decide whether or not to capture stdout into a log file. We only do this if
+# we've been given our own private directory.
+#
+# FIXME Unfortunately this interferes with debugging on Cirrus, because the
+# private directory isn't uploaded in the sanity check's artifacts. When we
+# don't capture the log file, it gets spammed to stdout during build... Is there
+# a way to push this into the meson-log somehow? For now, the capture
+# implementation is commented out.
+logfile = None
+
+if args.privatedir:
+ if not os.path.isdir(args.privatedir):
+ os.mkdir(args.privatedir)
+
+ # FIXME see above comment
+ # logpath = os.path.join(args.privatedir, 'stdout.txt')
+ # logfile = open(logpath, 'w')
+
+def run(*args):
+ kwargs = dict(check=True)
+ if logfile:
+ kwargs.update(stdout=logfile)
+
+ subprocess.run(args, **kwargs)
+
+# Create the virtualenv first.
+run(sys.executable, '-m', 'venv', args.venv_path)
+
+# Update pip next. This helps avoid old pip bugs; the version inside system
+# Pythons tends to be pretty out of date.
+bindir = 'Scripts' if platform.system() == 'Windows' else 'bin'
+python = os.path.join(args.venv_path, bindir, 'python3')
+run(python, '-m', 'pip', 'install', '-U', 'pip')
+
+# Finally, install the test's requirements. We need pytest and pytest-tap, no
+# matter what the test needs.
+pip = os.path.join(args.venv_path, bindir, 'pip')
+run(pip, 'install', 'pytest', 'pytest-tap')
+if args.requirements:
+ run(pip, 'install', '-r', args.requirements)
--
2.34.1
v18-0006-Introduce-OAuth-validator-libraries.patch (application/octet-stream, 32.0K) download
From 7a42365d62fd13bc4d5125017ebc8857f0855406 Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Wed, 21 Feb 2024 17:04:26 +0100
Subject: [PATCH v18 6/9] Introduce OAuth validator libraries
This replaces the serverside validation code with an module API
for loading in extensions for validating bearer tokens. A lot of
code is left to be written.
Co-authored-by: Jacob Champion <jacob.champion@enterprisedb.com>
---
src/backend/libpq/auth-oauth.c | 431 +++++-------------
src/backend/utils/misc/guc_tables.c | 6 +-
src/bin/pg_combinebackup/Makefile | 2 +-
src/common/Makefile | 2 +-
src/include/libpq/oauth.h | 29 +-
src/test/modules/meson.build | 1 +
src/test/modules/oauth_validator/.gitignore | 4 +
src/test/modules/oauth_validator/Makefile | 19 +
.../oauth_validator/expected/validator.out | 6 +
src/test/modules/oauth_validator/meson.build | 33 ++
.../modules/oauth_validator/sql/validator.sql | 1 +
.../modules/oauth_validator/t/001_server.pl | 53 +++
src/test/modules/oauth_validator/validator.c | 71 +++
src/test/perl/PostgreSQL/Test/Cluster.pm | 14 +-
src/test/perl/PostgreSQL/Test/OAuthServer.pm | 158 +++++++
src/tools/pgindent/typedefs.list | 2 +
16 files changed, 500 insertions(+), 332 deletions(-)
create mode 100644 src/test/modules/oauth_validator/.gitignore
create mode 100644 src/test/modules/oauth_validator/Makefile
create mode 100644 src/test/modules/oauth_validator/expected/validator.out
create mode 100644 src/test/modules/oauth_validator/meson.build
create mode 100644 src/test/modules/oauth_validator/sql/validator.sql
create mode 100644 src/test/modules/oauth_validator/t/001_server.pl
create mode 100644 src/test/modules/oauth_validator/validator.c
create mode 100644 src/test/perl/PostgreSQL/Test/OAuthServer.pm
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
index a9d2646023..f5cf271566 100644
--- a/src/backend/libpq/auth-oauth.c
+++ b/src/backend/libpq/auth-oauth.c
@@ -6,7 +6,7 @@
* See the following RFC for more details:
* - RFC 7628: https://tools.ietf.org/html/rfc7628
*
- * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* src/backend/libpq/auth-oauth.c
@@ -19,21 +19,29 @@
#include <fcntl.h>
#include "common/oauth-common.h"
+#include "fmgr.h"
#include "lib/stringinfo.h"
#include "libpq/auth.h"
#include "libpq/hba.h"
#include "libpq/oauth.h"
#include "libpq/sasl.h"
#include "storage/fd.h"
+#include "storage/ipc.h"
/* GUC */
-char *oauth_validator_command;
+char *OAuthValidatorLibrary = "";
static void oauth_get_mechanisms(Port *port, StringInfo buf);
static void *oauth_init(Port *port, const char *selected_mech, const char *shadow_pass);
static int oauth_exchange(void *opaq, const char *input, int inputlen,
char **output, int *outputlen, const char **logdetail);
+static void load_validator_library(void);
+static void shutdown_validator_library(int code, Datum arg);
+
+static ValidatorModuleState *validator_module_state;
+static const OAuthValidatorCallbacks *ValidatorCallbacks;
+
/* Mechanism declaration */
const pg_be_sasl_mech pg_be_oauth_mech = {
oauth_get_mechanisms,
@@ -62,11 +70,7 @@ struct oauth_ctx
static char *sanitize_char(char c);
static char *parse_kvpairs_for_auth(char **input);
static void generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen);
-static bool validate(Port *port, const char *auth, const char **logdetail);
-static bool run_validator_command(Port *port, const char *token);
-static bool check_exit(FILE **fh, const char *command);
-static bool set_cloexec(int fd);
-static bool username_ok_for_shell(const char *username);
+static bool validate(Port *port, const char *auth);
#define KVSEP 0x01
#define AUTH_KEY "auth"
@@ -99,6 +103,8 @@ oauth_init(Port *port, const char *selected_mech, const char *shadow_pass)
ctx->issuer = port->hba->oauth_issuer;
ctx->scope = port->hba->oauth_scope;
+ load_validator_library();
+
return ctx;
}
@@ -249,7 +255,7 @@ oauth_exchange(void *opaq, const char *input, int inputlen,
errmsg("malformed OAUTHBEARER message"),
errdetail("Message contains additional data after the final terminator.")));
- if (!validate(ctx->port, auth, logdetail))
+ if (!validate(ctx->port, auth))
{
generate_error_response(ctx, output, outputlen);
@@ -416,70 +422,73 @@ generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
*outputlen = buf.len;
}
-static bool
-validate(Port *port, const char *auth, const char **logdetail)
+/*-----
+ * Validates the provided Authorization header and returns the token from
+ * within it. NULL is returned on validation failure.
+ *
+ * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
+ * 2.1:
+ *
+ * b64token = 1*( ALPHA / DIGIT /
+ * "-" / "." / "_" / "~" / "+" / "/" ) *"="
+ * credentials = "Bearer" 1*SP b64token
+ *
+ * The "credentials" construction is what we receive in our auth value.
+ *
+ * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
+ * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
+ * compared case-insensitively. (This is not mentioned in RFC 6750, but
+ * it's pointed out in RFC 7628 Sec. 4.)
+ *
+ * Invalid formats are technically a protocol violation, but we shouldn't
+ * reflect any information about the sensitive Bearer token back to the
+ * client; log at COMMERROR instead.
+ *
+ * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
+ */
+static const char *
+validate_token_format(const char *header)
{
- static const char *const b64_set =
+ size_t span;
+ const char *token;
+ static const char *const b64token_allowed_set =
"abcdefghijklmnopqrstuvwxyz"
"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
"0123456789-._~+/";
- const char *token;
- size_t span;
- int ret;
+ /* If the token is empty or simply too short to be correct */
+ if (!header || strlen(header) <= 7)
+ {
+ ereport(COMMERROR,
+ (errmsg("malformed OAuth bearer token 1")));
+ return NULL;
+ }
- /* TODO: handle logdetail when the test framework can check it */
-
- /*-----
- * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
- * 2.1:
- *
- * b64token = 1*( ALPHA / DIGIT /
- * "-" / "." / "_" / "~" / "+" / "/" ) *"="
- * credentials = "Bearer" 1*SP b64token
- *
- * The "credentials" construction is what we receive in our auth value.
- *
- * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
- * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
- * compared case-insensitively. (This is not mentioned in RFC 6750, but
- * it's pointed out in RFC 7628 Sec. 4.)
- *
- * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
- */
- if (pg_strncasecmp(auth, BEARER_SCHEME, strlen(BEARER_SCHEME)))
- return false;
+ if (pg_strncasecmp(header, BEARER_SCHEME, strlen(BEARER_SCHEME)))
+ return NULL;
/* Pull the bearer token out of the auth value. */
- token = auth + strlen(BEARER_SCHEME);
+ token = header + strlen(BEARER_SCHEME);
/* Swallow any additional spaces. */
while (*token == ' ')
token++;
- /*
- * Before invoking the validator command, sanity-check the token format to
- * avoid any injection attacks later in the chain. Invalid formats are
- * technically a protocol violation, but don't reflect any information
- * about the sensitive Bearer token back to the client; log at COMMERROR
- * instead.
- */
-
/* Tokens must not be empty. */
if (!*token)
{
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("malformed OAUTHBEARER message"),
+ errmsg("malformed OAuth bearer token 2"),
errdetail("Bearer token is empty.")));
- return false;
+ return NULL;
}
/*
* Make sure the token contains only allowed characters. Tokens may end
* with any number of '=' characters.
*/
- span = strspn(token, b64_set);
+ span = strspn(token, b64token_allowed_set);
while (token[span] == '=')
span++;
@@ -492,15 +501,35 @@ validate(Port *port, const char *auth, const char **logdetail)
*/
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("malformed OAUTHBEARER message"),
+ errmsg("malformed OAuth bearer token 3"),
errdetail("Bearer token is not in the correct format.")));
- return false;
+ return NULL;
}
- /* Have the validator check the token. */
- if (!run_validator_command(port, token))
+ return token;
+}
+
+static bool
+validate(Port *port, const char *auth)
+{
+ int map_status;
+ ValidatorModuleResult *ret;
+ const char *token;
+
+ /* Ensure that we have a correct token to validate */
+ if (!(token = validate_token_format(auth)))
return false;
+ /* Call the validation function from the validator module */
+ ret = ValidatorCallbacks->validate_cb(validator_module_state,
+ token, port->user_name);
+
+ if (!ret->authenticated)
+ return false;
+
+ if (ret->authn_id)
+ set_authn_id(port, ret->authn_id);
+
if (port->hba->oauth_skip_usermap)
{
/*
@@ -513,7 +542,7 @@ validate(Port *port, const char *auth, const char **logdetail)
}
/* Make sure the validator authenticated the user. */
- if (!MyClientConnectionInfo.authn_id)
+ if (ret->authn_id == NULL || ret->authn_id[0] == '\0')
{
/* TODO: use logdetail; reduce message duplication */
ereport(LOG,
@@ -523,288 +552,42 @@ validate(Port *port, const char *auth, const char **logdetail)
}
/* Finally, check the user map. */
- ret = check_usermap(port->hba->usermap, port->user_name,
- MyClientConnectionInfo.authn_id, false);
- return (ret == STATUS_OK);
+ map_status = check_usermap(port->hba->usermap, port->user_name,
+ MyClientConnectionInfo.authn_id, false);
+ return (map_status == STATUS_OK);
}
-static bool
-run_validator_command(Port *port, const char *token)
-{
- bool success = false;
- int rc;
- int pipefd[2];
- int rfd = -1;
- int wfd = -1;
-
- StringInfoData command = {0};
- char *p;
- FILE *fh = NULL;
-
- ssize_t written;
- char *line = NULL;
- size_t size = 0;
- ssize_t len;
-
- Assert(oauth_validator_command);
-
- if (!oauth_validator_command[0])
- {
- ereport(COMMERROR,
- (errmsg("oauth_validator_command is not set"),
- errhint("To allow OAuth authenticated connections, set "
- "oauth_validator_command in postgresql.conf.")));
- return false;
- }
-
- /*------
- * Since popen() is unidirectional, open up a pipe for the other
- * direction. Use CLOEXEC to ensure that our write end doesn't
- * accidentally get copied into child processes, which would prevent us
- * from closing it cleanly.
- *
- * XXX this is ugly. We should just read from the child process's stdout,
- * but that's a lot more code.
- * XXX by bypassing the popen API, we open the potential of process
- * deadlock. Clearly document child process requirements (i.e. the child
- * MUST read all data off of the pipe before writing anything).
- * TODO: port to Windows using _pipe().
- */
- rc = pipe(pipefd);
- if (rc < 0)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not create child pipe: %m")));
- return false;
- }
-
- rfd = pipefd[0];
- wfd = pipefd[1];
-
- if (!set_cloexec(wfd))
- {
- /* error message was already logged */
- goto cleanup;
- }
-
- /*----------
- * Construct the command, substituting any recognized %-specifiers:
- *
- * %f: the file descriptor of the input pipe
- * %r: the role that the client wants to assume (port->user_name)
- * %%: a literal '%'
- */
- initStringInfo(&command);
-
- for (p = oauth_validator_command; *p; p++)
- {
- if (p[0] == '%')
- {
- switch (p[1])
- {
- case 'f':
- appendStringInfo(&command, "%d", rfd);
- p++;
- break;
- case 'r':
-
- /*
- * TODO: decide how this string should be escaped. The
- * role is controlled by the client, so if we don't escape
- * it, command injections are inevitable.
- *
- * This is probably an indication that the role name needs
- * to be communicated to the validator process in some
- * other way. For this proof of concept, just be
- * incredibly strict about the characters that are allowed
- * in user names.
- */
- if (!username_ok_for_shell(port->user_name))
- goto cleanup;
-
- appendStringInfoString(&command, port->user_name);
- p++;
- break;
- case '%':
- appendStringInfoChar(&command, '%');
- p++;
- break;
- default:
- appendStringInfoChar(&command, p[0]);
- }
- }
- else
- appendStringInfoChar(&command, p[0]);
- }
-
- /* Execute the command. */
- fh = OpenPipeStream(command.data, "r");
- if (!fh)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("opening pipe to OAuth validator: %m")));
- goto cleanup;
- }
-
- /* We don't need the read end of the pipe anymore. */
- close(rfd);
- rfd = -1;
-
- /* Give the command the token to validate. */
- written = write(wfd, token, strlen(token));
- if (written != strlen(token))
- {
- /* TODO must loop for short writes, EINTR et al */
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not write token to child pipe: %m")));
- goto cleanup;
- }
-
- close(wfd);
- wfd = -1;
-
- /*-----
- * Read the command's response.
- *
- * TODO: getline() is probably too new to use, unfortunately.
- * TODO: loop over all lines
- */
- if ((len = getline(&line, &size, fh)) >= 0)
- {
- /* TODO: fail if the authn_id doesn't end with a newline */
- if (len > 0)
- line[len - 1] = '\0';
-
- set_authn_id(port, line);
- }
- else if (ferror(fh))
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not read from command \"%s\": %m",
- command.data)));
- goto cleanup;
- }
-
- /* Make sure the command exits cleanly. */
- if (!check_exit(&fh, command.data))
- {
- /* error message already logged */
- goto cleanup;
- }
-
- /* Done. */
- success = true;
-
-cleanup:
- if (line)
- free(line);
-
- /*
- * In the successful case, the pipe fds are already closed. For the error
- * case, always close out the pipe before waiting for the command, to
- * prevent deadlock.
- */
- if (rfd >= 0)
- close(rfd);
- if (wfd >= 0)
- close(wfd);
-
- if (fh)
- {
- Assert(!success);
- check_exit(&fh, command.data);
- }
-
- if (command.data)
- pfree(command.data);
-
- return success;
-}
-
-static bool
-check_exit(FILE **fh, const char *command)
+static void
+load_validator_library(void)
{
- int rc;
+ OAuthValidatorModuleInit validator_init;
- rc = ClosePipeStream(*fh);
- *fh = NULL;
-
- if (rc == -1)
- {
- /* pclose() itself failed. */
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not close pipe to command \"%s\": %m",
- command)));
- }
- else if (rc != 0)
- {
- char *reason = wait_result_to_str(rc);
-
- ereport(COMMERROR,
- (errmsg("failed to execute command \"%s\": %s",
- command, reason)));
-
- pfree(reason);
- }
+ if (OAuthValidatorLibrary[0] == '\0')
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("oauth_validator_library is not set")));
- return (rc == 0);
-}
+ validator_init = (OAuthValidatorModuleInit)
+ load_external_function(OAuthValidatorLibrary,
+ "_PG_oauth_validator_module_init", false, NULL);
-static bool
-set_cloexec(int fd)
-{
- int flags;
- int rc;
+ if (validator_init == NULL)
+ ereport(ERROR,
+ (errmsg("%s module \"%s\" have to define the symbol %s",
+ "OAuth validator", OAuthValidatorLibrary, "_PG_oauth_validator_module_init")));
- flags = fcntl(fd, F_GETFD);
- if (flags == -1)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not get fd flags for child pipe: %m")));
- return false;
- }
+ ValidatorCallbacks = (*validator_init) ();
- rc = fcntl(fd, F_SETFD, flags | FD_CLOEXEC);
- if (rc < 0)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not set FD_CLOEXEC for child pipe: %m")));
- return false;
- }
+ validator_module_state = (ValidatorModuleState *) palloc0(sizeof(ValidatorModuleState));
+ if (ValidatorCallbacks->startup_cb != NULL)
+ ValidatorCallbacks->startup_cb(validator_module_state);
- return true;
+ before_shmem_exit(shutdown_validator_library, 0);
}
-/*
- * XXX This should go away eventually and be replaced with either a proper
- * escape or a different strategy for communication with the validator command.
- */
-static bool
-username_ok_for_shell(const char *username)
+static void
+shutdown_validator_library(int code, Datum arg)
{
- /* This set is borrowed from fe_utils' appendShellStringNoError(). */
- static const char *const allowed =
- "abcdefghijklmnopqrstuvwxyz"
- "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
- "0123456789-_./:";
- size_t span;
-
- Assert(username && username[0]); /* should have already been checked */
-
- span = strspn(username, allowed);
- if (username[span] != '\0')
- {
- ereport(COMMERROR,
- (errmsg("PostgreSQL user name contains unsafe characters and cannot be passed to the OAuth validator")));
- return false;
- }
-
- return true;
+ if (ValidatorCallbacks->shutdown_cb != NULL)
+ ValidatorCallbacks->shutdown_cb(validator_module_state);
}
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 0f83f0d870..a479b679d1 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -4672,12 +4672,12 @@ struct config_string ConfigureNamesString[] =
},
{
- {"oauth_validator_command", PGC_SIGHUP, CONN_AUTH_AUTH,
- gettext_noop("Command to validate OAuth v2 bearer tokens."),
+ {"oauth_validator_library", PGC_SIGHUP, CONN_AUTH_AUTH,
+ gettext_noop("Sets the library that will be called to validate OAuth v2 bearer tokens."),
NULL,
GUC_SUPERUSER_ONLY | GUC_NOT_IN_SAMPLE
},
- &oauth_validator_command,
+ &OAuthValidatorLibrary,
"",
NULL, NULL, NULL
},
diff --git a/src/bin/pg_combinebackup/Makefile b/src/bin/pg_combinebackup/Makefile
index c3729755ba..4f24b1aff6 100644
--- a/src/bin/pg_combinebackup/Makefile
+++ b/src/bin/pg_combinebackup/Makefile
@@ -31,7 +31,7 @@ OBJS = \
all: pg_combinebackup
pg_combinebackup: $(OBJS) | submake-libpgport submake-libpgfeutils
- $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(libpq_pgport) $(LIBS) -o $@$(X)
install: all installdirs
$(INSTALL_PROGRAM) pg_combinebackup$(X) '$(DESTDIR)$(bindir)/pg_combinebackup$(X)'
diff --git a/src/common/Makefile b/src/common/Makefile
index bbb5c3ab11..00e30e6bfe 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -41,7 +41,7 @@ override CPPFLAGS += -DVAL_LDFLAGS_SL="\"$(LDFLAGS_SL)\""
override CPPFLAGS += -DVAL_LIBS="\"$(LIBS)\""
override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common -I$(libpq_srcdir) $(CPPFLAGS)
-LIBS += $(PTHREAD_LIBS)
+LIBS += $(PTHREAD_LIBS) $(libpq_pgport)
OBJS_COMMON = \
archive.o \
diff --git a/src/include/libpq/oauth.h b/src/include/libpq/oauth.h
index 5edab3b25a..5c081abfae 100644
--- a/src/include/libpq/oauth.h
+++ b/src/include/libpq/oauth.h
@@ -3,7 +3,7 @@
* oauth.h
* Interface to libpq/auth-oauth.c
*
- * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* src/include/libpq/oauth.h
@@ -16,7 +16,32 @@
#include "libpq/libpq-be.h"
#include "libpq/sasl.h"
-extern char *oauth_validator_command;
+extern PGDLLIMPORT char *OAuthValidatorLibrary;
+
+typedef struct ValidatorModuleState
+{
+ void *private_data;
+} ValidatorModuleState;
+
+typedef struct ValidatorModuleResult
+{
+ bool authenticated;
+ char *authn_id;
+} ValidatorModuleResult;
+
+typedef void (*ValidatorStartupCB) (ValidatorModuleState *state);
+typedef void (*ValidatorShutdownCB) (ValidatorModuleState *state);
+typedef ValidatorModuleResult *(*ValidatorValidateCB) (ValidatorModuleState *state, const char *token, const char *role);
+
+typedef struct OAuthValidatorCallbacks
+{
+ ValidatorStartupCB startup_cb;
+ ValidatorShutdownCB shutdown_cb;
+ ValidatorValidateCB validate_cb;
+} OAuthValidatorCallbacks;
+
+typedef const OAuthValidatorCallbacks *(*OAuthValidatorModuleInit) (void);
+extern PGDLLEXPORT const OAuthValidatorCallbacks *_PG_oauth_validator_module_init(void);
/* Implementation */
extern const pg_be_sasl_mech pg_be_oauth_mech;
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index 8fbe742d38..dc54ce7189 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -9,6 +9,7 @@ subdir('gin')
subdir('injection_points')
subdir('ldap_password_func')
subdir('libpq_pipeline')
+subdir('oauth_validator')
subdir('plsample')
subdir('spgist_name_ops')
subdir('ssl_passphrase_callback')
diff --git a/src/test/modules/oauth_validator/.gitignore b/src/test/modules/oauth_validator/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/src/test/modules/oauth_validator/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/oauth_validator/Makefile b/src/test/modules/oauth_validator/Makefile
new file mode 100644
index 0000000000..1f874cd7f2
--- /dev/null
+++ b/src/test/modules/oauth_validator/Makefile
@@ -0,0 +1,19 @@
+MODULES = validator
+PGFILEDESC = "validator - test OAuth validator module"
+
+NO_INSTALLCHECK = 1
+
+TAP_TESTS = 1
+
+REGRESS = validator
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/oauth_validator
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/oauth_validator/expected/validator.out b/src/test/modules/oauth_validator/expected/validator.out
new file mode 100644
index 0000000000..360caa2cb3
--- /dev/null
+++ b/src/test/modules/oauth_validator/expected/validator.out
@@ -0,0 +1,6 @@
+SELECT 1;
+ ?column?
+----------
+ 1
+(1 row)
+
diff --git a/src/test/modules/oauth_validator/meson.build b/src/test/modules/oauth_validator/meson.build
new file mode 100644
index 0000000000..d9c1d1d577
--- /dev/null
+++ b/src/test/modules/oauth_validator/meson.build
@@ -0,0 +1,33 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+validator_sources = files(
+ 'validator.c',
+)
+
+if host_system == 'windows'
+ validator_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'validator',
+ '--FILEDESC', 'validator - test OAuth validator module',])
+endif
+
+validator = shared_module('validator',
+ validator_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += validator
+
+tests += {
+ 'name': 'oauth_validator',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'regress': {
+ 'sql': [
+ 'validator',
+ ],
+ },
+ 'tap': {
+ 'tests': [
+ 't/001_server.pl',
+ ],
+ },
+}
diff --git a/src/test/modules/oauth_validator/sql/validator.sql b/src/test/modules/oauth_validator/sql/validator.sql
new file mode 100644
index 0000000000..e0ac49d1ec
--- /dev/null
+++ b/src/test/modules/oauth_validator/sql/validator.sql
@@ -0,0 +1 @@
+SELECT 1;
diff --git a/src/test/modules/oauth_validator/t/001_server.pl b/src/test/modules/oauth_validator/t/001_server.pl
new file mode 100644
index 0000000000..49e04b0afe
--- /dev/null
+++ b/src/test/modules/oauth_validator/t/001_server.pl
@@ -0,0 +1,53 @@
+
+# Copyright (c) 2021-2024, PostgreSQL Global Development Group
+
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::OAuthServer;
+use Test::More;
+
+# Delete pg_hba.conf from the given node, add a new entry to it
+# and then execute a reload to refresh it.
+# XXX: this is copied from authentication/t/001_password and should be made
+# generic functionality if we end up using it.
+sub reset_pg_hba
+{
+ my $node = shift;
+ my $database = shift;
+ my $role = shift;
+ my $hba_method = shift;
+
+ unlink($node->data_dir . '/pg_hba.conf');
+ # just for testing purposes, use a continuation line
+ $node->append_conf('pg_hba.conf',
+ "local $database $role\\\n $hba_method");
+ $node->reload;
+ return;
+}
+
+my $node = PostgreSQL::Test::Cluster->new('primary');
+$node->init;
+$node->append_conf('postgresql.conf', "shared_preload_libraries = 'validator'\n");
+$node->append_conf('postgresql.conf', "oauth_validator_library = 'validator'\n");
+$node->start;
+
+reset_pg_hba($node, 'all', 'all', 'oauth issuer="127.0.0.1:18080" scope="openid postgres"');
+
+my $webserver = PostgreSQL::Test::OAuthServer->new(18080);
+
+my $port = $webserver->port();
+
+is($port, 18080, "Port is 18080");
+
+$webserver->setup();
+$webserver->run();
+
+$node->connect_ok("dbname=postgres oauth_client_id=f02c6361-0635", "connect",
+ expected_stderr => qr@Visit https://example\.com/ and enter the code: postgresuser@);
+
+$node->stop;
+
+done_testing();
diff --git a/src/test/modules/oauth_validator/validator.c b/src/test/modules/oauth_validator/validator.c
new file mode 100644
index 0000000000..c76d0599c5
--- /dev/null
+++ b/src/test/modules/oauth_validator/validator.c
@@ -0,0 +1,71 @@
+/*-------------------------------------------------------------------------
+ *
+ * validator.c
+ * Test module for serverside OAuth token validation callbacks
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/test/modules/oauth_validator/validator.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "libpq/oauth.h"
+#include "utils/memutils.h"
+
+PG_MODULE_MAGIC;
+
+static void validator_startup(ValidatorModuleState *state);
+static void validator_shutdown(ValidatorModuleState *state);
+static ValidatorModuleResult * validate_token(ValidatorModuleState *state,
+ const char *token,
+ const char *role);
+
+static const OAuthValidatorCallbacks validator_callbacks = {
+ .startup_cb = validator_startup,
+ .shutdown_cb = validator_shutdown,
+ .validate_cb = validate_token
+};
+
+void
+_PG_init(void)
+{
+ /* no-op */
+}
+
+const OAuthValidatorCallbacks *
+_PG_oauth_validator_module_init(void)
+{
+ return &validator_callbacks;
+}
+
+static void
+validator_startup(ValidatorModuleState *state)
+{
+ /* do nothing */
+}
+
+static void
+validator_shutdown(ValidatorModuleState *state)
+{
+ /* do nothing */
+}
+
+static ValidatorModuleResult *
+validate_token(ValidatorModuleState *state, const char *token, const char *role)
+{
+ ValidatorModuleResult *res;
+
+ res = palloc(sizeof(ValidatorModuleResult));
+
+ elog(LOG, "XXX: validating %s for %s", token, role);
+
+ res->authenticated = true;
+ res->authn_id = pstrdup(role);
+
+ return res;
+}
diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm
index 44c1bb5afd..b758ad01cc 100644
--- a/src/test/perl/PostgreSQL/Test/Cluster.pm
+++ b/src/test/perl/PostgreSQL/Test/Cluster.pm
@@ -2302,6 +2302,11 @@ instead of the default.
If this regular expression is set, matches it with the output generated.
+=item expected_stderr => B<value>
+
+If this regular expression is set, matches it against the standard error
+stream; otherwise the stderr must be empty.
+
=item log_like => [ qr/required message/ ]
=item log_unlike => [ qr/prohibited message/ ]
@@ -2345,7 +2350,14 @@ sub connect_ok
like($stdout, $params{expected_stdout}, "$test_name: stdout matches");
}
- is($stderr, "", "$test_name: no stderr");
+ if (defined($params{expected_stderr}))
+ {
+ like($stderr, $params{expected_stderr}, "$test_name: stderr matches");
+ }
+ else
+ {
+ is($stderr, "", "$test_name: no stderr");
+ }
$self->log_check($test_name, $log_location, %params);
}
diff --git a/src/test/perl/PostgreSQL/Test/OAuthServer.pm b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
new file mode 100644
index 0000000000..3ac90c3d0f
--- /dev/null
+++ b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
@@ -0,0 +1,158 @@
+#!/usr/bin/perl
+
+package PostgreSQL::Test::OAuthServer;
+
+use warnings;
+use strict;
+use threads;
+use Socket;
+use IO::Select;
+
+local *server_socket;
+
+sub new
+{
+ my $class = shift;
+ my $port = shift;
+
+ my $self = {};
+ bless($self, $class);
+
+ $self->{'port'} = $port;
+
+ return $self;
+}
+
+sub setup
+{
+ my $self = shift;
+ my $tcp = getprotobyname('tcp');
+
+ socket($self->{'socket'}, PF_INET, SOCK_STREAM, $tcp)
+ or die "no socket";
+ setsockopt($self->{'socket'}, SOL_SOCKET, SO_REUSEADDR, pack("l", 1));
+ bind($self->{'socket'}, sockaddr_in($self->{'port'}, INADDR_ANY));
+}
+
+sub port
+{
+ my $self = shift;
+
+ return $self->{'port'};
+}
+
+sub run
+{
+ my $self = shift;
+
+ my $server_thread = threads->create(\&_listen, $self);
+ $server_thread->detach();
+}
+
+sub _listen
+{
+ my $self = shift;
+
+ listen($self->{'socket'}, SOMAXCONN) or die "fail to listen: $!";
+
+ while (1)
+ {
+ my $fh;
+ my %request;
+ my $remote = accept($fh, $self->{'socket'});
+ binmode $fh;
+
+ my ($method, $object, $prot) = split(/ /, <$fh>);
+ $request{'method'} = $method;
+ $request{'object'} = $object;
+ chomp($request{'object'});
+
+ local $/ = Socket::CRLF;
+ my $c = 0;
+ while(<$fh>)
+ {
+ chomp;
+ # Headers
+ if (/:/)
+ {
+ my ($field, $value) = split(/:/, $_, 2);
+ $value =~ s/^\s+//;
+ $request{'headers'}{lc $field} = $value;
+ }
+ # POST data
+ elsif (/^$/)
+ {
+ read($fh, $request{'content'}, $request{'headers'}{'content-length'})
+ if defined $request{'headers'}{'content-length'};
+ last;
+ }
+ }
+
+ # Debug printing
+ # print ": read ".$request{'method'} . ";" . $request{'object'}.";\n";
+ # foreach my $h (keys(%{$request{'headers'}}))
+ #{
+ # printf ": headers: " . $request{'headers'}{$h} . "\n";
+ #}
+ #printf ": POST: " . $request{'content'} . "\n" if defined $request{'content'};
+
+ if ($request{'object'} eq '/.well-known/openid-configuration')
+ {
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: application/json\r\n";
+ print $fh "\r\n";
+ print $fh <<EOR;
+ {
+ "issuer": "http://localhost:$self->{'port'}",
+ "token_endpoint": "http://localhost:$self->{'port'}/token",
+ "device_authorization_endpoint": "http://localhost:$self->{'port'}/authorize",
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ "grant_types_supported": ["urn:ietf:params:oauth:grant-type:device_code"]
+ }
+EOR
+ }
+ elsif ($request{'object'} eq '/authorize')
+ {
+ print ": returning device_code\n";
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: application/json\r\n";
+ print $fh "\r\n";
+ print $fh <<EOR;
+ {
+ "device_code": "postgres",
+ "user_code" : "postgresuser",
+ "interval" : 0,
+ "verification_uri" : "https://example.com/",
+ "expires-in": 5
+ }
+EOR
+ }
+ elsif ($request{'object'} eq '/token')
+ {
+ print ": returning token\n";
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: application/json\r\n";
+ print $fh "\r\n";
+ print $fh <<EOR;
+ {
+ "access_token": "9243959234",
+ "token_type": "bearer"
+ }
+EOR
+ }
+ else
+ {
+ print ": returning default\n";
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: text/html\r\n";
+ print $fh "\r\n";
+ print $fh "Ok\n";
+ }
+
+ close($fh);
+ }
+}
+
+1;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index fe65a4222b..2bca506e16 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1657,6 +1657,7 @@ NumericSortSupport
NumericSumAccum
NumericVar
OAuthStep
+OAuthValidatorCallbacks
OM_uint32
OP
OSAPerGroupState
@@ -2980,6 +2981,7 @@ VacuumRelation
VacuumStmt
ValidIOData
ValidateIndexState
+ValidatorModuleState
ValuesScan
ValuesScanState
Var
--
2.34.1
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-03-01 17:46 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2024-03-01 17:46 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: pgsql-hackers; Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>; Andrew Dunstan <andrew@dunslane.net>
On Thu, Feb 29, 2024 at 5:08 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> We are now very, very close to green.
v19 gets us a bit closer by adding a missed import for Windows. I've
also removed iddawc support, so the client patch is lighter.
> The new oauth_validator tests can't work on Windows, since the client
> doesn't support OAuth there. The python/server tests can handle this
> case, since they emulate the client behavior; do we want to try
> something similar in Perl?
In addition to this question, I'm starting to notice intermittent
failures of the form
error: ... failed to fetch OpenID discovery document: failed to
queue HTTP request
This corresponds to a TODO in the libcurl implementation -- if the
initial call to curl_multi_socket_action() reports that no handles are
running, I treated that as an error. But it looks like it's possible
for libcurl to finish a request synchronously if the remote responds
quickly enough, so that needs to change.
--Jacob
Attachments: download all (10 files, .tar.gz)
since-v18.diff.txt (text/plain, 26.5K) download
1: e2a0b48561 = 1: ce06c03e2b common/jsonapi: support FRONTEND clients
2: db625e1d01 = 2: 6989b75153 Refactor SASL exchange to return tri-state status
3: e4ad0260d5 = 3: 783bfe0b95 Explicitly require password for SCRAM exchange
4: 229f602d5c ! 4: 77550a47db libpq: add OAUTHBEARER SASL mechanism
@@ Commit message
are currently implemented (but clients may provide their own flows; see
below).
- The client implementation requires either libcurl or libiddawc and their
- development headers. Pass `curl` or `iddawc` to --with-oauth/-Doauth
- during configuration.
+ The client implementation requires libcurl and its development headers.
+ Pass `curl` to --with-oauth/-Doauth during configuration.
Thomas Munro wrote the kqueue() implementation for oauth-curl; thanks!
@@ configure: Optional Packages:
--with-pam build with PAM support
--with-bsd-auth build with BSD Authentication support
--with-ldap build with LDAP support
-+ --with-oauth=LIB use LIB for OAuth 2.0 support (curl, iddawc)
++ --with-oauth=LIB use LIB for OAuth 2.0 support (curl)
--with-bonjour build with Bonjour support
--with-selinux build with SELinux support
--with-systemd build with systemd support
@@ configure: $as_echo "$with_ldap" >&6; }
+
+$as_echo "#define USE_OAUTH_CURL 1" >>confdefs.h
+
-+elif test x"$with_oauth" = x"iddawc"; then
-+
-+$as_echo "#define USE_OAUTH 1" >>confdefs.h
-+
-+
-+$as_echo "#define USE_OAUTH_IDDAWC 1" >>confdefs.h
-+
+elif test x"$with_oauth" != x"no"; then
-+ as_fn_error $? "--with-oauth must specify curl or iddawc" "$LINENO" 5
++ as_fn_error $? "--with-oauth must specify curl" "$LINENO" 5
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_oauth" >&5
@@ configure: fi
+ as_fn_error $? "library 'curl' is required for --with-oauth=curl" "$LINENO" 5
+fi
+
-+elif test "$with_oauth" = iddawc ; then
-+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for i_init_session in -liddawc" >&5
-+$as_echo_n "checking for i_init_session in -liddawc... " >&6; }
-+if ${ac_cv_lib_iddawc_i_init_session+:} false; then :
-+ $as_echo_n "(cached) " >&6
-+else
-+ ac_check_lib_save_LIBS=$LIBS
-+LIBS="-liddawc $LIBS"
-+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-+/* end confdefs.h. */
-+
-+/* Override any GCC internal prototype to avoid an error.
-+ Use char because int might match the return type of a GCC
-+ builtin and then its argument prototype would still apply. */
-+#ifdef __cplusplus
-+extern "C"
-+#endif
-+char i_init_session ();
-+int
-+main ()
-+{
-+return i_init_session ();
-+ ;
-+ return 0;
-+}
-+_ACEOF
-+if ac_fn_c_try_link "$LINENO"; then :
-+ ac_cv_lib_iddawc_i_init_session=yes
-+else
-+ ac_cv_lib_iddawc_i_init_session=no
-+fi
-+rm -f core conftest.err conftest.$ac_objext \
-+ conftest$ac_exeext conftest.$ac_ext
-+LIBS=$ac_check_lib_save_LIBS
-+fi
-+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_iddawc_i_init_session" >&5
-+$as_echo "$ac_cv_lib_iddawc_i_init_session" >&6; }
-+if test "x$ac_cv_lib_iddawc_i_init_session" = xyes; then :
-+ cat >>confdefs.h <<_ACEOF
-+#define HAVE_LIBIDDAWC 1
-+_ACEOF
-+
-+ LIBS="-liddawc $LIBS"
-+
-+else
-+ as_fn_error $? "library 'iddawc' is required for --with-oauth=iddawc" "$LINENO" 5
-+fi
-+
-+ # Check for an older spelling of i_get_openid_config
-+ for ac_func in i_load_openid_config
-+do :
-+ ac_fn_c_check_func "$LINENO" "i_load_openid_config" "ac_cv_func_i_load_openid_config"
-+if test "x$ac_cv_func_i_load_openid_config" = xyes; then :
-+ cat >>confdefs.h <<_ACEOF
-+#define HAVE_I_LOAD_OPENID_CONFIG 1
-+_ACEOF
-+
-+fi
-+done
-+
+fi
+
# for contrib/sepgsql
@@ configure: fi
+ as_fn_error $? "header file <curl/curl.h> is required for OAuth" "$LINENO" 5
+fi
+
-+
-+elif test "$with_oauth" = iddawc; then
-+ ac_fn_c_check_header_mongrel "$LINENO" "iddawc.h" "ac_cv_header_iddawc_h" "$ac_includes_default"
-+if test "x$ac_cv_header_iddawc_h" = xyes; then :
-+
-+else
-+ as_fn_error $? "header file <iddawc.h> is required for OAuth" "$LINENO" 5
-+fi
-+
+
fi
@@ configure.ac: AC_MSG_RESULT([$with_ldap])
+# OAuth 2.0
+#
+AC_MSG_CHECKING([whether to build with OAuth support])
-+PGAC_ARG_REQ(with, oauth, [LIB], [use LIB for OAuth 2.0 support (curl, iddawc)])
++PGAC_ARG_REQ(with, oauth, [LIB], [use LIB for OAuth 2.0 support (curl)])
+if test x"$with_oauth" = x"" ; then
+ with_oauth=no
+fi
@@ configure.ac: AC_MSG_RESULT([$with_ldap])
+if test x"$with_oauth" = x"curl"; then
+ AC_DEFINE([USE_OAUTH], 1, [Define to 1 to build with OAuth 2.0 support. (--with-oauth)])
+ AC_DEFINE([USE_OAUTH_CURL], 1, [Define to 1 to use libcurl for OAuth support.])
-+elif test x"$with_oauth" = x"iddawc"; then
-+ AC_DEFINE([USE_OAUTH], 1, [Define to 1 to build with OAuth 2.0 support. (--with-oauth)])
-+ AC_DEFINE([USE_OAUTH_IDDAWC], 1, [Define to 1 to use libiddawc for OAuth support.])
+elif test x"$with_oauth" != x"no"; then
-+ AC_MSG_ERROR([--with-oauth must specify curl or iddawc])
++ AC_MSG_ERROR([--with-oauth must specify curl])
+fi
+
+AC_MSG_RESULT([$with_oauth])
@@ configure.ac: fi
+if test "$with_oauth" = curl ; then
+ AC_CHECK_LIB(curl, curl_multi_init, [], [AC_MSG_ERROR([library 'curl' is required for --with-oauth=curl])])
-+elif test "$with_oauth" = iddawc ; then
-+ AC_CHECK_LIB(iddawc, i_init_session, [], [AC_MSG_ERROR([library 'iddawc' is required for --with-oauth=iddawc])])
-+ # Check for an older spelling of i_get_openid_config
-+ AC_CHECK_FUNCS([i_load_openid_config])
+fi
+
# for contrib/sepgsql
@@ configure.ac: elif test "$with_uuid" = ossp ; then
+if test "$with_oauth" = curl; then
+ AC_CHECK_HEADER(curl/curl.h, [], [AC_MSG_ERROR([header file <curl/curl.h> is required for OAuth])])
-+elif test "$with_oauth" = iddawc; then
-+ AC_CHECK_HEADER(iddawc.h, [], [AC_MSG_ERROR([header file <iddawc.h> is required for OAuth])])
+fi
+
if test "$PORTNAME" = "win32" ; then
@@ meson.build: endif
+ endif
+endif
+
-+if not oauth.found() and oauthopt in ['auto', 'iddawc']
-+ oauth = dependency('libiddawc', required: (oauthopt == 'iddawc'))
-+
-+ if oauth.found()
-+ oauth_library = 'iddawc'
-+ cdata.set('USE_OAUTH', 1)
-+ cdata.set('USE_OAUTH_IDDAWC', 1)
-+
-+ # Check for an older spelling of i_get_openid_config
-+ if cc.has_function('i_load_openid_config',
-+ dependencies: oauth, args: test_c_args)
-+ cdata.set('HAVE_I_LOAD_OPENID_CONFIG', 1)
-+ endif
-+ endif
-+endif
-+
+if oauthopt == 'auto' and auto_features.enabled() and not oauth.found()
+ error('no OAuth implementation library found')
+endif
@@ meson_options.txt: option('lz4', type: 'feature', value: 'auto',
option('nls', type: 'feature', value: 'auto',
description: 'Native language support')
-+option('oauth', type : 'combo', choices : ['auto', 'none', 'curl', 'iddawc'],
++option('oauth', type : 'combo', choices : ['auto', 'none', 'curl'],
+ value: 'auto',
-+ description: 'use LIB for OAuth 2.0 support (curl, iddawc)')
++ description: 'use LIB for OAuth 2.0 support (curl)')
+
option('pam', type: 'feature', value: 'auto',
description: 'PAM support')
@@ src/include/common/oauth-common.h (new)
+#endif /* OAUTH_COMMON_H */
## src/include/pg_config.h.in ##
-@@
- /* Define to 1 if __builtin_constant_p(x) implies "i"(x) acceptance. */
- #undef HAVE_I_CONSTRAINT__BUILTIN_CONSTANT_P
-
-+/* Define to 1 if you have the `i_load_openid_config' function. */
-+#undef HAVE_I_LOAD_OPENID_CONFIG
-+
- /* Define to 1 if you have the `kqueue' function. */
- #undef HAVE_KQUEUE
-
@@
/* Define to 1 if you have the `crypto' library (-lcrypto). */
#undef HAVE_LIBCRYPTO
+/* Define to 1 if you have the `curl' library (-lcurl). */
+#undef HAVE_LIBCURL
-+
-+/* Define to 1 if you have the `iddawc' library (-liddawc). */
-+#undef HAVE_LIBIDDAWC
+
/* Define to 1 if you have the `ldap' library (-lldap). */
#undef HAVE_LIBLDAP
@@ src/include/pg_config.h.in
+
+/* Define to 1 to use libcurl for OAuth support. */
+#undef USE_OAUTH_CURL
-+
-+/* Define to 1 to use libiddawc for OAuth support. */
-+#undef USE_OAUTH_IDDAWC
+
/* Define to 1 to build with OpenSSL support. (--with-ssl=openssl) */
#undef USE_OPENSSL
@@ src/interfaces/libpq/Makefile: OBJS += \
+ifneq ($(with_oauth),no)
+OBJS += fe-auth-oauth.o
+
-+ifeq ($(with_oauth),iddawc)
-+OBJS += fe-auth-oauth-iddawc.o
-+else
++ifeq ($(with_oauth),curl)
+OBJS += fe-auth-oauth-curl.o
+endif
+endif
@@ src/interfaces/libpq/Makefile: endif
SHLIB_LINK_INTERNAL = -lpgcommon_shlib -lpgport_shlib
ifneq ($(PORTNAME), win32)
-SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
-+SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lcurl -liddawc -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
++SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lcurl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
else
SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi32 -lssl -lsocket -lnsl -lresolv -lintl -lm $(PTHREAD_LIBS), $(LIBS)) $(LDAP_LIBS_FE)
endif
@@ src/interfaces/libpq/fe-auth-oauth-curl.c (new)
+
+ free_token(&tok);
+ return PGRES_POLLING_FAILED;
-+}
-
- ## src/interfaces/libpq/fe-auth-oauth-iddawc.c (new) ##
-@@
-+/*-------------------------------------------------------------------------
-+ *
-+ * fe-auth-oauth-iddawc.c
-+ * The libiddawc implementation of OAuth/OIDC authentication.
-+ *
-+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
-+ * Portions Copyright (c) 1994, Regents of the University of California
-+ *
-+ * IDENTIFICATION
-+ * src/interfaces/libpq/fe-auth-oauth-iddawc.c
-+ *
-+ *-------------------------------------------------------------------------
-+ */
-+
-+#include "postgres_fe.h"
-+
-+#include <iddawc.h>
-+
-+#include "fe-auth.h"
-+#include "fe-auth-oauth.h"
-+#include "libpq-int.h"
-+
-+#ifdef HAVE_I_LOAD_OPENID_CONFIG
-+/* Older versions of iddawc used 'load' instead of 'get' for some APIs. */
-+#define i_get_openid_config i_load_openid_config
-+#endif
-+
-+static const char *
-+iddawc_error_string(int errcode)
-+{
-+ switch (errcode)
-+ {
-+ case I_OK:
-+ return "I_OK";
-+
-+ case I_ERROR:
-+ return "I_ERROR";
-+
-+ case I_ERROR_PARAM:
-+ return "I_ERROR_PARAM";
-+
-+ case I_ERROR_MEMORY:
-+ return "I_ERROR_MEMORY";
-+
-+ case I_ERROR_UNAUTHORIZED:
-+ return "I_ERROR_UNAUTHORIZED";
-+
-+ case I_ERROR_SERVER:
-+ return "I_ERROR_SERVER";
-+ }
-+
-+ return "<unknown>";
-+}
-+
-+static void
-+iddawc_error(PGconn *conn, int errcode, const char *msg)
-+{
-+ appendPQExpBufferStr(&conn->errorMessage, libpq_gettext(msg));
-+ appendPQExpBuffer(&conn->errorMessage,
-+ libpq_gettext(" (iddawc error %s)\n"),
-+ iddawc_error_string(errcode));
-+}
-+
-+static void
-+iddawc_request_error(PGconn *conn, struct _i_session *i, int err, const char *msg)
-+{
-+ const char *error_code;
-+ const char *desc;
-+
-+ appendPQExpBuffer(&conn->errorMessage, "%s: ", libpq_gettext(msg));
-+
-+ error_code = i_get_str_parameter(i, I_OPT_ERROR);
-+ if (!error_code)
-+ {
-+ /*
-+ * The server didn't give us any useful information, so just print the
-+ * error code.
-+ */
-+ appendPQExpBuffer(&conn->errorMessage,
-+ libpq_gettext("(iddawc error %s)\n"),
-+ iddawc_error_string(err));
-+ return;
-+ }
-+
-+ /* If the server gave a string description, print that too. */
-+ desc = i_get_str_parameter(i, I_OPT_ERROR_DESCRIPTION);
-+ if (desc)
-+ appendPQExpBuffer(&conn->errorMessage, "%s ", desc);
-+
-+ appendPQExpBuffer(&conn->errorMessage, "(%s)\n", error_code);
-+}
-+
-+/*
-+ * Runs the device authorization flow using libiddawc. If successful, a malloc'd
-+ * token string in "Bearer xxxx..." format, suitable for sending to an
-+ * OAUTHBEARER server, is returned. NULL is returned on error.
-+ */
-+static char *
-+run_iddawc_auth_flow(PGconn *conn, const char *discovery_uri)
-+{
-+ struct _i_session session;
-+ PQExpBuffer token_buf = NULL;
-+ int err;
-+ int auth_method;
-+ bool user_prompted = false;
-+ const char *verification_uri;
-+ const char *user_code;
-+ const char *access_token;
-+ const char *token_type;
-+ char *token = NULL;
-+
-+ i_init_session(&session);
-+
-+ token_buf = createPQExpBuffer();
-+ if (!token_buf)
-+ goto cleanup;
-+
-+ err = i_set_str_parameter(&session, I_OPT_OPENID_CONFIG_ENDPOINT, discovery_uri);
-+ if (err)
-+ {
-+ iddawc_error(conn, err, "failed to set OpenID config endpoint");
-+ goto cleanup;
-+ }
-+
-+ err = i_get_openid_config(&session);
-+ if (err)
-+ {
-+ iddawc_error(conn, err, "failed to fetch OpenID discovery document");
-+ goto cleanup;
-+ }
-+
-+ if (!i_get_str_parameter(&session, I_OPT_TOKEN_ENDPOINT))
-+ {
-+ appendPQExpBufferStr(&conn->errorMessage,
-+ libpq_gettext("issuer has no token endpoint"));
-+ goto cleanup;
-+ }
-+
-+ if (!i_get_str_parameter(&session, I_OPT_DEVICE_AUTHORIZATION_ENDPOINT))
-+ {
-+ appendPQExpBufferStr(&conn->errorMessage,
-+ libpq_gettext("issuer does not support device authorization"));
-+ goto cleanup;
-+ }
-+
-+ err = i_set_response_type(&session, I_RESPONSE_TYPE_DEVICE_CODE);
-+ if (err)
-+ {
-+ iddawc_error(conn, err, "failed to set device code response type");
-+ goto cleanup;
-+ }
-+
-+ auth_method = I_TOKEN_AUTH_METHOD_NONE;
-+ if (conn->oauth_client_secret && *conn->oauth_client_secret)
-+ auth_method = I_TOKEN_AUTH_METHOD_SECRET_BASIC;
-+
-+ err = i_set_parameter_list(&session,
-+ I_OPT_CLIENT_ID, conn->oauth_client_id,
-+ I_OPT_CLIENT_SECRET, conn->oauth_client_secret,
-+ I_OPT_TOKEN_METHOD, auth_method,
-+ I_OPT_SCOPE, conn->oauth_scope,
-+ I_OPT_NONE
-+ );
-+ if (err)
-+ {
-+ iddawc_error(conn, err, "failed to set client identifier");
-+ goto cleanup;
-+ }
-+
-+ err = i_run_device_auth_request(&session);
-+ if (err)
-+ {
-+ iddawc_request_error(conn, &session, err,
-+ "failed to obtain device authorization");
-+ goto cleanup;
-+ }
-+
-+ verification_uri = i_get_str_parameter(&session, I_OPT_DEVICE_AUTH_VERIFICATION_URI);
-+ if (!verification_uri)
-+ {
-+ appendPQExpBufferStr(&conn->errorMessage,
-+ libpq_gettext("issuer did not provide a verification URI"));
-+ goto cleanup;
-+ }
-+
-+ user_code = i_get_str_parameter(&session, I_OPT_DEVICE_AUTH_USER_CODE);
-+ if (!user_code)
-+ {
-+ appendPQExpBufferStr(&conn->errorMessage,
-+ libpq_gettext("issuer did not provide a user code"));
-+ goto cleanup;
-+ }
-+
-+ /*
-+ * Poll the token endpoint until either the user logs in and authorizes
-+ * the use of a token, or a hard failure occurs. We perform one ping
-+ * _before_ prompting the user, so that we don't make them do the work of
-+ * logging in only to find that the token endpoint is completely
-+ * unreachable.
-+ */
-+ err = i_run_token_request(&session);
-+ while (err)
-+ {
-+ const char *error_code;
-+ uint interval;
-+
-+ error_code = i_get_str_parameter(&session, I_OPT_ERROR);
-+
-+ /*
-+ * authorization_pending and slow_down are the only acceptable errors;
-+ * anything else and we bail.
-+ */
-+ if (!error_code || (strcmp(error_code, "authorization_pending")
-+ && strcmp(error_code, "slow_down")))
-+ {
-+ iddawc_request_error(conn, &session, err,
-+ "failed to obtain access token");
-+ goto cleanup;
-+ }
-+
-+ if (!user_prompted)
-+ {
-+ int res;
-+ PQpromptOAuthDevice prompt = {
-+ .verification_uri = verification_uri,
-+ .user_code = user_code,
-+ /* TODO: optional fields */
-+ };
-+
-+ /*
-+ * Now that we know the token endpoint isn't broken, give the user
-+ * the login instructions.
-+ */
-+ res = PQauthDataHook(PQAUTHDATA_PROMPT_OAUTH_DEVICE, conn,
-+ &prompt);
-+
-+ if (!res)
-+ {
-+ fprintf(stderr, "Visit %s and enter the code: %s",
-+ prompt.verification_uri, prompt.user_code);
-+ }
-+ else if (res < 0)
-+ {
-+ appendPQExpBufferStr(&conn->errorMessage,
-+ libpq_gettext("device prompt failed\n"));
-+ goto cleanup;
-+ }
-+
-+ user_prompted = true;
-+ }
-+
-+ /*---
-+ * We are required to wait between polls; the server tells us how
-+ * long.
-+ * TODO: if interval's not set, we need to default to five seconds
-+ * TODO: sanity check the interval
-+ */
-+ interval = i_get_int_parameter(&session, I_OPT_DEVICE_AUTH_INTERVAL);
-+
-+ /*
-+ * A slow_down error requires us to permanently increase our retry
-+ * interval by five seconds. RFC 8628, Sec. 3.5.
-+ */
-+ if (!strcmp(error_code, "slow_down"))
-+ {
-+ interval += 5;
-+ i_set_int_parameter(&session, I_OPT_DEVICE_AUTH_INTERVAL, interval);
-+ }
-+
-+ sleep(interval);
-+
-+ /*
-+ * XXX Reset the error code before every call, because iddawc won't do
-+ * that for us. This matters if the server first sends a "pending"
-+ * error code, then later hard-fails without sending an error code to
-+ * overwrite the first one.
-+ *
-+ * That we have to do this at all seems like a bug in iddawc.
-+ */
-+ i_set_str_parameter(&session, I_OPT_ERROR, NULL);
-+
-+ err = i_run_token_request(&session);
-+ }
-+
-+ access_token = i_get_str_parameter(&session, I_OPT_ACCESS_TOKEN);
-+ token_type = i_get_str_parameter(&session, I_OPT_TOKEN_TYPE);
-+
-+ if (!access_token || !token_type || strcasecmp(token_type, "Bearer"))
-+ {
-+ appendPQExpBufferStr(&conn->errorMessage,
-+ libpq_gettext("issuer did not provide a bearer token"));
-+ goto cleanup;
-+ }
-+
-+ appendPQExpBufferStr(token_buf, "Bearer ");
-+ appendPQExpBufferStr(token_buf, access_token);
-+
-+ if (PQExpBufferBroken(token_buf))
-+ goto cleanup;
-+
-+ token = strdup(token_buf->data);
-+
-+cleanup:
-+ if (token_buf)
-+ destroyPQExpBuffer(token_buf);
-+ i_clean_session(&session);
-+
-+ return token;
-+}
-+
-+PostgresPollingStatusType
-+pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
-+{
-+ fe_oauth_state *state = conn->sasl_state;
-+
-+ /* TODO: actually make this asynchronous */
-+ state->token = run_iddawc_auth_flow(conn, conn->oauth_discovery_uri);
-+ return state->token ? PGRES_POLLING_OK : PGRES_POLLING_FAILED;
+}
## src/interfaces/libpq/fe-auth-oauth.c (new) ##
@@ src/interfaces/libpq/meson.build: if gssapi.found()
+if oauth.found()
+ libpq_sources += files('fe-auth-oauth.c')
-+ if oauth_library == 'iddawc'
-+ libpq_sources += files('fe-auth-oauth-iddawc.c')
-+ else
++ if oauth_library == 'curl'
+ libpq_sources += files('fe-auth-oauth-curl.c')
+ endif
+endif
5: 5c5a83e44e = 5: 12ae7c4355 backend: add OAUTHBEARER SASL mechanism
6: 7a42365d62 = 6: 707edf9314 Introduce OAuth validator libraries
7: 9c46ea6cf9 ! 7: 47236c5644 Add pytest suite for OAuth
@@ Commit message
option of the same name, which allows an ephemeral server to be spun up
during a test run.
- For iddawc, asynchronous tests still hang, as expected. Bad-interval
- tests fail because iddawc apparently doesn't care that the interval is
- bad.
-
TODOs:
- The --tap-stream option to pytest-tap is slightly broken during test
failures (it suppresses error information), which impedes debugging.
@@ src/test/python/client/test_oauth.py (new)
+ ),
+ pytest.param(
+ (400, {}),
-+ alt_patterns(
-+ r'failed to parse token error response: field "error" is missing',
-+ r"failed to obtain device authorization: \(iddawc error I_ERROR_PARAM\)",
-+ ),
++ r'failed to parse token error response: field "error" is missing',
+ id="broken error response",
+ ),
+ pytest.param(
+ (200, RawResponse(r'{ "interval": 3.5.8 }')),
-+ alt_patterns(
-+ r"failed to parse device authorization: Token .* is invalid",
-+ r"failed to obtain device authorization: \(iddawc error I_ERROR\)",
-+ ),
++ r"failed to parse device authorization: Token .* is invalid",
+ id="non-numeric interval",
+ ),
+ pytest.param(
+ (200, RawResponse(r'{ "interval": 08 }')),
-+ alt_patterns(
-+ r"failed to parse device authorization: Token .* is invalid",
-+ r"failed to obtain device authorization: \(iddawc error I_ERROR\)",
-+ ),
++ r"failed to parse device authorization: Token .* is invalid",
+ id="invalid numeric interval",
+ ),
+ ],
@@ src/test/python/client/test_oauth.py (new)
+ else:
+ assert False, "update error_pattern for new failure mode"
+
-+ # XXX iddawc doesn't really check for problems in the device authorization
-+ # response, leading to this patchwork:
-+ if field_name == "verification_uri":
-+ error_pattern = alt_patterns(
-+ error_pattern,
-+ "issuer did not provide a verification URI",
-+ )
-+ elif field_name == "user_code":
-+ error_pattern = alt_patterns(
-+ error_pattern,
-+ "issuer did not provide a user code",
-+ )
-+ else:
-+ error_pattern = alt_patterns(
-+ error_pattern,
-+ r"failed to obtain access token: \(iddawc error I_ERROR_PARAM\)",
-+ )
-+
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
@@ src/test/python/client/test_oauth.py (new)
+ ),
+ pytest.param(
+ (400, {}),
-+ alt_patterns(
-+ r'failed to parse token error response: field "error" is missing',
-+ r"failed to obtain access token: \(iddawc error I_ERROR_PARAM\)",
-+ ),
++ r'failed to parse token error response: field "error" is missing',
+ id="empty error response",
+ ),
+ pytest.param(
+ (200, {}, {}),
-+ alt_patterns(
-+ r"failed to parse access token response: no content type was provided",
-+ r"failed to obtain access token: \(iddawc error I_ERROR\)",
-+ ),
++ r"failed to parse access token response: no content type was provided",
+ id="missing content type",
+ ),
+ pytest.param(
+ (200, {"Content-Type": "text/plain"}, {}),
-+ alt_patterns(
-+ r"failed to parse access token response: unexpected content type",
-+ r"failed to obtain access token: \(iddawc error I_ERROR\)",
-+ ),
++ r"failed to parse access token response: unexpected content type",
+ id="wrong content type",
+ ),
+ ],
@@ src/test/python/client/test_oauth.py (new)
+ else:
+ assert False, "update error_pattern for new failure mode"
+
-+ # XXX iddawc is fairly silent on the topic.
-+ error_pattern = alt_patterns(
-+ error_pattern,
-+ r"failed to obtain access token: \(iddawc error I_ERROR_PARAM\)",
-+ )
-+
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
@@ src/test/python/client/test_oauth.py (new)
+
+ expect_disconnected_handshake(sock)
+
-+ # XXX iddawc doesn't differentiate...
-+ expected_error = alt_patterns(
-+ expected_error,
-+ r"failed to fetch OpenID discovery document \(iddawc error I_ERROR(_PARAM)?\)",
-+ )
-+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
@@ src/test/python/test_pq3.py (new)
+import contextlib
+import getpass
+import io
++import os
+import platform
+import struct
+import sys
8: 8ad4ce3068 = 8: 116e17eeee XXX temporary patches to build and test
9: 5630465578 = 9: 28756eda1c WIP: Python OAuth provider implementation
v19-0002-Refactor-SASL-exchange-to-return-tri-state-statu.patch (application/octet-stream, 9.9K) download
From 6989b751536149b0224bb1cd38c1f0142c781e62 Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <dgustafsson@postgresql.org>
Date: Fri, 23 Feb 2024 11:09:54 +0100
Subject: [PATCH v19 2/9] Refactor SASL exchange to return tri-state status
The SASL exchange callback returned state in to output variables:
done and success. This refactors that logic by introducing a new
return variable of type SASLStatus which makes the code easier to
read and understand, and prepares for future SASL exchanges which
operate asynchronously.
This was extracted from a larger patchset to introduce OAuthBearer
authentication and authorization.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://postgr.es/m/d1b467a78e0e36ed85a09adf979d04cf124a9d4b.camel@vmware.com
---
src/interfaces/libpq/fe-auth-sasl.h | 31 +++++++----
src/interfaces/libpq/fe-auth-scram.c | 78 +++++++++++++---------------
src/interfaces/libpq/fe-auth.c | 28 +++++-----
src/tools/pgindent/typedefs.list | 1 +
4 files changed, 71 insertions(+), 67 deletions(-)
diff --git a/src/interfaces/libpq/fe-auth-sasl.h b/src/interfaces/libpq/fe-auth-sasl.h
index ee5d1525b5..4eecf53a15 100644
--- a/src/interfaces/libpq/fe-auth-sasl.h
+++ b/src/interfaces/libpq/fe-auth-sasl.h
@@ -21,6 +21,17 @@
#include "libpq-fe.h"
+/*
+ * Possible states for the SASL exchange, see the comment on exchange for an
+ * explanation of these.
+ */
+typedef enum
+{
+ SASL_COMPLETE = 0,
+ SASL_FAILED,
+ SASL_CONTINUE,
+} SASLStatus;
+
/*
* Frontend SASL mechanism callbacks.
*
@@ -59,7 +70,8 @@ typedef struct pg_fe_sasl_mech
* Produces a client response to a server challenge. As a special case
* for client-first SASL mechanisms, exchange() is called with a NULL
* server response once at the start of the authentication exchange to
- * generate an initial response.
+ * generate an initial response. Returns a SASLStatus indicating the
+ * state and status of the exchange.
*
* Input parameters:
*
@@ -79,22 +91,23 @@ typedef struct pg_fe_sasl_mech
*
* output: A malloc'd buffer containing the client's response to
* the server (can be empty), or NULL if the exchange should
- * be aborted. (*success should be set to false in the
+ * be aborted. (The callback should return SASL_FAILED in the
* latter case.)
*
* outputlen: The length (0 or higher) of the client response buffer,
* ignored if output is NULL.
*
- * done: Set to true if the SASL exchange should not continue,
- * because the exchange is either complete or failed
+ * Return value:
*
- * success: Set to true if the SASL exchange completed successfully.
- * Ignored if *done is false.
+ * SASL_CONTINUE: The output buffer is filled with a client response.
+ * Additional server challenge is expected
+ * SASL_COMPLETE: The SASL exchange has completed successfully.
+ * SASL_FAILED: The exchange has failed and the connection should be
+ * dropped.
*--------
*/
- void (*exchange) (void *state, char *input, int inputlen,
- char **output, int *outputlen,
- bool *done, bool *success);
+ SASLStatus (*exchange) (void *state, char *input, int inputlen,
+ char **output, int *outputlen);
/*--------
* channel_bound()
diff --git a/src/interfaces/libpq/fe-auth-scram.c b/src/interfaces/libpq/fe-auth-scram.c
index 04f0e5713d..0bb820e0d9 100644
--- a/src/interfaces/libpq/fe-auth-scram.c
+++ b/src/interfaces/libpq/fe-auth-scram.c
@@ -24,9 +24,8 @@
/* The exported SCRAM callback mechanism. */
static void *scram_init(PGconn *conn, const char *password,
const char *sasl_mechanism);
-static void scram_exchange(void *opaq, char *input, int inputlen,
- char **output, int *outputlen,
- bool *done, bool *success);
+static SASLStatus scram_exchange(void *opaq, char *input, int inputlen,
+ char **output, int *outputlen);
static bool scram_channel_bound(void *opaq);
static void scram_free(void *opaq);
@@ -202,17 +201,14 @@ scram_free(void *opaq)
/*
* Exchange a SCRAM message with backend.
*/
-static void
+static SASLStatus
scram_exchange(void *opaq, char *input, int inputlen,
- char **output, int *outputlen,
- bool *done, bool *success)
+ char **output, int *outputlen)
{
fe_scram_state *state = (fe_scram_state *) opaq;
PGconn *conn = state->conn;
const char *errstr = NULL;
- *done = false;
- *success = false;
*output = NULL;
*outputlen = 0;
@@ -225,12 +221,12 @@ scram_exchange(void *opaq, char *input, int inputlen,
if (inputlen == 0)
{
libpq_append_conn_error(conn, "malformed SCRAM message (empty message)");
- goto error;
+ return SASL_FAILED;
}
if (inputlen != strlen(input))
{
libpq_append_conn_error(conn, "malformed SCRAM message (length mismatch)");
- goto error;
+ return SASL_FAILED;
}
}
@@ -240,61 +236,59 @@ scram_exchange(void *opaq, char *input, int inputlen,
/* Begin the SCRAM handshake, by sending client nonce */
*output = build_client_first_message(state);
if (*output == NULL)
- goto error;
+ return SASL_FAILED;
*outputlen = strlen(*output);
- *done = false;
state->state = FE_SCRAM_NONCE_SENT;
- break;
+ return SASL_CONTINUE;
case FE_SCRAM_NONCE_SENT:
/* Receive salt and server nonce, send response. */
if (!read_server_first_message(state, input))
- goto error;
+ return SASL_FAILED;
*output = build_client_final_message(state);
if (*output == NULL)
- goto error;
+ return SASL_FAILED;
*outputlen = strlen(*output);
- *done = false;
state->state = FE_SCRAM_PROOF_SENT;
- break;
+ return SASL_CONTINUE;
case FE_SCRAM_PROOF_SENT:
- /* Receive server signature */
- if (!read_server_final_message(state, input))
- goto error;
-
- /*
- * Verify server signature, to make sure we're talking to the
- * genuine server.
- */
- if (!verify_server_signature(state, success, &errstr))
- {
- libpq_append_conn_error(conn, "could not verify server signature: %s", errstr);
- goto error;
- }
-
- if (!*success)
{
- libpq_append_conn_error(conn, "incorrect server signature");
+ bool match;
+
+ /* Receive server signature */
+ if (!read_server_final_message(state, input))
+ return SASL_FAILED;
+
+ /*
+ * Verify server signature, to make sure we're talking to the
+ * genuine server.
+ */
+ if (!verify_server_signature(state, &match, &errstr))
+ {
+ libpq_append_conn_error(conn, "could not verify server signature: %s", errstr);
+ return SASL_FAILED;
+ }
+
+ if (!match)
+ {
+ libpq_append_conn_error(conn, "incorrect server signature");
+ }
+ state->state = FE_SCRAM_FINISHED;
+ state->conn->client_finished_auth = true;
+ return match ? SASL_COMPLETE : SASL_FAILED;
}
- *done = true;
- state->state = FE_SCRAM_FINISHED;
- state->conn->client_finished_auth = true;
- break;
default:
/* shouldn't happen */
libpq_append_conn_error(conn, "invalid SCRAM exchange state");
- goto error;
+ break;
}
- return;
-error:
- *done = true;
- *success = false;
+ return SASL_FAILED;
}
/*
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index 1a8e4f6fbf..cf8af4c62e 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -423,11 +423,10 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
char *initialresponse = NULL;
int initialresponselen;
- bool done;
- bool success;
const char *selected_mechanism;
PQExpBufferData mechanism_buf;
char *password;
+ SASLStatus status;
initPQExpBuffer(&mechanism_buf);
@@ -575,12 +574,11 @@ pg_SASL_init(PGconn *conn, int payloadlen)
goto oom_error;
/* Get the mechanism-specific Initial Client Response, if any */
- conn->sasl->exchange(conn->sasl_state,
- NULL, -1,
- &initialresponse, &initialresponselen,
- &done, &success);
+ status = conn->sasl->exchange(conn->sasl_state,
+ NULL, -1,
+ &initialresponse, &initialresponselen);
- if (done && !success)
+ if (status == SASL_FAILED)
goto error;
/*
@@ -629,10 +627,9 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
{
char *output;
int outputlen;
- bool done;
- bool success;
int res;
char *challenge;
+ SASLStatus status;
/* Read the SASL challenge from the AuthenticationSASLContinue message. */
challenge = malloc(payloadlen + 1);
@@ -651,13 +648,12 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
/* For safety and convenience, ensure the buffer is NULL-terminated. */
challenge[payloadlen] = '\0';
- conn->sasl->exchange(conn->sasl_state,
- challenge, payloadlen,
- &output, &outputlen,
- &done, &success);
+ status = conn->sasl->exchange(conn->sasl_state,
+ challenge, payloadlen,
+ &output, &outputlen);
free(challenge); /* don't need the input anymore */
- if (final && !done)
+ if (final && status == SASL_CONTINUE)
{
if (outputlen != 0)
free(output);
@@ -670,7 +666,7 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
* If the exchange is not completed yet, we need to make sure that the
* SASL mechanism has generated a message to send back.
*/
- if (output == NULL && !done)
+ if (output == NULL && status == SASL_CONTINUE)
{
libpq_append_conn_error(conn, "no client response found after SASL exchange success");
return STATUS_ERROR;
@@ -692,7 +688,7 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
return STATUS_ERROR;
}
- if (done && !success)
+ if (status == SASL_FAILED)
return STATUS_ERROR;
return STATUS_OK;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index ee40a341d3..2dfd8176a3 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2423,6 +2423,7 @@ RuleLock
RuleStmt
RunningTransactions
RunningTransactionsData
+SASLStatus
SC_HANDLE
SECURITY_ATTRIBUTES
SECURITY_STATUS
--
2.34.1
v19-0004-libpq-add-OAUTHBEARER-SASL-mechanism.patch (application/octet-stream, 106.3K) download
From 77550a47db85ec83edb4f6c6f83caff3bc8fd9b4 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 13 Apr 2021 10:27:27 -0700
Subject: [PATCH v19 4/9] libpq: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) and OAuth 2.0 Device Authorization
Grants (RFC 8628) on the client side. When speaking to a OAuth-enabled
server, it looks a bit like this:
$ psql 'host=example.org oauth_client_id=f02c6361-0635-...'
Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG
The OAuth issuer must support device authorization. No other OAuth flows
are currently implemented (but clients may provide their own flows; see
below).
The client implementation requires libcurl and its development headers.
Pass `curl` to --with-oauth/-Doauth during configuration.
Thomas Munro wrote the kqueue() implementation for oauth-curl; thanks!
= PQauthDataHook =
Clients may override two pieces of OAuth handling using the new
PQsetAuthDataHook():
- PQAUTHDATA_PROMPT_OAUTH_DEVICE: replaces the default user prompt to
standard error when using the builtin device authorization flow
- PQAUTHDATA_OAUTH_BEARER_TOKEN: replaces the entire OAuth flow with a
custom asynchronous implementation
In general, a hook implementation should examine the incoming `type` to
decide whether or not to handle a specific piece of authdata; if not, it
should delegate to the previous hook in the chain (retrievable via
PQgetAuthDataHook()). Otherwise, it should return an integer > 0 and
follow the authdata-specific instructions. Returning an integer < 0
signals an error condition and abandons the connection attempt.
== PQAUTHDATA_PROMPT_OAUTH_DEVICE ==
The hook should display the device prompt (URL + code) using whatever
method it prefers.
== PQAUTHDATA_OAUTH_BEARER_TOKEN ==
The hook should either directly return a Bearer token for the current
user/issuer/scope combination, if one is available without blocking, or
else set up an asynchronous callback to retrieve one. See the
documentation for PQoauthBearerRequest.
Several TODOs:
- don't retry forever if the server won't accept our token
- perform several sanity checks on the OAuth issuer's responses
- handle cases where the client has been set up with an issuer and
scope, but the Postgres server wants to use something different
- improve error debuggability during the OAuth handshake
- fix libcurl initialization thread-safety
- harden the libcurl flow implementation
- figure out pgsocket/int difference on Windows
- fix intermittent failure in the cleanup callback tests (race
condition?)
- support require_auth
- ...and more.
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
---
configure | 110 ++
configure.ac | 28 +
meson.build | 29 +
meson_options.txt | 4 +
src/Makefile.global.in | 1 +
src/include/common/oauth-common.h | 19 +
src/include/pg_config.h.in | 9 +
src/interfaces/libpq/Makefile | 10 +-
src/interfaces/libpq/exports.txt | 3 +
src/interfaces/libpq/fe-auth-oauth-curl.c | 1982 +++++++++++++++++++++
src/interfaces/libpq/fe-auth-oauth.c | 659 +++++++
src/interfaces/libpq/fe-auth-oauth.h | 42 +
src/interfaces/libpq/fe-auth-sasl.h | 10 +-
src/interfaces/libpq/fe-auth-scram.c | 6 +-
src/interfaces/libpq/fe-auth.c | 105 +-
src/interfaces/libpq/fe-auth.h | 9 +-
src/interfaces/libpq/fe-connect.c | 85 +-
src/interfaces/libpq/fe-misc.c | 7 +-
src/interfaces/libpq/libpq-fe.h | 77 +-
src/interfaces/libpq/libpq-int.h | 14 +
src/interfaces/libpq/meson.build | 7 +
src/makefiles/meson.build | 1 +
src/tools/pgindent/typedefs.list | 10 +
23 files changed, 3200 insertions(+), 27 deletions(-)
create mode 100644 src/include/common/oauth-common.h
create mode 100644 src/interfaces/libpq/fe-auth-oauth-curl.c
create mode 100644 src/interfaces/libpq/fe-auth-oauth.c
create mode 100644 src/interfaces/libpq/fe-auth-oauth.h
diff --git a/configure b/configure
index 46859a4244..2ccfb01b7a 100755
--- a/configure
+++ b/configure
@@ -712,6 +712,7 @@ with_uuid
with_readline
with_systemd
with_selinux
+with_oauth
with_ldap
with_krb_srvnam
krb_srvtab
@@ -858,6 +859,7 @@ with_krb_srvnam
with_pam
with_bsd_auth
with_ldap
+with_oauth
with_bonjour
with_selinux
with_systemd
@@ -1568,6 +1570,7 @@ Optional Packages:
--with-pam build with PAM support
--with-bsd-auth build with BSD Authentication support
--with-ldap build with LDAP support
+ --with-oauth=LIB use LIB for OAuth 2.0 support (curl)
--with-bonjour build with Bonjour support
--with-selinux build with SELinux support
--with-systemd build with systemd support
@@ -8485,6 +8488,52 @@ $as_echo "$with_ldap" >&6; }
+#
+# OAuth 2.0
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with OAuth support" >&5
+$as_echo_n "checking whether to build with OAuth support... " >&6; }
+
+
+
+# Check whether --with-oauth was given.
+if test "${with_oauth+set}" = set; then :
+ withval=$with_oauth;
+ case $withval in
+ yes)
+ as_fn_error $? "argument required for --with-oauth option" "$LINENO" 5
+ ;;
+ no)
+ as_fn_error $? "argument required for --with-oauth option" "$LINENO" 5
+ ;;
+ *)
+
+ ;;
+ esac
+
+fi
+
+
+if test x"$with_oauth" = x"" ; then
+ with_oauth=no
+fi
+
+if test x"$with_oauth" = x"curl"; then
+
+$as_echo "#define USE_OAUTH 1" >>confdefs.h
+
+
+$as_echo "#define USE_OAUTH_CURL 1" >>confdefs.h
+
+elif test x"$with_oauth" != x"no"; then
+ as_fn_error $? "--with-oauth must specify curl" "$LINENO" 5
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_oauth" >&5
+$as_echo "$with_oauth" >&6; }
+
+
+
#
# Bonjour
#
@@ -13037,6 +13086,56 @@ fi
+if test "$with_oauth" = curl ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for curl_multi_init in -lcurl" >&5
+$as_echo_n "checking for curl_multi_init in -lcurl... " >&6; }
+if ${ac_cv_lib_curl_curl_multi_init+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lcurl $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char curl_multi_init ();
+int
+main ()
+{
+return curl_multi_init ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_curl_curl_multi_init=yes
+else
+ ac_cv_lib_curl_curl_multi_init=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_curl_curl_multi_init" >&5
+$as_echo "$ac_cv_lib_curl_curl_multi_init" >&6; }
+if test "x$ac_cv_lib_curl_curl_multi_init" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBCURL 1
+_ACEOF
+
+ LIBS="-lcurl $LIBS"
+
+else
+ as_fn_error $? "library 'curl' is required for --with-oauth=curl" "$LINENO" 5
+fi
+
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for security_compute_create_name in -lselinux" >&5
@@ -14062,6 +14161,17 @@ fi
done
+fi
+
+if test "$with_oauth" = curl; then
+ ac_fn_c_check_header_mongrel "$LINENO" "curl/curl.h" "ac_cv_header_curl_curl_h" "$ac_includes_default"
+if test "x$ac_cv_header_curl_curl_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <curl/curl.h> is required for OAuth" "$LINENO" 5
+fi
+
+
fi
if test "$PORTNAME" = "win32" ; then
diff --git a/configure.ac b/configure.ac
index 88b75a7696..4a80c97d5b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -927,6 +927,26 @@ AC_MSG_RESULT([$with_ldap])
AC_SUBST(with_ldap)
+#
+# OAuth 2.0
+#
+AC_MSG_CHECKING([whether to build with OAuth support])
+PGAC_ARG_REQ(with, oauth, [LIB], [use LIB for OAuth 2.0 support (curl)])
+if test x"$with_oauth" = x"" ; then
+ with_oauth=no
+fi
+
+if test x"$with_oauth" = x"curl"; then
+ AC_DEFINE([USE_OAUTH], 1, [Define to 1 to build with OAuth 2.0 support. (--with-oauth)])
+ AC_DEFINE([USE_OAUTH_CURL], 1, [Define to 1 to use libcurl for OAuth support.])
+elif test x"$with_oauth" != x"no"; then
+ AC_MSG_ERROR([--with-oauth must specify curl])
+fi
+
+AC_MSG_RESULT([$with_oauth])
+AC_SUBST(with_oauth)
+
+
#
# Bonjour
#
@@ -1423,6 +1443,10 @@ fi
AC_SUBST(LDAP_LIBS_FE)
AC_SUBST(LDAP_LIBS_BE)
+if test "$with_oauth" = curl ; then
+ AC_CHECK_LIB(curl, curl_multi_init, [], [AC_MSG_ERROR([library 'curl' is required for --with-oauth=curl])])
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
AC_CHECK_LIB(selinux, security_compute_create_name, [],
@@ -1614,6 +1638,10 @@ elif test "$with_uuid" = ossp ; then
[AC_MSG_ERROR([header file <ossp/uuid.h> or <uuid.h> is required for OSSP UUID])])])
fi
+if test "$with_oauth" = curl; then
+ AC_CHECK_HEADER(curl/curl.h, [], [AC_MSG_ERROR([header file <curl/curl.h> is required for OAuth])])
+fi
+
if test "$PORTNAME" = "win32" ; then
AC_CHECK_HEADERS(crtdefs.h)
fi
diff --git a/meson.build b/meson.build
index a198eca25d..45b20d11c1 100644
--- a/meson.build
+++ b/meson.build
@@ -830,6 +830,33 @@ endif
+###############################################################
+# Library: oauth
+###############################################################
+
+oauth = not_found_dep
+oauth_library = 'none'
+oauthopt = get_option('oauth')
+
+if oauthopt == 'auto' and auto_features.disabled()
+ oauthopt = 'none'
+endif
+
+if oauthopt in ['auto', 'curl']
+ oauth = dependency('libcurl', required: (oauthopt == 'curl'))
+
+ if oauth.found()
+ oauth_library = 'curl'
+ cdata.set('USE_OAUTH', 1)
+ cdata.set('USE_OAUTH_CURL', 1)
+ endif
+endif
+
+if oauthopt == 'auto' and auto_features.enabled() and not oauth.found()
+ error('no OAuth implementation library found')
+endif
+
+
###############################################################
# Library: Tcl (for pltcl)
#
@@ -2834,6 +2861,7 @@ libpq_deps += [
gssapi,
ldap_r,
libintl,
+ oauth,
ssl,
]
@@ -3435,6 +3463,7 @@ if meson.version().version_compare('>=0.57')
'llvm': llvm,
'lz4': lz4,
'nls': libintl,
+ 'oauth': oauth,
'openssl': ssl,
'pam': pam,
'plperl': perl_dep,
diff --git a/meson_options.txt b/meson_options.txt
index 249ecc5ffd..3248b9cc1c 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -121,6 +121,10 @@ option('lz4', type: 'feature', value: 'auto',
option('nls', type: 'feature', value: 'auto',
description: 'Native language support')
+option('oauth', type : 'combo', choices : ['auto', 'none', 'curl'],
+ value: 'auto',
+ description: 'use LIB for OAuth 2.0 support (curl)')
+
option('pam', type: 'feature', value: 'auto',
description: 'PAM support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 8b3f8c24e0..79b3647834 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -193,6 +193,7 @@ with_ldap = @with_ldap@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
+with_oauth = @with_oauth@
with_system_tzdata = @with_system_tzdata@
with_uuid = @with_uuid@
with_zlib = @with_zlib@
diff --git a/src/include/common/oauth-common.h b/src/include/common/oauth-common.h
new file mode 100644
index 0000000000..5ff3488bfb
--- /dev/null
+++ b/src/include/common/oauth-common.h
@@ -0,0 +1,19 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth-common.h
+ * Declarations for helper functions used for OAuth/OIDC authentication
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/common/oauth-common.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef OAUTH_COMMON_H
+#define OAUTH_COMMON_H
+
+/* Name of SASL mechanism per IANA */
+#define OAUTHBEARER_NAME "OAUTHBEARER"
+
+#endif /* OAUTH_COMMON_H */
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 07e73567dc..a5e6f99ba4 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -243,6 +243,9 @@
/* Define to 1 if you have the `crypto' library (-lcrypto). */
#undef HAVE_LIBCRYPTO
+/* Define to 1 if you have the `curl' library (-lcurl). */
+#undef HAVE_LIBCURL
+
/* Define to 1 if you have the `ldap' library (-lldap). */
#undef HAVE_LIBLDAP
@@ -711,6 +714,12 @@
/* Define to select named POSIX semaphores. */
#undef USE_NAMED_POSIX_SEMAPHORES
+/* Define to 1 to build with OAuth 2.0 support. (--with-oauth) */
+#undef USE_OAUTH
+
+/* Define to 1 to use libcurl for OAuth support. */
+#undef USE_OAUTH_CURL
+
/* Define to 1 to build with OpenSSL support. (--with-ssl=openssl) */
#undef USE_OPENSSL
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index fe2af575c5..2618c293af 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -61,6 +61,14 @@ OBJS += \
fe-secure-gssapi.o
endif
+ifneq ($(with_oauth),no)
+OBJS += fe-auth-oauth.o
+
+ifeq ($(with_oauth),curl)
+OBJS += fe-auth-oauth-curl.o
+endif
+endif
+
ifeq ($(PORTNAME), cygwin)
override shlib = cyg$(NAME)$(DLSUFFIX)
endif
@@ -79,7 +87,7 @@ endif
# that are built correctly for use in a shlib.
SHLIB_LINK_INTERNAL = -lpgcommon_shlib -lpgport_shlib
ifneq ($(PORTNAME), win32)
-SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
+SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lcurl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
else
SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi32 -lssl -lsocket -lnsl -lresolv -lintl -lm $(PTHREAD_LIBS), $(LIBS)) $(LDAP_LIBS_FE)
endif
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 088592deb1..0f8f5e3125 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -193,3 +193,6 @@ PQsendClosePrepared 190
PQsendClosePortal 191
PQchangePassword 192
PQsendPipelineSync 193
+PQsetAuthDataHook 194
+PQgetAuthDataHook 195
+PQdefaultAuthDataHook 196
diff --git a/src/interfaces/libpq/fe-auth-oauth-curl.c b/src/interfaces/libpq/fe-auth-oauth-curl.c
new file mode 100644
index 0000000000..0504f96e4e
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth-curl.c
@@ -0,0 +1,1982 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth-curl.c
+ * The libcurl implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth-curl.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <curl/curl.h>
+#include <math.h>
+#ifdef HAVE_SYS_EPOLL_H
+#include <sys/epoll.h>
+#include <sys/timerfd.h>
+#endif
+#ifdef HAVE_SYS_EVENT_H
+#include <sys/event.h>
+#endif
+#include <unistd.h>
+
+#include "common/jsonapi.h"
+#include "fe-auth.h"
+#include "fe-auth-oauth.h"
+#include "libpq-int.h"
+#include "mb/pg_wchar.h"
+
+/*
+ * Parsed JSON Representations
+ *
+ * As a general rule, we parse and cache only the fields we're currently using.
+ * When adding new fields, ensure the corresponding free_*() function is updated
+ * too.
+ */
+
+/*
+ * The OpenID Provider configuration (alternatively named "authorization server
+ * metadata") jointly described by OpenID Connect Discovery 1.0 and RFC 8414:
+ *
+ * https://openid.net/specs/openid-connect-discovery-1_0.html
+ * https://www.rfc-editor.org/rfc/rfc8414#section-3.2
+ */
+struct provider
+{
+ char *issuer;
+ char *token_endpoint;
+ char *device_authorization_endpoint;
+ struct curl_slist *grant_types_supported;
+};
+
+static void
+free_provider(struct provider *provider)
+{
+ free(provider->issuer);
+ free(provider->token_endpoint);
+ free(provider->device_authorization_endpoint);
+ curl_slist_free_all(provider->grant_types_supported);
+}
+
+/*
+ * The Device Authorization response, described by RFC 8628:
+ *
+ * https://www.rfc-editor.org/rfc/rfc8628#section-3.2
+ */
+struct device_authz
+{
+ char *device_code;
+ char *user_code;
+ char *verification_uri;
+ char *interval_str;
+
+ /* Fields below are parsed from the corresponding string above. */
+ int interval;
+};
+
+static void
+free_device_authz(struct device_authz *authz)
+{
+ free(authz->device_code);
+ free(authz->user_code);
+ free(authz->verification_uri);
+ free(authz->interval_str);
+}
+
+/*
+ * The Token Endpoint error response, as described by RFC 6749:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-5.2
+ *
+ * Note that this response type can also be returned from the Device
+ * Authorization Endpoint.
+ */
+struct token_error
+{
+ char *error;
+ char *error_description;
+};
+
+static void
+free_token_error(struct token_error *err)
+{
+ free(err->error);
+ free(err->error_description);
+}
+
+/*
+ * The Access Token response, as described by RFC 6749:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-4.1.4
+ *
+ * During the Device Authorization flow, several temporary errors are expected
+ * as part of normal operation. To make it easy to handle these in the happy
+ * path, this contains an embedded token_error that is filled in if needed.
+ */
+struct token
+{
+ /* for successful responses */
+ char *access_token;
+ char *token_type;
+
+ /* for error responses */
+ struct token_error err;
+};
+
+static void
+free_token(struct token *tok)
+{
+ free(tok->access_token);
+ free(tok->token_type);
+ free_token_error(&tok->err);
+}
+
+/*
+ * Asynchronous State
+ */
+
+/* States for the overall async machine. */
+typedef enum
+{
+ OAUTH_STEP_INIT,
+ OAUTH_STEP_DISCOVERY,
+ OAUTH_STEP_DEVICE_AUTHORIZATION,
+ OAUTH_STEP_TOKEN_REQUEST,
+ OAUTH_STEP_WAIT_INTERVAL,
+} OAuthStep;
+
+/*
+ * The async_ctx holds onto state that needs to persist across multiple calls to
+ * pg_fe_run_oauth_flow(). Almost everything interacts with this in some way.
+ */
+struct async_ctx
+{
+ OAuthStep step; /* where are we in the flow? */
+
+#ifdef HAVE_SYS_EPOLL_H
+ int timerfd; /* a timerfd for signaling async timeouts */
+#endif
+ pgsocket mux; /* the multiplexer socket containing all
+ * descriptors tracked by cURL, plus the
+ * timerfd */
+ CURLM *curlm; /* top-level multi handle for cURL operations */
+ CURL *curl; /* the (single) easy handle for serial
+ * requests */
+
+ struct curl_slist *headers; /* common headers for all requests */
+ PQExpBufferData work_data; /* scratch buffer for general use (remember to
+ * clear out prior contents first!) */
+
+ /*------
+ * Since a single logical operation may stretch across multiple calls to
+ * our entry point, errors have three parts:
+ *
+ * - errctx: an optional static string, describing the global operation
+ * currently in progress. It'll be translated for you.
+ *
+ * - errbuf: contains the actual error message. Generally speaking, use
+ * actx_error[_str] to manipulate this. This must be filled
+ * with something useful on an error.
+ *
+ * - curl_err: an optional static error buffer used by cURL to put
+ * detailed information about failures. Unfortunately
+ * untranslatable.
+ *
+ * These pieces will be combined into a single error message looking
+ * something like the following, with errctx and/or curl_err omitted when
+ * absent:
+ *
+ * connection to server ... failed: errctx: errbuf (curl_err)
+ */
+ const char *errctx; /* not freed; must point to static allocation */
+ PQExpBufferData errbuf;
+ char curl_err[CURL_ERROR_SIZE];
+
+ /*
+ * These documents need to survive over multiple calls, and are therefore
+ * cached directly in the async_ctx.
+ */
+ struct provider provider;
+ struct device_authz authz;
+
+ bool user_prompted; /* have we already sent the authz prompt? */
+};
+
+/*
+ * Frees the async_ctx, which is stored directly on the PGconn. This is called
+ * during pqDropConnection() so that we don't leak resources even if
+ * PQconnectPoll() never calls us back.
+ *
+ * TODO: we should probably call this at the end of a successful authentication,
+ * too, to proactively free up resources.
+ */
+static void
+free_curl_async_ctx(PGconn *conn, void *ctx)
+{
+ struct async_ctx *actx = ctx;
+
+ Assert(actx); /* oauth_free() shouldn't call us otherwise */
+
+ /*
+ * TODO: in general, none of the error cases below should ever happen if
+ * we have no bugs above. But if we do hit them, surfacing those errors
+ * somehow might be the only way to have a chance to debug them. What's
+ * the best way to do that? Assertions? Spraying messages on stderr?
+ * Bubbling an error code to the top? Appending to the connection's error
+ * message only helps if the bug caused a connection failure; otherwise
+ * it'll be buried...
+ */
+
+ if (actx->curlm && actx->curl)
+ {
+ CURLMcode err = curl_multi_remove_handle(actx->curlm, actx->curl);
+
+ if (err)
+ libpq_append_conn_error(conn,
+ "cURL easy handle removal failed: %s",
+ curl_multi_strerror(err));
+ }
+
+ if (actx->curl)
+ {
+ /*
+ * curl_multi_cleanup() doesn't free any associated easy handles; we
+ * need to do that separately. We only ever have one easy handle per
+ * multi handle.
+ */
+ curl_easy_cleanup(actx->curl);
+ }
+
+ if (actx->curlm)
+ {
+ CURLMcode err = curl_multi_cleanup(actx->curlm);
+
+ if (err)
+ libpq_append_conn_error(conn,
+ "cURL multi handle cleanup failed: %s",
+ curl_multi_strerror(err));
+ }
+
+ free_provider(&actx->provider);
+ free_device_authz(&actx->authz);
+
+ curl_slist_free_all(actx->headers);
+ termPQExpBuffer(&actx->work_data);
+ termPQExpBuffer(&actx->errbuf);
+
+ if (actx->mux != PGINVALID_SOCKET)
+ close(actx->mux);
+#ifdef HAVE_SYS_EPOLL_H
+ if (actx->timerfd >= 0)
+ close(actx->timerfd);
+#endif
+
+ free(actx);
+}
+
+/*
+ * Macros for manipulating actx->errbuf. actx_error() translates and formats a
+ * string for you; actx_error_str() appends a string directly without
+ * translation.
+ */
+
+#define actx_error(ACTX, FMT, ...) \
+ appendPQExpBuffer(&(ACTX)->errbuf, libpq_gettext(FMT), ##__VA_ARGS__)
+
+#define actx_error_str(ACTX, S) \
+ appendPQExpBufferStr(&(ACTX)->errbuf, S)
+
+/*
+ * Macros for getting and setting state for the connection's two cURL handles,
+ * so you don't have to write out the error handling every time.
+ */
+
+#define CHECK_MSETOPT(ACTX, OPT, VAL, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLMcode _setopterr = curl_multi_setopt(_actx->curlm, OPT, VAL); \
+ if (_setopterr) { \
+ actx_error(_actx, "failed to set %s on OAuth connection: %s",\
+ #OPT, curl_multi_strerror(_setopterr)); \
+ FAILACTION; \
+ } \
+ } while (0)
+
+#define CHECK_SETOPT(ACTX, OPT, VAL, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLcode _setopterr = curl_easy_setopt(_actx->curl, OPT, VAL); \
+ if (_setopterr) { \
+ actx_error(_actx, "failed to set %s on OAuth connection: %s",\
+ #OPT, curl_easy_strerror(_setopterr)); \
+ FAILACTION; \
+ } \
+ } while (0)
+
+#define CHECK_GETINFO(ACTX, INFO, OUT, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLcode _getinfoerr = curl_easy_getinfo(_actx->curl, INFO, OUT); \
+ if (_getinfoerr) { \
+ actx_error(_actx, "failed to get %s from OAuth response: %s",\
+ #INFO, curl_easy_strerror(_getinfoerr)); \
+ FAILACTION; \
+ } \
+ } while (0)
+
+/*
+ * General JSON Parsing for OAuth Responses
+ */
+
+/*
+ * Represents a single name/value pair in a JSON object. This is the primary
+ * interface to parse_oauth_json().
+ *
+ * All fields are stored internally as strings or lists of strings, so clients
+ * have to explicitly parse other scalar types (though they will have gone
+ * through basic lexical validation). Storing nested objects is not currently
+ * supported, nor is parsing arrays of anything other than strings.
+ */
+struct json_field
+{
+ const char *name; /* name (key) of the member */
+
+ JsonTokenType type; /* currently supports JSON_TOKEN_STRING,
+ * JSON_TOKEN_NUMBER, and
+ * JSON_TOKEN_ARRAY_START */
+ union
+ {
+ char **scalar; /* for all scalar types */
+ struct curl_slist **array; /* for type == JSON_TOKEN_ARRAY_START */
+ };
+
+ bool required; /* REQUIRED field, or just OPTIONAL? */
+};
+
+/* Documentation macros for json_field.required. */
+#define REQUIRED true
+#define OPTIONAL false
+
+/* Parse state for parse_oauth_json(). */
+struct oauth_parse
+{
+ PQExpBuffer errbuf; /* detail message for JSON_SEM_ACTION_FAILED */
+ int nested; /* nesting level (zero is the top) */
+
+ const struct json_field *fields; /* field definition array */
+ const struct json_field *active; /* points inside the fields array */
+};
+
+#define oauth_parse_set_error(ctx, fmt, ...) \
+ appendPQExpBuffer((ctx)->errbuf, libpq_gettext(fmt), ##__VA_ARGS__)
+
+static void
+report_type_mismatch(struct oauth_parse *ctx)
+{
+ char *msgfmt;
+
+ Assert(ctx->active);
+
+ /*
+ * At the moment, the only fields we're interested in are strings,
+ * numbers, and arrays of strings.
+ */
+ switch (ctx->active->type)
+ {
+ case JSON_TOKEN_STRING:
+ msgfmt = "field \"%s\" must be a string";
+ break;
+
+ case JSON_TOKEN_NUMBER:
+ msgfmt = "field \"%s\" must be a number";
+ break;
+
+ case JSON_TOKEN_ARRAY_START:
+ msgfmt = "field \"%s\" must be an array of strings";
+ break;
+
+ default:
+ Assert(false);
+ msgfmt = "field \"%s\" has unexpected type";
+ }
+
+ oauth_parse_set_error(ctx, msgfmt, ctx->active->name);
+}
+
+static JsonParseErrorType
+oauth_json_object_start(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (ctx->active)
+ {
+ /*
+ * Currently, none of the fields we're interested in can be or contain
+ * objects, so we can reject this case outright.
+ */
+ report_type_mismatch(ctx);
+ return JSON_SEM_ACTION_FAILED;
+ }
+
+ ++ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_field_start(void *state, char *name, bool isnull)
+{
+ struct oauth_parse *ctx = state;
+
+ /* We care only about the top-level fields. */
+ if (ctx->nested == 1)
+ {
+ const struct json_field *field = ctx->fields;
+
+ /*
+ * We should never start parsing a new field while a previous one is
+ * still active.
+ *
+ * TODO: this code relies on assertions too much. We need to exit
+ * sanely on internal logic errors, to avoid turning bugs into
+ * vulnerabilities.
+ */
+ Assert(!ctx->active);
+
+ while (field->name)
+ {
+ if (strcmp(name, field->name) == 0)
+ {
+ ctx->active = field;
+ break;
+ }
+
+ ++field;
+ }
+ }
+
+ free(name);
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_end(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_start(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (!ctx->nested)
+ {
+ oauth_parse_set_error(ctx, "top-level element must be an object");
+ return JSON_SEM_ACTION_FAILED;
+ }
+
+ if (ctx->active)
+ {
+ if (ctx->active->type != JSON_TOKEN_ARRAY_START
+ /* The arrays we care about must not have arrays as values. */
+ || ctx->nested > 1)
+ {
+ report_type_mismatch(ctx);
+ return JSON_SEM_ACTION_FAILED;
+ }
+ }
+
+ ++ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_end(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (ctx->active)
+ {
+ /*
+ * This assumes that no target arrays can contain other arrays, which
+ * we check in the array_start callback.
+ */
+ Assert(ctx->nested == 2);
+ Assert(ctx->active->type == JSON_TOKEN_ARRAY_START);
+
+ ctx->active = NULL;
+ }
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_scalar(void *state, char *token, JsonTokenType type)
+{
+ struct oauth_parse *ctx = state;
+ JsonParseErrorType result = JSON_SUCCESS;
+
+ if (!ctx->nested)
+ {
+ oauth_parse_set_error(ctx, "top-level element must be an object");
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ if (ctx->active)
+ {
+ JsonTokenType expected;
+
+ /*
+ * Make sure this matches what the active field expects. Arrays must
+ * contain only strings with the current implementation.
+ */
+ if (ctx->active->type == JSON_TOKEN_ARRAY_START)
+ expected = JSON_TOKEN_STRING;
+ else
+ expected = ctx->active->type;
+
+ if (type != expected)
+ {
+ report_type_mismatch(ctx);
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ /*
+ * FIXME if the JSON field is duplicated, we'll leak the prior value.
+ * Error out in that case instead.
+ */
+ if (ctx->active->type != JSON_TOKEN_ARRAY_START)
+ {
+ Assert(ctx->nested == 1);
+
+ *ctx->active->scalar = token;
+ ctx->active = NULL;
+
+ return JSON_SUCCESS; /* don't free the token */
+ }
+ else /* ctx->target_array */
+ {
+ struct curl_slist *temp;
+
+ Assert(ctx->nested == 2);
+
+ temp = curl_slist_append(*ctx->active->array, token);
+ if (!temp)
+ {
+ oauth_parse_set_error(ctx, "out of memory");
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ *ctx->active->array = temp;
+
+ /*
+ * Note that curl_slist_append() makes a copy of the token, so we
+ * can free it below.
+ */
+ }
+ }
+ else
+ {
+ /* otherwise we just ignore it */
+ }
+
+cleanup:
+ free(token);
+ return result;
+}
+
+/*
+ * A helper function for general JSON parsing. fields is the array of field
+ * definitions with their backing pointers. The response will be parsed from
+ * actx->curl and actx->work_data (as set up by start_request()), and any
+ * parsing errors will be placed into actx->errbuf.
+ */
+static bool
+parse_oauth_json(struct async_ctx *actx, const struct json_field *fields)
+{
+ PQExpBuffer resp = &actx->work_data;
+ char *content_type;
+ JsonLexContext lex = {0};
+ JsonSemAction sem = {0};
+ JsonParseErrorType err;
+ struct oauth_parse ctx = {0};
+ bool success = false;
+
+ /* Make sure the server thinks it's given us JSON. */
+ CHECK_GETINFO(actx, CURLINFO_CONTENT_TYPE, &content_type, return false);
+
+ if (!content_type)
+ {
+ actx_error(actx, "no content type was provided");
+ goto cleanup;
+ }
+ else if (strcasecmp(content_type, "application/json") != 0)
+ {
+ actx_error(actx, "unexpected content type \"%s\"", content_type);
+ goto cleanup;
+ }
+
+ if (strlen(resp->data) != resp->len)
+ {
+ actx_error(actx, "response contains embedded NULLs");
+ goto cleanup;
+ }
+
+ makeJsonLexContextCstringLen(&lex, resp->data, resp->len, PG_UTF8, true);
+
+ ctx.errbuf = &actx->errbuf;
+ ctx.fields = fields;
+ sem.semstate = &ctx;
+
+ sem.object_start = oauth_json_object_start;
+ sem.object_field_start = oauth_json_object_field_start;
+ sem.object_end = oauth_json_object_end;
+ sem.array_start = oauth_json_array_start;
+ sem.array_end = oauth_json_array_end;
+ sem.scalar = oauth_json_scalar;
+
+ err = pg_parse_json(&lex, &sem);
+
+ if (err != JSON_SUCCESS)
+ {
+ /*
+ * For JSON_SEM_ACTION_FAILED, we've already written the error
+ * message. Other errors come directly from pg_parse_json(), already
+ * translated.
+ */
+ if (err != JSON_SEM_ACTION_FAILED)
+ actx_error_str(actx, json_errdetail(err, &lex));
+
+ goto cleanup;
+ }
+
+ /* Check all required fields. */
+ while (fields->name)
+ {
+ if (fields->required && !*fields->scalar && !*fields->array)
+ {
+ actx_error(actx, "field \"%s\" is missing", fields->name);
+ goto cleanup;
+ }
+
+ fields++;
+ }
+
+ success = true;
+
+cleanup:
+ freeJsonLexContext(&lex);
+ return success;
+}
+
+/*
+ * JSON Parser Definitions
+ */
+
+static bool
+parse_provider(struct async_ctx *actx, struct provider *provider)
+{
+ struct json_field fields[] = {
+ {"issuer", JSON_TOKEN_STRING, {&provider->issuer}, REQUIRED},
+ {"token_endpoint", JSON_TOKEN_STRING, {&provider->token_endpoint}, REQUIRED},
+
+ /*----
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - jwks_uri
+ * - response_types_supported
+ * - subject_types_supported
+ * - id_token_signing_alg_values_supported
+ */
+
+ {"device_authorization_endpoint", JSON_TOKEN_STRING, {&provider->device_authorization_endpoint}, OPTIONAL},
+ {"grant_types_supported", JSON_TOKEN_ARRAY_START, {.array = &provider->grant_types_supported}, OPTIONAL},
+
+ {0},
+ };
+
+ return parse_oauth_json(actx, fields);
+}
+
+/*
+ * Parses the "interval" JSON number, corresponding to the number of seconds to
+ * wait between token endpoint requests.
+ *
+ * RFC 8628 is pretty silent on sanity checks for the interval. As a matter of
+ * practicality, round any fractional intervals up to the next second, and clamp
+ * the result at a minimum of one. (Zero-second intervals would result in an
+ * expensive network polling loop.)
+ *
+ * TODO: maybe clamp the upper bound too, based on the libpq timeout and/or the
+ * code expiration time?
+ */
+static int
+parse_interval(const char *interval_str)
+{
+ double parsed;
+ int cnt;
+
+ /*
+ * The JSON lexer has already validated the number, which is stricter than
+ * the %f format, so we should be good to use sscanf().
+ */
+ cnt = sscanf(interval_str, "%lf", &parsed);
+
+ if (cnt != 1)
+ {
+ /*
+ * Either the lexer screwed up or our assumption above isn't true, and
+ * either way a developer needs to take a look.
+ */
+ Assert(cnt == 1);
+ return 1; /* don't fall through in release builds */
+ }
+
+ parsed = ceil(parsed);
+
+ if (parsed < 1)
+ return 1; /* TODO this slows down the tests
+ * considerably... */
+ else if (INT_MAX <= parsed)
+ return INT_MAX;
+
+ return parsed;
+}
+
+static bool
+parse_device_authz(struct async_ctx *actx, struct device_authz *authz)
+{
+ struct json_field fields[] = {
+ {"device_code", JSON_TOKEN_STRING, {&authz->device_code}, REQUIRED},
+ {"user_code", JSON_TOKEN_STRING, {&authz->user_code}, REQUIRED},
+ {"verification_uri", JSON_TOKEN_STRING, {&authz->verification_uri}, REQUIRED},
+
+ /*
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - expires_in
+ */
+
+ {"interval", JSON_TOKEN_NUMBER, {&authz->interval_str}, OPTIONAL},
+
+ {0},
+ };
+
+ if (!parse_oauth_json(actx, fields))
+ return false;
+
+ /*
+ * Parse our numeric fields. Lexing has already completed by this time, so
+ * we at least know they're valid JSON numbers.
+ */
+ if (authz->interval_str)
+ authz->interval = parse_interval(authz->interval_str);
+ else
+ {
+ /* TODO: handle default interval of 5 seconds */
+ }
+
+ return true;
+}
+
+static bool
+parse_token_error(struct async_ctx *actx, struct token_error *err)
+{
+ bool result;
+ struct json_field fields[] = {
+ {"error", JSON_TOKEN_STRING, {&err->error}, REQUIRED},
+
+ {"error_description", JSON_TOKEN_STRING, {&err->error_description}, OPTIONAL},
+
+ {0},
+ };
+
+ result = parse_oauth_json(actx, fields);
+
+ /*
+ * Since token errors are parsed during other active error paths, only
+ * override the errctx if parsing explicitly fails.
+ */
+ if (!result)
+ actx->errctx = "failed to parse token error response";
+
+ return result;
+}
+
+static bool
+parse_access_token(struct async_ctx *actx, struct token *tok)
+{
+ struct json_field fields[] = {
+ {"access_token", JSON_TOKEN_STRING, {&tok->access_token}, REQUIRED},
+ {"token_type", JSON_TOKEN_STRING, {&tok->token_type}, REQUIRED},
+
+ /*
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - scope (only required if different than requested -- TODO check)
+ */
+
+ {0},
+ };
+
+ return parse_oauth_json(actx, fields);
+}
+
+/*
+ * cURL Multi Setup/Callbacks
+ */
+
+/*
+ * Sets up the actx->mux, which is the altsock that PQconnectPoll clients will
+ * select() on instead of the Postgres socket during OAuth negotiation.
+ *
+ * This is just an epoll set or kqueue abstracting multiple other descriptors.
+ * A timerfd is always part of the set when using epoll; it's just disabled
+ * when we're not using it.
+ */
+static bool
+setup_multiplexer(struct async_ctx *actx)
+{
+#ifdef HAVE_SYS_EPOLL_H
+ struct epoll_event ev = {.events = EPOLLIN};
+
+ actx->mux = epoll_create1(EPOLL_CLOEXEC);
+ if (actx->mux < 0)
+ {
+ actx_error(actx, "failed to create epoll set: %m");
+ return false;
+ }
+
+ actx->timerfd = timerfd_create(CLOCK_MONOTONIC, TFD_CLOEXEC);
+ if (actx->timerfd < 0)
+ {
+ actx_error(actx, "failed to create timerfd: %m");
+ return false;
+ }
+
+ if (epoll_ctl(actx->mux, EPOLL_CTL_ADD, actx->timerfd, &ev) < 0)
+ {
+ actx_error(actx, "failed to add timerfd to epoll set: %m");
+ return false;
+ }
+
+ return true;
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ actx->mux = kqueue();
+ if (actx->mux < 0)
+ {
+ actx_error(actx, "failed to create kqueue: %m");
+ return false;
+ }
+
+ return true;
+#endif
+
+ actx_error(actx, "here's a nickel kid, get yourself a better computer");
+ return false;
+}
+
+/*
+ * Adds and removes sockets from the multiplexer set, as directed by the
+ * cURL multi handle.
+ */
+static int
+register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
+ void *socketp)
+{
+#ifdef HAVE_SYS_EPOLL_H
+ struct async_ctx *actx = ctx;
+ struct epoll_event ev = {0};
+ int res;
+ int op = EPOLL_CTL_ADD;
+
+ switch (what)
+ {
+ case CURL_POLL_IN:
+ ev.events = EPOLLIN;
+ break;
+
+ case CURL_POLL_OUT:
+ ev.events = EPOLLOUT;
+ break;
+
+ case CURL_POLL_INOUT:
+ ev.events = EPOLLIN | EPOLLOUT;
+ break;
+
+ case CURL_POLL_REMOVE:
+ op = EPOLL_CTL_DEL;
+ break;
+
+ default:
+ actx_error(actx, "unknown cURL socket operation (%d)", what);
+ return -1;
+ }
+
+ res = epoll_ctl(actx->mux, op, socket, &ev);
+ if (res < 0 && errno == EEXIST)
+ {
+ /* We already had this socket in the pollset. */
+ op = EPOLL_CTL_MOD;
+ res = epoll_ctl(actx->mux, op, socket, &ev);
+ }
+
+ if (res < 0)
+ {
+ switch (op)
+ {
+ case EPOLL_CTL_ADD:
+ actx_error(actx, "could not add to epoll set: %m");
+ break;
+
+ case EPOLL_CTL_DEL:
+ actx_error(actx, "could not delete from epoll set: %m");
+ break;
+
+ default:
+ actx_error(actx, "could not update epoll set: %m");
+ }
+
+ return -1;
+ }
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct async_ctx *actx = ctx;
+ struct kevent ev[2] = {{0}};
+ struct kevent ev_out[2];
+ struct timespec timeout = {0};
+ int nev = 0;
+ int res;
+
+ switch (what)
+ {
+ case CURL_POLL_IN:
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_OUT:
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_INOUT:
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_REMOVE:
+
+ /*
+ * We don't know which of these is currently registered, perhaps
+ * both, so we try to remove both. This means we need to tolerate
+ * ENOENT below.
+ */
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_DELETE | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_DELETE | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ default:
+ actx_error(actx, "unknown cURL socket operation (%d)", what);
+ return -1;
+ }
+
+ res = kevent(actx->mux, ev, nev, ev_out, lengthof(ev_out), &timeout);
+ if (res < 0)
+ {
+ actx_error(actx, "could not modify kqueue: %m");
+ return -1;
+ }
+
+ /*
+ * We can't use the simple errno version of kevent, because we need to
+ * skip over ENOENT while still allowing a second change to be processed.
+ * So we need a longer-form error checking loop.
+ */
+ for (int i = 0; i < res; ++i)
+ {
+ /*
+ * EV_RECEIPT should guarantee one EV_ERROR result for every change,
+ * whether successful or not. Failed entries contain a non-zero errno
+ * in the `data` field.
+ */
+ Assert(ev_out[i].flags & EV_ERROR);
+
+ errno = ev_out[i].data;
+ if (errno && errno != ENOENT)
+ {
+ switch (what)
+ {
+ case CURL_POLL_REMOVE:
+ actx_error(actx, "could not delete from kqueue: %m");
+ break;
+ default:
+ actx_error(actx, "could not add to kqueue: %m");
+ }
+ return -1;
+ }
+ }
+#endif
+
+ return 0;
+}
+
+/*
+ * Adds or removes timeouts from the multiplexer set, as directed by the
+ * cURL multi handle. Rather than continually adding and removing the timer,
+ * we keep it in the set at all times and just disarm it when it's not
+ * needed.
+ */
+static int
+register_timer(CURLM *curlm, long timeout, void *ctx)
+{
+#if HAVE_SYS_EPOLL_H
+ struct async_ctx *actx = ctx;
+ struct itimerspec spec = {0};
+
+ if (timeout < 0)
+ {
+ /* the zero itimerspec will disarm the timer below */
+ }
+ else if (timeout == 0)
+ {
+ /*
+ * A zero timeout means cURL wants us to call back immediately. That's
+ * not technically an option for timerfd, but we can make the timeout
+ * ridiculously short.
+ *
+ * TODO: maybe just signal drive_request() to immediately call back in
+ * this case?
+ */
+ spec.it_value.tv_nsec = 1;
+ }
+ else
+ {
+ spec.it_value.tv_sec = timeout / 1000;
+ spec.it_value.tv_nsec = (timeout % 1000) * 1000000;
+ }
+
+ if (timerfd_settime(actx->timerfd, 0 /* no flags */ , &spec, NULL) < 0)
+ {
+ actx_error(actx, "setting timerfd to %ld: %m", timeout);
+ return -1;
+ }
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct async_ctx *actx = ctx;
+ struct kevent ev;
+
+ EV_SET(&ev, 1, EVFILT_TIMER, timeout < 0 ? EV_DELETE : EV_ADD,
+ 0, timeout, 0);
+ if (kevent(actx->mux, &ev, 1, NULL, 0, NULL) < 0 && errno != ENOENT)
+ {
+ actx_error(actx, "setting kqueue timer to %ld: %m", timeout);
+ return -1;
+ }
+#endif
+
+ return 0;
+}
+
+/*
+ * Initializes the two cURL handles in the async_ctx. The multi handle,
+ * actx->curlm, is what drives the asynchronous engine and tells us what to do
+ * next. The easy handle, actx->curl, encapsulates the state for a single
+ * request/response. It's added to the multi handle as needed, during
+ * start_request().
+ */
+static bool
+setup_curl_handles(struct async_ctx *actx)
+{
+ curl_version_info_data *curl_info;
+
+ /*
+ * Create our multi handle. This encapsulates the entire conversation with
+ * cURL for this connection.
+ */
+ actx->curlm = curl_multi_init();
+ if (!actx->curlm)
+ {
+ /* We don't get a lot of feedback on the failure reason. */
+ actx_error(actx, "failed to create cURL multi handle");
+ return false;
+ }
+
+ /*
+ * Extract information about the libcurl we are linked against.
+ */
+ curl_info = curl_version_info(CURLVERSION_NOW);
+
+ /*
+ * The multi handle tells us what to wait on using two callbacks. These
+ * will manipulate actx->mux as needed.
+ */
+ CHECK_MSETOPT(actx, CURLMOPT_SOCKETFUNCTION, register_socket, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_SOCKETDATA, actx, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_TIMERFUNCTION, register_timer, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_TIMERDATA, actx, return false);
+
+ /*
+ * Set up an easy handle. All of our requests are made serially, so we
+ * only ever need to keep track of one.
+ */
+ actx->curl = curl_easy_init();
+ if (!actx->curl)
+ {
+ actx_error(actx, "failed to create cURL handle");
+ return false;
+ }
+
+ /*
+ * Multi-threaded applications must set CURLOPT_NOSIGNAL. This requires us
+ * to handle the possibility of SIGPIPE ourselves.
+ *
+ * TODO: handle SIGPIPE via pq_block_sigpipe(), or via a
+ * CURLOPT_SOCKOPTFUNCTION maybe...
+ */
+ CHECK_SETOPT(actx, CURLOPT_NOSIGNAL, 1L, return false);
+ if (!curl_info->ares_num)
+ {
+ /* No alternative resolver, TODO: warn about timeouts */
+ }
+
+ /* TODO investigate using conn->Pfdebug and CURLOPT_DEBUGFUNCTION here */
+ CHECK_SETOPT(actx, CURLOPT_VERBOSE, 1L, return false);
+ CHECK_SETOPT(actx, CURLOPT_ERRORBUFFER, actx->curl_err, return false);
+
+ /*
+ * Only HTTP[S] is allowed. TODO: disallow HTTP without user opt-in
+ */
+ CHECK_SETOPT(actx, CURLOPT_PROTOCOLS, CURLPROTO_HTTP | CURLPROTO_HTTPS, return false);
+
+ /*
+ * Suppress the Accept header to make our request as minimal as possible.
+ * (Ideally we would set it to "application/json" instead, but OpenID is
+ * pretty strict when it comes to provider behavior, so we have to check
+ * what comes back anyway.)
+ */
+ actx->headers = curl_slist_append(actx->headers, "Accept:"); /* TODO: check result */
+ CHECK_SETOPT(actx, CURLOPT_HTTPHEADER, actx->headers, return false);
+
+ return true;
+}
+
+/*
+ * Generic HTTP Request Handlers
+ */
+
+/*
+ * Response callback from cURL; appends the response body into actx->work_data.
+ * See start_request().
+ */
+static size_t
+append_data(char *buf, size_t size, size_t nmemb, void *userdata)
+{
+ PQExpBuffer resp = userdata;
+ size_t len = size * nmemb;
+
+ /* TODO: cap the maximum size */
+ appendBinaryPQExpBuffer(resp, buf, len);
+ /* TODO: check for broken buffer */
+
+ return len;
+}
+
+/*
+ * Begins an HTTP request on the multi handle. The caller should have set up all
+ * request-specific options on actx->curl first. The server's response body will
+ * be accumulated in actx->work_data (which will be reset, so don't store
+ * anything important there across this call).
+ *
+ * Once a request is queued, it can be driven to completion via drive_request().
+ */
+static bool
+start_request(struct async_ctx *actx)
+{
+ CURLMcode err;
+ int running;
+
+ resetPQExpBuffer(&actx->work_data);
+ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data, return false);
+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, &actx->work_data, return false);
+
+ err = curl_multi_add_handle(actx->curlm, actx->curl);
+ if (err)
+ {
+ actx_error(actx, "failed to queue HTTP request: %s",
+ curl_multi_strerror(err));
+ return false;
+ }
+
+ err = curl_multi_socket_action(actx->curlm, CURL_SOCKET_TIMEOUT, 0, &running);
+ if (err)
+ {
+ actx_error(actx, "asynchronous HTTP request failed: %s",
+ curl_multi_strerror(err));
+ return false;
+ }
+
+ /*
+ * Sanity check.
+ *
+ * TODO: even though this is nominally an asynchronous process, there are
+ * apparently operations that can synchronously fail by this point, such
+ * as connections to closed local ports. Maybe we need to let this case
+ * fall through to drive_request instead, or else perform a
+ * curl_multi_info_read immediately.
+ */
+ if (running != 1)
+ {
+ actx_error(actx, "failed to queue HTTP request");
+ return false;
+ }
+
+ return true;
+}
+
+/*
+ * Drives the multi handle towards completion. The caller should have already
+ * set up an asynchronous request via start_request().
+ */
+static PostgresPollingStatusType
+drive_request(struct async_ctx *actx)
+{
+ CURLMcode err;
+ int running;
+ CURLMsg *msg;
+ int msgs_left;
+ bool done;
+
+ err = curl_multi_socket_all(actx->curlm, &running);
+ if (err)
+ {
+ actx_error(actx, "asynchronous HTTP request failed: %s",
+ curl_multi_strerror(err));
+ return PGRES_POLLING_FAILED;
+ }
+
+ if (running)
+ {
+ /* We'll come back again. */
+ return PGRES_POLLING_READING;
+ }
+
+ done = false;
+ while ((msg = curl_multi_info_read(actx->curlm, &msgs_left)) != NULL)
+ {
+ if (msg->msg != CURLMSG_DONE)
+ {
+ /*
+ * Future cURL versions may define new message types; we don't
+ * know how to handle them, so we'll ignore them.
+ */
+ continue;
+ }
+
+ /* First check the status of the request itself. */
+ if (msg->data.result != CURLE_OK)
+ {
+ actx_error_str(actx, curl_easy_strerror(msg->data.result));
+ return PGRES_POLLING_FAILED;
+ }
+
+ /* Now remove the finished handle; we'll add it back later if needed. */
+ err = curl_multi_remove_handle(actx->curlm, msg->easy_handle);
+ if (err)
+ {
+ actx_error(actx, "cURL easy handle removal failed: %s",
+ curl_multi_strerror(err));
+ return PGRES_POLLING_FAILED;
+ }
+
+ done = true;
+ }
+
+ /* Sanity check. */
+ if (!done)
+ {
+ actx_error(actx, "no result was retrieved for the finished handle");
+ return PGRES_POLLING_FAILED;
+ }
+
+ return PGRES_POLLING_OK;
+}
+
+/*
+ * Specific HTTP Request Handlers
+ *
+ * This is finally the beginning of the actual application logic. Generally
+ * speaking, a single request consists of a start_* and a finish_* step, with
+ * drive_request() pumping the machine in between.
+ */
+
+/*
+ * Queue an OpenID Provider Configuration Request:
+ *
+ * https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderConfigurationRequest
+ * https://www.rfc-editor.org/rfc/rfc8414#section-3.1
+ *
+ * This is done first to get the endpoint URIs we need to contact and to make
+ * sure the provider provides a device authorization flow. finish_discovery()
+ * will fill in actx->provider.
+ */
+static bool
+start_discovery(struct async_ctx *actx, const char *discovery_uri)
+{
+ CHECK_SETOPT(actx, CURLOPT_HTTPGET, 1L, return false);
+ CHECK_SETOPT(actx, CURLOPT_URL, discovery_uri, return false);
+
+ return start_request(actx);
+}
+
+static bool
+finish_discovery(struct async_ctx *actx)
+{
+ long response_code;
+
+ /*----
+ * Now check the response. OIDC Discovery 1.0 is pretty strict:
+ *
+ * A successful response MUST use the 200 OK HTTP status code and
+ * return a JSON object using the application/json content type that
+ * contains a set of Claims as its members that are a subset of the
+ * Metadata values defined in Section 3.
+ *
+ * Compared to standard HTTP semantics, this makes life easy -- we don't
+ * need to worry about redirections (which would call the Issuer host
+ * validation into question), or non-authoritative responses, or any other
+ * complications.
+ */
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ if (response_code != 200)
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ actx->errctx = "failed to parse OpenID discovery document";
+ if (!parse_provider(actx, &actx->provider))
+ return false; /* error message already set */
+
+ /*
+ * Fill in any defaults for OPTIONAL/RECOMMENDED fields we care about.
+ */
+ if (!actx->provider.grant_types_supported)
+ {
+ /*
+ * Per Section 3, the default is ["authorization_code", "implicit"].
+ */
+ struct curl_slist *temp = actx->provider.grant_types_supported;
+
+ temp = curl_slist_append(temp, "authorization_code");
+ if (temp)
+ {
+ temp = curl_slist_append(temp, "implicit");
+ }
+
+ if (!temp)
+ {
+ actx_error(actx, "out of memory");
+ return false;
+ }
+
+ actx->provider.grant_types_supported = temp;
+ }
+
+ return true;
+}
+
+#define OAUTH_GRANT_TYPE_DEVICE_CODE "urn:ietf:params:oauth:grant-type:device_code"
+
+/*
+ * Ensure that the provider supports the Device Authorization flow (i.e. it
+ * accepts the device_code grant type and provides an authorization endpoint).
+ */
+static bool
+check_for_device_flow(struct async_ctx *actx)
+{
+ const struct provider *provider = &actx->provider;
+ const struct curl_slist *grant;
+ bool device_grant_found = false;
+
+ Assert(provider->issuer); /* ensured by get_discovery_document() */
+
+ /*------
+ * First, sanity checks for discovery contents that are OPTIONAL in the
+ * spec but required for our flow:
+ * - the issuer must support the device_code grant
+ * - the issuer must have actually given us a
+ * device_authorization_endpoint
+ */
+
+ grant = provider->grant_types_supported;
+ while (grant)
+ {
+ if (strcmp(grant->data, OAUTH_GRANT_TYPE_DEVICE_CODE) == 0)
+ {
+ device_grant_found = true;
+ break;
+ }
+
+ grant = grant->next;
+ }
+
+ if (!device_grant_found)
+ {
+ actx_error(actx, "issuer \"%s\" does not support device code grants",
+ provider->issuer);
+ return false;
+ }
+
+ if (!provider->device_authorization_endpoint)
+ {
+ actx_error(actx,
+ "issuer \"%s\" does not provide a device authorization endpoint",
+ provider->issuer);
+ return false;
+ }
+
+ /* TODO: check that the endpoint uses HTTPS */
+
+ return true;
+}
+
+/*
+ * Queue a Device Authorization Request:
+ *
+ * https://www.rfc-editor.org/rfc/rfc8628#section-3.1
+ *
+ * This is the second step. We ask the provider to verify the end user out of
+ * band and authorize us to act on their behalf; it will give us the required
+ * nonces for us to later poll the request status, which we'll grab in
+ * finish_device_authz().
+ */
+static bool
+start_device_authz(struct async_ctx *actx, PGconn *conn)
+{
+ const char *device_authz_uri = actx->provider.device_authorization_endpoint;
+ PQExpBuffer work_buffer = &actx->work_data;
+
+ Assert(conn->oauth_client_id); /* ensured by get_auth_token() */
+ Assert(device_authz_uri); /* ensured by check_for_device_flow() */
+
+ /* Construct our request body. TODO: url-encode */
+ resetPQExpBuffer(work_buffer);
+ appendPQExpBuffer(work_buffer, "client_id=%s", conn->oauth_client_id);
+ if (conn->oauth_scope)
+ appendPQExpBuffer(work_buffer, "&scope=%s", conn->oauth_scope);
+ /* TODO check for broken buffer */
+
+ /* Make our request. */
+ CHECK_SETOPT(actx, CURLOPT_URL, device_authz_uri, return false);
+ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data, return false);
+
+ if (conn->oauth_client_secret)
+ {
+ /*----
+ * Use HTTP Basic auth to send the password. Per RFC 6749, Sec. 2.3.1,
+ *
+ * Including the client credentials in the request-body using the
+ * two parameters is NOT RECOMMENDED and SHOULD be limited to
+ * clients unable to directly utilize the HTTP Basic authentication
+ * scheme (or other password-based HTTP authentication schemes).
+ *
+ * TODO: should we omit client_id from the body in this case?
+ */
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC, return false);
+ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id, return false);
+ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret, return false);
+ }
+ else
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE, return false);
+
+ return start_request(actx);
+}
+
+static bool
+finish_device_authz(struct async_ctx *actx)
+{
+ long response_code;
+
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ /*
+ * The device authorization endpoint uses the same error response as the
+ * token endpoint, so we have to handle 400/401 here too.
+ */
+ if (response_code != 200
+ && response_code != 400
+ /* && response_code != 401 TODO */ )
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ if (response_code != 200)
+ {
+ struct token_error err = {0};
+
+ if (!parse_token_error(actx, &err))
+ {
+ free_token_error(&err);
+ return false;
+ }
+
+ if (err.error_description)
+ appendPQExpBuffer(&actx->errbuf, "%s ", err.error_description);
+
+ appendPQExpBuffer(&actx->errbuf, "(%s)", err.error);
+
+ free_token_error(&err);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ actx->errctx = "failed to parse device authorization";
+ if (!parse_device_authz(actx, &actx->authz))
+ return false; /* error message already set */
+
+ return true;
+}
+
+/*
+ * Queue an Access Token Request:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-4.1.3
+ *
+ * This is the final step. We continually poll the token endpoint to see if the
+ * user has authorized us yet. finish_token_request() will pull either the token
+ * or a (ideally temporary) error status from the provider.
+ */
+static bool
+start_token_request(struct async_ctx *actx, PGconn *conn)
+{
+ const char *token_uri = actx->provider.token_endpoint;
+ const char *device_code = actx->authz.device_code;
+ PQExpBuffer work_buffer = &actx->work_data;
+
+ Assert(conn->oauth_client_id); /* ensured by get_auth_token() */
+ Assert(token_uri); /* ensured by get_discovery_document() */
+ Assert(device_code); /* ensured by run_device_authz() */
+
+ /* Construct our request body. TODO: url-encode */
+ resetPQExpBuffer(work_buffer);
+ appendPQExpBuffer(work_buffer, "client_id=%s", conn->oauth_client_id);
+ appendPQExpBuffer(work_buffer, "&device_code=%s", device_code);
+ appendPQExpBuffer(work_buffer, "&grant_type=%s",
+ OAUTH_GRANT_TYPE_DEVICE_CODE);
+ /* TODO check for broken buffer */
+
+ /* Make our request. */
+ CHECK_SETOPT(actx, CURLOPT_URL, token_uri, return false);
+ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data, return false);
+
+ if (conn->oauth_client_secret)
+ {
+ /*----
+ * Use HTTP Basic auth to send the password. Per RFC 6749, Sec. 2.3.1,
+ *
+ * Including the client credentials in the request-body using the
+ * two parameters is NOT RECOMMENDED and SHOULD be limited to
+ * clients unable to directly utilize the HTTP Basic authentication
+ * scheme (or other password-based HTTP authentication schemes).
+ *
+ * TODO: should we omit client_id from the body in this case?
+ */
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC, return false);
+ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id, return false);
+ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret, return false);
+ }
+ else
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE, return false);
+
+ resetPQExpBuffer(work_buffer);
+ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data, return false);
+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, work_buffer, return false);
+
+ return start_request(actx);
+}
+
+static bool
+finish_token_request(struct async_ctx *actx, struct token *tok)
+{
+ long response_code;
+
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ /*
+ * Per RFC 6749, Section 5, a successful response uses 200 OK. An error
+ * response uses either 400 Bad Request or 401 Unauthorized.
+ *
+ * TODO: there are references online to 403 appearing in the wild...
+ */
+ if (response_code != 200
+ && response_code != 400
+ /* && response_code != 401 TODO */ )
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ if (response_code == 200)
+ {
+ actx->errctx = "failed to parse access token response";
+ if (!parse_access_token(actx, tok))
+ return false; /* error message already set */
+ }
+ else if (!parse_token_error(actx, &tok->err))
+ return false;
+
+ return true;
+}
+
+/*
+ * The top-level, nonblocking entry point for the cURL implementation. This will
+ * be called several times to pump the async engine.
+ *
+ * The architecture is based on PQconnectPoll(). The first half drives the
+ * connection state forward as necessary, returning if we're not ready to
+ * proceed to the next step yet. The second half performs the actual transition
+ * between states.
+ *
+ * You can trace the overall OAuth flow through the second half. It's linear
+ * until we get to the end, where we flip back and forth between
+ * OAUTH_STEP_TOKEN_REQUEST and OAUTH_STEP_WAIT_INTERVAL to regularly ping the
+ * provider.
+ */
+PostgresPollingStatusType
+pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
+{
+ fe_oauth_state *state = conn->sasl_state;
+ struct async_ctx *actx;
+
+ struct token tok = {0};
+
+ /*
+ * XXX This is not safe. cURL has stringent requirements for the thread
+ * context in which you call curl_global_init(), because it's going to try
+ * initializing a bunch of other libraries (OpenSSL, Winsock...). And we
+ * probably need to consider both the TLS backend libcurl is compiled
+ * against and what the user has asked us to do via PQinit[Open]SSL.
+ *
+ * Recent versions of libcurl have improved the thread-safety situation,
+ * but you apparently can't check at compile time whether the
+ * implementation is thread-safe, and there's a chicken-and-egg problem
+ * where you can't check the thread safety until you've initialized cURL,
+ * which you can't do before you've made sure it's thread-safe...
+ *
+ * We know we've already initialized Winsock by this point, so we should
+ * be able to safely skip that bit. But we have to tell cURL to initialize
+ * everything else, because other pieces of our client executable may
+ * already be using cURL for their own purposes. If we initialize libcurl
+ * first, with only a subset of its features, we could break those other
+ * clients nondeterministically, and that would probably be a nightmare to
+ * debug.
+ */
+ curl_global_init(CURL_GLOBAL_ALL
+ & ~CURL_GLOBAL_WIN32); /* we already initialized Winsock */
+
+ if (!state->async_ctx)
+ {
+ /*
+ * Create our asynchronous state, and hook it into the upper-level
+ * OAuth state immediately, so any failures below won't leak the
+ * context allocation.
+ */
+ actx = calloc(1, sizeof(*actx));
+ if (!actx)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return PGRES_POLLING_FAILED;
+ }
+
+ actx->mux = PGINVALID_SOCKET;
+#ifdef HAVE_SYS_EPOLL_H
+ actx->timerfd = -1;
+#endif
+
+ state->async_ctx = actx;
+ state->free_async_ctx = free_curl_async_ctx;
+
+ initPQExpBuffer(&actx->work_data);
+ initPQExpBuffer(&actx->errbuf);
+
+ if (!setup_multiplexer(actx))
+ goto error_return;
+
+ if (!setup_curl_handles(actx))
+ goto error_return;
+ }
+
+ actx = state->async_ctx;
+
+ /* By default, the multiplexer is the altsock. Reassign as desired. */
+ *altsock = actx->mux;
+
+ switch (actx->step)
+ {
+ case OAUTH_STEP_INIT:
+ break;
+
+ case OAUTH_STEP_DISCOVERY:
+ case OAUTH_STEP_DEVICE_AUTHORIZATION:
+ case OAUTH_STEP_TOKEN_REQUEST:
+ {
+ PostgresPollingStatusType status;
+
+ status = drive_request(actx);
+
+ if (status == PGRES_POLLING_FAILED)
+ goto error_return;
+ else if (status != PGRES_POLLING_OK)
+ {
+ /* not done yet */
+ free_token(&tok);
+ return status;
+ }
+ }
+
+ case OAUTH_STEP_WAIT_INTERVAL:
+ /* TODO check that the timer has expired */
+ break;
+ }
+
+ switch (actx->step)
+ {
+ case OAUTH_STEP_INIT:
+ actx->errctx = "failed to fetch OpenID discovery document";
+ if (!start_discovery(actx, conn->oauth_discovery_uri))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_DISCOVERY;
+ break;
+
+ case OAUTH_STEP_DISCOVERY:
+ if (!finish_discovery(actx))
+ goto error_return;
+
+ /* TODO: check issuer */
+
+ actx->errctx = "cannot run OAuth device authorization";
+ if (!check_for_device_flow(actx))
+ goto error_return;
+
+ actx->errctx = "failed to obtain device authorization";
+ if (!start_device_authz(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_DEVICE_AUTHORIZATION;
+ break;
+
+ case OAUTH_STEP_DEVICE_AUTHORIZATION:
+ if (!finish_device_authz(actx))
+ goto error_return;
+
+ actx->errctx = "failed to obtain access token";
+ if (!start_token_request(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_TOKEN_REQUEST;
+ break;
+
+ case OAUTH_STEP_TOKEN_REQUEST:
+ {
+ const struct token_error *err;
+#ifdef HAVE_SYS_EPOLL_H
+ struct itimerspec spec = {0};
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct kevent ev = {0};
+#endif
+
+ if (!finish_token_request(actx, &tok))
+ goto error_return;
+
+ if (!actx->user_prompted)
+ {
+ int res;
+ PQpromptOAuthDevice prompt = {
+ .verification_uri = actx->authz.verification_uri,
+ .user_code = actx->authz.user_code,
+ /* TODO: optional fields */
+ };
+
+ /*
+ * Now that we know the token endpoint isn't broken, give
+ * the user the login instructions.
+ */
+ res = PQauthDataHook(PQAUTHDATA_PROMPT_OAUTH_DEVICE, conn,
+ &prompt);
+
+ if (!res)
+ {
+ fprintf(stderr, "Visit %s and enter the code: %s",
+ prompt.verification_uri, prompt.user_code);
+ }
+ else if (res < 0)
+ {
+ actx_error(actx, "device prompt failed");
+ goto error_return;
+ }
+
+ actx->user_prompted = true;
+ }
+
+ if (tok.access_token)
+ {
+ /* Construct our Bearer token. */
+ resetPQExpBuffer(&actx->work_data);
+ appendPQExpBuffer(&actx->work_data, "Bearer %s",
+ tok.access_token);
+
+ if (PQExpBufferDataBroken(actx->work_data))
+ {
+ actx_error(actx, "out of memory");
+ goto error_return;
+ }
+
+ state->token = strdup(actx->work_data.data);
+ break;
+ }
+
+ /*
+ * authorization_pending and slow_down are the only acceptable
+ * errors; anything else and we bail.
+ */
+ err = &tok.err;
+ if (!err->error || (strcmp(err->error, "authorization_pending")
+ && strcmp(err->error, "slow_down")))
+ {
+ /* TODO handle !err->error */
+ if (err->error_description)
+ appendPQExpBuffer(&actx->errbuf, "%s ",
+ err->error_description);
+
+ appendPQExpBuffer(&actx->errbuf, "(%s)", err->error);
+
+ goto error_return;
+ }
+
+ /*
+ * A slow_down error requires us to permanently increase our
+ * retry interval by five seconds. RFC 8628, Sec. 3.5.
+ */
+ if (strcmp(err->error, "slow_down") == 0)
+ {
+ actx->authz.interval += 5; /* TODO check for overflow? */
+ }
+
+ /*
+ * Wait for the required interval before issuing the next
+ * request.
+ */
+ Assert(actx->authz.interval > 0);
+#ifdef HAVE_SYS_EPOLL_H
+ spec.it_value.tv_sec = actx->authz.interval;
+
+ if (timerfd_settime(actx->timerfd, 0 /* no flags */ , &spec, NULL) < 0)
+ {
+ actx_error(actx, "failed to set timerfd: %m");
+ goto error_return;
+ }
+
+ *altsock = actx->timerfd;
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ /* XXX: I guess this wants to be hidden in a routine */
+ EV_SET(&ev, 1, EVFILT_TIMER, EV_ADD, 0,
+ actx->authz.interval * 1000, 0);
+ if (kevent(actx->mux, &ev, 1, NULL, 0, NULL) < 0)
+ {
+ actx_error(actx, "failed to set kqueue timer: %m");
+ goto error_return;
+ }
+ /* XXX: why did we change the altsock in the epoll version? */
+#endif
+ actx->step = OAUTH_STEP_WAIT_INTERVAL;
+ break;
+ }
+
+ case OAUTH_STEP_WAIT_INTERVAL:
+ actx->errctx = "failed to obtain access token";
+ if (!start_token_request(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_TOKEN_REQUEST;
+ break;
+ }
+
+ free_token(&tok);
+
+ /* If we've stored a token, we're done. Otherwise come back later. */
+ return state->token ? PGRES_POLLING_OK : PGRES_POLLING_READING;
+
+error_return:
+
+ /*
+ * Assemble the three parts of our error: context, body, and detail. See
+ * also the documentation for struct async_ctx.
+ */
+ if (actx->errctx)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext(actx->errctx));
+ appendPQExpBufferStr(&conn->errorMessage, ": ");
+ }
+
+ if (PQExpBufferDataBroken(actx->errbuf))
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("out of memory"));
+ else
+ appendPQExpBufferStr(&conn->errorMessage, actx->errbuf.data);
+
+ if (actx->curl_err[0])
+ {
+ size_t len;
+
+ appendPQExpBuffer(&conn->errorMessage, " (%s)", actx->curl_err);
+
+ /* Sometimes libcurl adds a newline to the error buffer. :( */
+ len = conn->errorMessage.len;
+ if (len >= 2 && conn->errorMessage.data[len - 2] == '\n')
+ {
+ conn->errorMessage.data[len - 2] = ')';
+ conn->errorMessage.data[len - 1] = '\0';
+ conn->errorMessage.len--;
+ }
+ }
+
+ appendPQExpBufferStr(&conn->errorMessage, "\n");
+
+ free_token(&tok);
+ return PGRES_POLLING_FAILED;
+}
diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
new file mode 100644
index 0000000000..66ee8ff076
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth.c
@@ -0,0 +1,659 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth.c
+ * The front-end (client) implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "common/base64.h"
+#include "common/hmac.h"
+#include "common/jsonapi.h"
+#include "common/oauth-common.h"
+#include "fe-auth.h"
+#include "fe-auth-oauth.h"
+#include "mb/pg_wchar.h"
+
+/* The exported OAuth callback mechanism. */
+static void *oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism);
+static SASLStatus oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen);
+static bool oauth_channel_bound(void *opaq);
+static void oauth_free(void *opaq);
+
+const pg_fe_sasl_mech pg_oauth_mech = {
+ oauth_init,
+ oauth_exchange,
+ oauth_channel_bound,
+ oauth_free,
+};
+
+static void *
+oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism)
+{
+ fe_oauth_state *state;
+
+ /*
+ * We only support one SASL mechanism here; anything else is programmer
+ * error.
+ */
+ Assert(sasl_mechanism != NULL);
+ Assert(strcmp(sasl_mechanism, OAUTHBEARER_NAME) == 0);
+
+ state = calloc(1, sizeof(*state));
+ if (!state)
+ return NULL;
+
+ state->state = FE_OAUTH_INIT;
+ state->conn = conn;
+
+ return state;
+}
+
+#define kvsep "\x01"
+
+static char *
+client_initial_response(PGconn *conn, const char *token)
+{
+ static const char *const resp_format = "n,," kvsep "auth=%s" kvsep kvsep;
+
+ PQExpBufferData buf;
+ char *response = NULL;
+
+ if (!token)
+ {
+ /*
+ * Either programmer error, or something went badly wrong during the
+ * asynchronous fetch.
+ *
+ * TODO: users shouldn't see this; what action should they take if
+ * they do?
+ */
+ libpq_append_conn_error(conn, "no OAuth token was set for the connection");
+ return NULL;
+ }
+
+ initPQExpBuffer(&buf);
+ appendPQExpBuffer(&buf, resp_format, token);
+
+ if (!PQExpBufferDataBroken(buf))
+ response = strdup(buf.data);
+
+ termPQExpBuffer(&buf);
+ return response;
+}
+
+#define ERROR_STATUS_FIELD "status"
+#define ERROR_SCOPE_FIELD "scope"
+#define ERROR_OPENID_CONFIGURATION_FIELD "openid-configuration"
+
+struct json_ctx
+{
+ char *errmsg; /* any non-NULL value stops all processing */
+ PQExpBufferData errbuf; /* backing memory for errmsg */
+ int nested; /* nesting level (zero is the top) */
+
+ const char *target_field_name; /* points to a static allocation */
+ char **target_field; /* see below */
+
+ /* target_field, if set, points to one of the following: */
+ char *status;
+ char *scope;
+ char *discovery_uri;
+};
+
+#define oauth_json_has_error(ctx) \
+ (PQExpBufferDataBroken((ctx)->errbuf) || (ctx)->errmsg)
+
+#define oauth_json_set_error(ctx, ...) \
+ do { \
+ appendPQExpBuffer(&(ctx)->errbuf, __VA_ARGS__); \
+ (ctx)->errmsg = (ctx)->errbuf.data; \
+ } while (0)
+
+static JsonParseErrorType
+oauth_json_object_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ ++ctx->nested;
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_end(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_field_start(void *state, char *name, bool isnull)
+{
+ struct json_ctx *ctx = state;
+
+ if (ctx->nested == 1)
+ {
+ if (strcmp(name, ERROR_STATUS_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_STATUS_FIELD;
+ ctx->target_field = &ctx->status;
+ }
+ else if (strcmp(name, ERROR_SCOPE_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_SCOPE_FIELD;
+ ctx->target_field = &ctx->scope;
+ }
+ else if (strcmp(name, ERROR_OPENID_CONFIGURATION_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_OPENID_CONFIGURATION_FIELD;
+ ctx->target_field = &ctx->discovery_uri;
+ }
+ }
+
+ free(name);
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_scalar(void *state, char *token, JsonTokenType type)
+{
+ struct json_ctx *ctx = state;
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ if (type == JSON_TOKEN_STRING)
+ {
+ *ctx->target_field = token;
+
+ ctx->target_field = NULL;
+ ctx->target_field_name = NULL;
+
+ return JSON_SUCCESS; /* don't free the token we're using */
+ }
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ free(token);
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static bool
+handle_oauth_sasl_error(PGconn *conn, char *msg, int msglen)
+{
+ JsonLexContext lex = {0};
+ JsonSemAction sem = {0};
+ JsonParseErrorType err;
+ struct json_ctx ctx = {0};
+ char *errmsg = NULL;
+
+ /* Sanity check. */
+ if (strlen(msg) != msglen)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server's error message contained an embedded NULL, and was discarded"));
+ return false;
+ }
+
+ makeJsonLexContextCstringLen(&lex, msg, msglen, PG_UTF8, true);
+
+ initPQExpBuffer(&ctx.errbuf);
+ sem.semstate = &ctx;
+
+ sem.object_start = oauth_json_object_start;
+ sem.object_end = oauth_json_object_end;
+ sem.object_field_start = oauth_json_object_field_start;
+ sem.array_start = oauth_json_array_start;
+ sem.scalar = oauth_json_scalar;
+
+ err = pg_parse_json(&lex, &sem);
+
+ if (err == JSON_SEM_ACTION_FAILED)
+ {
+ if (PQExpBufferDataBroken(ctx.errbuf))
+ errmsg = libpq_gettext("out of memory");
+ else if (ctx.errmsg)
+ errmsg = ctx.errmsg;
+ else
+ {
+ /*
+ * Developer error: one of the action callbacks didn't call
+ * oauth_json_set_error() before erroring out.
+ */
+ Assert(oauth_json_has_error(&ctx));
+ errmsg = "<unexpected empty error>";
+ }
+ }
+ else if (err != JSON_SUCCESS)
+ errmsg = json_errdetail(err, &lex);
+
+ if (errmsg)
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("failed to parse server's error response: %s"),
+ errmsg);
+
+ /* Don't need the error buffer or the JSON lexer anymore. */
+ termPQExpBuffer(&ctx.errbuf);
+ freeJsonLexContext(&lex);
+
+ if (errmsg)
+ return false;
+
+ /* TODO: what if these override what the user already specified? */
+ if (ctx.discovery_uri)
+ {
+ if (conn->oauth_discovery_uri)
+ free(conn->oauth_discovery_uri);
+
+ conn->oauth_discovery_uri = ctx.discovery_uri;
+ }
+
+ if (ctx.scope)
+ {
+ if (conn->oauth_scope)
+ free(conn->oauth_scope);
+
+ conn->oauth_scope = ctx.scope;
+ }
+ /* TODO: missing error scope should clear any existing connection scope */
+
+ if (!ctx.status)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("server sent error response without a status"));
+ return false;
+ }
+
+ if (strcmp(ctx.status, "invalid_token") == 0)
+ {
+ /*
+ * invalid_token is the only error code we'll automatically retry for,
+ * but only if we have enough information to do so.
+ */
+ if (conn->oauth_discovery_uri)
+ conn->oauth_want_retry = true;
+ }
+ /* TODO: include status in hard failure message */
+
+ return true;
+}
+
+static void
+free_request(PGconn *conn, void *vreq)
+{
+ PQoauthBearerRequest *request = vreq;
+
+ if (request->cleanup)
+ request->cleanup(conn, request);
+
+ free(request);
+}
+
+static PostgresPollingStatusType
+run_user_oauth_flow(PGconn *conn, pgsocket *altsock)
+{
+ fe_oauth_state *state = conn->sasl_state;
+ PQoauthBearerRequest *request = state->async_ctx;
+ PostgresPollingStatusType status;
+
+ if (!request->async)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow provided neither a token nor an async callback");
+ return PGRES_POLLING_FAILED;
+ }
+
+ status = request->async(conn, request, altsock);
+ if (status == PGRES_POLLING_FAILED)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow failed");
+ return status;
+ }
+ else if (status == PGRES_POLLING_OK)
+ {
+ /*
+ * We already have a token, so copy it into the state. (We can't hold
+ * onto the original string, since it may not be safe for us to free()
+ * it.)
+ */
+ PQExpBufferData token;
+
+ if (!request->token)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow did not provide a token");
+ return PGRES_POLLING_FAILED;
+ }
+
+ initPQExpBuffer(&token);
+ appendPQExpBuffer(&token, "Bearer %s", request->token);
+
+ if (PQExpBufferDataBroken(token))
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return PGRES_POLLING_FAILED;
+ }
+
+ state->token = token.data;
+ return PGRES_POLLING_OK;
+ }
+
+ /* TODO: what if no altsock was set? */
+ return status;
+}
+
+static bool
+setup_token_request(PGconn *conn, fe_oauth_state *state)
+{
+ int res;
+ PQoauthBearerRequest request = {
+ .openid_configuration = conn->oauth_discovery_uri,
+ .scope = conn->oauth_scope,
+ };
+
+ Assert(request.openid_configuration);
+
+ /* The client may have overridden the OAuth flow. */
+ res = PQauthDataHook(PQAUTHDATA_OAUTH_BEARER_TOKEN, conn, &request);
+ if (res > 0)
+ {
+ PQoauthBearerRequest *request_copy;
+
+ if (request.token)
+ {
+ /*
+ * We already have a token, so copy it into the state. (We can't
+ * hold onto the original string, since it may not be safe for us
+ * to free() it.)
+ */
+ PQExpBufferData token;
+
+ initPQExpBuffer(&token);
+ appendPQExpBuffer(&token, "Bearer %s", request.token);
+
+ if (PQExpBufferDataBroken(token))
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ goto fail;
+ }
+
+ state->token = token.data;
+
+ /* short-circuit */
+ if (request.cleanup)
+ request.cleanup(conn, &request);
+ return true;
+ }
+
+ request_copy = malloc(sizeof(*request_copy));
+ if (!request_copy)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ goto fail;
+ }
+
+ memcpy(request_copy, &request, sizeof(request));
+
+ conn->async_auth = run_user_oauth_flow;
+ state->async_ctx = request_copy;
+ state->free_async_ctx = free_request;
+ }
+ else if (res < 0)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow failed");
+ goto fail;
+ }
+ else
+ {
+ /* Use our built-in OAuth flow. */
+ conn->async_auth = pg_fe_run_oauth_flow;
+ }
+
+ return true;
+
+fail:
+ if (request.cleanup)
+ request.cleanup(conn, &request);
+ return false;
+}
+
+static bool
+derive_discovery_uri(PGconn *conn)
+{
+ PQExpBufferData discovery_buf;
+
+ if (conn->oauth_discovery_uri || !conn->oauth_issuer)
+ {
+ /*
+ * Either we already have one, or we aren't able to derive one
+ * ourselves. The latter case is not an error condition; we'll just
+ * ask the server to provide one for us.
+ */
+ return true;
+ }
+
+ initPQExpBuffer(&discovery_buf);
+
+ Assert(!conn->oauth_discovery_uri);
+ Assert(conn->oauth_issuer);
+
+ /*
+ * If we don't yet have a discovery URI, but the user gave us an explicit
+ * issuer, use the .well-known discovery URI for that issuer.
+ */
+ appendPQExpBufferStr(&discovery_buf, conn->oauth_issuer);
+ appendPQExpBufferStr(&discovery_buf, "/.well-known/openid-configuration");
+
+ if (PQExpBufferDataBroken(discovery_buf))
+ goto cleanup;
+
+ conn->oauth_discovery_uri = strdup(discovery_buf.data);
+
+cleanup:
+ termPQExpBuffer(&discovery_buf);
+ return (conn->oauth_discovery_uri != NULL);
+}
+
+static SASLStatus
+oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen)
+{
+ fe_oauth_state *state = opaq;
+ PGconn *conn = state->conn;
+
+ *output = NULL;
+ *outputlen = 0;
+
+ switch (state->state)
+ {
+ case FE_OAUTH_INIT:
+ Assert(inputlen == -1);
+
+ if (!derive_discovery_uri(conn))
+ return SASL_FAILED;
+
+ if (conn->oauth_discovery_uri)
+ {
+ if (!conn->oauth_client_id)
+ {
+ /* We can't talk to a server without a client identifier. */
+ libpq_append_conn_error(conn, "no oauth_client_id is set for the connection");
+ return SASL_FAILED;
+ }
+
+ /*
+ * Decide whether we're using a user-provided OAuth flow, or
+ * the one we have built in.
+ */
+ if (!setup_token_request(conn, state))
+ return SASL_FAILED;
+
+ if (state->token)
+ {
+ /*
+ * A really smart user implementation may have already
+ * given us the token (e.g. if there was an unexpired copy
+ * already cached). In that case, we can just fall
+ * through.
+ */
+ }
+ else
+ {
+ /*
+ * Otherwise, we have to hand the connection over to our
+ * OAuth implementation. This involves a number of HTTP
+ * connections and timed waits, so we escape the
+ * synchronous auth processing and tell PQconnectPoll to
+ * transfer control to our async implementation.
+ */
+ Assert(conn->async_auth); /* should have been set
+ * already */
+ state->state = FE_OAUTH_REQUESTING_TOKEN;
+ return SASL_ASYNC;
+ }
+ }
+ else
+ {
+ /*
+ * If we don't have a discovery URI to be able to request a
+ * token, we ask the server for one explicitly with an empty
+ * token. This doesn't require any asynchronous work.
+ */
+ state->token = strdup("");
+ if (!state->token)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return SASL_FAILED;
+ }
+ }
+
+ /* fall through */
+
+ case FE_OAUTH_REQUESTING_TOKEN:
+ /* We should still be in the initial response phase. */
+ Assert(inputlen == -1);
+
+ *output = client_initial_response(conn, state->token);
+ if (!*output)
+ return SASL_FAILED;
+
+ *outputlen = strlen(*output);
+ state->state = FE_OAUTH_BEARER_SENT;
+
+ return SASL_CONTINUE;
+
+ case FE_OAUTH_BEARER_SENT:
+ if (final)
+ {
+ /* TODO: ensure there is no message content here. */
+ return SASL_COMPLETE;
+ }
+
+ /*
+ * Error message sent by the server.
+ */
+ if (!handle_oauth_sasl_error(conn, input, inputlen))
+ return SASL_FAILED;
+
+ /*
+ * Respond with the required dummy message (RFC 7628, sec. 3.2.3).
+ */
+ *output = strdup(kvsep);
+ *outputlen = strlen(*output); /* == 1 */
+
+ state->state = FE_OAUTH_SERVER_ERROR;
+ return SASL_CONTINUE;
+
+ case FE_OAUTH_SERVER_ERROR:
+
+ /*
+ * After an error, the server should send an error response to
+ * fail the SASL handshake, which is handled in higher layers.
+ *
+ * If we get here, the server either sent *another* challenge
+ * which isn't defined in the RFC, or completed the handshake
+ * successfully after telling us it was going to fail. Neither is
+ * acceptable.
+ */
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server sent additional OAuth data after error\n"));
+ return SASL_FAILED;
+
+ default:
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("invalid OAuth exchange state\n"));
+ break;
+ }
+
+ return SASL_FAILED;
+}
+
+static bool
+oauth_channel_bound(void *opaq)
+{
+ /* This mechanism does not support channel binding. */
+ return false;
+}
+
+static void
+oauth_free(void *opaq)
+{
+ fe_oauth_state *state = opaq;
+
+ free(state->token);
+ if (state->async_ctx)
+ state->free_async_ctx(state->conn, state->async_ctx);
+
+ free(state);
+}
diff --git a/src/interfaces/libpq/fe-auth-oauth.h b/src/interfaces/libpq/fe-auth-oauth.h
new file mode 100644
index 0000000000..8d4ea45aa8
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth.h
@@ -0,0 +1,42 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth.h
+ *
+ * Definitions for OAuth authentication implementations
+ *
+ * Portions Copyright (c) 2023, PostgreSQL Global Development Group
+ *
+ * src/interfaces/libpq/fe-auth-oauth.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FE_AUTH_OAUTH_H
+#define FE_AUTH_OAUTH_H
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+
+
+typedef enum
+{
+ FE_OAUTH_INIT,
+ FE_OAUTH_REQUESTING_TOKEN,
+ FE_OAUTH_BEARER_SENT,
+ FE_OAUTH_SERVER_ERROR,
+} fe_oauth_state_enum;
+
+typedef struct
+{
+ fe_oauth_state_enum state;
+
+ PGconn *conn;
+ char *token;
+
+ void *async_ctx;
+ void (*free_async_ctx) (PGconn *conn, void *ctx);
+} fe_oauth_state;
+
+extern PostgresPollingStatusType pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock);
+
+#endif /* FE_AUTH_OAUTH_H */
diff --git a/src/interfaces/libpq/fe-auth-sasl.h b/src/interfaces/libpq/fe-auth-sasl.h
index 4eecf53a15..83bdab1f40 100644
--- a/src/interfaces/libpq/fe-auth-sasl.h
+++ b/src/interfaces/libpq/fe-auth-sasl.h
@@ -30,6 +30,7 @@ typedef enum
SASL_COMPLETE = 0,
SASL_FAILED,
SASL_CONTINUE,
+ SASL_ASYNC,
} SASLStatus;
/*
@@ -77,6 +78,8 @@ typedef struct pg_fe_sasl_mech
*
* state: The opaque mechanism state returned by init()
*
+ * final: true if the server has sent a final exchange outcome
+ *
* input: The challenge data sent by the server, or NULL when
* generating a client-first initial response (that is, when
* the server expects the client to send a message to start
@@ -101,12 +104,17 @@ typedef struct pg_fe_sasl_mech
*
* SASL_CONTINUE: The output buffer is filled with a client response.
* Additional server challenge is expected
+ * SASL_ASYNC: Some asynchronous processing external to the
+ * connection needs to be done before a response can be
+ * generated. The mechanism is responsible for setting up
+ * conn->async_auth appropriately before returning.
* SASL_COMPLETE: The SASL exchange has completed successfully.
* SASL_FAILED: The exchange has failed and the connection should be
* dropped.
*--------
*/
- SASLStatus (*exchange) (void *state, char *input, int inputlen,
+ SASLStatus (*exchange) (void *state, bool final,
+ char *input, int inputlen,
char **output, int *outputlen);
/*--------
diff --git a/src/interfaces/libpq/fe-auth-scram.c b/src/interfaces/libpq/fe-auth-scram.c
index 0bb820e0d9..da168eb2f5 100644
--- a/src/interfaces/libpq/fe-auth-scram.c
+++ b/src/interfaces/libpq/fe-auth-scram.c
@@ -24,7 +24,8 @@
/* The exported SCRAM callback mechanism. */
static void *scram_init(PGconn *conn, const char *password,
const char *sasl_mechanism);
-static SASLStatus scram_exchange(void *opaq, char *input, int inputlen,
+static SASLStatus scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen);
static bool scram_channel_bound(void *opaq);
static void scram_free(void *opaq);
@@ -202,7 +203,8 @@ scram_free(void *opaq)
* Exchange a SCRAM message with backend.
*/
static SASLStatus
-scram_exchange(void *opaq, char *input, int inputlen,
+scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen)
{
fe_scram_state *state = (fe_scram_state *) opaq;
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index 81ec08485d..9cd5c8cfb1 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -39,6 +39,7 @@
#endif
#include "common/md5.h"
+#include "common/oauth-common.h"
#include "common/scram-common.h"
#include "fe-auth.h"
#include "fe-auth-sasl.h"
@@ -419,7 +420,7 @@ pg_SSPI_startup(PGconn *conn, int use_negotiate, int payloadlen)
* Initialize SASL authentication exchange.
*/
static int
-pg_SASL_init(PGconn *conn, int payloadlen)
+pg_SASL_init(PGconn *conn, int payloadlen, bool *async)
{
char *initialresponse = NULL;
int initialresponselen;
@@ -437,7 +438,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
goto error;
}
- if (conn->sasl_state)
+ if (conn->sasl_state && !conn->async_auth)
{
libpq_append_conn_error(conn, "duplicate SASL authentication request");
goto error;
@@ -524,6 +525,15 @@ pg_SASL_init(PGconn *conn, int payloadlen)
conn->sasl = &pg_scram_mech;
conn->password_needed = true;
}
+#ifdef USE_OAUTH
+ else if (strcmp(mechanism_buf.data, OAUTHBEARER_NAME) == 0 &&
+ !selected_mechanism)
+ {
+ selected_mechanism = OAUTHBEARER_NAME;
+ conn->sasl = &pg_oauth_mech;
+ conn->password_needed = false;
+ }
+#endif
}
if (!selected_mechanism)
@@ -563,26 +573,48 @@ pg_SASL_init(PGconn *conn, int payloadlen)
Assert(conn->sasl);
- /*
- * Initialize the SASL state information with all the information gathered
- * during the initial exchange.
- *
- * Note: Only tls-unique is supported for the moment.
- */
- conn->sasl_state = conn->sasl->init(conn,
- password,
- selected_mechanism);
if (!conn->sasl_state)
- goto oom_error;
+ {
+ /*
+ * Initialize the SASL state information with all the information
+ * gathered during the initial exchange.
+ *
+ * Note: Only tls-unique is supported for the moment.
+ */
+ conn->sasl_state = conn->sasl->init(conn,
+ password,
+ selected_mechanism);
+ if (!conn->sasl_state)
+ goto oom_error;
+ }
+ else
+ {
+ /*
+ * This is only possible if we're returning from an async loop.
+ * Disconnect it now.
+ */
+ Assert(conn->async_auth);
+ conn->async_auth = NULL;
+ }
/* Get the mechanism-specific Initial Client Response, if any */
- status = conn->sasl->exchange(conn->sasl_state,
+ status = conn->sasl->exchange(conn->sasl_state, false,
NULL, -1,
&initialresponse, &initialresponselen);
if (status == SASL_FAILED)
goto error;
+ if (status == SASL_ASYNC)
+ {
+ /*
+ * The mechanism should have set up the necessary callbacks; all we
+ * need to do is signal the caller.
+ */
+ *async = true;
+ return STATUS_OK;
+ }
+
/*
* Build a SASLInitialResponse message, and send it.
*/
@@ -625,7 +657,7 @@ oom_error:
* the protocol.
*/
static int
-pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
+pg_SASL_continue(PGconn *conn, int payloadlen, bool final, bool *async)
{
char *output;
int outputlen;
@@ -650,11 +682,21 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
/* For safety and convenience, ensure the buffer is NULL-terminated. */
challenge[payloadlen] = '\0';
- status = conn->sasl->exchange(conn->sasl_state,
+ status = conn->sasl->exchange(conn->sasl_state, final,
challenge, payloadlen,
&output, &outputlen);
free(challenge); /* don't need the input anymore */
+ if (status == SASL_ASYNC)
+ {
+ /*
+ * The mechanism should have set up the necessary callbacks; all we
+ * need to do is signal the caller.
+ */
+ *async = true;
+ return STATUS_OK;
+ }
+
if (final && status == SASL_CONTINUE)
{
if (outputlen != 0)
@@ -955,12 +997,18 @@ check_expected_areq(AuthRequest areq, PGconn *conn)
* it. We are responsible for reading any remaining extra data, specific
* to the authentication method. 'payloadlen' is the remaining length in
* the message.
+ *
+ * If *async is set to true on return, the client doesn't yet have enough
+ * information to respond, and the caller must temporarily switch to
+ * conn->async_auth() to continue driving the exchange.
*/
int
-pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
+pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn, bool *async)
{
int oldmsglen;
+ *async = false;
+
if (!check_expected_areq(areq, conn))
return STATUS_ERROR;
@@ -1118,7 +1166,7 @@ pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
* The request contains the name (as assigned by IANA) of the
* authentication mechanism.
*/
- if (pg_SASL_init(conn, payloadlen) != STATUS_OK)
+ if (pg_SASL_init(conn, payloadlen, async) != STATUS_OK)
{
/* pg_SASL_init already set the error message */
return STATUS_ERROR;
@@ -1135,7 +1183,8 @@ pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
}
oldmsglen = conn->errorMessage.len;
if (pg_SASL_continue(conn, payloadlen,
- (areq == AUTH_REQ_SASL_FIN)) != STATUS_OK)
+ (areq == AUTH_REQ_SASL_FIN),
+ async) != STATUS_OK)
{
/* Use this message if pg_SASL_continue didn't supply one */
if (conn->errorMessage.len == oldmsglen)
@@ -1451,3 +1500,23 @@ PQchangePassword(PGconn *conn, const char *user, const char *passwd)
}
}
}
+
+PQauthDataHook_type PQauthDataHook = PQdefaultAuthDataHook;
+
+PQauthDataHook_type
+PQgetAuthDataHook(void)
+{
+ return PQauthDataHook;
+}
+
+void
+PQsetAuthDataHook(PQauthDataHook_type hook)
+{
+ PQauthDataHook = hook ? hook : PQdefaultAuthDataHook;
+}
+
+int
+PQdefaultAuthDataHook(PGAuthData type, PGconn *conn, void *data)
+{
+ return 0; /* handle nothing */
+}
diff --git a/src/interfaces/libpq/fe-auth.h b/src/interfaces/libpq/fe-auth.h
index a18c508688..3e2bc1333f 100644
--- a/src/interfaces/libpq/fe-auth.h
+++ b/src/interfaces/libpq/fe-auth.h
@@ -18,8 +18,12 @@
#include "libpq-int.h"
+extern PQauthDataHook_type PQauthDataHook;
+
+
/* Prototypes for functions in fe-auth.c */
-extern int pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn);
+extern int pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn,
+ bool *async);
extern char *pg_fe_getusername(uid_t user_id, PQExpBuffer errorMessage);
extern char *pg_fe_getauthname(PQExpBuffer errorMessage);
@@ -29,4 +33,7 @@ extern char *pg_fe_scram_build_secret(const char *password,
int iterations,
const char **errstr);
+/* Mechanisms in fe-auth-oauth.c */
+extern const pg_fe_sasl_mech pg_oauth_mech;
+
#endif /* FE_AUTH_H */
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index d4e10a0c4f..15ceb73d01 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -359,6 +359,23 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"Load-Balance-Hosts", "", 8, /* sizeof("disable") = 8 */
offsetof(struct pg_conn, load_balance_hosts)},
+ /* OAuth v2 */
+ {"oauth_issuer", NULL, NULL, NULL,
+ "OAuth-Issuer", "", 40,
+ offsetof(struct pg_conn, oauth_issuer)},
+
+ {"oauth_client_id", NULL, NULL, NULL,
+ "OAuth-Client-ID", "", 40,
+ offsetof(struct pg_conn, oauth_client_id)},
+
+ {"oauth_client_secret", NULL, NULL, NULL,
+ "OAuth-Client-Secret", "", 40,
+ offsetof(struct pg_conn, oauth_client_secret)},
+
+ {"oauth_scope", NULL, NULL, NULL,
+ "OAuth-Scope", "", 15,
+ offsetof(struct pg_conn, oauth_scope)},
+
/* Terminating entry --- MUST BE LAST */
{NULL, NULL, NULL, NULL,
NULL, NULL, 0}
@@ -618,6 +635,7 @@ pqDropServerData(PGconn *conn)
conn->write_err_msg = NULL;
conn->be_pid = 0;
conn->be_key = 0;
+ /* conn->oauth_want_retry = false; TODO */
}
@@ -2536,6 +2554,7 @@ PQconnectPoll(PGconn *conn)
case CONNECTION_NEEDED:
case CONNECTION_GSS_STARTUP:
case CONNECTION_CHECK_TARGET:
+ case CONNECTION_AUTHENTICATING:
break;
default:
@@ -3517,6 +3536,7 @@ keep_going: /* We will come back to here until there is
int avail;
AuthRequest areq;
int res;
+ bool async;
/*
* Scan the message from current point (note that if we find
@@ -3672,6 +3692,16 @@ keep_going: /* We will come back to here until there is
/* Check to see if we should mention pgpassfile */
pgpassfileWarning(conn);
+#ifdef USE_OAUTH
+ if (conn->sasl == &pg_oauth_mech
+ && conn->oauth_want_retry)
+ {
+ /* TODO: only allow retry once */
+ need_new_connection = true;
+ goto keep_going;
+ }
+#endif
+
#ifdef ENABLE_GSS
/*
@@ -3753,7 +3783,17 @@ keep_going: /* We will come back to here until there is
* Note that conn->pghost must be non-NULL if we are going to
* avoid the Kerberos code doing a hostname look-up.
*/
- res = pg_fe_sendauth(areq, msgLength, conn);
+ res = pg_fe_sendauth(areq, msgLength, conn, &async);
+
+ if (async && (res == STATUS_OK))
+ {
+ /*
+ * We'll come back later once we're ready to respond.
+ * Don't consume the request yet.
+ */
+ conn->status = CONNECTION_AUTHENTICATING;
+ goto keep_going;
+ }
/* OK, we have processed the message; mark data consumed */
conn->inStart = conn->inCursor;
@@ -3786,6 +3826,41 @@ keep_going: /* We will come back to here until there is
goto keep_going;
}
+ case CONNECTION_AUTHENTICATING:
+ {
+ PostgresPollingStatusType status;
+ pgsocket altsock = PGINVALID_SOCKET;
+
+ if (!conn->async_auth)
+ {
+ /* programmer error; should not happen */
+ libpq_append_conn_error(conn, "async authentication has no handler");
+ goto error_return;
+ }
+
+ status = conn->async_auth(conn, &altsock);
+
+ if (status == PGRES_POLLING_FAILED)
+ goto error_return;
+
+ if (status == PGRES_POLLING_OK)
+ {
+ /*
+ * Reenter the authentication exchange with the server. We
+ * didn't consume the message that started external
+ * authentication, so it'll be reprocessed as if we just
+ * received it.
+ */
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ conn->altsock = PGINVALID_SOCKET; /* TODO: what frees
+ * this? */
+ goto keep_going;
+ }
+
+ conn->altsock = altsock;
+ return status;
+ }
+
case CONNECTION_AUTH_OK:
{
/*
@@ -4285,6 +4360,7 @@ pqMakeEmptyPGconn(void)
conn->verbosity = PQERRORS_DEFAULT;
conn->show_context = PQSHOW_CONTEXT_ERRORS;
conn->sock = PGINVALID_SOCKET;
+ conn->altsock = PGINVALID_SOCKET;
conn->Pfdebug = NULL;
/*
@@ -4400,6 +4476,11 @@ freePGconn(PGconn *conn)
free(conn->rowBuf);
free(conn->target_session_attrs);
free(conn->load_balance_hosts);
+ free(conn->oauth_issuer);
+ free(conn->oauth_discovery_uri);
+ free(conn->oauth_client_id);
+ free(conn->oauth_client_secret);
+ free(conn->oauth_scope);
termPQExpBuffer(&conn->errorMessage);
termPQExpBuffer(&conn->workBuffer);
@@ -6868,6 +6949,8 @@ PQsocket(const PGconn *conn)
{
if (!conn)
return -1;
+ if (conn->altsock != PGINVALID_SOCKET)
+ return conn->altsock;
return (conn->sock != PGINVALID_SOCKET) ? conn->sock : -1;
}
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index f2fc78a481..663b1c1acf 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -1039,10 +1039,13 @@ static int
pqSocketCheck(PGconn *conn, int forRead, int forWrite, time_t end_time)
{
int result;
+ pgsocket sock;
if (!conn)
return -1;
- if (conn->sock == PGINVALID_SOCKET)
+
+ sock = (conn->altsock != PGINVALID_SOCKET) ? conn->altsock : conn->sock;
+ if (sock == PGINVALID_SOCKET)
{
libpq_append_conn_error(conn, "invalid socket");
return -1;
@@ -1059,7 +1062,7 @@ pqSocketCheck(PGconn *conn, int forRead, int forWrite, time_t end_time)
/* We will retry as long as we get EINTR */
do
- result = pqSocketPoll(conn->sock, forRead, forWrite, end_time);
+ result = pqSocketPoll(sock, forRead, forWrite, end_time);
while (result < 0 && SOCK_ERRNO == EINTR);
if (result < 0)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index defc415fa3..d095351c66 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -38,6 +38,8 @@ extern "C"
#define LIBPQ_HAS_TRACE_FLAGS 1
/* Indicates that PQsslAttribute(NULL, "library") is useful */
#define LIBPQ_HAS_SSL_LIBRARY_DETECTION 1
+/* Indicates presence of the PQAUTHDATA_PROMPT_OAUTH_DEVICE authdata hook */
+#define LIBPQ_HAS_PROMPT_OAUTH_DEVICE 1
/*
* Option flags for PQcopyResult
@@ -78,7 +80,9 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_AUTHENTICATING, /* Authentication is in progress with some
+ * external system. */
} ConnStatusType;
typedef enum
@@ -160,6 +164,13 @@ typedef enum
PQ_PIPELINE_ABORTED
} PGpipelineStatus;
+typedef enum
+{
+ PQAUTHDATA_PROMPT_OAUTH_DEVICE, /* user must visit a device-authorization
+ * URL */
+ PQAUTHDATA_OAUTH_BEARER_TOKEN, /* server requests an OAuth Bearer token */
+} PGAuthData;
+
/* PGconn encapsulates a connection to the backend.
* The contents of this struct are not supposed to be known to applications.
*/
@@ -658,10 +669,74 @@ extern int PQenv2encoding(void);
/* === in fe-auth.c === */
+typedef struct _PQpromptOAuthDevice
+{
+ const char *verification_uri; /* verification URI to visit */
+ const char *user_code; /* user code to enter */
+} PQpromptOAuthDevice;
+
+typedef struct _PQoauthBearerRequest
+{
+ /* Hook inputs (constant across all calls) */
+ const char *const openid_configuration; /* OIDC discovery URI */
+ const char *const scope; /* required scope(s), or NULL */
+
+ /* Hook outputs */
+
+ /*---------
+ * Callback implementing a custom asynchronous OAuth flow.
+ *
+ * The callback may return
+ * - PGRES_POLLING_READING/WRITING, to indicate that a file descriptor
+ * has been stored in *altsock and libpq should wait until it is
+ * readable or writable before calling back;
+ * - PGRES_POLLING_OK, to indicate that the flow is complete and
+ * request->token has been set; or
+ * - PGRES_POLLING_FAILED, to indicate that token retrieval has failed.
+ *
+ * This callback is optional. If the token can be obtained without
+ * blocking during the original call to the PQAUTHDATA_OAUTH_BEARER_TOKEN
+ * hook, it may be returned directly, but one of request->async or
+ * request->token must be set by the hook.
+ */
+ PostgresPollingStatusType (*async) (PGconn *conn,
+ struct _PQoauthBearerRequest *request,
+ int *altsock);
+
+ /*
+ * Callback to clean up custom allocations. A hook implementation may use
+ * this to free request->token and any resources in request->user.
+ *
+ * This is technically optional, but highly recommended, because there is
+ * no other indication as to when it is safe to free the token.
+ */
+ void (*cleanup) (PGconn *conn, struct _PQoauthBearerRequest *request);
+
+ /*
+ * The hook should set this to the Bearer token contents for the
+ * connection, once the flow is completed. The token contents must remain
+ * available to libpq until the hook's cleanup callback is called.
+ */
+ char *token;
+
+ /*
+ * Hook-defined data. libpq will not modify this pointer across calls to
+ * the async callback, so it can be used to keep track of
+ * application-specific state. Resources allocated here should be freed by
+ * the cleanup callback.
+ */
+ void *user;
+} PQoauthBearerRequest;
+
extern char *PQencryptPassword(const char *passwd, const char *user);
extern char *PQencryptPasswordConn(PGconn *conn, const char *passwd, const char *user, const char *algorithm);
extern PGresult *PQchangePassword(PGconn *conn, const char *user, const char *passwd);
+typedef int (*PQauthDataHook_type) (PGAuthData type, PGconn *conn, void *data);
+extern void PQsetAuthDataHook(PQauthDataHook_type hook);
+extern PQauthDataHook_type PQgetAuthDataHook(void);
+extern int PQdefaultAuthDataHook(PGAuthData type, PGconn *conn, void *data);
+
/* === in encnames.c === */
extern int pg_char_to_encoding(const char *name);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 82c18f870d..cf26c693e3 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -351,6 +351,8 @@ typedef struct pg_conn_host
* found in password file. */
} pg_conn_host;
+typedef PostgresPollingStatusType (*AsyncAuthFunc) (PGconn *conn, pgsocket *altsock);
+
/*
* PGconn stores all the state data associated with a single connection
* to a backend.
@@ -409,6 +411,15 @@ struct pg_conn
char *require_auth; /* name of the expected auth method */
char *load_balance_hosts; /* load balance over hosts */
+ /* OAuth v2 */
+ char *oauth_issuer; /* token issuer URL */
+ char *oauth_discovery_uri; /* URI of the issuer's discovery
+ * document */
+ char *oauth_client_id; /* client identifier */
+ char *oauth_client_secret; /* client secret */
+ char *oauth_scope; /* access token scope */
+ bool oauth_want_retry; /* should we retry on failure? */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -477,6 +488,9 @@ struct pg_conn
bool client_finished_auth; /* have we finished our half of the
* authentication exchange? */
+ AsyncAuthFunc async_auth; /* callback for external async authentication */
+ pgsocket altsock; /* alternative socket for client to poll */
+
/* Transient state needed while establishing connection */
PGTargetServerType target_server_type; /* desired session properties */
diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build
index be6fadaea2..0d4b7ac17d 100644
--- a/src/interfaces/libpq/meson.build
+++ b/src/interfaces/libpq/meson.build
@@ -40,6 +40,13 @@ if gssapi.found()
)
endif
+if oauth.found()
+ libpq_sources += files('fe-auth-oauth.c')
+ if oauth_library == 'curl'
+ libpq_sources += files('fe-auth-oauth-curl.c')
+ endif
+endif
+
export_file = custom_target('libpq.exports',
kwargs: gen_export_kwargs,
)
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index b0f4178b3d..f803c1200b 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -231,6 +231,7 @@ pgxs_deps = {
'llvm': llvm,
'lz4': lz4,
'nls': libintl,
+ 'oauth': oauth,
'pam': pam,
'perl': perl_dep,
'python': python3_dep,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 2dfd8176a3..c45aae6f9e 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -153,6 +153,7 @@ ArrayMetaState
ArraySubWorkspace
ArrayToken
ArrayType
+AsyncAuthFunc
AsyncQueueControl
AsyncQueueEntry
AsyncRequest
@@ -354,6 +355,8 @@ CState
CTECycleClause
CTEMaterialize
CTESearchClause
+CURL
+CURLM
CV
CachedExpression
CachedPlan
@@ -1653,6 +1656,7 @@ NumericDigit
NumericSortSupport
NumericSumAccum
NumericVar
+OAuthStep
OM_uint32
OP
OSAPerGroupState
@@ -1718,6 +1722,7 @@ PFN
PGAlignedBlock
PGAlignedXLogBlock
PGAsyncStatusType
+PGAuthData
PGCALL2
PGChecksummablePage
PGContextVisibility
@@ -1877,11 +1882,14 @@ PQArgBlock
PQEnvironmentOption
PQExpBuffer
PQExpBufferData
+PQauthDataHook_type
PQcommMethods
PQconninfoOption
PQnoticeProcessor
PQnoticeReceiver
+PQoauthBearerRequest
PQprintOpt
+PQpromptOAuthDevice
PQsslKeyPassHook_OpenSSL_type
PREDICATELOCK
PREDICATELOCKTAG
@@ -3343,6 +3351,8 @@ explain_get_index_name_hook_type
f_smgr
fasthash_state
fd_set
+fe_oauth_state
+fe_oauth_state_enum
fe_scram_state
fe_scram_state_enum
fetch_range_request
--
2.34.1
v19-0001-common-jsonapi-support-FRONTEND-clients.patch (application/octet-stream, 20.4K) download
From ce06c03e2b5c50675d0bd8961c858bd0165cfabd Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Mon, 3 May 2021 15:38:26 -0700
Subject: [PATCH v19 1/9] common/jsonapi: support FRONTEND clients
Based on a patch by Michael Paquier.
For frontend code, use PQExpBuffer instead of StringInfo. This requires
us to track allocation failures so that we can return JSON_OUT_OF_MEMORY
as needed. json_errdetail() now allocates its error message inside
memory owned by the JsonLexContext, so clients don't need to worry about
freeing it.
We can now partially revert b44669b2ca, now that json_errdetail() works
correctly.
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
---
src/bin/pg_verifybackup/t/005_bad_manifest.pl | 3 +-
src/common/Makefile | 2 +-
src/common/jsonapi.c | 268 +++++++++++++-----
src/common/meson.build | 8 +-
src/common/parse_manifest.c | 2 +-
src/common/stringinfo.c | 7 +
src/include/common/jsonapi.h | 18 +-
src/include/lib/stringinfo.h | 2 +
8 files changed, 225 insertions(+), 85 deletions(-)
diff --git a/src/bin/pg_verifybackup/t/005_bad_manifest.pl b/src/bin/pg_verifybackup/t/005_bad_manifest.pl
index e278ccea5a..e2a297930e 100644
--- a/src/bin/pg_verifybackup/t/005_bad_manifest.pl
+++ b/src/bin/pg_verifybackup/t/005_bad_manifest.pl
@@ -13,7 +13,8 @@ use Test::More;
my $tempdir = PostgreSQL::Test::Utils::tempdir;
test_bad_manifest('input string ended unexpectedly',
- qr/could not parse backup manifest: parsing failed/, <<EOM);
+ qr/could not parse backup manifest: The input string ended unexpectedly/,
+ <<EOM);
{
EOM
diff --git a/src/common/Makefile b/src/common/Makefile
index 2ba5069dca..bbb5c3ab11 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -40,7 +40,7 @@ override CPPFLAGS += -DVAL_LDFLAGS_EX="\"$(LDFLAGS_EX)\""
override CPPFLAGS += -DVAL_LDFLAGS_SL="\"$(LDFLAGS_SL)\""
override CPPFLAGS += -DVAL_LIBS="\"$(LIBS)\""
-override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common $(CPPFLAGS)
+override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common -I$(libpq_srcdir) $(CPPFLAGS)
LIBS += $(PTHREAD_LIBS)
OBJS_COMMON = \
diff --git a/src/common/jsonapi.c b/src/common/jsonapi.c
index 32931ded82..2d1f30353a 100644
--- a/src/common/jsonapi.c
+++ b/src/common/jsonapi.c
@@ -21,10 +21,43 @@
#include "mb/pg_wchar.h"
#include "port/pg_lfind.h"
-#ifndef FRONTEND
+#ifdef FRONTEND
+#include "pqexpbuffer.h"
+#else
+#include "lib/stringinfo.h"
#include "miscadmin.h"
#endif
+/*
+ * In backend, we will use palloc/pfree along with StringInfo. In frontend, use
+ * malloc and PQExpBuffer, and return JSON_OUT_OF_MEMORY on out-of-memory.
+ */
+#ifdef FRONTEND
+
+#define STRDUP(s) strdup(s)
+#define ALLOC(size) malloc(size)
+
+#define appendStrVal appendPQExpBuffer
+#define appendBinaryStrVal appendBinaryPQExpBuffer
+#define appendStrValChar appendPQExpBufferChar
+#define createStrVal createPQExpBuffer
+#define resetStrVal resetPQExpBuffer
+#define destroyStrVal destroyPQExpBuffer
+
+#else /* !FRONTEND */
+
+#define STRDUP(s) pstrdup(s)
+#define ALLOC(size) palloc(size)
+
+#define appendStrVal appendStringInfo
+#define appendBinaryStrVal appendBinaryStringInfo
+#define appendStrValChar appendStringInfoChar
+#define createStrVal makeStringInfo
+#define resetStrVal resetStringInfo
+#define destroyStrVal destroyStringInfo
+
+#endif
+
/*
* The context of the parser is maintained by the recursive descent
* mechanism, but is passed explicitly to the error reporting routine
@@ -167,9 +200,16 @@ makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
lex->input_encoding = encoding;
if (need_escapes)
{
- lex->strval = makeStringInfo();
+ /*
+ * This call can fail in FRONTEND code. We defer error handling to
+ * time of use (json_lex_string()) since we might not need to parse
+ * any strings anyway.
+ */
+ lex->strval = createStrVal();
lex->flags |= JSONLEX_FREE_STRVAL;
+ lex->parse_strval = true;
}
+ lex->errormsg = NULL;
return lex;
}
@@ -182,13 +222,18 @@ makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
void
freeJsonLexContext(JsonLexContext *lex)
{
+ static const JsonLexContext empty = {0};
+
if (lex->flags & JSONLEX_FREE_STRVAL)
- {
- pfree(lex->strval->data);
- pfree(lex->strval);
- }
+ destroyStrVal(lex->strval);
+
+ if (lex->errormsg)
+ destroyStrVal(lex->errormsg);
+
if (lex->flags & JSONLEX_FREE_STRUCT)
pfree(lex);
+ else
+ *lex = empty;
}
/*
@@ -254,7 +299,7 @@ json_count_array_elements(JsonLexContext *lex, int *elements)
* etc, so doing this with a copy makes that safe.
*/
memcpy(©lex, lex, sizeof(JsonLexContext));
- copylex.strval = NULL; /* not interested in values here */
+ copylex.parse_strval = false; /* not interested in values here */
copylex.lex_level++;
count = 0;
@@ -316,14 +361,21 @@ parse_scalar(JsonLexContext *lex, JsonSemAction *sem)
/* extract the de-escaped string value, or the raw lexeme */
if (lex_peek(lex) == JSON_TOKEN_STRING)
{
- if (lex->strval != NULL)
- val = pstrdup(lex->strval->data);
+ if (lex->parse_strval)
+ {
+ val = STRDUP(lex->strval->data);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
}
else
{
int len = (lex->token_terminator - lex->token_start);
- val = palloc(len + 1);
+ val = ALLOC(len + 1);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+
memcpy(val, lex->token_start, len);
val[len] = '\0';
}
@@ -357,8 +409,12 @@ parse_object_field(JsonLexContext *lex, JsonSemAction *sem)
if (lex_peek(lex) != JSON_TOKEN_STRING)
return report_parse_error(JSON_PARSE_STRING, lex);
- if ((ostart != NULL || oend != NULL) && lex->strval != NULL)
- fname = pstrdup(lex->strval->data);
+ if ((ostart != NULL || oend != NULL) && lex->parse_strval)
+ {
+ fname = STRDUP(lex->strval->data);
+ if (fname == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
result = json_lex(lex);
if (result != JSON_SUCCESS)
return result;
@@ -414,6 +470,11 @@ parse_object(JsonLexContext *lex, JsonSemAction *sem)
JsonParseErrorType result;
#ifndef FRONTEND
+
+ /*
+ * TODO: clients need some way to put a bound on stack growth. Parse level
+ * limits maybe?
+ */
check_stack_depth();
#endif
@@ -762,8 +823,15 @@ json_lex_string(JsonLexContext *lex)
return code; \
} while (0)
- if (lex->strval != NULL)
- resetStringInfo(lex->strval);
+ if (lex->parse_strval)
+ {
+#ifdef FRONTEND
+ /* make sure initialization succeeded */
+ if (lex->strval == NULL)
+ return JSON_OUT_OF_MEMORY;
+#endif
+ resetStrVal(lex->strval);
+ }
Assert(lex->input_length > 0);
s = lex->token_start;
@@ -800,7 +868,7 @@ json_lex_string(JsonLexContext *lex)
else
FAIL_AT_CHAR_END(JSON_UNICODE_ESCAPE_FORMAT);
}
- if (lex->strval != NULL)
+ if (lex->parse_strval)
{
/*
* Combine surrogate pairs.
@@ -857,19 +925,19 @@ json_lex_string(JsonLexContext *lex)
unicode_to_utf8(ch, (unsigned char *) utf8str);
utf8len = pg_utf_mblen((unsigned char *) utf8str);
- appendBinaryStringInfo(lex->strval, utf8str, utf8len);
+ appendBinaryPQExpBuffer(lex->strval, utf8str, utf8len);
}
else if (ch <= 0x007f)
{
/* The ASCII range is the same in all encodings */
- appendStringInfoChar(lex->strval, (char) ch);
+ appendPQExpBufferChar(lex->strval, (char) ch);
}
else
FAIL_AT_CHAR_END(JSON_UNICODE_HIGH_ESCAPE);
#endif /* FRONTEND */
}
}
- else if (lex->strval != NULL)
+ else if (lex->parse_strval)
{
if (hi_surrogate != -1)
FAIL_AT_CHAR_END(JSON_UNICODE_LOW_SURROGATE);
@@ -879,22 +947,22 @@ json_lex_string(JsonLexContext *lex)
case '"':
case '\\':
case '/':
- appendStringInfoChar(lex->strval, *s);
+ appendStrValChar(lex->strval, *s);
break;
case 'b':
- appendStringInfoChar(lex->strval, '\b');
+ appendStrValChar(lex->strval, '\b');
break;
case 'f':
- appendStringInfoChar(lex->strval, '\f');
+ appendStrValChar(lex->strval, '\f');
break;
case 'n':
- appendStringInfoChar(lex->strval, '\n');
+ appendStrValChar(lex->strval, '\n');
break;
case 'r':
- appendStringInfoChar(lex->strval, '\r');
+ appendStrValChar(lex->strval, '\r');
break;
case 't':
- appendStringInfoChar(lex->strval, '\t');
+ appendStrValChar(lex->strval, '\t');
break;
default:
@@ -929,7 +997,7 @@ json_lex_string(JsonLexContext *lex)
/*
* Skip to the first byte that requires special handling, so we
- * can batch calls to appendBinaryStringInfo.
+ * can batch calls to appendBinaryStrVal.
*/
while (p < end - sizeof(Vector8) &&
!pg_lfind8('\\', (uint8 *) p, sizeof(Vector8)) &&
@@ -953,8 +1021,8 @@ json_lex_string(JsonLexContext *lex)
}
}
- if (lex->strval != NULL)
- appendBinaryStringInfo(lex->strval, s, p - s);
+ if (lex->parse_strval)
+ appendBinaryStrVal(lex->strval, s, p - s);
/*
* s will be incremented at the top of the loop, so set it to just
@@ -970,6 +1038,11 @@ json_lex_string(JsonLexContext *lex)
return JSON_UNICODE_LOW_SURROGATE;
}
+#ifdef FRONTEND
+ if (lex->parse_strval && PQExpBufferBroken(lex->strval))
+ return JSON_OUT_OF_MEMORY;
+#endif
+
/* Hooray, we found the end of the string! */
lex->prev_token_terminator = lex->token_terminator;
lex->token_terminator = s + 1;
@@ -1145,72 +1218,93 @@ report_parse_error(JsonParseContext ctx, JsonLexContext *lex)
return JSON_SUCCESS; /* silence stupider compilers */
}
-
-#ifndef FRONTEND
-/*
- * Extract the current token from a lexing context, for error reporting.
- */
-static char *
-extract_token(JsonLexContext *lex)
-{
- int toklen = lex->token_terminator - lex->token_start;
- char *token = palloc(toklen + 1);
-
- memcpy(token, lex->token_start, toklen);
- token[toklen] = '\0';
- return token;
-}
-
/*
* Construct an (already translated) detail message for a JSON error.
*
- * Note that the error message generated by this routine may not be
- * palloc'd, making it unsafe for frontend code as there is no way to
- * know if this can be safely pfree'd or not.
+ * The returned allocation is either static or owned by the JsonLexContext and
+ * should not be freed.
*/
char *
json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
{
+ int toklen = lex->token_terminator - lex->token_start;
+
+ if (error == JSON_OUT_OF_MEMORY)
+ {
+ /* Short circuit. Allocating anything for this case is unhelpful. */
+ return _("out of memory");
+ }
+
+ if (lex->errormsg)
+ resetStrVal(lex->errormsg);
+ else
+ lex->errormsg = createStrVal();
+
switch (error)
{
case JSON_SUCCESS:
/* fall through to the error code after switch */
break;
case JSON_ESCAPING_INVALID:
- return psprintf(_("Escape sequence \"\\%s\" is invalid."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Escape sequence \"\\%.*s\" is invalid."),
+ toklen, lex->token_start);
+ break;
case JSON_ESCAPING_REQUIRED:
- return psprintf(_("Character with value 0x%02x must be escaped."),
- (unsigned char) *(lex->token_terminator));
+ appendStrVal(lex->errormsg,
+ _("Character with value 0x%02x must be escaped."),
+ (unsigned char) *(lex->token_terminator));
+ break;
case JSON_EXPECTED_END:
- return psprintf(_("Expected end of input, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected end of input, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_ARRAY_FIRST:
- return psprintf(_("Expected array element or \"]\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected array element or \"]\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_ARRAY_NEXT:
- return psprintf(_("Expected \",\" or \"]\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \",\" or \"]\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_COLON:
- return psprintf(_("Expected \":\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \":\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_JSON:
- return psprintf(_("Expected JSON value, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected JSON value, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_MORE:
return _("The input string ended unexpectedly.");
case JSON_EXPECTED_OBJECT_FIRST:
- return psprintf(_("Expected string or \"}\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected string or \"}\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_OBJECT_NEXT:
- return psprintf(_("Expected \",\" or \"}\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \",\" or \"}\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_STRING:
- return psprintf(_("Expected string, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected string, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_INVALID_TOKEN:
- return psprintf(_("Token \"%s\" is invalid."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Token \"%.*s\" is invalid."),
+ toklen, lex->token_start);
+ break;
+ case JSON_OUT_OF_MEMORY:
+ /* should have been handled above; use the error path */
+ break;
case JSON_UNICODE_CODE_POINT_ZERO:
return _("\\u0000 cannot be converted to text.");
case JSON_UNICODE_ESCAPE_FORMAT:
@@ -1219,9 +1313,19 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
/* note: this case is only reachable in frontend not backend */
return _("Unicode escape values cannot be used for code point values above 007F when the encoding is not UTF8.");
case JSON_UNICODE_UNTRANSLATABLE:
- /* note: this case is only reachable in backend not frontend */
+
+ /*
+ * note: this case is only reachable in backend not frontend.
+ * #ifdef it away so the frontend doesn't try to link against
+ * backend functionality.
+ */
+#ifndef FRONTEND
return psprintf(_("Unicode escape value could not be translated to the server's encoding %s."),
GetDatabaseEncodingName());
+#else
+ Assert(false);
+ break;
+#endif
case JSON_UNICODE_HIGH_SURROGATE:
return _("Unicode high surrogate must not follow a high surrogate.");
case JSON_UNICODE_LOW_SURROGATE:
@@ -1231,12 +1335,22 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
break;
}
- /*
- * We don't use a default: case, so that the compiler will warn about
- * unhandled enum values. But this needs to be here anyway to cover the
- * possibility of an incorrect input.
- */
- elog(ERROR, "unexpected json parse error type: %d", (int) error);
- return NULL;
-}
+ /* Note that lex->errormsg can be NULL in FRONTEND code. */
+ if (lex->errormsg && !lex->errormsg->data[0])
+ {
+ /*
+ * We don't use a default: case, so that the compiler will warn about
+ * unhandled enum values. But this needs to be here anyway to cover
+ * the possibility of an incorrect input.
+ */
+ appendStrVal(lex->errormsg,
+ "unexpected json parse error type: %d", (int) error);
+ }
+
+#ifdef FRONTEND
+ if (PQExpBufferBroken(lex->errormsg))
+ return _("out of memory while constructing error description");
#endif
+
+ return lex->errormsg->data;
+}
diff --git a/src/common/meson.build b/src/common/meson.build
index 4eb16024cb..5d2c7abaa6 100644
--- a/src/common/meson.build
+++ b/src/common/meson.build
@@ -124,13 +124,18 @@ common_sources_frontend_static += files(
# least cryptohash_openssl.c, hmac_openssl.c depend on it.
# controldata_utils.c depends on wait_event_types_h. That's arguably a
# layering violation, but ...
+#
+# XXX Frontend builds need libpq's pqexpbuffer.h, so adjust the include paths
+# appropriately. This seems completely broken.
pgcommon = {}
pgcommon_variants = {
'_srv': internal_lib_args + {
+ 'include_directories': include_directories('.'),
'sources': common_sources + [lwlocknames_h] + [wait_event_types_h],
'dependencies': [backend_common_code],
},
'': default_lib_args + {
+ 'include_directories': include_directories('../interfaces/libpq', '.'),
'sources': common_sources_frontend_static,
'dependencies': [frontend_common_code],
# Files in libpgcommon.a should use/export the "xxx_private" versions
@@ -139,6 +144,7 @@ pgcommon_variants = {
},
'_shlib': default_lib_args + {
'pic': true,
+ 'include_directories': include_directories('../interfaces/libpq', '.'),
'sources': common_sources_frontend_shlib,
'dependencies': [frontend_common_code],
},
@@ -156,7 +162,6 @@ foreach name, opts : pgcommon_variants
c_args = opts.get('c_args', []) + common_cflags[cflagname]
cflag_libs += static_library('libpgcommon@0@_@1@'.format(name, cflagname),
c_pch: pch_c_h,
- include_directories: include_directories('.'),
kwargs: opts + {
'sources': sources,
'c_args': c_args,
@@ -169,7 +174,6 @@ foreach name, opts : pgcommon_variants
lib = static_library('libpgcommon@0@'.format(name),
link_with: cflag_libs,
c_pch: pch_c_h,
- include_directories: include_directories('.'),
kwargs: opts + {
'dependencies': opts['dependencies'] + [ssl],
}
diff --git a/src/common/parse_manifest.c b/src/common/parse_manifest.c
index 92a97714f3..62d93989be 100644
--- a/src/common/parse_manifest.c
+++ b/src/common/parse_manifest.c
@@ -147,7 +147,7 @@ json_parse_manifest(JsonManifestParseContext *context, char *buffer,
/* Run the actual JSON parser. */
json_error = pg_parse_json(lex, &sem);
if (json_error != JSON_SUCCESS)
- json_manifest_parse_failure(context, "parsing failed");
+ json_manifest_parse_failure(context, json_errdetail(json_error, lex));
if (parse.state != JM_EXPECT_EOF)
json_manifest_parse_failure(context, "manifest ended unexpectedly");
diff --git a/src/common/stringinfo.c b/src/common/stringinfo.c
index c61d5c58f3..09419f6042 100644
--- a/src/common/stringinfo.c
+++ b/src/common/stringinfo.c
@@ -350,3 +350,10 @@ enlargeStringInfo(StringInfo str, int needed)
str->maxlen = newlen;
}
+
+void
+destroyStringInfo(StringInfo str)
+{
+ pfree(str->data);
+ pfree(str);
+}
diff --git a/src/include/common/jsonapi.h b/src/include/common/jsonapi.h
index 02943cdad8..75d444c17a 100644
--- a/src/include/common/jsonapi.h
+++ b/src/include/common/jsonapi.h
@@ -14,8 +14,6 @@
#ifndef JSONAPI_H
#define JSONAPI_H
-#include "lib/stringinfo.h"
-
typedef enum JsonTokenType
{
JSON_TOKEN_INVALID,
@@ -48,6 +46,7 @@ typedef enum JsonParseErrorType
JSON_EXPECTED_OBJECT_NEXT,
JSON_EXPECTED_STRING,
JSON_INVALID_TOKEN,
+ JSON_OUT_OF_MEMORY,
JSON_UNICODE_CODE_POINT_ZERO,
JSON_UNICODE_ESCAPE_FORMAT,
JSON_UNICODE_HIGH_ESCAPE,
@@ -57,6 +56,17 @@ typedef enum JsonParseErrorType
JSON_SEM_ACTION_FAILED, /* error should already be reported */
} JsonParseErrorType;
+/*
+ * Don't depend on the internal type header for strval; if callers need access
+ * then they can include the appropriate header themselves.
+ */
+#ifdef FRONTEND
+#define StrValType PQExpBufferData
+#else
+#define StrValType StringInfoData
+#endif
+
+typedef struct StrValType StrValType;
/*
* All the fields in this structure should be treated as read-only.
@@ -88,7 +98,9 @@ typedef struct JsonLexContext
bits32 flags;
int line_number; /* line number, starting from 1 */
char *line_start; /* where that line starts within input */
- StringInfo strval;
+ bool parse_strval;
+ StrValType *strval; /* only used if parse_strval == true */
+ StrValType *errormsg;
} JsonLexContext;
typedef JsonParseErrorType (*json_struct_action) (void *state);
diff --git a/src/include/lib/stringinfo.h b/src/include/lib/stringinfo.h
index 2cd636b01c..64ec6419af 100644
--- a/src/include/lib/stringinfo.h
+++ b/src/include/lib/stringinfo.h
@@ -233,4 +233,6 @@ extern void appendBinaryStringInfoNT(StringInfo str,
*/
extern void enlargeStringInfo(StringInfo str, int needed);
+
+extern void destroyStringInfo(StringInfo str);
#endif /* STRINGINFO_H */
--
2.34.1
v19-0003-Explicitly-require-password-for-SCRAM-exchange.patch (application/octet-stream, 3.2K) download
From 783bfe0b953bf9c6591f1aa5cb747f5c0f96a8d9 Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <dgustafsson@postgresql.org>
Date: Fri, 23 Feb 2024 11:19:55 +0100
Subject: [PATCH v19 3/9] Explicitly require password for SCRAM exchange
This refactors the SASL init flow to set password_needed on the two
SCRAM exchanges currently supported. The code already required this
but was set up in such a way that all SASL exchanges required using
a password, a restriction which may not hold for all exchanges (the
example at hand being the proposed OAuthbearer exchange).
This was extracted from a larger patchset to introduce OAuthBearer
authentication and authorization.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://postgr.es/m/d1b467a78e0e36ed85a09adf979d04cf124a9d4b.camel@vmware.com
---
src/interfaces/libpq/fe-auth.c | 28 +++++++++++++++-------------
1 file changed, 15 insertions(+), 13 deletions(-)
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index cf8af4c62e..81ec08485d 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -425,7 +425,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
int initialresponselen;
const char *selected_mechanism;
PQExpBufferData mechanism_buf;
- char *password;
+ char *password = NULL;
SASLStatus status;
initPQExpBuffer(&mechanism_buf);
@@ -446,8 +446,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
/*
* Parse the list of SASL authentication mechanisms in the
* AuthenticationSASL message, and select the best mechanism that we
- * support. SCRAM-SHA-256-PLUS and SCRAM-SHA-256 are the only ones
- * supported at the moment, listed by order of decreasing importance.
+ * support. Mechanisms are listed by order of decreasing importance.
*/
selected_mechanism = NULL;
for (;;)
@@ -487,6 +486,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
selected_mechanism = SCRAM_SHA_256_PLUS_NAME;
conn->sasl = &pg_scram_mech;
+ conn->password_needed = true;
}
#else
/*
@@ -522,6 +522,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
selected_mechanism = SCRAM_SHA_256_NAME;
conn->sasl = &pg_scram_mech;
+ conn->password_needed = true;
}
}
@@ -545,18 +546,19 @@ pg_SASL_init(PGconn *conn, int payloadlen)
/*
* First, select the password to use for the exchange, complaining if
- * there isn't one. Currently, all supported SASL mechanisms require a
- * password, so we can just go ahead here without further distinction.
+ * there isn't one and the selected SASL mechanism needs it.
*/
- conn->password_needed = true;
- password = conn->connhost[conn->whichhost].password;
- if (password == NULL)
- password = conn->pgpass;
- if (password == NULL || password[0] == '\0')
+ if (conn->password_needed)
{
- appendPQExpBufferStr(&conn->errorMessage,
- PQnoPasswordSupplied);
- goto error;
+ password = conn->connhost[conn->whichhost].password;
+ if (password == NULL)
+ password = conn->pgpass;
+ if (password == NULL || password[0] == '\0')
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ PQnoPasswordSupplied);
+ goto error;
+ }
}
Assert(conn->sasl);
--
2.34.1
v19-0005-backend-add-OAUTHBEARER-SASL-mechanism.patch (application/octet-stream, 39.9K) download
From 12ae7c4355bffd95741b7f7990067c963d593b93 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 4 May 2021 16:21:11 -0700
Subject: [PATCH v19 5/9] backend: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) on the server side. This adds a new
auth method, oauth, to pg_hba.
Because OAuth implementations vary so wildly, and bearer token
validation is heavily dependent on the issuing party, authn/z is done by
communicating with an external program: the oauth_validator_command.
This command must do the following:
1. Receive the bearer token by reading its contents from a file
descriptor passed from the server. (The numeric value of this
descriptor may be inserted into the oauth_validator_command using the
%f specifier.)
This MUST be the first action the command performs. The server will
not begin reading stdout from the command until the token has been
read in full, so if the command tries to print anything and hits a
buffer limit, the backend will deadlock and time out.
2. Validate the bearer token. The correct way to do this depends on the
issuer, but it generally involves either cryptographic operations to
prove that the token was issued by a trusted party, or the
presentation of the bearer token to some other party so that _it_ can
perform validation.
The command MUST maintain confidentiality of the bearer token, since
in most cases it can be used just like a password. (There are ways to
cryptographically bind tokens to client certificates, but they are
way beyond the scope of this commit message.)
If the token cannot be validated, the command must exit with a
non-zero status. Further authentication/authorization is pointless if
the bearer token wasn't issued by someone you trust.
3. Authenticate the user, authorize the user, or both:
a. To authenticate the user, use the bearer token to retrieve some
trusted identifier string for the end user. The exact process for
this is, again, issuer-dependent. The command should print the
authenticated identity string to stdout, followed by a newline.
If the user cannot be authenticated, the validator should not
print anything to stdout. It should also exit with a non-zero
status, unless the token may be used to authorize the connection
through some other means (see below).
On a success, the command may then exit with a zero success code.
By default, the server will then check to make sure the identity
string matches the role that is being used (or matches a usermap
entry, if one is in use).
b. To optionally authorize the user, in combination with the HBA
option trust_validator_authz=1 (see below), the validator simply
returns a zero exit code if the client should be allowed to
connect with its presented role (which can be passed to the
command using the %r specifier), or a non-zero code otherwise.
The hard part is in determining whether the given token truly
authorizes the client to use the given role, which must
unfortunately be left as an exercise to the reader.
This obviously requires some care, as a poorly implemented token
validator may silently open the entire database to anyone with a
bearer token. But it may be a more portable approach, since OAuth
is designed as an authorization framework, not an authentication
framework. For example, the user's bearer token could carry an
"allow_superuser_access" claim, which would authorize pseudonymous
database access as any role. It's then up to the OAuth system
administrators to ensure that allow_superuser_access is doled out
only to the proper users.
c. It's possible that the user can be successfully authenticated but
isn't authorized to connect. In this case, the command may print
the authenticated ID and then fail with a non-zero exit code.
(This makes it easier to see what's going on in the Postgres
logs.)
4. Token validators may optionally log to stderr. This will be printed
verbatim into the Postgres server logs.
The oauth method supports the following HBA options (but note that two
of them are not optional, since we have no way of choosing sensible
defaults):
issuer: Required. The URL of the OAuth issuing party, which the client
must contact to receive a bearer token.
Some real-world examples as of time of writing:
- https://accounts.google.com
- https://login.microsoft.com/[tenant-id]/v2.0
scope: Required. The OAuth scope(s) required for the server to
authenticate and/or authorize the user. This is heavily
deployment-specific, but a simple example is "openid email".
map: Optional. Specify a standard PostgreSQL user map; this works
the same as with other auth methods such as peer. If a map is
not specified, the user ID returned by the token validator
must exactly match the role that's being requested (but see
trust_validator_authz, below).
trust_validator_authz:
Optional. When set to 1, this allows the token validator to
take full control of the authorization process. Standard user
mapping is skipped: if the validator command succeeds, the
client is allowed to connect under its desired role and no
further checks are done.
Unlike the client, servers support OAuth without needing to be built
against libiddawc (since the responsibility for "speaking" OAuth/OIDC
correctly is delegated entirely to the oauth_validator_command).
Several TODOs:
- port to platforms other than "modern Linux/BSD"
- overhaul the communication with oauth_validator_command, which is
currently a bad hack on OpenPipeStream()
- implement more sanity checks on the OAUTHBEARER message format and
tokens sent by the client
- implement more helpful handling of HBA misconfigurations
- properly interpolate JSON when generating error responses
- use logdetail during auth failures
- deal with role names that can't be safely passed to system() without
shell-escaping
- allow passing the configured issuer to the oauth_validator_command, to
deal with multi-issuer setups
- ...and more.
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
---
src/backend/libpq/Makefile | 1 +
src/backend/libpq/auth-oauth.c | 810 ++++++++++++++++++++++++++++
src/backend/libpq/auth-sasl.c | 10 +-
src/backend/libpq/auth-scram.c | 4 +-
src/backend/libpq/auth.c | 26 +-
src/backend/libpq/hba.c | 31 +-
src/backend/libpq/meson.build | 1 +
src/backend/utils/misc/guc_tables.c | 12 +
src/include/libpq/auth.h | 17 +
src/include/libpq/hba.h | 6 +-
src/include/libpq/oauth.h | 24 +
src/include/libpq/sasl.h | 11 +
src/tools/pgindent/typedefs.list | 1 +
13 files changed, 922 insertions(+), 32 deletions(-)
create mode 100644 src/backend/libpq/auth-oauth.c
create mode 100644 src/include/libpq/oauth.h
diff --git a/src/backend/libpq/Makefile b/src/backend/libpq/Makefile
index 6d385fd6a4..98eb2a8242 100644
--- a/src/backend/libpq/Makefile
+++ b/src/backend/libpq/Makefile
@@ -15,6 +15,7 @@ include $(top_builddir)/src/Makefile.global
# be-fsstubs is here for historical reasons, probably belongs elsewhere
OBJS = \
+ auth-oauth.o \
auth-sasl.o \
auth-scram.o \
auth.o \
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
new file mode 100644
index 0000000000..a9d2646023
--- /dev/null
+++ b/src/backend/libpq/auth-oauth.c
@@ -0,0 +1,810 @@
+/*-------------------------------------------------------------------------
+ *
+ * auth-oauth.c
+ * Server-side implementation of the SASL OAUTHBEARER mechanism.
+ *
+ * See the following RFC for more details:
+ * - RFC 7628: https://tools.ietf.org/html/rfc7628
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/backend/libpq/auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <unistd.h>
+#include <fcntl.h>
+
+#include "common/oauth-common.h"
+#include "lib/stringinfo.h"
+#include "libpq/auth.h"
+#include "libpq/hba.h"
+#include "libpq/oauth.h"
+#include "libpq/sasl.h"
+#include "storage/fd.h"
+
+/* GUC */
+char *oauth_validator_command;
+
+static void oauth_get_mechanisms(Port *port, StringInfo buf);
+static void *oauth_init(Port *port, const char *selected_mech, const char *shadow_pass);
+static int oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, const char **logdetail);
+
+/* Mechanism declaration */
+const pg_be_sasl_mech pg_be_oauth_mech = {
+ oauth_get_mechanisms,
+ oauth_init,
+ oauth_exchange,
+
+ PG_MAX_AUTH_TOKEN_LENGTH,
+};
+
+
+typedef enum
+{
+ OAUTH_STATE_INIT = 0,
+ OAUTH_STATE_ERROR,
+ OAUTH_STATE_FINISHED,
+} oauth_state;
+
+struct oauth_ctx
+{
+ oauth_state state;
+ Port *port;
+ const char *issuer;
+ const char *scope;
+};
+
+static char *sanitize_char(char c);
+static char *parse_kvpairs_for_auth(char **input);
+static void generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen);
+static bool validate(Port *port, const char *auth, const char **logdetail);
+static bool run_validator_command(Port *port, const char *token);
+static bool check_exit(FILE **fh, const char *command);
+static bool set_cloexec(int fd);
+static bool username_ok_for_shell(const char *username);
+
+#define KVSEP 0x01
+#define AUTH_KEY "auth"
+#define BEARER_SCHEME "Bearer "
+
+static void
+oauth_get_mechanisms(Port *port, StringInfo buf)
+{
+ /* Only OAUTHBEARER is supported. */
+ appendStringInfoString(buf, OAUTHBEARER_NAME);
+ appendStringInfoChar(buf, '\0');
+}
+
+static void *
+oauth_init(Port *port, const char *selected_mech, const char *shadow_pass)
+{
+ struct oauth_ctx *ctx;
+
+ if (strcmp(selected_mech, OAUTHBEARER_NAME))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("client selected an invalid SASL authentication mechanism")));
+
+ ctx = palloc0(sizeof(*ctx));
+
+ ctx->state = OAUTH_STATE_INIT;
+ ctx->port = port;
+
+ Assert(port->hba);
+ ctx->issuer = port->hba->oauth_issuer;
+ ctx->scope = port->hba->oauth_scope;
+
+ return ctx;
+}
+
+static int
+oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, const char **logdetail)
+{
+ char *p;
+ char cbind_flag;
+ char *auth;
+
+ struct oauth_ctx *ctx = opaq;
+
+ *output = NULL;
+ *outputlen = -1;
+
+ /*
+ * If the client didn't include an "Initial Client Response" in the
+ * SASLInitialResponse message, send an empty challenge, to which the
+ * client will respond with the same data that usually comes in the
+ * Initial Client Response.
+ */
+ if (input == NULL)
+ {
+ Assert(ctx->state == OAUTH_STATE_INIT);
+
+ *output = pstrdup("");
+ *outputlen = 0;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ /*
+ * Check that the input length agrees with the string length of the input.
+ */
+ if (inputlen == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The message is empty.")));
+ if (inputlen != strlen(input))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message length does not match input length.")));
+
+ switch (ctx->state)
+ {
+ case OAUTH_STATE_INIT:
+ /* Handle this case below. */
+ break;
+
+ case OAUTH_STATE_ERROR:
+
+ /*
+ * Only one response is valid for the client during authentication
+ * failure: a single kvsep.
+ */
+ if (inputlen != 1 || *input != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Client did not send a kvsep response.")));
+
+ /* The (failed) handshake is now complete. */
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_FAILURE;
+
+ default:
+ elog(ERROR, "invalid OAUTHBEARER exchange state");
+ return PG_SASL_EXCHANGE_FAILURE;
+ }
+
+ /* Handle the client's initial message. */
+ p = pstrdup(input);
+
+ /*
+ * OAUTHBEARER does not currently define a channel binding (so there is no
+ * OAUTHBEARER-PLUS, and we do not accept a 'p' specifier). We accept a
+ * 'y' specifier purely for the remote chance that a future specification
+ * could define one; then future clients can still interoperate with this
+ * server implementation. 'n' is the expected case.
+ */
+ cbind_flag = *p;
+ switch (cbind_flag)
+ {
+ case 'p':
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The server does not support channel binding for OAuth, but the client message includes channel binding data.")));
+ break;
+
+ case 'y': /* fall through */
+ case 'n':
+ p++;
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Comma expected, but found character \"%s\".",
+ sanitize_char(*p))));
+ p++;
+ break;
+
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected channel-binding flag %s.",
+ sanitize_char(cbind_flag))));
+ }
+
+ /*
+ * Forbid optional authzid (authorization identity). We don't support it.
+ */
+ if (*p == 'a')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("client uses authorization identity, but it is not supported")));
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected attribute %s in client-first-message.",
+ sanitize_char(*p))));
+ p++;
+
+ /* All remaining fields are separated by the RFC's kvsep (\x01). */
+ if (*p != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Key-value separator expected, but found character %s.",
+ sanitize_char(*p))));
+ p++;
+
+ auth = parse_kvpairs_for_auth(&p);
+ if (!auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message does not contain an auth value.")));
+
+ /* We should be at the end of our message. */
+ if (*p)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains additional data after the final terminator.")));
+
+ if (!validate(ctx->port, auth, logdetail))
+ {
+ generate_error_response(ctx, output, outputlen);
+
+ ctx->state = OAUTH_STATE_ERROR;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_SUCCESS;
+}
+
+/*
+ * Convert an arbitrary byte to printable form. For error messages.
+ *
+ * If it's a printable ASCII character, print it as a single character.
+ * otherwise, print it in hex.
+ *
+ * The returned pointer points to a static buffer.
+ */
+static char *
+sanitize_char(char c)
+{
+ static char buf[5];
+
+ if (c >= 0x21 && c <= 0x7E)
+ snprintf(buf, sizeof(buf), "'%c'", c);
+ else
+ snprintf(buf, sizeof(buf), "0x%02x", (unsigned char) c);
+ return buf;
+}
+
+/*
+ * Consumes all kvpairs in an OAUTHBEARER exchange message. If the "auth" key is
+ * found, its value is returned.
+ */
+static char *
+parse_kvpairs_for_auth(char **input)
+{
+ char *pos = *input;
+ char *auth = NULL;
+
+ /*----
+ * The relevant ABNF, from Sec. 3.1:
+ *
+ * kvsep = %x01
+ * key = 1*(ALPHA)
+ * value = *(VCHAR / SP / HTAB / CR / LF )
+ * kvpair = key "=" value kvsep
+ * ;;gs2-header = See RFC 5801
+ * client-resp = (gs2-header kvsep *kvpair kvsep) / kvsep
+ *
+ * By the time we reach this code, the gs2-header and initial kvsep have
+ * already been validated. We start at the beginning of the first kvpair.
+ */
+
+ while (*pos)
+ {
+ char *end;
+ char *sep;
+ char *key;
+ char *value;
+
+ /*
+ * Find the end of this kvpair. Note that input is null-terminated by
+ * the SASL code, so the strchr() is bounded.
+ */
+ end = strchr(pos, KVSEP);
+ if (!end)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an unterminated key/value pair.")));
+ *end = '\0';
+
+ if (pos == end)
+ {
+ /* Empty kvpair, signifying the end of the list. */
+ *input = pos + 1;
+ return auth;
+ }
+
+ /*
+ * Find the end of the key name.
+ *
+ * TODO further validate the key/value grammar? empty keys, bad
+ * chars...
+ */
+ sep = strchr(pos, '=');
+ if (!sep)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains a key without a value.")));
+ *sep = '\0';
+
+ /* Both key and value are now safely terminated. */
+ key = pos;
+ value = sep + 1;
+
+ if (!strcmp(key, AUTH_KEY))
+ {
+ if (auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains multiple auth values.")));
+
+ auth = value;
+ }
+ else
+ {
+ /*
+ * The RFC also defines the host and port keys, but they are not
+ * required for OAUTHBEARER and we do not use them. Also, per Sec.
+ * 3.1, any key/value pairs we don't recognize must be ignored.
+ */
+ }
+
+ /* Move to the next pair. */
+ pos = end + 1;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message did not contain a final terminator.")));
+
+ return NULL; /* unreachable */
+}
+
+static void
+generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
+{
+ StringInfoData buf;
+
+ /*
+ * The admin needs to set an issuer and scope for OAuth to work. There's
+ * not really a way to hide this from the user, either, because we can't
+ * choose a "default" issuer, so be honest in the failure message.
+ *
+ * TODO: see if there's a better place to fail, earlier than this.
+ */
+ if (!ctx->issuer || !ctx->scope)
+ ereport(FATAL,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("OAuth is not properly configured for this user"),
+ errdetail_log("The issuer and scope parameters must be set in pg_hba.conf.")));
+
+
+ initStringInfo(&buf);
+
+ /*
+ * TODO: JSON escaping
+ */
+ appendStringInfo(&buf,
+ "{ "
+ "\"status\": \"invalid_token\", "
+ "\"openid-configuration\": \"%s/.well-known/openid-configuration\", "
+ "\"scope\": \"%s\" "
+ "}",
+ ctx->issuer, ctx->scope);
+
+ *output = buf.data;
+ *outputlen = buf.len;
+}
+
+static bool
+validate(Port *port, const char *auth, const char **logdetail)
+{
+ static const char *const b64_set =
+ "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-._~+/";
+
+ const char *token;
+ size_t span;
+ int ret;
+
+ /* TODO: handle logdetail when the test framework can check it */
+
+ /*-----
+ * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
+ * 2.1:
+ *
+ * b64token = 1*( ALPHA / DIGIT /
+ * "-" / "." / "_" / "~" / "+" / "/" ) *"="
+ * credentials = "Bearer" 1*SP b64token
+ *
+ * The "credentials" construction is what we receive in our auth value.
+ *
+ * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
+ * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
+ * compared case-insensitively. (This is not mentioned in RFC 6750, but
+ * it's pointed out in RFC 7628 Sec. 4.)
+ *
+ * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
+ */
+ if (pg_strncasecmp(auth, BEARER_SCHEME, strlen(BEARER_SCHEME)))
+ return false;
+
+ /* Pull the bearer token out of the auth value. */
+ token = auth + strlen(BEARER_SCHEME);
+
+ /* Swallow any additional spaces. */
+ while (*token == ' ')
+ token++;
+
+ /*
+ * Before invoking the validator command, sanity-check the token format to
+ * avoid any injection attacks later in the chain. Invalid formats are
+ * technically a protocol violation, but don't reflect any information
+ * about the sensitive Bearer token back to the client; log at COMMERROR
+ * instead.
+ */
+
+ /* Tokens must not be empty. */
+ if (!*token)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Bearer token is empty.")));
+ return false;
+ }
+
+ /*
+ * Make sure the token contains only allowed characters. Tokens may end
+ * with any number of '=' characters.
+ */
+ span = strspn(token, b64_set);
+ while (token[span] == '=')
+ span++;
+
+ if (token[span] != '\0')
+ {
+ /*
+ * This error message could be more helpful by printing the
+ * problematic character(s), but that'd be a bit like printing a piece
+ * of someone's password into the logs.
+ */
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Bearer token is not in the correct format.")));
+ return false;
+ }
+
+ /* Have the validator check the token. */
+ if (!run_validator_command(port, token))
+ return false;
+
+ if (port->hba->oauth_skip_usermap)
+ {
+ /*
+ * If the validator is our authorization authority, we're done.
+ * Authentication may or may not have been performed depending on the
+ * validator implementation; all that matters is that the validator
+ * says the user can log in with the target role.
+ */
+ return true;
+ }
+
+ /* Make sure the validator authenticated the user. */
+ if (!MyClientConnectionInfo.authn_id)
+ {
+ /* TODO: use logdetail; reduce message duplication */
+ ereport(LOG,
+ (errmsg("OAuth bearer authentication failed for user \"%s\": validator provided no identity",
+ port->user_name)));
+ return false;
+ }
+
+ /* Finally, check the user map. */
+ ret = check_usermap(port->hba->usermap, port->user_name,
+ MyClientConnectionInfo.authn_id, false);
+ return (ret == STATUS_OK);
+}
+
+static bool
+run_validator_command(Port *port, const char *token)
+{
+ bool success = false;
+ int rc;
+ int pipefd[2];
+ int rfd = -1;
+ int wfd = -1;
+
+ StringInfoData command = {0};
+ char *p;
+ FILE *fh = NULL;
+
+ ssize_t written;
+ char *line = NULL;
+ size_t size = 0;
+ ssize_t len;
+
+ Assert(oauth_validator_command);
+
+ if (!oauth_validator_command[0])
+ {
+ ereport(COMMERROR,
+ (errmsg("oauth_validator_command is not set"),
+ errhint("To allow OAuth authenticated connections, set "
+ "oauth_validator_command in postgresql.conf.")));
+ return false;
+ }
+
+ /*------
+ * Since popen() is unidirectional, open up a pipe for the other
+ * direction. Use CLOEXEC to ensure that our write end doesn't
+ * accidentally get copied into child processes, which would prevent us
+ * from closing it cleanly.
+ *
+ * XXX this is ugly. We should just read from the child process's stdout,
+ * but that's a lot more code.
+ * XXX by bypassing the popen API, we open the potential of process
+ * deadlock. Clearly document child process requirements (i.e. the child
+ * MUST read all data off of the pipe before writing anything).
+ * TODO: port to Windows using _pipe().
+ */
+ rc = pipe(pipefd);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create child pipe: %m")));
+ return false;
+ }
+
+ rfd = pipefd[0];
+ wfd = pipefd[1];
+
+ if (!set_cloexec(wfd))
+ {
+ /* error message was already logged */
+ goto cleanup;
+ }
+
+ /*----------
+ * Construct the command, substituting any recognized %-specifiers:
+ *
+ * %f: the file descriptor of the input pipe
+ * %r: the role that the client wants to assume (port->user_name)
+ * %%: a literal '%'
+ */
+ initStringInfo(&command);
+
+ for (p = oauth_validator_command; *p; p++)
+ {
+ if (p[0] == '%')
+ {
+ switch (p[1])
+ {
+ case 'f':
+ appendStringInfo(&command, "%d", rfd);
+ p++;
+ break;
+ case 'r':
+
+ /*
+ * TODO: decide how this string should be escaped. The
+ * role is controlled by the client, so if we don't escape
+ * it, command injections are inevitable.
+ *
+ * This is probably an indication that the role name needs
+ * to be communicated to the validator process in some
+ * other way. For this proof of concept, just be
+ * incredibly strict about the characters that are allowed
+ * in user names.
+ */
+ if (!username_ok_for_shell(port->user_name))
+ goto cleanup;
+
+ appendStringInfoString(&command, port->user_name);
+ p++;
+ break;
+ case '%':
+ appendStringInfoChar(&command, '%');
+ p++;
+ break;
+ default:
+ appendStringInfoChar(&command, p[0]);
+ }
+ }
+ else
+ appendStringInfoChar(&command, p[0]);
+ }
+
+ /* Execute the command. */
+ fh = OpenPipeStream(command.data, "r");
+ if (!fh)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("opening pipe to OAuth validator: %m")));
+ goto cleanup;
+ }
+
+ /* We don't need the read end of the pipe anymore. */
+ close(rfd);
+ rfd = -1;
+
+ /* Give the command the token to validate. */
+ written = write(wfd, token, strlen(token));
+ if (written != strlen(token))
+ {
+ /* TODO must loop for short writes, EINTR et al */
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write token to child pipe: %m")));
+ goto cleanup;
+ }
+
+ close(wfd);
+ wfd = -1;
+
+ /*-----
+ * Read the command's response.
+ *
+ * TODO: getline() is probably too new to use, unfortunately.
+ * TODO: loop over all lines
+ */
+ if ((len = getline(&line, &size, fh)) >= 0)
+ {
+ /* TODO: fail if the authn_id doesn't end with a newline */
+ if (len > 0)
+ line[len - 1] = '\0';
+
+ set_authn_id(port, line);
+ }
+ else if (ferror(fh))
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from command \"%s\": %m",
+ command.data)));
+ goto cleanup;
+ }
+
+ /* Make sure the command exits cleanly. */
+ if (!check_exit(&fh, command.data))
+ {
+ /* error message already logged */
+ goto cleanup;
+ }
+
+ /* Done. */
+ success = true;
+
+cleanup:
+ if (line)
+ free(line);
+
+ /*
+ * In the successful case, the pipe fds are already closed. For the error
+ * case, always close out the pipe before waiting for the command, to
+ * prevent deadlock.
+ */
+ if (rfd >= 0)
+ close(rfd);
+ if (wfd >= 0)
+ close(wfd);
+
+ if (fh)
+ {
+ Assert(!success);
+ check_exit(&fh, command.data);
+ }
+
+ if (command.data)
+ pfree(command.data);
+
+ return success;
+}
+
+static bool
+check_exit(FILE **fh, const char *command)
+{
+ int rc;
+
+ rc = ClosePipeStream(*fh);
+ *fh = NULL;
+
+ if (rc == -1)
+ {
+ /* pclose() itself failed. */
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close pipe to command \"%s\": %m",
+ command)));
+ }
+ else if (rc != 0)
+ {
+ char *reason = wait_result_to_str(rc);
+
+ ereport(COMMERROR,
+ (errmsg("failed to execute command \"%s\": %s",
+ command, reason)));
+
+ pfree(reason);
+ }
+
+ return (rc == 0);
+}
+
+static bool
+set_cloexec(int fd)
+{
+ int flags;
+ int rc;
+
+ flags = fcntl(fd, F_GETFD);
+ if (flags == -1)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not get fd flags for child pipe: %m")));
+ return false;
+ }
+
+ rc = fcntl(fd, F_SETFD, flags | FD_CLOEXEC);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not set FD_CLOEXEC for child pipe: %m")));
+ return false;
+ }
+
+ return true;
+}
+
+/*
+ * XXX This should go away eventually and be replaced with either a proper
+ * escape or a different strategy for communication with the validator command.
+ */
+static bool
+username_ok_for_shell(const char *username)
+{
+ /* This set is borrowed from fe_utils' appendShellStringNoError(). */
+ static const char *const allowed =
+ "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-_./:";
+ size_t span;
+
+ Assert(username && username[0]); /* should have already been checked */
+
+ span = strspn(username, allowed);
+ if (username[span] != '\0')
+ {
+ ereport(COMMERROR,
+ (errmsg("PostgreSQL user name contains unsafe characters and cannot be passed to the OAuth validator")));
+ return false;
+ }
+
+ return true;
+}
diff --git a/src/backend/libpq/auth-sasl.c b/src/backend/libpq/auth-sasl.c
index 08b24d90b4..4039e7fa3e 100644
--- a/src/backend/libpq/auth-sasl.c
+++ b/src/backend/libpq/auth-sasl.c
@@ -20,14 +20,6 @@
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
-/*
- * Maximum accepted size of SASL messages.
- *
- * The messages that the server or libpq generate are much smaller than this,
- * but have some headroom.
- */
-#define PG_MAX_SASL_MESSAGE_LENGTH 1024
-
/*
* Perform a SASL exchange with a libpq client, using a specific mechanism
* implementation.
@@ -103,7 +95,7 @@ CheckSASLAuth(const pg_be_sasl_mech *mech, Port *port, char *shadow_pass,
/* Get the actual SASL message */
initStringInfo(&buf);
- if (pq_getmessage(&buf, PG_MAX_SASL_MESSAGE_LENGTH))
+ if (pq_getmessage(&buf, mech->max_message_length))
{
/* EOF - pq_getmessage already logged error */
pfree(buf.data);
diff --git a/src/backend/libpq/auth-scram.c b/src/backend/libpq/auth-scram.c
index 2abb1a9b3a..aa6b5020dc 100644
--- a/src/backend/libpq/auth-scram.c
+++ b/src/backend/libpq/auth-scram.c
@@ -118,7 +118,9 @@ static int scram_exchange(void *opaq, const char *input, int inputlen,
const pg_be_sasl_mech pg_be_scram_mech = {
scram_get_mechanisms,
scram_init,
- scram_exchange
+ scram_exchange,
+
+ PG_MAX_SASL_MESSAGE_LENGTH
};
/*
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 9bbdc4beb0..db7c77da86 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -29,6 +29,7 @@
#include "libpq/auth.h"
#include "libpq/crypt.h"
#include "libpq/libpq.h"
+#include "libpq/oauth.h"
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
#include "libpq/scram.h"
@@ -47,7 +48,6 @@
*/
static void auth_failed(Port *port, int status, const char *logdetail);
static char *recv_password_packet(Port *port);
-static void set_authn_id(Port *port, const char *id);
/*----------------------------------------------------------------
@@ -203,22 +203,6 @@ static int CheckRADIUSAuth(Port *port);
static int PerformRadiusTransaction(const char *server, const char *secret, const char *portstr, const char *identifier, const char *user_name, const char *passwd);
-/*
- * Maximum accepted size of GSS and SSPI authentication tokens.
- * We also use this as a limit on ordinary password packet lengths.
- *
- * Kerberos tickets are usually quite small, but the TGTs issued by Windows
- * domain controllers include an authorization field known as the Privilege
- * Attribute Certificate (PAC), which contains the user's Windows permissions
- * (group memberships etc.). The PAC is copied into all tickets obtained on
- * the basis of this TGT (even those issued by Unix realms which the Windows
- * realm trusts), and can be several kB in size. The maximum token size
- * accepted by Windows systems is determined by the MaxAuthToken Windows
- * registry setting. Microsoft recommends that it is not set higher than
- * 65535 bytes, so that seems like a reasonable limit for us as well.
- */
-#define PG_MAX_AUTH_TOKEN_LENGTH 65535
-
/*----------------------------------------------------------------
* Global authentication functions
*----------------------------------------------------------------
@@ -307,6 +291,9 @@ auth_failed(Port *port, int status, const char *logdetail)
case uaRADIUS:
errstr = gettext_noop("RADIUS authentication failed for user \"%s\"");
break;
+ case uaOAuth:
+ errstr = gettext_noop("OAuth bearer authentication failed for user \"%s\"");
+ break;
default:
errstr = gettext_noop("authentication failed for user \"%s\": invalid authentication method");
break;
@@ -342,7 +329,7 @@ auth_failed(Port *port, int status, const char *logdetail)
* lifetime of MyClientConnectionInfo, so it is safe to pass a string that is
* managed by an external library.
*/
-static void
+void
set_authn_id(Port *port, const char *id)
{
Assert(id);
@@ -629,6 +616,9 @@ ClientAuthentication(Port *port)
case uaTrust:
status = STATUS_OK;
break;
+ case uaOAuth:
+ status = CheckSASLAuth(&pg_be_oauth_mech, port, NULL, NULL);
+ break;
}
if ((status == STATUS_OK && port->hba->clientcert == clientCertFull)
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index 8004d102ad..03c3f038c7 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -119,7 +119,8 @@ static const char *const UserAuthName[] =
"ldap",
"cert",
"radius",
- "peer"
+ "peer",
+ "oauth",
};
/*
@@ -1748,6 +1749,8 @@ parse_hba_line(TokenizedAuthLine *tok_line, int elevel)
#endif
else if (strcmp(token->string, "radius") == 0)
parsedline->auth_method = uaRADIUS;
+ else if (strcmp(token->string, "oauth") == 0)
+ parsedline->auth_method = uaOAuth;
else
{
ereport(elevel,
@@ -2067,8 +2070,9 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->auth_method != uaPeer &&
hbaline->auth_method != uaGSS &&
hbaline->auth_method != uaSSPI &&
- hbaline->auth_method != uaCert)
- INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, and cert"));
+ hbaline->auth_method != uaCert &&
+ hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, cert, and oauth"));
hbaline->usermap = pstrdup(val);
}
else if (strcmp(name, "clientcert") == 0)
@@ -2451,6 +2455,27 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->radiusidentifiers = parsed_identifiers;
hbaline->radiusidentifiers_s = pstrdup(val);
}
+ else if (strcmp(name, "issuer") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("issuer", gettext_noop("oauth"));
+ hbaline->oauth_issuer = pstrdup(val);
+ }
+ else if (strcmp(name, "scope") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("scope", gettext_noop("oauth"));
+ hbaline->oauth_scope = pstrdup(val);
+ }
+ else if (strcmp(name, "trust_validator_authz") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("trust_validator_authz", gettext_noop("oauth"));
+ if (strcmp(val, "1") == 0)
+ hbaline->oauth_skip_usermap = true;
+ else
+ hbaline->oauth_skip_usermap = false;
+ }
else
{
ereport(elevel,
diff --git a/src/backend/libpq/meson.build b/src/backend/libpq/meson.build
index 7c65314512..c85527fb01 100644
--- a/src/backend/libpq/meson.build
+++ b/src/backend/libpq/meson.build
@@ -1,6 +1,7 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
backend_sources += files(
+ 'auth-oauth.c',
'auth-sasl.c',
'auth-scram.c',
'auth.c',
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 93ded31ed9..0f83f0d870 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -48,6 +48,7 @@
#include "jit/jit.h"
#include "libpq/auth.h"
#include "libpq/libpq.h"
+#include "libpq/oauth.h"
#include "libpq/scram.h"
#include "nodes/queryjumble.h"
#include "optimizer/cost.h"
@@ -4670,6 +4671,17 @@ struct config_string ConfigureNamesString[] =
check_debug_io_direct, assign_debug_io_direct, NULL
},
+ {
+ {"oauth_validator_command", PGC_SIGHUP, CONN_AUTH_AUTH,
+ gettext_noop("Command to validate OAuth v2 bearer tokens."),
+ NULL,
+ GUC_SUPERUSER_ONLY | GUC_NOT_IN_SAMPLE
+ },
+ &oauth_validator_command,
+ "",
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, NULL, NULL, NULL, NULL
diff --git a/src/include/libpq/auth.h b/src/include/libpq/auth.h
index 227b41daf6..22f6ab9f1d 100644
--- a/src/include/libpq/auth.h
+++ b/src/include/libpq/auth.h
@@ -16,6 +16,22 @@
#include "libpq/libpq-be.h"
+/*
+ * Maximum accepted size of GSS and SSPI authentication tokens.
+ * We also use this as a limit on ordinary password packet lengths.
+ *
+ * Kerberos tickets are usually quite small, but the TGTs issued by Windows
+ * domain controllers include an authorization field known as the Privilege
+ * Attribute Certificate (PAC), which contains the user's Windows permissions
+ * (group memberships etc.). The PAC is copied into all tickets obtained on
+ * the basis of this TGT (even those issued by Unix realms which the Windows
+ * realm trusts), and can be several kB in size. The maximum token size
+ * accepted by Windows systems is determined by the MaxAuthToken Windows
+ * registry setting. Microsoft recommends that it is not set higher than
+ * 65535 bytes, so that seems like a reasonable limit for us as well.
+ */
+#define PG_MAX_AUTH_TOKEN_LENGTH 65535
+
extern PGDLLIMPORT char *pg_krb_server_keyfile;
extern PGDLLIMPORT bool pg_krb_caseins_users;
extern PGDLLIMPORT bool pg_gss_accept_delegation;
@@ -23,6 +39,7 @@ extern PGDLLIMPORT bool pg_gss_accept_delegation;
extern void ClientAuthentication(Port *port);
extern void sendAuthRequest(Port *port, AuthRequest areq, const char *extradata,
int extralen);
+extern void set_authn_id(Port *port, const char *id);
/* Hook for plugins to get control in ClientAuthentication() */
typedef void (*ClientAuthentication_hook_type) (Port *, int);
diff --git a/src/include/libpq/hba.h b/src/include/libpq/hba.h
index 8ea837ae82..53c999a69f 100644
--- a/src/include/libpq/hba.h
+++ b/src/include/libpq/hba.h
@@ -39,7 +39,8 @@ typedef enum UserAuth
uaCert,
uaRADIUS,
uaPeer,
-#define USER_AUTH_LAST uaPeer /* Must be last value of this enum */
+ uaOAuth,
+#define USER_AUTH_LAST uaOAuth /* Must be last value of this enum */
} UserAuth;
/*
@@ -135,6 +136,9 @@ typedef struct HbaLine
char *radiusidentifiers_s;
List *radiusports;
char *radiusports_s;
+ char *oauth_issuer;
+ char *oauth_scope;
+ bool oauth_skip_usermap;
} HbaLine;
typedef struct IdentLine
diff --git a/src/include/libpq/oauth.h b/src/include/libpq/oauth.h
new file mode 100644
index 0000000000..5edab3b25a
--- /dev/null
+++ b/src/include/libpq/oauth.h
@@ -0,0 +1,24 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth.h
+ * Interface to libpq/auth-oauth.c
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/libpq/oauth.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_OAUTH_H
+#define PG_OAUTH_H
+
+#include "libpq/libpq-be.h"
+#include "libpq/sasl.h"
+
+extern char *oauth_validator_command;
+
+/* Implementation */
+extern const pg_be_sasl_mech pg_be_oauth_mech;
+
+#endif /* PG_OAUTH_H */
diff --git a/src/include/libpq/sasl.h b/src/include/libpq/sasl.h
index 7a1f970cca..3f2c02b8f2 100644
--- a/src/include/libpq/sasl.h
+++ b/src/include/libpq/sasl.h
@@ -26,6 +26,14 @@
#define PG_SASL_EXCHANGE_SUCCESS 1
#define PG_SASL_EXCHANGE_FAILURE 2
+/*
+ * Maximum accepted size of SASL messages.
+ *
+ * The messages that the server or libpq generate are much smaller than this,
+ * but have some headroom.
+ */
+#define PG_MAX_SASL_MESSAGE_LENGTH 1024
+
/*
* Backend SASL mechanism callbacks.
*
@@ -127,6 +135,9 @@ typedef struct pg_be_sasl_mech
const char *input, int inputlen,
char **output, int *outputlen,
const char **logdetail);
+
+ /* The maximum size allowed for client SASLResponses. */
+ int max_message_length;
} pg_be_sasl_mech;
/* Common implementation for auth.c */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index c45aae6f9e..a0acd33ccf 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3543,6 +3543,7 @@ normal_rand_fctx
nsphash_hash
ntile_context
numeric
+oauth_state
object_access_hook_type
object_access_hook_type_str
off_t
--
2.34.1
v19-0007-Add-pytest-suite-for-OAuth.patch (application/octet-stream, 171.8K) download
From 47236c5644f33b18b2b7379a35467eeed2177e80 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Fri, 4 Jun 2021 09:06:38 -0700
Subject: [PATCH v19 7/9] Add pytest suite for OAuth
Requires Python 3. On the first run of `make installcheck` or `meson
test` the dependencies will be installed into a local virtualenv for
you. See the README for more details.
Cirrus has been updated to build OAuth support on Debian and FreeBSD.
The suite contains a --temp-instance option, analogous to pg_regress's
option of the same name, which allows an ephemeral server to be spun up
during a test run.
TODOs:
- The --tap-stream option to pytest-tap is slightly broken during test
failures (it suppresses error information), which impedes debugging.
- pyca/cryptography is pinned at an old version. Since we use it for
testing and not security, this isn't a critical problem yet, but it's
not ideal. Newer versions require a Rust compiler to build, and while
many platforms have precompiled wheels, some (FreeBSD) do not. Even
with the Rust pieces bypassed, compilation on FreeBSD takes a while.
- The with_oauth test skip logic should probably be integrated into the
Makefile side as well...
- See if 32-bit tests can be enabled with a 32-bit Python.
---
.cirrus.tasks.yml | 22 +-
meson.build | 107 ++
src/test/meson.build | 1 +
src/test/python/.gitignore | 2 +
src/test/python/Makefile | 38 +
src/test/python/README | 66 +
src/test/python/client/__init__.py | 0
src/test/python/client/conftest.py | 137 ++
src/test/python/client/test_client.py | 180 +++
src/test/python/client/test_oauth.py | 1720 +++++++++++++++++++++++++
src/test/python/conftest.py | 34 +
src/test/python/meson.build | 46 +
src/test/python/pq3.py | 732 +++++++++++
src/test/python/pytest.ini | 4 +
src/test/python/requirements.txt | 9 +
src/test/python/server/__init__.py | 0
src/test/python/server/conftest.py | 141 ++
src/test/python/server/meson.build | 18 +
src/test/python/server/oauthtest.c | 118 ++
src/test/python/server/test_oauth.py | 945 ++++++++++++++
src/test/python/server/test_server.py | 21 +
src/test/python/test_internals.py | 138 ++
src/test/python/test_pq3.py | 564 ++++++++
src/test/python/tls.py | 195 +++
src/tools/make_venv | 56 +
25 files changed, 5287 insertions(+), 7 deletions(-)
create mode 100644 src/test/python/.gitignore
create mode 100644 src/test/python/Makefile
create mode 100644 src/test/python/README
create mode 100644 src/test/python/client/__init__.py
create mode 100644 src/test/python/client/conftest.py
create mode 100644 src/test/python/client/test_client.py
create mode 100644 src/test/python/client/test_oauth.py
create mode 100644 src/test/python/conftest.py
create mode 100644 src/test/python/meson.build
create mode 100644 src/test/python/pq3.py
create mode 100644 src/test/python/pytest.ini
create mode 100644 src/test/python/requirements.txt
create mode 100644 src/test/python/server/__init__.py
create mode 100644 src/test/python/server/conftest.py
create mode 100644 src/test/python/server/meson.build
create mode 100644 src/test/python/server/oauthtest.c
create mode 100644 src/test/python/server/test_oauth.py
create mode 100644 src/test/python/server/test_server.py
create mode 100644 src/test/python/test_internals.py
create mode 100644 src/test/python/test_pq3.py
create mode 100644 src/test/python/tls.py
create mode 100755 src/tools/make_venv
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 3b5b54df58..2501743b31 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -20,7 +20,7 @@ env:
MTEST_ARGS: --print-errorlogs --no-rebuild -C build
PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests
TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf
- PG_TEST_EXTRA: kerberos ldap ssl load_balance
+ PG_TEST_EXTRA: kerberos ldap ssl load_balance python
# What files to preserve in case tests fail
@@ -165,7 +165,7 @@ task:
chown root:postgres /tmp/cores
sysctl kern.corefile='/tmp/cores/%N.%P.core'
setup_additional_packages_script: |
- #pkg install -y ...
+ pkg install -y curl
# NB: Intentionally build without -Dllvm. The freebsd image size is already
# large enough to make VM startup slow, and even without llvm freebsd
@@ -177,6 +177,7 @@ task:
-Dcassert=true -Dinjection_points=true \
-Duuid=bsd -Dtcl_version=tcl86 -Ddtrace=auto \
-DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
+ -Doauth=curl \
-Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \
build
EOF
@@ -225,6 +226,7 @@ LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
--with-libxslt
--with-llvm
--with-lz4
+ --with-oauth=curl
--with-pam
--with-perl
--with-python
@@ -237,6 +239,7 @@ LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
LINUX_MESON_FEATURES: &LINUX_MESON_FEATURES >-
-Dllvm=enabled
+ -Doauth=curl
-Duuid=e2fs
@@ -312,8 +315,11 @@ task:
EOF
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install \
+ libcurl4-openssl-dev \
+ libcurl4-openssl-dev:i386 \
+ python3-venv \
matrix:
- name: Linux - Debian Bullseye - Autoconf
@@ -368,6 +374,8 @@ task:
# Also build & test in a 32bit build - it's gotten rare to test that
# locally.
+ # XXX 32-bit Python tests are currently disabled, as the system's 64-bit
+ # Python modules can't link against libpq.
configure_32_script: |
su postgres <<-EOF
export CC='ccache gcc -m32'
@@ -378,7 +386,7 @@ task:
-Dllvm=disabled \
--pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
-DPERL=perl5.32-i386-linux-gnu \
- -DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
+ -DPG_TEST_EXTRA="${PG_TEST_EXTRA//"python"}" \
build-32
EOF
@@ -676,8 +684,8 @@ task:
folder: $CCACHE_DIR
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install libcurl4-openssl-dev
###
# Test that code can be built with gcc/clang without warnings
diff --git a/meson.build b/meson.build
index 45b20d11c1..8567355a25 100644
--- a/meson.build
+++ b/meson.build
@@ -3174,6 +3174,9 @@ else
endif
testwrap = files('src/tools/testwrap')
+make_venv = files('src/tools/make_venv')
+
+checked_working_venv = false
foreach test_dir : tests
testwrap_base = [
@@ -3335,6 +3338,110 @@ foreach test_dir : tests
)
endforeach
install_suites += test_group
+ elif kind == 'pytest'
+ venv_name = test_dir['name'] + '_venv'
+ venv_path = meson.build_root() / venv_name
+
+ # The Python tests require a working venv module. This is part of the
+ # standard library, but some platforms disable it until a separate package
+ # is installed. Those same platforms don't provide an easy way to check
+ # whether the venv command will work until the first time you try it, so
+ # we decide whether or not to enable these tests on the fly.
+ if not checked_working_venv
+ cmd = run_command(python, '-m', 'venv', venv_path, check: false)
+
+ have_working_venv = (cmd.returncode() == 0)
+ if not have_working_venv
+ warning('A working Python venv module is required to run Python tests.')
+ endif
+
+ checked_working_venv = true
+ endif
+
+ if not have_working_venv
+ continue
+ endif
+
+ # Make sure the temporary installation is in PATH (necessary both for
+ # --temp-instance and for any pip modules compiling against libpq, like
+ # psycopg2).
+ env = test_env
+ env.prepend('PATH', temp_install_bindir, test_dir['bd'])
+
+ foreach name, value : t.get('env', {})
+ env.set(name, value)
+ endforeach
+
+ if get_option('PG_TEST_EXTRA').contains('python')
+ reqs = files(t['requirements'])
+ test('install_' + venv_name,
+ python,
+ args: [ make_venv, '--requirements', reqs, venv_path ],
+ env: env,
+ priority: setup_tests_priority - 1, # must run after tmp_install
+ is_parallel: false,
+ suite: ['setup'],
+ timeout: 60, # 30s is too short for the cryptography package compile
+ )
+ endif
+
+ test_group = test_dir['name']
+ test_output = test_result_dir / test_group / kind
+ test_kwargs = {
+ #'protocol': 'tap',
+ 'suite': test_group,
+ 'timeout': 1000,
+ 'depends': test_deps,
+ 'env': env,
+ }
+
+ if fs.is_dir(venv_path / 'Scripts')
+ # Windows virtualenv layout
+ pytest = venv_path / 'Scripts' / 'py.test'
+ else
+ pytest = venv_path / 'bin' / 'py.test'
+ endif
+
+ test_command = [
+ pytest,
+ # Avoid running these tests against an existing database.
+ '--temp-instance', test_output / 'data',
+
+ # FIXME pytest-tap's stream feature accidentally suppresses errors that
+ # are critical for debugging:
+ # https://github.com/python-tap/pytest-tap/issues/30
+ # Don't use the meson TAP protocol for now...
+ #'--tap-stream',
+ ]
+
+ foreach pyt : t['tests']
+ # Similarly to TAP, strip ./ and .py to make the names prettier
+ pyt_p = pyt
+ if pyt_p.startswith('./')
+ pyt_p = pyt_p.split('./')[1]
+ endif
+ if pyt_p.endswith('.py')
+ pyt_p = fs.stem(pyt_p)
+ endif
+
+ testwrap_pytest = testwrap_base + [
+ '--testgroup', test_group,
+ '--testname', pyt_p,
+ ]
+ if not get_option('PG_TEST_EXTRA').contains('python')
+ testwrap_pytest += ['--skip', '"python" tests not enabled in PG_TEST_EXTRA']
+ endif
+
+ test(test_group / pyt_p,
+ python,
+ kwargs: test_kwargs,
+ args: testwrap_pytest + [
+ '--', test_command,
+ test_dir['sd'] / pyt,
+ ],
+ )
+ endforeach
+ install_suites += test_group
else
error('unknown kind @0@ of test in @1@'.format(kind, test_dir['sd']))
endif
diff --git a/src/test/meson.build b/src/test/meson.build
index c3d0dfedf1..f401ec179e 100644
--- a/src/test/meson.build
+++ b/src/test/meson.build
@@ -7,6 +7,7 @@ subdir('authentication')
subdir('recovery')
subdir('subscription')
subdir('modules')
+subdir('python')
if ssl.found()
subdir('ssl')
diff --git a/src/test/python/.gitignore b/src/test/python/.gitignore
new file mode 100644
index 0000000000..0e8f027b2e
--- /dev/null
+++ b/src/test/python/.gitignore
@@ -0,0 +1,2 @@
+__pycache__/
+/venv/
diff --git a/src/test/python/Makefile b/src/test/python/Makefile
new file mode 100644
index 0000000000..b0695b6287
--- /dev/null
+++ b/src/test/python/Makefile
@@ -0,0 +1,38 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+# Only Python 3 is supported, but if it's named something different on your
+# system you can override it with the PYTHON3 variable.
+PYTHON3 := python3
+
+# All dependencies are placed into this directory. The default is .gitignored
+# for you, but you can override it if you'd like.
+VENV := ./venv
+
+override VBIN := $(VENV)/bin
+override PIP := $(VBIN)/pip
+override PYTEST := $(VBIN)/py.test
+override ISORT := $(VBIN)/isort
+override BLACK := $(VBIN)/black
+
+.PHONY: installcheck indent
+
+installcheck: $(PYTEST)
+ $(PYTEST) -v -rs
+
+indent: $(ISORT) $(BLACK)
+ $(ISORT) --profile black *.py client/*.py server/*.py
+ $(BLACK) *.py client/*.py server/*.py
+
+$(PYTEST) $(ISORT) $(BLACK) &: requirements.txt | $(PIP)
+ $(PIP) install --force-reinstall -r $<
+
+$(PIP):
+ $(PYTHON3) -m venv $(VENV)
+
+# A convenience recipe to rebuild psycopg2 against the local libpq.
+.PHONY: rebuild-psycopg2
+rebuild-psycopg2: | $(PIP)
+ $(PIP) install --force-reinstall --no-binary :all: $(shell grep psycopg2 requirements.txt)
diff --git a/src/test/python/README b/src/test/python/README
new file mode 100644
index 0000000000..acf339a589
--- /dev/null
+++ b/src/test/python/README
@@ -0,0 +1,66 @@
+A test suite for exercising both the libpq client and the server backend at the
+protocol level, based on pytest and Construct.
+
+WARNING! This suite takes superuser-level control of the cluster under test,
+writing to the server config, creating and destroying databases, etc. It also
+spins up various ephemeral TCP services. This is not safe for production servers
+and therefore must be explicitly opted into by setting PG_TEST_EXTRA=python in
+the environment.
+
+The test suite currently assumes that the standard PG* environment variables
+point to the database under test and are sufficient to log in a superuser on
+that system. In other words, a bare `psql` needs to Just Work before the test
+suite can do its thing. For a newly built dev cluster, typically all that I need
+to do is a
+
+ export PGDATABASE=postgres
+
+but you can adjust as needed for your setup. See also 'Advanced Usage' below.
+
+## Requirements
+
+A supported version (3.6+) of Python.
+
+The first run of
+
+ make installcheck PG_TEST_EXTRA=python
+
+will install a local virtual environment and all needed dependencies. During
+development, if libpq changes incompatibly, you can issue
+
+ $ make rebuild-psycopg2
+
+to force a rebuild of the client library.
+
+## Hacking
+
+The code style is enforced by a _very_ opinionated autoformatter. Running the
+
+ make indent
+
+recipe will invoke it for you automatically. Don't fight the tool; part of the
+zen is in knowing that if the formatter makes your code ugly, there's probably a
+cleaner way to write your code.
+
+## Advanced Usage
+
+The Makefile is there for convenience, but you don't have to use it. Activate
+the virtualenv to be able to use pytest directly:
+
+ $ export PG_TEST_EXTRA=python
+ $ source venv/bin/activate
+ $ py.test -k oauth
+ ...
+ $ py.test ./server/test_server.py
+ ...
+ $ deactivate # puts the PATH et al back the way it was before
+
+To make quick smoke tests possible, slow tests have been marked explicitly. You
+can skip them by saying e.g.
+
+ $ py.test -m 'not slow'
+
+If you'd rather not test against an existing server, you can have the suite spin
+up a temporary one using whatever pg_ctl it finds in PATH:
+
+ $ py.test --temp-instance=./tmp_check
diff --git a/src/test/python/client/__init__.py b/src/test/python/client/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/src/test/python/client/conftest.py b/src/test/python/client/conftest.py
new file mode 100644
index 0000000000..94f3620af3
--- /dev/null
+++ b/src/test/python/client/conftest.py
@@ -0,0 +1,137 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import socket
+import sys
+import threading
+
+import psycopg2
+import psycopg2.extras
+import pytest
+
+import pq3
+
+BLOCKING_TIMEOUT = 2 # the number of seconds to wait for blocking calls
+
+
+@pytest.fixture
+def server_socket(unused_tcp_port_factory):
+ """
+ Returns a listening socket bound to an ephemeral port.
+ """
+ with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+ s.bind(("127.0.0.1", unused_tcp_port_factory()))
+ s.listen(1)
+ s.settimeout(BLOCKING_TIMEOUT)
+ yield s
+
+
+class ClientHandshake(threading.Thread):
+ """
+ A thread that connects to a local Postgres server using psycopg2. Once the
+ opening handshake completes, the connection will be immediately closed.
+ """
+
+ def __init__(self, *, port, **kwargs):
+ super().__init__()
+
+ kwargs["port"] = port
+ self._kwargs = kwargs
+
+ self.exception = None
+
+ def run(self):
+ try:
+ conn = psycopg2.connect(host="127.0.0.1", **self._kwargs)
+ self._pump_async(conn)
+ conn.close()
+ except Exception as e:
+ self.exception = e
+
+ def check_completed(self, timeout=BLOCKING_TIMEOUT):
+ """
+ Joins the client thread. Raises an exception if the thread could not be
+ joined, or if it threw an exception itself. (The exception will be
+ cleared, so future calls to check_completed will succeed.)
+ """
+ self.join(timeout)
+
+ if self.is_alive():
+ raise TimeoutError("client thread did not handshake within the timeout")
+ elif self.exception:
+ e = self.exception
+ self.exception = None
+ raise e
+
+ def _pump_async(self, conn):
+ """
+ Polls a psycopg2 connection until it's completed. (Synchronous
+ connections will work here too; they'll just immediately return OK.)
+ """
+ psycopg2.extras.wait_select(conn)
+
+
+@pytest.fixture
+def accept(server_socket):
+ """
+ Returns a factory function that, when called, returns a pair (sock, client)
+ where sock is a server socket that has accepted a connection from client,
+ and client is an instance of ClientHandshake. Clients will complete their
+ handshakes and cleanly disconnect.
+
+ The default connstring options may be extended or overridden by passing
+ arbitrary keyword arguments. Keep in mind that you generally should not
+ override the host or port, since they point to the local test server.
+
+ For situations where a client needs to connect more than once to complete a
+ handshake, the accept function may be called more than once. (The client
+ returned for subsequent calls will always be the same client that was
+ returned for the first call.)
+
+ Tests must either complete the handshake so that the client thread can be
+ automatically joined during teardown, or else call client.check_completed()
+ and manually handle any expected errors.
+ """
+ _, port = server_socket.getsockname()
+
+ client = None
+ default_opts = dict(
+ port=port,
+ user=pq3.pguser(),
+ sslmode="disable",
+ )
+
+ def factory(**kwargs):
+ nonlocal client
+
+ if client is None:
+ opts = dict(default_opts)
+ opts.update(kwargs)
+
+ # The server_socket is already listening, so the client thread can
+ # be safely started; it'll block on the connection until we accept.
+ client = ClientHandshake(**opts)
+ client.start()
+
+ sock, _ = server_socket.accept()
+ return sock, client
+
+ yield factory
+
+ if client is not None:
+ client.check_completed()
+
+
+@pytest.fixture
+def conn(accept):
+ """
+ Returns an accepted, wrapped pq3 connection to a psycopg2 client. The socket
+ will be closed when the test finishes, and the client will be checked for a
+ cleanly completed handshake.
+ """
+ sock, client = accept()
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ yield conn
diff --git a/src/test/python/client/test_client.py b/src/test/python/client/test_client.py
new file mode 100644
index 0000000000..c4c946fda4
--- /dev/null
+++ b/src/test/python/client/test_client.py
@@ -0,0 +1,180 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import sys
+
+import psycopg2
+import pytest
+from cryptography.hazmat.primitives import hashes, hmac
+
+import pq3
+
+
+def finish_handshake(conn):
+ """
+ Sends the AuthenticationOK message and the standard opening salvo of server
+ messages, then asserts that the client immediately sends a Terminate message
+ to close the connection cleanly.
+ """
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.OK)
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"client_encoding", value=b"UTF-8")
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"DateStyle", value=b"ISO, MDY")
+ pq3.send(conn, pq3.types.ReadyForQuery, status=b"I")
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.Terminate
+
+
+def test_handshake(conn):
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ finish_handshake(conn)
+
+
+def test_aborted_connection(accept):
+ """
+ Make sure the client correctly reports an early close during handshakes.
+ """
+ sock, client = accept()
+ sock.close()
+
+ expected = "server closed the connection unexpectedly"
+ with pytest.raises(psycopg2.OperationalError, match=expected):
+ client.check_completed()
+
+
+#
+# SCRAM-SHA-256 (see RFC 5802: https://tools.ietf.org/html/rfc5802)
+#
+
+
+@pytest.fixture
+def password():
+ """
+ Returns a password for use by both client and server.
+ """
+ # TODO: parameterize this with passwords that require SASLprep.
+ return "secret"
+
+
+@pytest.fixture
+def pwconn(accept, password):
+ """
+ Like the conn fixture, but uses a password in the connection.
+ """
+ sock, client = accept(password=password)
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ yield conn
+
+
+def sha256(data):
+ """The H(str) function from Section 2.2."""
+ digest = hashes.Hash(hashes.SHA256())
+ digest.update(data)
+ return digest.finalize()
+
+
+def hmac_256(key, data):
+ """The HMAC(key, str) function from Section 2.2."""
+ h = hmac.HMAC(key, hashes.SHA256())
+ h.update(data)
+ return h.finalize()
+
+
+def xor(a, b):
+ """The XOR operation from Section 2.2."""
+ res = bytearray(a)
+ for i, byte in enumerate(b):
+ res[i] ^= byte
+ return bytes(res)
+
+
+def h_i(data, salt, i):
+ """The Hi(str, salt, i) function from Section 2.2."""
+ assert i > 0
+
+ acc = hmac_256(data, salt + b"\x00\x00\x00\x01")
+ last = acc
+ i -= 1
+
+ while i:
+ u = hmac_256(data, last)
+ acc = xor(acc, u)
+
+ last = u
+ i -= 1
+
+ return acc
+
+
+def test_scram(pwconn, password):
+ startup = pq3.recv1(pwconn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ pwconn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"SCRAM-SHA-256", b""],
+ )
+
+ # Get the client-first-message.
+ pkt = pq3.recv1(pwconn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ initial = pq3.SASLInitialResponse.parse(pkt.payload)
+ assert initial.name == b"SCRAM-SHA-256"
+
+ c_bind, authzid, c_name, c_nonce = initial.data.split(b",")
+ assert c_bind == b"n" # no channel bindings on a plaintext connection
+ assert authzid == b"" # we don't support authzid currently
+ assert c_name == b"n=" # libpq doesn't honor the GS2 username
+ assert c_nonce.startswith(b"r=")
+
+ # Send the server-first-message.
+ salt = b"12345"
+ iterations = 2
+
+ s_nonce = c_nonce + b"somenonce"
+ s_salt = b"s=" + base64.b64encode(salt)
+ s_iterations = b"i=%d" % iterations
+
+ msg = b",".join([s_nonce, s_salt, s_iterations])
+ pq3.send(pwconn, pq3.types.AuthnRequest, type=pq3.authn.SASLContinue, body=msg)
+
+ # Get the client-final-message.
+ pkt = pq3.recv1(pwconn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ c_bind_final, c_nonce_final, c_proof = pkt.payload.split(b",")
+ assert c_bind_final == b"c=" + base64.b64encode(c_bind + b"," + authzid + b",")
+ assert c_nonce_final == s_nonce
+
+ # Calculate what the client proof should be.
+ salted_password = h_i(password.encode("ascii"), salt, iterations)
+ client_key = hmac_256(salted_password, b"Client Key")
+ stored_key = sha256(client_key)
+
+ auth_message = b",".join(
+ [c_name, c_nonce, s_nonce, s_salt, s_iterations, c_bind_final, c_nonce_final]
+ )
+ client_signature = hmac_256(stored_key, auth_message)
+ client_proof = xor(client_key, client_signature)
+
+ expected = b"p=" + base64.b64encode(client_proof)
+ assert c_proof == expected
+
+ # Send the correct server signature.
+ server_key = hmac_256(salted_password, b"Server Key")
+ server_signature = hmac_256(server_key, auth_message)
+
+ s_verify = b"v=" + base64.b64encode(server_signature)
+ pq3.send(pwconn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal, body=s_verify)
+
+ # Done!
+ finish_handshake(pwconn)
diff --git a/src/test/python/client/test_oauth.py b/src/test/python/client/test_oauth.py
new file mode 100644
index 0000000000..aee8ecdf5e
--- /dev/null
+++ b/src/test/python/client/test_oauth.py
@@ -0,0 +1,1720 @@
+#
+# Copyright 2021 VMware, Inc.
+# Portions Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import collections
+import ctypes
+import http.server
+import json
+import logging
+import os
+import platform
+import secrets
+import sys
+import threading
+import time
+import traceback
+import types
+import urllib.parse
+from numbers import Number
+
+import psycopg2
+import pytest
+
+import pq3
+
+from .conftest import BLOCKING_TIMEOUT
+
+# The client tests need libpq to have been compiled with OAuth support; skip
+# them otherwise.
+pytestmark = pytest.mark.skipif(
+ os.getenv("with_oauth") == "none",
+ reason="OAuth client tests require --with-oauth support",
+)
+
+if platform.system() == "Darwin":
+ libpq = ctypes.cdll.LoadLibrary("libpq.5.dylib")
+elif platform.system() == "Windows":
+ pass # TODO
+else:
+ libpq = ctypes.cdll.LoadLibrary("libpq.so.5")
+
+
+def finish_handshake(conn):
+ """
+ Sends the AuthenticationOK message and the standard opening salvo of server
+ messages, then asserts that the client immediately sends a Terminate message
+ to close the connection cleanly.
+ """
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.OK)
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"client_encoding", value=b"UTF-8")
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"DateStyle", value=b"ISO, MDY")
+ pq3.send(conn, pq3.types.ReadyForQuery, status=b"I")
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.Terminate
+
+
+#
+# OAUTHBEARER (see RFC 7628: https://tools.ietf.org/html/rfc7628)
+#
+
+
+def start_oauth_handshake(conn):
+ """
+ Negotiates an OAUTHBEARER SASL challenge. Returns the client's initial
+ response data.
+ """
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn, pq3.types.AuthnRequest, type=pq3.authn.SASL, body=[b"OAUTHBEARER", b""]
+ )
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ initial = pq3.SASLInitialResponse.parse(pkt.payload)
+ assert initial.name == b"OAUTHBEARER"
+
+ return initial.data
+
+
+def get_auth_value(initial):
+ """
+ Finds the auth value (e.g. "Bearer somedata..." in the client's initial SASL
+ response.
+ """
+ kvpairs = initial.split(b"\x01")
+ assert kvpairs[0] == b"n,," # no channel binding or authzid
+ assert kvpairs[2] == b"" # ends with an empty kvpair
+ assert kvpairs[3] == b"" # ...and there's nothing after it
+ assert len(kvpairs) == 4
+
+ key, value = kvpairs[1].split(b"=", 2)
+ assert key == b"auth"
+
+ return value
+
+
+def xtest_oauth_success(conn): # TODO
+ initial = start_oauth_handshake(conn)
+
+ auth = get_auth_value(initial)
+ assert auth.startswith(b"Bearer ")
+
+ # Accept the token. TODO actually validate
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+class RawResponse(str):
+ """
+ Returned by registered endpoint callbacks to take full control of the
+ response. Usually, return values are converted to JSON; a RawResponse body
+ will be passed to the client as-is, allowing endpoint implementations to
+ issue invalid JSON.
+ """
+
+ pass
+
+
+class OpenIDProvider(threading.Thread):
+ """
+ A thread that runs a mock OpenID provider server.
+ """
+
+ def __init__(self, *, port):
+ super().__init__()
+
+ self.exception = None
+
+ addr = ("", port)
+ self.server = self._Server(addr, self._Handler)
+
+ # TODO: allow HTTPS only, somehow
+ oauth = self._OAuthState()
+ oauth.host = f"localhost:{port}"
+ oauth.issuer = f"http://localhost:{port}"
+
+ # The following endpoints are required to be advertised by providers,
+ # even though our chosen client implementation does not actually make
+ # use of them.
+ oauth.register_endpoint(
+ "authorization_endpoint", "POST", "/authorize", self._authorization_handler
+ )
+ oauth.register_endpoint("jwks_uri", "GET", "/keys", self._jwks_handler)
+
+ self.server.oauth = oauth
+
+ def run(self):
+ try:
+ # XXX socketserver.serve_forever() has a serious architectural
+ # issue: its select loop wakes up every `poll_interval` seconds to
+ # see if the server is shutting down. The default, 500 ms, only lets
+ # us run two tests every second. But the faster we go, the more CPU
+ # we burn unnecessarily...
+ self.server.serve_forever(poll_interval=0.01)
+ except Exception as e:
+ self.exception = e
+
+ def stop(self, timeout=BLOCKING_TIMEOUT):
+ """
+ Shuts down the server and joins its thread. Raises an exception if the
+ thread could not be joined, or if it threw an exception itself. Must
+ only be called once, after start().
+ """
+ self.server.shutdown()
+ self.join(timeout)
+
+ if self.is_alive():
+ raise TimeoutError("client thread did not handshake within the timeout")
+ elif self.exception:
+ e = self.exception
+ raise e
+
+ class _OAuthState(object):
+ def __init__(self):
+ self.endpoint_paths = {}
+ self._endpoints = {}
+
+ # Provide a standard discovery document by default; tests can
+ # override it.
+ self.register_endpoint(
+ None,
+ "GET",
+ "/.well-known/openid-configuration",
+ self._default_discovery_handler,
+ )
+
+ def register_endpoint(self, name, method, path, func):
+ if method not in self._endpoints:
+ self._endpoints[method] = {}
+
+ self._endpoints[method][path] = func
+
+ if name is not None:
+ self.endpoint_paths[name] = path
+
+ def endpoint(self, method, path):
+ if method not in self._endpoints:
+ return None
+
+ return self._endpoints[method].get(path)
+
+ def _default_discovery_handler(self, headers, params):
+ doc = {
+ "issuer": self.issuer,
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ "grant_types_supported": [
+ "urn:ietf:params:oauth:grant-type:device_code"
+ ],
+ }
+
+ for name, path in self.endpoint_paths.items():
+ doc[name] = self.issuer + path
+
+ return 200, doc
+
+ class _Server(http.server.HTTPServer):
+ def handle_error(self, request, addr):
+ self.shutdown_request(request)
+ raise
+
+ @staticmethod
+ def _jwks_handler(headers, params):
+ return 200, {"keys": []}
+
+ @staticmethod
+ def _authorization_handler(headers, params):
+ # We don't actually want this to be called during these tests -- we
+ # should be using the device authorization endpoint instead.
+ assert (
+ False
+ ), "authorization handler called instead of device authorization handler"
+
+ class _Handler(http.server.BaseHTTPRequestHandler):
+ timeout = BLOCKING_TIMEOUT
+
+ def _handle(self, *, params=None, handler=None):
+ oauth = self.server.oauth
+ assert self.headers["Host"] == oauth.host
+
+ if handler is None:
+ handler = oauth.endpoint(self.command, self.path)
+ assert (
+ handler is not None
+ ), f"no registered endpoint for {self.command} {self.path}"
+
+ result = handler(self.headers, params)
+
+ if len(result) == 2:
+ headers = {"Content-Type": "application/json"}
+ code, resp = result
+ else:
+ code, headers, resp = result
+
+ self.send_response(code)
+ for h, v in headers.items():
+ self.send_header(h, v)
+ self.end_headers()
+
+ if resp is not None:
+ if not isinstance(resp, RawResponse):
+ resp = json.dumps(resp)
+ resp = resp.encode("utf-8")
+ self.wfile.write(resp)
+
+ self.close_connection = True
+
+ def do_GET(self):
+ self._handle()
+
+ def _request_body(self):
+ length = self.headers["Content-Length"]
+
+ # Handle only an explicit content-length.
+ assert length is not None
+ length = int(length)
+
+ return self.rfile.read(length).decode("utf-8")
+
+ def do_POST(self):
+ assert self.headers["Content-Type"] == "application/x-www-form-urlencoded"
+
+ body = self._request_body()
+ params = urllib.parse.parse_qs(body)
+
+ self._handle(params=params)
+
+
+@pytest.fixture
+def openid_provider(unused_tcp_port_factory):
+ """
+ A fixture that returns the OAuth state of a running OpenID provider server. The
+ server will be stopped when the fixture is torn down.
+ """
+ thread = OpenIDProvider(port=unused_tcp_port_factory())
+ thread.start()
+
+ try:
+ yield thread.server.oauth
+ finally:
+ thread.stop()
+
+
+#
+# PQAuthDataHook implementation, matching libpq.h
+#
+
+
+PQAUTHDATA_PROMPT_OAUTH_DEVICE = 0
+PQAUTHDATA_OAUTH_BEARER_TOKEN = 1
+
+PGRES_POLLING_FAILED = 0
+PGRES_POLLING_READING = 1
+PGRES_POLLING_WRITING = 2
+PGRES_POLLING_OK = 3
+
+
+class PQPromptOAuthDevice(ctypes.Structure):
+ _fields_ = [
+ ("verification_uri", ctypes.c_char_p),
+ ("user_code", ctypes.c_char_p),
+ ]
+
+
+class PQOAuthBearerRequest(ctypes.Structure):
+ pass
+
+
+PQOAuthBearerRequest._fields_ = [
+ ("openid_configuration", ctypes.c_char_p),
+ ("scope", ctypes.c_char_p),
+ (
+ "async_",
+ ctypes.CFUNCTYPE(
+ ctypes.c_int,
+ ctypes.c_void_p,
+ ctypes.POINTER(PQOAuthBearerRequest),
+ ctypes.POINTER(ctypes.c_int),
+ ),
+ ),
+ (
+ "cleanup",
+ ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(PQOAuthBearerRequest)),
+ ),
+ ("token", ctypes.c_char_p),
+ ("user", ctypes.c_void_p),
+]
+
+
+@pytest.fixture
+def auth_data_cb():
+ """
+ Tracks calls to the libpq authdata hook. The yielded object contains a calls
+ member that records the data sent to the hook. If a test needs to perform
+ custom actions during a call, it can set the yielded object's impl callback;
+ beware that the callback takes place on a different thread.
+
+ This is done differently from the other callback implementations on purpose.
+ For the others, we can declare test-specific callbacks and have them perform
+ direct assertions on the data they receive. But that won't work for a C
+ callback, because there's no way for us to bubble up the assertion through
+ libpq. Instead, this mock-style approach is taken, where we just record the
+ calls and let the test examine them later.
+ """
+
+ class _Call:
+ pass
+
+ class _cb(object):
+ def __init__(self):
+ self.calls = []
+
+ cb = _cb()
+ cb.impl = None
+
+ # The callback will occur on a different thread, so protect the cb object.
+ cb_lock = threading.Lock()
+
+ @ctypes.CFUNCTYPE(ctypes.c_int, ctypes.c_byte, ctypes.c_void_p, ctypes.c_void_p)
+ def auth_data_cb(typ, pgconn, data):
+ handle_by_default = 0 # does an implementation have to be provided?
+
+ if typ == PQAUTHDATA_PROMPT_OAUTH_DEVICE:
+ cls = PQPromptOAuthDevice
+ handle_by_default = 1
+ elif typ == PQAUTHDATA_OAUTH_BEARER_TOKEN:
+ cls = PQOAuthBearerRequest
+ else:
+ return 0
+
+ call = _Call()
+ call.type = typ
+
+ # The lifetime of the underlying data being pointed to doesn't
+ # necessarily match the lifetime of the Python object, so we can't
+ # reference a Structure's fields after returning. Explicitly copy the
+ # contents over, field by field.
+ data = ctypes.cast(data, ctypes.POINTER(cls))
+ for name, _ in cls._fields_:
+ setattr(call, name, getattr(data.contents, name))
+
+ with cb_lock:
+ cb.calls.append(call)
+
+ if cb.impl:
+ # Pass control back to the test.
+ try:
+ return cb.impl(typ, pgconn, data.contents)
+ except Exception:
+ # This can't escape into the C stack, but we can fail the flow
+ # and hope the traceback gives us enough detail.
+ logging.error(
+ "Exception during authdata hook callback:\n"
+ + traceback.format_exc()
+ )
+ return -1
+
+ return handle_by_default
+
+ libpq.PQsetAuthDataHook(auth_data_cb)
+ try:
+ yield cb
+ finally:
+ # The callback is about to go out of scope, so make sure libpq is
+ # disconnected from it. (We wouldn't want to accidentally influence
+ # later tests anyway.)
+ libpq.PQsetAuthDataHook(None)
+
+
+@pytest.mark.parametrize("secret", [None, "", "hunter2"])
+@pytest.mark.parametrize("scope", [None, "", "openid email"])
+@pytest.mark.parametrize("retries", [0, 1])
+@pytest.mark.parametrize(
+ "asynchronous",
+ [
+ pytest.param(False, id="synchronous"),
+ pytest.param(True, id="asynchronous"),
+ ],
+)
+def test_oauth_with_explicit_issuer(
+ accept, openid_provider, asynchronous, retries, scope, secret, auth_data_cb
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ oauth_client_secret=secret,
+ oauth_scope=scope,
+ async_=asynchronous,
+ )
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+ verification_url = "https://example.com/device"
+
+ access_token = secrets.token_urlsafe()
+
+ def check_client_authn(headers, params):
+ if not secret:
+ assert params["client_id"] == [client_id]
+ return
+
+ # Require the client to use Basic authn; request-body credentials are
+ # NOT RECOMMENDED (RFC 6749, Sec. 2.3.1).
+ assert "Authorization" in headers
+
+ method, creds = headers["Authorization"].split()
+ assert method == "Basic"
+
+ expected = f"{client_id}:{secret}"
+ assert base64.b64decode(creds) == expected.encode("ascii")
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ check_client_authn(headers, params)
+
+ if scope:
+ assert params["scope"] == [scope]
+ else:
+ assert "scope" not in params
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": verification_url,
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ attempts = 0
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ check_client_authn(headers, params)
+
+ assert params["grant_type"] == ["urn:ietf:params:oauth:grant-type:device_code"]
+ assert params["device_code"] == [device_code]
+
+ now = time.monotonic()
+
+ with retry_lock:
+ nonlocal attempts
+
+ # If the test wants to force the client to retry, return an
+ # authorization_pending response and decrement the retry count.
+ if attempts < retries:
+ attempts += 1
+ return 400, {"error": "authorization_pending"}
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+ if retries:
+ # Finally, make sure that the client prompted the user once with the
+ # expected authorization URL and user code.
+ assert len(auth_data_cb.calls) == 2
+
+ # First call should have been for a custom flow, which we ignored.
+ assert auth_data_cb.calls[0].type == PQAUTHDATA_OAUTH_BEARER_TOKEN
+
+ # Second call is for our user prompt.
+ call = auth_data_cb.calls[1]
+ assert call.type == PQAUTHDATA_PROMPT_OAUTH_DEVICE
+ assert call.verification_uri.decode() == verification_url
+ assert call.user_code.decode() == user_code
+
+
+def expect_disconnected_handshake(sock):
+ """
+ Helper for any tests that expect the client to disconnect immediately after
+ being sent the OAUTHBEARER SASL method. Generally speaking, this requires
+ the client to have an oauth_issuer set so that it doesn't try to go through
+ discovery.
+ """
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake.
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"OAUTHBEARER", b""],
+ )
+
+ # The client should disconnect at this point.
+ assert not conn.read()
+
+
+def test_oauth_requires_client_id(accept, openid_provider):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ # Do not set a client ID; this should cause a client error after the
+ # server asks for OAUTHBEARER and the client tries to contact the
+ # issuer.
+ )
+
+ expect_disconnected_handshake(sock)
+
+ expected_error = "no oauth_client_id is set"
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.slow
+@pytest.mark.parametrize("error_code", ["authorization_pending", "slow_down"])
+@pytest.mark.parametrize("retries", [1, 2])
+def test_oauth_retry_interval(accept, openid_provider, retries, error_code):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id="some-id",
+ )
+
+ expected_retry_interval = 1
+ access_token = secrets.token_urlsafe()
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ resp = {
+ "device_code": "my-device-code",
+ "user_code": "my-user-code",
+ "interval": expected_retry_interval,
+ "verification_uri": "https://example.com",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ attempts = 0
+ last_retry = None
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ now = time.monotonic()
+
+ with retry_lock:
+ nonlocal attempts, last_retry, expected_retry_interval
+
+ # Make sure the retry interval is being respected by the client.
+ if last_retry is not None:
+ interval = now - last_retry
+ assert interval >= expected_retry_interval
+
+ last_retry = now
+
+ # If the test wants to force the client to retry, return the desired
+ # error response and decrement the retry count.
+ if attempts < retries:
+ attempts += 1
+
+ # A slow_down code requires the client to additionally increase
+ # its interval by five seconds.
+ if error_code == "slow_down":
+ expected_retry_interval += 5
+
+ return 400, {"error": error_code}
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+@pytest.fixture
+def self_pipe():
+ """
+ Yields a pipe fd pair.
+ """
+
+ class _Pipe:
+ pass
+
+ p = _Pipe()
+ p.readfd, p.writefd = os.pipe()
+
+ try:
+ yield p
+ finally:
+ os.close(p.readfd)
+ os.close(p.writefd)
+
+
+@pytest.mark.parametrize("scope", [None, "", "openid email"])
+@pytest.mark.parametrize(
+ "retries",
+ [
+ -1, # no async callback
+ 0, # async callback immediately returns token
+ 1, # async callback waits on altsock once
+ 2, # async callback waits on altsock twice
+ ],
+)
+@pytest.mark.parametrize(
+ "asynchronous",
+ [
+ pytest.param(False, id="synchronous"),
+ pytest.param(True, id="asynchronous"),
+ ],
+)
+def test_user_defined_flow(
+ accept, auth_data_cb, self_pipe, scope, retries, asynchronous
+):
+ issuer = "http://localhost"
+ discovery_uri = issuer + "/.well-known/openid-configuration"
+ access_token = secrets.token_urlsafe()
+
+ sock, _ = accept(
+ oauth_issuer=issuer,
+ oauth_client_id="some-id",
+ oauth_scope=scope,
+ async_=asynchronous,
+ )
+
+ # Track callbacks.
+ attempts = 0
+ wakeup_called = False
+ cleanup_calls = 0
+ lock = threading.Lock()
+
+ def wakeup():
+ """Writes a byte to the wakeup pipe."""
+ nonlocal wakeup_called
+ with lock:
+ wakeup_called = True
+ os.write(self_pipe.writefd, b"\0")
+
+ def get_token(pgconn, request, p_altsock):
+ """
+ Async token callback. While attempts < retries, libpq will be instructed
+ to wait on the self_pipe. When attempts == retries, the token will be
+ set.
+
+ Note that assertions and exceptions raised here are allowed but not very
+ helpful, since they can't bubble through the libpq stack to be collected
+ by the test suite. Try not to rely too heavily on them.
+ """
+ # Make sure libpq passed our user data through.
+ assert request.user == 42
+
+ with lock:
+ nonlocal attempts, wakeup_called
+
+ if attempts:
+ # If we've already started the timer, we shouldn't get a
+ # call back before it trips.
+ assert wakeup_called, "authdata hook was called before the timer"
+
+ # Drain the wakeup byte.
+ os.read(self_pipe.readfd, 1)
+
+ if attempts < retries:
+ attempts += 1
+
+ # Wake up the client in a little bit of time.
+ wakeup_called = False
+ threading.Timer(0.1, wakeup).start()
+
+ # Tell libpq to wait on the other end of the wakeup pipe.
+ p_altsock[0] = self_pipe.readfd
+ return PGRES_POLLING_READING
+
+ # Done!
+ request.token = access_token.encode()
+ return PGRES_POLLING_OK
+
+ @ctypes.CFUNCTYPE(
+ ctypes.c_int,
+ ctypes.c_void_p,
+ ctypes.POINTER(PQOAuthBearerRequest),
+ ctypes.POINTER(ctypes.c_int),
+ )
+ def get_token_wrapper(pgconn, p_request, p_altsock):
+ """
+ Translation layer between C and Python for the async callback.
+ Assertions and exceptions will be swallowed at the boundary, so make
+ sure they don't escape here.
+ """
+ try:
+ return get_token(pgconn, p_request.contents, p_altsock)
+ except Exception:
+ logging.error("Exception during async callback:\n" + traceback.format_exc())
+ return PGRES_POLLING_FAILED
+
+ @ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(PQOAuthBearerRequest))
+ def cleanup(pgconn, p_request):
+ """
+ Should be called exactly once per connection.
+ """
+ nonlocal cleanup_calls
+ with lock:
+ cleanup_calls += 1
+
+ def bearer_hook(typ, pgconn, request):
+ """
+ Implementation of the PQAuthDataHook, which either sets up an async
+ callback or returns the token directly, depending on the value of
+ retries.
+
+ As above, try not to rely too much on assertions/exceptions here.
+ """
+ assert typ == PQAUTHDATA_OAUTH_BEARER_TOKEN
+ request.cleanup = cleanup
+
+ if retries < 0:
+ # Special case: return a token immediately without a callback.
+ request.token = access_token.encode()
+ return 1
+
+ # Tell libpq to call us back.
+ request.async_ = get_token_wrapper
+ request.user = ctypes.c_void_p(42) # will be checked in the callback
+ return 1
+
+ auth_data_cb.impl = bearer_hook
+
+ # Now drive the server side.
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in our custom callback
+ # being invoked to fetch the token.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+ # Check the data provided to the hook.
+ assert len(auth_data_cb.calls) == 1
+
+ call = auth_data_cb.calls[0]
+ assert call.type == PQAUTHDATA_OAUTH_BEARER_TOKEN
+ assert call.openid_configuration.decode() == discovery_uri
+ assert call.scope == (None if scope is None else scope.encode())
+
+ # Make sure we cleaned up after ourselves.
+ assert cleanup_calls == 1
+
+
+def alt_patterns(*patterns):
+ """
+ Just combines multiple alternative regexes into one. It's not very efficient
+ but IMO it's easier to read and maintain.
+ """
+ pat = ""
+
+ for p in patterns:
+ if pat:
+ pat += "|"
+ pat += f"({p})"
+
+ return pat
+
+
+@pytest.mark.parametrize(
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
+ (
+ 400,
+ {
+ "error": "invalid_client",
+ "error_description": "client authentication failed",
+ },
+ ),
+ r"failed to obtain device authorization: client authentication failed \(invalid_client\)",
+ id="authentication failure with description",
+ ),
+ pytest.param(
+ (400, {"error": "invalid_request"}),
+ r"failed to obtain device authorization: \(invalid_request\)",
+ id="invalid request without description",
+ ),
+ pytest.param(
+ (400, {}),
+ r'failed to parse token error response: field "error" is missing',
+ id="broken error response",
+ ),
+ pytest.param(
+ (200, RawResponse(r'{ "interval": 3.5.8 }')),
+ r"failed to parse device authorization: Token .* is invalid",
+ id="non-numeric interval",
+ ),
+ pytest.param(
+ (200, RawResponse(r'{ "interval": 08 }')),
+ r"failed to parse device authorization: Token .* is invalid",
+ id="invalid numeric interval",
+ ),
+ ],
+)
+def test_oauth_device_authorization_failures(
+ accept, openid_provider, failure_mode, error_pattern
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ )
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ return failure_mode
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert False, "token endpoint was invoked unexpectedly"
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+Missing = object() # sentinel for test_oauth_device_authorization_bad_json()
+
+
+@pytest.mark.parametrize(
+ "bad_value",
+ [
+ pytest.param({"device_code": 3}, id="object"),
+ pytest.param([1, 2, 3], id="array"),
+ pytest.param("some string", id="string"),
+ pytest.param(4, id="numeric"),
+ pytest.param(False, id="boolean"),
+ pytest.param(None, id="null"),
+ pytest.param(Missing, id="missing"),
+ ],
+)
+@pytest.mark.parametrize(
+ "field_name,ok_type,required",
+ [
+ ("device_code", str, True),
+ ("user_code", str, True),
+ ("verification_uri", str, True),
+ ("interval", int, False),
+ ],
+)
+def test_oauth_device_authorization_bad_json_schema(
+ accept, openid_provider, field_name, ok_type, required, bad_value
+):
+ # To make the test matrix easy, just skip the tests that aren't actually
+ # interesting (field of the correct type, missing optional field).
+ if bad_value is Missing and not required:
+ pytest.skip("not interesting: optional field")
+ elif type(bad_value) == ok_type: # not isinstance(), because bool is an int
+ pytest.skip("not interesting: correct type")
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=secrets.token_hex(),
+ )
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ # Begin with an acceptable base response...
+ resp = {
+ "device_code": "my-device-code",
+ "user_code": "my-user-code",
+ "interval": 0,
+ "verification_uri": "https://example.com",
+ "expires_in": 5,
+ }
+
+ # ...then tweak it so the client fails.
+ if bad_value is Missing:
+ del resp[field_name]
+ else:
+ resp[field_name] = bad_value
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert False, "token endpoint was invoked unexpectedly"
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ if bad_value is Missing:
+ error_pattern = f'field "{field_name}" is missing'
+ elif ok_type == str:
+ error_pattern = f'field "{field_name}" must be a string'
+ elif ok_type == int:
+ error_pattern = f'field "{field_name}" must be a number'
+ else:
+ assert False, "update error_pattern for new failure mode"
+
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
+ (
+ 400,
+ {
+ "error": "expired_token",
+ "error_description": "the device code has expired",
+ },
+ ),
+ r"failed to obtain access token: the device code has expired \(expired_token\)",
+ id="expired token with description",
+ ),
+ pytest.param(
+ (400, {"error": "access_denied"}),
+ r"failed to obtain access token: \(access_denied\)",
+ id="access denied without description",
+ ),
+ pytest.param(
+ (400, {}),
+ r'failed to parse token error response: field "error" is missing',
+ id="empty error response",
+ ),
+ pytest.param(
+ (200, {}, {}),
+ r"failed to parse access token response: no content type was provided",
+ id="missing content type",
+ ),
+ pytest.param(
+ (200, {"Content-Type": "text/plain"}, {}),
+ r"failed to parse access token response: unexpected content type",
+ id="wrong content type",
+ ),
+ ],
+)
+@pytest.mark.parametrize("retries", [0, 1])
+def test_oauth_token_failures(
+ accept, openid_provider, retries, failure_mode, error_pattern
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ )
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ assert params["client_id"] == [client_id]
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": "https://example.com/device",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ retry_lock = threading.Lock()
+ final_sent = False
+
+ def token_endpoint(headers, params):
+ with retry_lock:
+ nonlocal retries, final_sent
+
+ # If the test wants to force the client to retry, return an
+ # authorization_pending response and decrement the retry count.
+ if retries > 0:
+ retries -= 1
+ return 400, {"error": "authorization_pending"}
+
+ # We should only return our failure_mode response once; any further
+ # requests indicate that the client isn't correctly bailing out.
+ assert not final_sent, "client continued after token error"
+
+ final_sent = True
+
+ return failure_mode
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "bad_value",
+ [
+ pytest.param({"device_code": 3}, id="object"),
+ pytest.param([1, 2, 3], id="array"),
+ pytest.param("some string", id="string"),
+ pytest.param(4, id="numeric"),
+ pytest.param(False, id="boolean"),
+ pytest.param(None, id="null"),
+ pytest.param(Missing, id="missing"),
+ ],
+)
+@pytest.mark.parametrize(
+ "field_name,ok_type,required",
+ [
+ ("access_token", str, True),
+ ("token_type", str, True),
+ ],
+)
+def test_oauth_token_bad_json_schema(
+ accept, openid_provider, field_name, ok_type, required, bad_value
+):
+ # To make the test matrix easy, just skip the tests that aren't actually
+ # interesting (field of the correct type, missing optional field).
+ if bad_value is Missing and not required:
+ pytest.skip("not interesting: optional field")
+ elif type(bad_value) == ok_type: # not isinstance(), because bool is an int
+ pytest.skip("not interesting: correct type")
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=secrets.token_hex(),
+ )
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ resp = {
+ "device_code": "my-device-code",
+ "user_code": "my-user-code",
+ "interval": 0,
+ "verification_uri": "https://example.com",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ # Begin with an acceptable base response...
+ resp = {
+ "access_token": secrets.token_urlsafe(),
+ "token_type": "bearer",
+ }
+
+ # ...then tweak it so the client fails.
+ if bad_value is Missing:
+ del resp[field_name]
+ else:
+ resp[field_name] = bad_value
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ error_pattern = "failed to parse access token response: "
+ if bad_value is Missing:
+ error_pattern += f'field "{field_name}" is missing'
+ elif ok_type == str:
+ error_pattern += f'field "{field_name}" must be a string'
+ elif ok_type == int:
+ error_pattern += f'field "{field_name}" must be a number'
+ else:
+ assert False, "update error_pattern for new failure mode"
+
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize("scope", [None, "openid email"])
+@pytest.mark.parametrize(
+ "base_response",
+ [
+ {"status": "invalid_token"},
+ {"extra_object": {"key": "value"}, "status": "invalid_token"},
+ {"extra_object": {"status": 1}, "status": "invalid_token"},
+ ],
+)
+def test_oauth_discovery(accept, openid_provider, base_response, scope):
+ sock, client = accept(oauth_client_id=secrets.token_hex())
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+ verification_url = "https://example.com/device"
+
+ access_token = secrets.token_urlsafe()
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ if scope:
+ assert params["scope"] == [scope]
+ else:
+ assert "scope" not in params
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": verification_url,
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert params["grant_type"] == ["urn:ietf:params:oauth:grant-type:device_code"]
+ assert params["device_code"] == [device_code]
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # For discovery, the client should send an empty auth header. See
+ # RFC 7628, Sec. 4.3.
+ auth = get_auth_value(initial)
+ assert auth == b""
+
+ # We will fail the first SASL exchange. First return a link to the
+ # discovery document, pointing to the test provider server.
+ resp = dict(base_response)
+
+ discovery_uri = f"{openid_provider.issuer}/.well-known/openid-configuration"
+ resp["openid-configuration"] = discovery_uri
+
+ if scope:
+ resp["scope"] = scope
+
+ resp = json.dumps(resp)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASLContinue,
+ body=resp.encode("ascii"),
+ )
+
+ # Per RFC, the client is required to send a dummy ^A response.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+ assert pkt.payload == b"\x01"
+
+ # Now fail the SASL exchange.
+ pq3.send(
+ conn,
+ pq3.types.ErrorResponse,
+ fields=[
+ b"SFATAL",
+ b"C28000",
+ b"Mdoesn't matter",
+ b"",
+ ],
+ )
+
+ # The client will connect to us a second time, using the parameters we sent
+ # it.
+ sock, _ = accept()
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+@pytest.mark.parametrize(
+ "response,expected_error",
+ [
+ pytest.param(
+ "abcde",
+ 'Token "abcde" is invalid',
+ id="bad JSON: invalid syntax",
+ ),
+ pytest.param(
+ '"abcde"',
+ "top-level element must be an object",
+ id="bad JSON: top-level element is a string",
+ ),
+ pytest.param(
+ "[]",
+ "top-level element must be an object",
+ id="bad JSON: top-level element is an array",
+ ),
+ pytest.param(
+ "{}",
+ "server sent error response without a status",
+ id="bad JSON: no status member",
+ ),
+ pytest.param(
+ '{ "status": null }',
+ 'field "status" must be a string',
+ id="bad JSON: null status member",
+ ),
+ pytest.param(
+ '{ "status": 0 }',
+ 'field "status" must be a string',
+ id="bad JSON: int status member",
+ ),
+ pytest.param(
+ '{ "status": [ "bad" ] }',
+ 'field "status" must be a string',
+ id="bad JSON: array status member",
+ ),
+ pytest.param(
+ '{ "status": { "bad": "bad" } }',
+ 'field "status" must be a string',
+ id="bad JSON: object status member",
+ ),
+ pytest.param(
+ '{ "nested": { "status": "bad" } }',
+ "server sent error response without a status",
+ id="bad JSON: nested status",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token" ',
+ "The input string ended unexpectedly",
+ id="bad JSON: unterminated object",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token" } { }',
+ 'Expected end of input, but found "{"',
+ id="bad JSON: trailing data",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "openid-configuration": 1 }',
+ 'field "openid-configuration" must be a string',
+ id="bad JSON: int openid-configuration member",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "openid-configuration": 1 }',
+ 'field "openid-configuration" must be a string',
+ id="bad JSON: int openid-configuration member",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "scope": 1 }',
+ 'field "scope" must be a string',
+ id="bad JSON: int scope member",
+ ),
+ ],
+)
+def test_oauth_discovery_server_error(accept, response, expected_error):
+ sock, client = accept(oauth_client_id=secrets.token_hex())
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # Fail the SASL exchange with an invalid JSON response.
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASLContinue,
+ body=response.encode("utf-8"),
+ )
+
+ # The client should disconnect, so the socket is closed here. (If
+ # the client doesn't disconnect, it will report a different error
+ # below and the test will fail.)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "bad_response,expected_error",
+ [
+ pytest.param(
+ (200, {"Content-Type": "text/plain"}, {}),
+ r'failed to parse OpenID discovery document: unexpected content type "text/plain"',
+ id="not JSON",
+ ),
+ pytest.param(
+ (200, {}, {}),
+ r"failed to parse OpenID discovery document: no content type was provided",
+ id="no Content-Type",
+ ),
+ pytest.param(
+ (204, {}, None),
+ r"failed to fetch OpenID discovery document: unexpected response code 204",
+ id="no content",
+ ),
+ pytest.param(
+ (301, {"Location": "https://localhost/"}, None),
+ r"failed to fetch OpenID discovery document: unexpected response code 301",
+ id="redirection",
+ ),
+ pytest.param(
+ (404, {}),
+ r"failed to fetch OpenID discovery document: unexpected response code 404",
+ id="not found",
+ ),
+ pytest.param(
+ (200, RawResponse("blah\x00blah")),
+ r"failed to parse OpenID discovery document: response contains embedded NULLs",
+ id="NULL bytes in document",
+ ),
+ pytest.param(
+ (200, 123),
+ r"failed to parse OpenID discovery document: top-level element must be an object",
+ id="scalar at top level",
+ ),
+ pytest.param(
+ (200, []),
+ r"failed to parse OpenID discovery document: top-level element must be an object",
+ id="array at top level",
+ ),
+ pytest.param(
+ (200, RawResponse("{")),
+ r"failed to parse OpenID discovery document.* input string ended unexpectedly",
+ id="unclosed object",
+ ),
+ pytest.param(
+ (200, RawResponse(r'{ "hello": ] }')),
+ r"failed to parse OpenID discovery document.* Expected JSON value",
+ id="bad array",
+ ),
+ pytest.param(
+ (200, {"issuer": 123}),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="non-string issuer",
+ ),
+ pytest.param(
+ (200, {"issuer": ["something"]}),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="issuer array",
+ ),
+ pytest.param(
+ (200, {"issuer": {}}),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="issuer object",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": 123}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="scalar grant types field",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": {}}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="object grant types field",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": [123]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="non-string grant types",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": ["something", 123]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="non-string grant types later in the list",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": ["something", {}]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="object grant types later in the list",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": ["something", ["something"]]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="embedded array grant types later in the list",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "grant_types_supported": ["something"],
+ "token_endpoint": "https://example.com/",
+ "issuer": 123,
+ },
+ ),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="non-string issuer after other valid fields",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "ignored": {"grant_types_supported": 123, "token_endpoint": 123},
+ "issuer": 123,
+ },
+ ),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="non-string issuer after other ignored fields",
+ ),
+ pytest.param(
+ (200, {"token_endpoint": "https://example.com/"}),
+ r'failed to parse OpenID discovery document: field "issuer" is missing',
+ id="missing issuer",
+ ),
+ pytest.param(
+ (200, {"issuer": "https://example.com/"}),
+ r'failed to parse OpenID discovery document: field "token_endpoint" is missing',
+ id="missing token endpoint",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "issuer": "https://example.com",
+ "token_endpoint": "https://example.com/token",
+ "device_authorization_endpoint": "https://example.com/dev",
+ },
+ ),
+ r'cannot run OAuth device authorization: issuer "https://example.com" does not support device code grants',
+ id="missing device code grants",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "issuer": "https://example.com",
+ "token_endpoint": "https://example.com/token",
+ "grant_types_supported": [
+ "urn:ietf:params:oauth:grant-type:device_code"
+ ],
+ },
+ ),
+ r'cannot run OAuth device authorization: issuer "https://example.com" does not provide a device authorization endpoint',
+ id="missing device_authorization_endpoint",
+ ),
+ #
+ # Exercise HTTP-level failures by breaking the protocol. Note that the
+ # error messages here are implementation-dependent.
+ #
+ pytest.param(
+ (1000, {}),
+ r"failed to fetch OpenID discovery document: Unsupported protocol \(.*\)",
+ id="invalid HTTP response code",
+ ),
+ pytest.param(
+ (200, {"Content-Length": -1}, {}),
+ r"failed to fetch OpenID discovery document: Weird server reply \(.*Content-Length.*\)",
+ id="bad HTTP Content-Length",
+ ),
+ ],
+)
+def test_oauth_discovery_provider_failure(
+ accept, openid_provider, bad_response, expected_error
+):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=secrets.token_hex(),
+ )
+
+ def failing_discovery_handler(headers, params):
+ return bad_response
+
+ openid_provider.register_endpoint(
+ None,
+ "GET",
+ "/.well-known/openid-configuration",
+ failing_discovery_handler,
+ )
+
+ expect_disconnected_handshake(sock)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "sasl_err,resp_type,resp_payload,expected_error",
+ [
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.ErrorResponse,
+ dict(
+ fields=[b"SFATAL", b"C28000", b"Mexpected error message", b""],
+ ),
+ "expected error message",
+ id="standard server error: invalid_request",
+ ),
+ pytest.param(
+ {"status": "invalid_token"},
+ pq3.types.ErrorResponse,
+ dict(
+ fields=[b"SFATAL", b"C28000", b"Mexpected error message", b""],
+ ),
+ "expected error message",
+ id="standard server error: invalid_token without discovery URI",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASLContinue, body=b""),
+ "server sent additional OAuth data",
+ id="broken server: additional challenge after error",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASLFinal),
+ "server sent additional OAuth data",
+ id="broken server: SASL success after error",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASL, body=[b"OAUTHBEARER", b""]),
+ "duplicate SASL authentication request",
+ id="broken server: SASL reinitialization after error",
+ ),
+ ],
+)
+def test_oauth_server_error(accept, sasl_err, resp_type, resp_payload, expected_error):
+ sock, client = accept()
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ start_oauth_handshake(conn)
+
+ # Ignore the client data. Return an error "challenge".
+ resp = json.dumps(sasl_err)
+ resp = resp.encode("utf-8")
+
+ pq3.send(
+ conn, pq3.types.AuthnRequest, type=pq3.authn.SASLContinue, body=resp
+ )
+
+ # Per RFC, the client is required to send a dummy ^A response.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+ assert pkt.payload == b"\x01"
+
+ # Now fail the SASL exchange (in either a valid way, or an invalid
+ # one, depending on the test).
+ pq3.send(conn, resp_type, **resp_payload)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
diff --git a/src/test/python/conftest.py b/src/test/python/conftest.py
new file mode 100644
index 0000000000..1a73865ee4
--- /dev/null
+++ b/src/test/python/conftest.py
@@ -0,0 +1,34 @@
+#
+# Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import os
+
+import pytest
+
+
+def pytest_addoption(parser):
+ """
+ Adds custom command line options to py.test. We add one to signal temporary
+ Postgres instance creation for the server tests.
+
+ Per pytest documentation, this must live in the top level test directory.
+ """
+ parser.addoption(
+ "--temp-instance",
+ metavar="DIR",
+ help="create a temporary Postgres instance in DIR",
+ )
+
+
+@pytest.fixture(scope="session", autouse=True)
+def _check_PG_TEST_EXTRA(request):
+ """
+ Automatically skips the whole suite if PG_TEST_EXTRA doesn't contain
+ 'python'. pytestmark doesn't seem to work in a top-level conftest.py, so
+ I've made this an autoused fixture instead.
+ """
+ extra_tests = os.getenv("PG_TEST_EXTRA", "").split()
+ if "python" not in extra_tests:
+ pytest.skip("Potentially unsafe test 'python' not enabled in PG_TEST_EXTRA")
diff --git a/src/test/python/meson.build b/src/test/python/meson.build
new file mode 100644
index 0000000000..045526a43a
--- /dev/null
+++ b/src/test/python/meson.build
@@ -0,0 +1,46 @@
+# Copyright (c) 2023, PostgreSQL Global Development Group
+
+subdir('server')
+
+pytest_env = {
+ 'with_oauth': oauth_library,
+
+ # Point to the default database; the tests will create their own databases as
+ # needed.
+ 'PGDATABASE': 'postgres',
+
+ # Avoid the need for a Rust compiler on platforms without prebuilt wheels for
+ # pyca/cryptography.
+ 'CRYPTOGRAPHY_DONT_BUILD_RUST': '1',
+}
+
+# Some modules (psycopg2) need OpenSSL at compile time; for platforms where we
+# might have multiple implementations installed (macOS+brew), try to use the
+# same one that libpq is using.
+if ssl.found()
+ pytest_incdir = ssl.get_variable(pkgconfig: 'includedir', default_value: '')
+ if pytest_incdir != ''
+ pytest_env += { 'CPPFLAGS': '-I@0@'.format(pytest_incdir) }
+ endif
+
+ pytest_libdir = ssl.get_variable(pkgconfig: 'libdir', default_value: '')
+ if pytest_libdir != ''
+ pytest_env += { 'LDFLAGS': '-L@0@'.format(pytest_libdir) }
+ endif
+endif
+
+tests += {
+ 'name': 'python',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'pytest': {
+ 'requirements': meson.current_source_dir() / 'requirements.txt',
+ 'tests': [
+ './client',
+ './server',
+ './test_internals.py',
+ './test_pq3.py',
+ ],
+ 'env': pytest_env,
+ },
+}
diff --git a/src/test/python/pq3.py b/src/test/python/pq3.py
new file mode 100644
index 0000000000..a2d2812f0e
--- /dev/null
+++ b/src/test/python/pq3.py
@@ -0,0 +1,732 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import getpass
+import io
+import os
+import platform
+import ssl
+import sys
+import textwrap
+
+from construct import *
+
+import tls
+
+
+def protocol(major, minor):
+ """
+ Returns the protocol version, in integer format, corresponding to the given
+ major and minor version numbers.
+ """
+ return (major << 16) | minor
+
+
+# Startup
+
+StringList = GreedyRange(NullTerminated(GreedyBytes))
+
+
+class KeyValueAdapter(Adapter):
+ """
+ Turns a key-value store into a null-terminated list of null-terminated
+ strings, as presented on the wire in the startup packet.
+ """
+
+ def _encode(self, obj, context, path):
+ if isinstance(obj, list):
+ return obj
+
+ l = []
+
+ for k, v in obj.items():
+ if isinstance(k, str):
+ k = k.encode("utf-8")
+ l.append(k)
+
+ if isinstance(v, str):
+ v = v.encode("utf-8")
+ l.append(v)
+
+ l.append(b"")
+ return l
+
+ def _decode(self, obj, context, path):
+ # TODO: turn a list back into a dict
+ return obj
+
+
+KeyValues = KeyValueAdapter(StringList)
+
+_startup_payload = Switch(
+ this.proto,
+ {
+ protocol(3, 0): KeyValues,
+ },
+ default=GreedyBytes,
+)
+
+
+def _default_protocol(this):
+ try:
+ if isinstance(this.payload, (list, dict)):
+ return protocol(3, 0)
+ except AttributeError:
+ pass # no payload passed during build
+
+ return 0
+
+
+def _startup_payload_len(this):
+ """
+ The payload field has a fixed size based on the length of the packet. But
+ if the caller hasn't supplied an explicit length at build time, we have to
+ build the payload to figure out how long it is, which requires us to know
+ the length first... This function exists solely to break the cycle.
+ """
+ assert this._building, "_startup_payload_len() cannot be called during parsing"
+
+ try:
+ payload = this.payload
+ except AttributeError:
+ return 0 # no payload
+
+ if isinstance(payload, bytes):
+ # already serialized; just use the given length
+ return len(payload)
+
+ try:
+ proto = this.proto
+ except AttributeError:
+ proto = _default_protocol(this)
+
+ data = _startup_payload.build(payload, proto=proto)
+ return len(data)
+
+
+Startup = Struct(
+ "len" / Default(Int32sb, lambda this: _startup_payload_len(this) + 8),
+ "proto" / Default(Hex(Int32sb), _default_protocol),
+ "payload" / FixedSized(this.len - 8, Default(_startup_payload, b"")),
+)
+
+# Pq3
+
+
+# Adapted from construct.core.EnumIntegerString
+class EnumNamedByte:
+ def __init__(self, val, name):
+ self._val = val
+ self._name = name
+
+ def __int__(self):
+ return ord(self._val)
+
+ def __str__(self):
+ return "(enum) %s %r" % (self._name, self._val)
+
+ def __repr__(self):
+ return "EnumNamedByte(%r)" % self._val
+
+ def __eq__(self, other):
+ if isinstance(other, EnumNamedByte):
+ other = other._val
+ if not isinstance(other, bytes):
+ return NotImplemented
+
+ return self._val == other
+
+ def __hash__(self):
+ return hash(self._val)
+
+
+# Adapted from construct.core.Enum
+class ByteEnum(Adapter):
+ def __init__(self, **mapping):
+ super(ByteEnum, self).__init__(Byte)
+ self.namemapping = {k: EnumNamedByte(v, k) for k, v in mapping.items()}
+ self.decmapping = {v: EnumNamedByte(v, k) for k, v in mapping.items()}
+
+ def __getattr__(self, name):
+ if name in self.namemapping:
+ return self.decmapping[self.namemapping[name]]
+ raise AttributeError
+
+ def _decode(self, obj, context, path):
+ b = bytes([obj])
+ try:
+ return self.decmapping[b]
+ except KeyError:
+ return EnumNamedByte(b, "(unknown)")
+
+ def _encode(self, obj, context, path):
+ if isinstance(obj, int):
+ return obj
+ elif isinstance(obj, bytes):
+ return ord(obj)
+ return int(obj)
+
+
+types = ByteEnum(
+ ErrorResponse=b"E",
+ ReadyForQuery=b"Z",
+ Query=b"Q",
+ EmptyQueryResponse=b"I",
+ AuthnRequest=b"R",
+ PasswordMessage=b"p",
+ BackendKeyData=b"K",
+ CommandComplete=b"C",
+ ParameterStatus=b"S",
+ DataRow=b"D",
+ Terminate=b"X",
+)
+
+
+authn = Enum(
+ Int32ub,
+ OK=0,
+ SASL=10,
+ SASLContinue=11,
+ SASLFinal=12,
+)
+
+
+_authn_body = Switch(
+ this.type,
+ {
+ authn.OK: Terminated,
+ authn.SASL: StringList,
+ },
+ default=GreedyBytes,
+)
+
+
+def _data_len(this):
+ assert this._building, "_data_len() cannot be called during parsing"
+
+ if not hasattr(this, "data") or this.data is None:
+ return -1
+
+ return len(this.data)
+
+
+# The protocol reuses the PasswordMessage for several authentication response
+# types, and there's no good way to figure out which is which without keeping
+# state for the entire stream. So this is a separate Construct that can be
+# explicitly parsed/built by code that knows it's needed.
+SASLInitialResponse = Struct(
+ "name" / NullTerminated(GreedyBytes),
+ "len" / Default(Int32sb, lambda this: _data_len(this)),
+ "data"
+ / IfThenElse(
+ # Allow tests to explicitly pass an incorrect length during testing, by
+ # not enforcing a FixedSized during build. (The len calculation above
+ # defaults to the correct size.)
+ this._building,
+ Optional(GreedyBytes),
+ If(this.len != -1, Default(FixedSized(this.len, GreedyBytes), b"")),
+ ),
+ Terminated, # make sure the entire response is consumed
+)
+
+
+_column = FocusedSeq(
+ "data",
+ "len" / Default(Int32sb, lambda this: _data_len(this)),
+ "data" / If(this.len != -1, FixedSized(this.len, GreedyBytes)),
+)
+
+
+_payload_map = {
+ types.ErrorResponse: Struct("fields" / StringList),
+ types.ReadyForQuery: Struct("status" / Bytes(1)),
+ types.Query: Struct("query" / NullTerminated(GreedyBytes)),
+ types.EmptyQueryResponse: Terminated,
+ types.AuthnRequest: Struct("type" / authn, "body" / Default(_authn_body, b"")),
+ types.BackendKeyData: Struct("pid" / Int32ub, "key" / Hex(Int32ub)),
+ types.CommandComplete: Struct("tag" / NullTerminated(GreedyBytes)),
+ types.ParameterStatus: Struct(
+ "name" / NullTerminated(GreedyBytes), "value" / NullTerminated(GreedyBytes)
+ ),
+ types.DataRow: Struct("columns" / Default(PrefixedArray(Int16sb, _column), b"")),
+ types.Terminate: Terminated,
+}
+
+
+_payload = FocusedSeq(
+ "_payload",
+ "_payload"
+ / Switch(
+ this._.type,
+ _payload_map,
+ default=GreedyBytes,
+ ),
+ Terminated, # make sure every payload consumes the entire packet
+)
+
+
+def _payload_len(this):
+ """
+ See _startup_payload_len() for an explanation.
+ """
+ assert this._building, "_payload_len() cannot be called during parsing"
+
+ try:
+ payload = this.payload
+ except AttributeError:
+ return 0 # no payload
+
+ if isinstance(payload, bytes):
+ # already serialized; just use the given length
+ return len(payload)
+
+ data = _payload.build(payload, type=this.type)
+ return len(data)
+
+
+Pq3 = Struct(
+ "type" / types,
+ "len" / Default(Int32ub, lambda this: _payload_len(this) + 4),
+ "payload" / FixedSized(this.len - 4, Default(_payload, b"")),
+)
+
+
+# Environment
+
+
+def pghost():
+ return os.environ.get("PGHOST", default="localhost")
+
+
+def pgport():
+ return int(os.environ.get("PGPORT", default=5432))
+
+
+def pguser():
+ try:
+ return os.environ["PGUSER"]
+ except KeyError:
+ if platform.system() == "Windows":
+ # libpq defaults to GetUserName() on Windows.
+ return os.getlogin()
+ return getpass.getuser()
+
+
+def pgdatabase():
+ return os.environ.get("PGDATABASE", default="postgres")
+
+
+# Connections
+
+
+def _hexdump_translation_map():
+ """
+ For hexdumps. Translates any unprintable or non-ASCII bytes into '.'.
+ """
+ input = bytearray()
+
+ for i in range(128):
+ c = chr(i)
+
+ if not c.isprintable():
+ input += bytes([i])
+
+ input += bytes(range(128, 256))
+
+ return bytes.maketrans(input, b"." * len(input))
+
+
+class _DebugStream(object):
+ """
+ Wraps a file-like object and adds hexdumps of the read and write data. Call
+ end_packet() on a _DebugStream to write the accumulated hexdumps to the
+ output stream, along with the packet that was sent.
+ """
+
+ _translation_map = _hexdump_translation_map()
+
+ def __init__(self, stream, out=sys.stdout):
+ """
+ Creates a new _DebugStream wrapping the given stream (which must have
+ been created by wrap()). All attributes not provided by the _DebugStream
+ are delegated to the wrapped stream. out is the text stream to which
+ hexdumps are written.
+ """
+ self.raw = stream
+ self._out = out
+ self._rbuf = io.BytesIO()
+ self._wbuf = io.BytesIO()
+
+ def __getattr__(self, name):
+ return getattr(self.raw, name)
+
+ def __setattr__(self, name, value):
+ if name in ("raw", "_out", "_rbuf", "_wbuf"):
+ return object.__setattr__(self, name, value)
+
+ setattr(self.raw, name, value)
+
+ def read(self, *args, **kwargs):
+ buf = self.raw.read(*args, **kwargs)
+
+ self._rbuf.write(buf)
+ return buf
+
+ def write(self, b):
+ self._wbuf.write(b)
+ return self.raw.write(b)
+
+ def recv(self, *args):
+ buf = self.raw.recv(*args)
+
+ self._rbuf.write(buf)
+ return buf
+
+ def _flush(self, buf, prefix):
+ width = 16
+ hexwidth = width * 3 - 1
+
+ count = 0
+ buf.seek(0)
+
+ while True:
+ line = buf.read(16)
+
+ if not line:
+ if count:
+ self._out.write("\n") # separate the output block with a newline
+ return
+
+ self._out.write("%s %04X:\t" % (prefix, count))
+ self._out.write("%*s\t" % (-hexwidth, line.hex(" ")))
+ self._out.write(line.translate(self._translation_map).decode("ascii"))
+ self._out.write("\n")
+
+ count += 16
+
+ def print_debug(self, obj, *, prefix=""):
+ contents = ""
+ if obj is not None:
+ contents = str(obj)
+
+ for line in contents.splitlines():
+ self._out.write("%s%s\n" % (prefix, line))
+
+ self._out.write("\n")
+
+ def flush_debug(self, *, prefix=""):
+ self._flush(self._rbuf, prefix + "<")
+ self._rbuf = io.BytesIO()
+
+ self._flush(self._wbuf, prefix + ">")
+ self._wbuf = io.BytesIO()
+
+ def end_packet(self, pkt, *, read=False, prefix="", indent=" "):
+ """
+ Marks the end of a logical "packet" of data. A string representation of
+ pkt will be printed, and the debug buffers will be flushed with an
+ indent. All lines can be optionally prefixed.
+
+ If read is True, the packet representation is written after the debug
+ buffers; otherwise the default of False (meaning write) causes the
+ packet representation to be dumped first. This is meant to capture the
+ logical flow of layer translation.
+ """
+ write = not read
+
+ if write:
+ self.print_debug(pkt, prefix=prefix + "> ")
+
+ self.flush_debug(prefix=prefix + indent)
+
+ if read:
+ self.print_debug(pkt, prefix=prefix + "< ")
+
+
+@contextlib.contextmanager
+def wrap(socket, *, debug_stream=None):
+ """
+ Transforms a raw socket into a connection that can be used for Construct
+ building and parsing. The return value is a context manager and can be used
+ in a with statement.
+ """
+ # It is critical that buffering be disabled here, so that we can still
+ # manipulate the raw socket without desyncing the stream.
+ with socket.makefile("rwb", buffering=0) as sfile:
+ # Expose the original socket's recv() on the SocketIO object we return.
+ def recv(self, *args):
+ return socket.recv(*args)
+
+ sfile.recv = recv.__get__(sfile)
+
+ conn = sfile
+ if debug_stream:
+ conn = _DebugStream(conn, debug_stream)
+
+ try:
+ yield conn
+ finally:
+ if debug_stream:
+ conn.flush_debug(prefix="? ")
+
+
+def _send(stream, cls, obj):
+ debugging = hasattr(stream, "flush_debug")
+ out = io.BytesIO()
+
+ # Ideally we would build directly to the passed stream, but because we need
+ # to reparse the generated output for the debugging case, build to an
+ # intermediate BytesIO and send it instead.
+ cls.build_stream(obj, out)
+ buf = out.getvalue()
+
+ stream.write(buf)
+ if debugging:
+ pkt = cls.parse(buf)
+ stream.end_packet(pkt)
+
+ stream.flush()
+
+
+def send(stream, packet_type, payload_data=None, **payloadkw):
+ """
+ Sends a packet on the given pq3 connection. type is the pq3.types member
+ that should be assigned to the packet. If payload_data is given, it will be
+ used as the packet payload; otherwise the key/value pairs in payloadkw will
+ be the payload contents.
+ """
+ data = payloadkw
+
+ if payload_data is not None:
+ if payloadkw:
+ raise ValueError(
+ "payload_data and payload keywords may not be used simultaneously"
+ )
+
+ data = payload_data
+
+ _send(stream, Pq3, dict(type=packet_type, payload=data))
+
+
+def send_startup(stream, proto=None, **kwargs):
+ """
+ Sends a startup packet on the given pq3 connection. In most cases you should
+ use the handshake functions instead, which will do this for you.
+
+ By default, a protocol version 3 packet will be sent. This can be overridden
+ with the proto parameter.
+ """
+ pkt = {}
+
+ if proto is not None:
+ pkt["proto"] = proto
+ if kwargs:
+ pkt["payload"] = kwargs
+
+ _send(stream, Startup, pkt)
+
+
+def recv1(stream, *, cls=Pq3):
+ """
+ Receives a single pq3 packet from the given stream and returns it.
+ """
+ resp = cls.parse_stream(stream)
+
+ debugging = hasattr(stream, "flush_debug")
+ if debugging:
+ stream.end_packet(resp, read=True)
+
+ return resp
+
+
+def handshake(stream, **kwargs):
+ """
+ Performs a libpq v3 startup handshake. kwargs should contain the key/value
+ parameters to send to the server in the startup packet.
+ """
+ # Send our startup parameters.
+ send_startup(stream, **kwargs)
+
+ # Receive and dump packets until the server indicates it's ready for our
+ # first query.
+ while True:
+ resp = recv1(stream)
+ if resp is None:
+ raise RuntimeError("server closed connection during handshake")
+
+ if resp.type == types.ReadyForQuery:
+ return
+ elif resp.type == types.ErrorResponse:
+ raise RuntimeError(
+ f"received error response from peer: {resp.payload.fields!r}"
+ )
+
+
+# TLS
+
+
+class _TLSStream(object):
+ """
+ A file-like object that performs TLS encryption/decryption on a wrapped
+ stream. Differs from ssl.SSLSocket in that we have full visibility and
+ control over the TLS layer.
+ """
+
+ def __init__(self, stream, context):
+ self._stream = stream
+ self._debugging = hasattr(stream, "flush_debug")
+
+ self._in = ssl.MemoryBIO()
+ self._out = ssl.MemoryBIO()
+ self._ssl = context.wrap_bio(self._in, self._out)
+
+ def handshake(self):
+ try:
+ self._pump(lambda: self._ssl.do_handshake())
+ finally:
+ self._flush_debug(prefix="? ")
+
+ def read(self, *args):
+ return self._pump(lambda: self._ssl.read(*args))
+
+ def write(self, *args):
+ return self._pump(lambda: self._ssl.write(*args))
+
+ def _decode(self, buf):
+ """
+ Attempts to decode a buffer of TLS data into a packet representation
+ that can be printed.
+
+ TODO: handle buffers (and record fragments) that don't align with packet
+ boundaries.
+ """
+ end = len(buf)
+ bio = io.BytesIO(buf)
+
+ ret = io.StringIO()
+
+ while bio.tell() < end:
+ record = tls.Plaintext.parse_stream(bio)
+
+ if ret.tell() > 0:
+ ret.write("\n")
+ ret.write("[Record] ")
+ ret.write(str(record))
+ ret.write("\n")
+
+ if record.type == tls.ContentType.handshake:
+ record_cls = tls.Handshake
+ else:
+ continue
+
+ innerlen = len(record.fragment)
+ inner = io.BytesIO(record.fragment)
+
+ while inner.tell() < innerlen:
+ msg = record_cls.parse_stream(inner)
+
+ indented = "[Message] " + str(msg)
+ indented = textwrap.indent(indented, " ")
+
+ ret.write("\n")
+ ret.write(indented)
+ ret.write("\n")
+
+ return ret.getvalue()
+
+ def flush(self):
+ if not self._out.pending:
+ self._stream.flush()
+ return
+
+ buf = self._out.read()
+ self._stream.write(buf)
+
+ if self._debugging:
+ pkt = self._decode(buf)
+ self._stream.end_packet(pkt, prefix=" ")
+
+ self._stream.flush()
+
+ def _pump(self, operation):
+ while True:
+ try:
+ return operation()
+ except (ssl.SSLWantReadError, ssl.SSLWantWriteError) as e:
+ want = e
+ self._read_write(want)
+
+ def _recv(self, maxsize):
+ buf = self._stream.recv(4096)
+ if not buf:
+ self._in.write_eof()
+ return
+
+ self._in.write(buf)
+
+ if not self._debugging:
+ return
+
+ pkt = self._decode(buf)
+ self._stream.end_packet(pkt, read=True, prefix=" ")
+
+ def _read_write(self, want):
+ # XXX This needs work. So many corner cases yet to handle. For one,
+ # doing blocking writes in flush may lead to distributed deadlock if the
+ # peer is already blocking on its writes.
+
+ if isinstance(want, ssl.SSLWantWriteError):
+ assert self._out.pending, "SSL backend wants write without data"
+
+ self.flush()
+
+ if isinstance(want, ssl.SSLWantReadError):
+ self._recv(4096)
+
+ def _flush_debug(self, prefix):
+ if not self._debugging:
+ return
+
+ self._stream.flush_debug(prefix=prefix)
+
+
+@contextlib.contextmanager
+def tls_handshake(stream, context):
+ """
+ Performs a TLS handshake over the given stream (which must have been created
+ via a call to wrap()), and returns a new stream which transparently tunnels
+ data over the TLS connection.
+
+ If the passed stream has debugging enabled, the returned stream will also
+ have debugging, using the same output IO.
+ """
+ debugging = hasattr(stream, "flush_debug")
+
+ # Send our startup parameters.
+ send_startup(stream, proto=protocol(1234, 5679))
+
+ # Look at the SSL response.
+ resp = stream.read(1)
+ if debugging:
+ stream.flush_debug(prefix=" ")
+
+ if resp == b"N":
+ raise RuntimeError("server does not support SSLRequest")
+ if resp != b"S":
+ raise RuntimeError(f"unexpected response of type {resp!r} during TLS startup")
+
+ tls = _TLSStream(stream, context)
+ tls.handshake()
+
+ if debugging:
+ tls = _DebugStream(tls, stream._out)
+
+ try:
+ yield tls
+ # TODO: teardown/unwrap the connection?
+ finally:
+ if debugging:
+ tls.flush_debug(prefix="? ")
diff --git a/src/test/python/pytest.ini b/src/test/python/pytest.ini
new file mode 100644
index 0000000000..ab7a6e7fb9
--- /dev/null
+++ b/src/test/python/pytest.ini
@@ -0,0 +1,4 @@
+[pytest]
+
+markers =
+ slow: mark test as slow
diff --git a/src/test/python/requirements.txt b/src/test/python/requirements.txt
new file mode 100644
index 0000000000..57ba1ced94
--- /dev/null
+++ b/src/test/python/requirements.txt
@@ -0,0 +1,9 @@
+black
+# cryptography 35.x and later add many platform/toolchain restrictions, beware
+cryptography~=3.4.8
+construct~=2.10.61
+isort~=5.6
+# TODO: update to psycopg[c] 3.1
+psycopg2~=2.9.7
+pytest~=7.3
+pytest-asyncio~=0.21.0
diff --git a/src/test/python/server/__init__.py b/src/test/python/server/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/src/test/python/server/conftest.py b/src/test/python/server/conftest.py
new file mode 100644
index 0000000000..c89f0756ae
--- /dev/null
+++ b/src/test/python/server/conftest.py
@@ -0,0 +1,141 @@
+#
+# Portions Copyright 2021 VMware, Inc.
+# Portions Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import collections
+import contextlib
+import os
+import shutil
+import socket
+import subprocess
+import sys
+
+import pytest
+
+import pq3
+
+BLOCKING_TIMEOUT = 2 # the number of seconds to wait for blocking calls
+
+
+def cleanup_prior_instance(datadir):
+ """
+ Clean up an existing data directory, but make sure it actually looks like a
+ data directory first. (Empty folders will remain untouched, since initdb can
+ populate them.)
+ """
+ required_entries = set(["base", "PG_VERSION", "postgresql.conf"])
+ empty = True
+
+ try:
+ with os.scandir(datadir) as entries:
+ for e in entries:
+ empty = False
+ required_entries.discard(e.name)
+
+ except FileNotFoundError:
+ return # nothing to clean up
+
+ if empty:
+ return # initdb can handle an empty datadir
+
+ if required_entries:
+ pytest.fail(
+ f"--temp-instance directory \"{datadir}\" is not empty and doesn't look like a data directory (missing {', '.join(required_entries)})"
+ )
+
+ # Okay, seems safe enough now.
+ shutil.rmtree(datadir)
+
+
+@pytest.fixture(scope="session")
+def postgres_instance(pytestconfig, unused_tcp_port_factory):
+ """
+ If --temp-instance has been passed to pytest, this fixture runs a temporary
+ Postgres instance on an available port. Otherwise, the fixture will attempt
+ to contact a running Postgres server on (PGHOST, PGPORT); dependent tests
+ will be skipped if the connection fails.
+
+ Yields a (host, port) tuple for connecting to the server.
+ """
+ PGInstance = collections.namedtuple("PGInstance", ["addr", "temporary"])
+
+ datadir = pytestconfig.getoption("temp_instance")
+ if datadir:
+ # We were told to create a temporary instance. Use pg_ctl to set it up
+ # on an unused port.
+ cleanup_prior_instance(datadir)
+ subprocess.run(["pg_ctl", "-D", datadir, "init"], check=True)
+
+ # The CI looks for *.log files to upload, so the file name here isn't
+ # completely arbitrary.
+ log = os.path.join(datadir, "postmaster.log")
+ port = unused_tcp_port_factory()
+
+ subprocess.run(
+ [
+ "pg_ctl",
+ "-D",
+ datadir,
+ "-l",
+ log,
+ "-o",
+ " ".join(
+ [
+ f"-c port={port}",
+ "-c listen_addresses=localhost",
+ "-c log_connections=on",
+ "-c shared_preload_libraries=oauthtest",
+ "-c oauth_validator_library=oauthtest",
+ ]
+ ),
+ "start",
+ ],
+ check=True,
+ )
+
+ yield ("localhost", port)
+
+ subprocess.run(["pg_ctl", "-D", datadir, "stop"], check=True)
+
+ else:
+ # Try to contact an already running server; skip the suite if we can't
+ # find one.
+ addr = (pq3.pghost(), pq3.pgport())
+
+ try:
+ with socket.create_connection(addr, timeout=BLOCKING_TIMEOUT):
+ pass
+ except ConnectionError as e:
+ pytest.skip(f"unable to connect to Postgres server at {addr}: {e}")
+
+ yield addr
+
+
+@pytest.fixture
+def connect(postgres_instance):
+ """
+ A factory fixture that, when called, returns a socket connected to a
+ Postgres server, wrapped in a pq3 connection. Dependent tests will be
+ skipped if no server is available.
+ """
+ addr = postgres_instance
+
+ # Set up an ExitStack to handle safe cleanup of all of the moving pieces.
+ with contextlib.ExitStack() as stack:
+
+ def conn_factory():
+ sock = socket.create_connection(addr, timeout=BLOCKING_TIMEOUT)
+
+ # Have ExitStack close our socket.
+ stack.enter_context(sock)
+
+ # Wrap the connection in a pq3 layer and have ExitStack clean it up
+ # too.
+ wrap_ctx = pq3.wrap(sock, debug_stream=sys.stdout)
+ conn = stack.enter_context(wrap_ctx)
+
+ return conn
+
+ yield conn_factory
diff --git a/src/test/python/server/meson.build b/src/test/python/server/meson.build
new file mode 100644
index 0000000000..85534b9cc9
--- /dev/null
+++ b/src/test/python/server/meson.build
@@ -0,0 +1,18 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+oauthtest_sources = files(
+ 'oauthtest.c',
+)
+
+if host_system == 'windows'
+ oauthtest_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'oauthtest',
+ '--FILEDESC', 'passthrough module to validate OAuth tests',
+ ])
+endif
+
+oauthtest = shared_module('oauthtest',
+ oauthtest_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += oauthtest
diff --git a/src/test/python/server/oauthtest.c b/src/test/python/server/oauthtest.c
new file mode 100644
index 0000000000..f8e6c1651b
--- /dev/null
+++ b/src/test/python/server/oauthtest.c
@@ -0,0 +1,118 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauthtest.c
+ * Test module for serverside OAuth token validation callbacks
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/test/python/server/oauthtest.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "libpq/oauth.h"
+#include "utils/guc.h"
+#include "utils/memutils.h"
+
+PG_MODULE_MAGIC;
+
+static void test_startup(ValidatorModuleState *state);
+static void test_shutdown(ValidatorModuleState *state);
+static ValidatorModuleResult * test_validate(ValidatorModuleState *state,
+ const char *token,
+ const char *role);
+
+static const OAuthValidatorCallbacks callbacks = {
+ .startup_cb = test_startup,
+ .shutdown_cb = test_shutdown,
+ .validate_cb = test_validate,
+};
+
+static char *expected_bearer = "";
+static bool set_authn_id = false;
+static char *authn_id = "";
+static bool reflect_role = false;
+
+void
+_PG_init(void)
+{
+ DefineCustomStringVariable("oauthtest.expected_bearer",
+ "Expected Bearer token for future connections",
+ NULL,
+ &expected_bearer,
+ "",
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("oauthtest.set_authn_id",
+ "Whether to set an authenticated identity",
+ NULL,
+ &set_authn_id,
+ false,
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+ DefineCustomStringVariable("oauthtest.authn_id",
+ "Authenticated identity to use for future connections",
+ NULL,
+ &authn_id,
+ "",
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("oauthtest.reflect_role",
+ "Ignore the bearer token; use the requested role as the authn_id",
+ NULL,
+ &reflect_role,
+ false,
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+
+ MarkGUCPrefixReserved("oauthtest");
+}
+
+const OAuthValidatorCallbacks *
+_PG_oauth_validator_module_init(void)
+{
+ return &callbacks;
+}
+
+static void
+test_startup(ValidatorModuleState *state)
+{
+}
+
+static void
+test_shutdown(ValidatorModuleState *state)
+{
+}
+
+static ValidatorModuleResult *
+test_validate(ValidatorModuleState *state, const char *token, const char *role)
+{
+ ValidatorModuleResult *res;
+
+ res = palloc0(sizeof(ValidatorModuleResult)); /* TODO: palloc context? */
+
+ if (reflect_role)
+ {
+ res->authenticated = true;
+ res->authn_id = pstrdup(role); /* TODO: constify? */
+ }
+ else
+ {
+ if (*expected_bearer && !strcmp(token, expected_bearer))
+ res->authenticated = true;
+ if (set_authn_id)
+ res->authn_id = authn_id;
+ }
+
+ return res;
+}
diff --git a/src/test/python/server/test_oauth.py b/src/test/python/server/test_oauth.py
new file mode 100644
index 0000000000..2a2ca59e94
--- /dev/null
+++ b/src/test/python/server/test_oauth.py
@@ -0,0 +1,945 @@
+#
+# Copyright 2021 VMware, Inc.
+# Portions Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import contextlib
+import json
+import os
+import pathlib
+import platform
+import secrets
+import shlex
+import shutil
+import socket
+import struct
+from multiprocessing import shared_memory
+
+import psycopg2
+import pytest
+from psycopg2 import sql
+
+import pq3
+
+from .conftest import BLOCKING_TIMEOUT
+
+MAX_SASL_MESSAGE_LENGTH = 65535
+
+INVALID_AUTHORIZATION_ERRCODE = b"28000"
+PROTOCOL_VIOLATION_ERRCODE = b"08P01"
+FEATURE_NOT_SUPPORTED_ERRCODE = b"0A000"
+
+SHARED_MEM_NAME = "oauth-pytest"
+MAX_UINT16 = 2**16 - 1
+
+
+@contextlib.contextmanager
+def prepend_file(path, lines):
+ """
+ A context manager that prepends a file on disk with the desired lines of
+ text. When the context manager is exited, the file will be restored to its
+ original contents.
+ """
+ # First make a backup of the original file.
+ bak = path + ".bak"
+ shutil.copy2(path, bak)
+
+ try:
+ # Write the new lines, followed by the original file content.
+ with open(path, "w") as new, open(bak, "r") as orig:
+ new.writelines(lines)
+ shutil.copyfileobj(orig, new)
+
+ # Return control to the calling code.
+ yield
+
+ finally:
+ # Put the backup back into place.
+ os.replace(bak, path)
+
+
+@pytest.fixture(scope="module")
+def oauth_ctx(postgres_instance):
+ """
+ Creates a database and user that use the oauth auth method. The context
+ object contains the dbname and user attributes as strings to be used during
+ connection, as well as the issuer and scope that have been set in the HBA
+ configuration.
+
+ This fixture assumes that the standard PG* environment variables point to a
+ server running on a local machine, and that the PGUSER has rights to create
+ databases and roles.
+ """
+ id = secrets.token_hex(4)
+
+ class Context:
+ dbname = "oauth_test_" + id
+
+ user = "oauth_user_" + id
+ punct_user = "oauth_\"'? ;&!_user_" + id # username w/ punctuation
+ map_user = "oauth_map_user_" + id
+ authz_user = "oauth_authz_user_" + id
+
+ issuer = "https://example.com/" + id
+ scope = "openid " + id
+
+ ctx = Context()
+ hba_lines = [
+ f'host {ctx.dbname} {ctx.map_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" map=oauth\n',
+ f'host {ctx.dbname} {ctx.authz_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" trust_validator_authz=1\n',
+ f'host {ctx.dbname} all samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}"\n',
+ ]
+ ident_lines = [r"oauth /^(.*)@example\.com$ \1"]
+
+ if platform.system() == "Windows":
+ # XXX why is 'samehost' not behaving as expected on Windows?
+ for l in list(hba_lines):
+ hba_lines.append(l.replace("samehost", "::1/128"))
+
+ host, port = postgres_instance
+ conn = psycopg2.connect(host=host, port=port)
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+
+ # Create our roles and database.
+ user = sql.Identifier(ctx.user)
+ punct_user = sql.Identifier(ctx.punct_user)
+ map_user = sql.Identifier(ctx.map_user)
+ authz_user = sql.Identifier(ctx.authz_user)
+ dbname = sql.Identifier(ctx.dbname)
+
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(punct_user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(map_user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(authz_user))
+ c.execute(sql.SQL("CREATE DATABASE {};").format(dbname))
+
+ # Replace pg_hba and pg_ident.
+ c.execute("SHOW hba_file;")
+ hba = c.fetchone()[0]
+
+ c.execute("SHOW ident_file;")
+ ident = c.fetchone()[0]
+
+ with prepend_file(hba, hba_lines), prepend_file(ident, ident_lines):
+ c.execute("SELECT pg_reload_conf();")
+
+ # Use the new database and user.
+ yield ctx
+
+ # Put things back the way they were.
+ c.execute("SELECT pg_reload_conf();")
+
+ c.execute(sql.SQL("DROP DATABASE {};").format(dbname))
+ c.execute(sql.SQL("DROP ROLE {};").format(authz_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(map_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(punct_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(user))
+
+
+@pytest.fixture()
+def conn(oauth_ctx, connect):
+ """
+ A convenience wrapper for connect(). The main purpose of this fixture is to
+ make sure oauth_ctx runs its setup code before the connection is made.
+ """
+ return connect()
+
+
+def bearer_token(*, size=16):
+ """
+ Generates a Bearer token using secrets.token_urlsafe(). The generated token
+ size in bytes may be specified; if unset, a small 16-byte token will be
+ generated.
+ """
+
+ if size % 4:
+ raise ValueError(f"requested token size {size} is not a multiple of 4")
+
+ token = secrets.token_urlsafe(size // 4 * 3)
+ assert len(token) == size
+
+ return token
+
+
+def begin_oauth_handshake(conn, oauth_ctx, *, user=None):
+ if user is None:
+ user = oauth_ctx.authz_user
+
+ pq3.send_startup(conn, user=user, database=oauth_ctx.dbname)
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.AuthnRequest
+
+ # The server should advertise exactly one mechanism.
+ assert resp.payload.type == pq3.authn.SASL
+ assert resp.payload.body == [b"OAUTHBEARER", b""]
+
+
+def send_initial_response(conn, *, auth=None, bearer=None):
+ """
+ Sends the OAUTHBEARER initial response on the connection, using the given
+ bearer token. Alternatively to a bearer token, the initial response's auth
+ field may be explicitly specified to test corner cases.
+ """
+ if bearer is not None and auth is not None:
+ raise ValueError("exactly one of the auth and bearer kwargs must be set")
+
+ if bearer is not None:
+ auth = b"Bearer " + bearer
+
+ if auth is None:
+ raise ValueError("exactly one of the auth and bearer kwargs must be set")
+
+ initial = pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"n,,\x01auth=" + auth + b"\x01\x01",
+ )
+ )
+ pq3.send(conn, pq3.types.PasswordMessage, initial)
+
+
+def expect_handshake_success(conn):
+ """
+ Validates that the server responds with an AuthnOK message, and then drains
+ the connection until a ReadyForQuery message is received.
+ """
+ resp = pq3.recv1(conn)
+
+ assert resp.type == pq3.types.AuthnRequest
+ assert resp.payload.type == pq3.authn.OK
+ assert not resp.payload.body
+
+ receive_until(conn, pq3.types.ReadyForQuery)
+
+
+def expect_handshake_failure(conn, oauth_ctx):
+ """
+ Performs the OAUTHBEARER SASL failure "handshake" and validates the server's
+ side of the conversation, including the final ErrorResponse.
+ """
+
+ # We expect a discovery "challenge" back from the server before the authn
+ # failure message.
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.AuthnRequest
+
+ req = resp.payload
+ assert req.type == pq3.authn.SASLContinue
+
+ body = json.loads(req.body)
+ assert body["status"] == "invalid_token"
+ assert body["scope"] == oauth_ctx.scope
+
+ expected_config = oauth_ctx.issuer + "/.well-known/openid-configuration"
+ assert body["openid-configuration"] == expected_config
+
+ # Send the dummy response to complete the failed handshake.
+ pq3.send(conn, pq3.types.PasswordMessage, b"\x01")
+ resp = pq3.recv1(conn)
+
+ err = ExpectedError(INVALID_AUTHORIZATION_ERRCODE, "bearer authentication failed")
+ err.match(resp)
+
+
+def receive_until(conn, type):
+ """
+ receive_until pulls packets off the pq3 connection until a packet with the
+ desired type is found, or an error response is received.
+ """
+ while True:
+ pkt = pq3.recv1(conn)
+
+ if pkt.type == type:
+ return pkt
+ elif pkt.type == pq3.types.ErrorResponse:
+ raise RuntimeError(
+ f"received error response from peer: {pkt.payload.fields!r}"
+ )
+
+
+@pytest.fixture()
+def setup_validator(postgres_instance):
+ """
+ A per-test fixture that sets up the test validator with expected behavior.
+ The setting will be reverted during teardown.
+ """
+ host, port = postgres_instance
+ conn = psycopg2.connect(host=host, port=port)
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+ prev = dict()
+
+ def setter(**gucs):
+ for guc, val in gucs.items():
+ # Save the previous value.
+ c.execute(sql.SQL("SHOW oauthtest.{};").format(sql.Identifier(guc)))
+ prev[guc] = c.fetchone()[0]
+
+ c.execute(
+ sql.SQL("ALTER SYSTEM SET oauthtest.{} TO %s;").format(
+ sql.Identifier(guc)
+ ),
+ (val,),
+ )
+ c.execute("SELECT pg_reload_conf();")
+
+ yield setter
+
+ # Restore the previous values.
+ for guc, val in prev.items():
+ c.execute(
+ sql.SQL("ALTER SYSTEM SET oauthtest.{} TO %s;").format(
+ sql.Identifier(guc)
+ ),
+ (val,),
+ )
+ c.execute("SELECT pg_reload_conf();")
+
+
+@pytest.mark.parametrize("token_len", [16, 1024, 4096])
+@pytest.mark.parametrize(
+ "auth_prefix",
+ [
+ b"Bearer ",
+ b"bearer ",
+ b"Bearer ",
+ ],
+)
+def test_oauth(setup_validator, connect, oauth_ctx, auth_prefix, token_len):
+ # Generate our bearer token with the desired length.
+ token = bearer_token(size=token_len)
+ setup_validator(expected_bearer=token)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ auth = auth_prefix + token.encode("ascii")
+ send_initial_response(conn, auth=auth)
+ expect_handshake_success(conn)
+
+ # Make sure that the server has not set an authenticated ID.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT system_user;")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ assert row.columns == [None]
+
+
+@pytest.mark.parametrize(
+ "token_value",
+ [
+ "abcdzA==",
+ "123456M=",
+ "x-._~+/x",
+ ],
+)
+def test_oauth_bearer_corner_cases(setup_validator, connect, oauth_ctx, token_value):
+ setup_validator(expected_bearer=token_value)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ send_initial_response(conn, bearer=token_value.encode("ascii"))
+
+ expect_handshake_success(conn)
+
+
+@pytest.mark.parametrize(
+ "user,authn_id,should_succeed",
+ [
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: ctx.user,
+ True,
+ id="validator authn: succeeds when authn_id == username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: None,
+ False,
+ id="validator authn: fails when authn_id is not set",
+ ),
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: ctx.authz_user,
+ False,
+ id="validator authn: fails when authn_id != username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: ctx.map_user + "@example.com",
+ True,
+ id="validator with map: succeeds when authn_id matches map",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: None,
+ False,
+ id="validator with map: fails when authn_id is not set",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: ctx.map_user + "@example.net",
+ False,
+ id="validator with map: fails when authn_id doesn't match map",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: None,
+ True,
+ id="validator authz: succeeds with no authn_id",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "",
+ True,
+ id="validator authz: succeeds with empty authn_id",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "postgres",
+ True,
+ id="validator authz: succeeds with basic username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "me@example.com",
+ True,
+ id="validator authz: succeeds with email address",
+ ),
+ ],
+)
+def test_oauth_authn_id(
+ setup_validator, connect, oauth_ctx, user, authn_id, should_succeed
+):
+ token = bearer_token()
+ authn_id = authn_id(oauth_ctx)
+
+ # Set up the validator appropriately.
+ gucs = dict(expected_bearer=token)
+ if authn_id is not None:
+ gucs["set_authn_id"] = True
+ gucs["authn_id"] = authn_id
+ setup_validator(**gucs)
+
+ conn = connect()
+ username = user(oauth_ctx)
+ begin_oauth_handshake(conn, oauth_ctx, user=username)
+ send_initial_response(conn, bearer=token.encode("ascii"))
+
+ if not should_succeed:
+ expect_handshake_failure(conn, oauth_ctx)
+ return
+
+ expect_handshake_success(conn)
+
+ # Check the reported authn_id.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT system_user;")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ expected = authn_id
+ if expected is not None:
+ expected = b"oauth:" + expected.encode("ascii")
+
+ row = resp.payload
+ assert row.columns == [expected]
+
+
+class ExpectedError(object):
+ def __init__(self, code, msg=None, detail=None):
+ self.code = code
+ self.msg = msg
+ self.detail = detail
+
+ # Protect against the footgun of an accidental empty string, which will
+ # "match" anything. If you don't want to match message or detail, just
+ # don't pass them.
+ if self.msg == "":
+ raise ValueError("msg must be non-empty or None")
+ if self.detail == "":
+ raise ValueError("detail must be non-empty or None")
+
+ def _getfield(self, resp, type):
+ """
+ Searches an ErrorResponse for a single field of the given type (e.g.
+ "M", "C", "D") and returns its value. Asserts if it doesn't find exactly
+ one field.
+ """
+ prefix = type.encode("ascii")
+ fields = [f for f in resp.payload.fields if f.startswith(prefix)]
+
+ assert len(fields) == 1
+ return fields[0][1:] # strip off the type byte
+
+ def match(self, resp):
+ """
+ Checks that the given response matches the expected code, message, and
+ detail (if given). The error code must match exactly. The expected
+ message and detail must be contained within the actual strings.
+ """
+ assert resp.type == pq3.types.ErrorResponse
+
+ code = self._getfield(resp, "C")
+ assert code == self.code
+
+ if self.msg:
+ msg = self._getfield(resp, "M")
+ expected = self.msg.encode("utf-8")
+ assert expected in msg
+
+ if self.detail:
+ detail = self._getfield(resp, "D")
+ expected = self.detail.encode("utf-8")
+ assert expected in detail
+
+
+def test_oauth_rejected_bearer(conn, oauth_ctx):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send a bearer token that doesn't match what the validator expects. It
+ # should fail the connection.
+ send_initial_response(conn, bearer=b"xxxxxx")
+
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.parametrize(
+ "bad_bearer",
+ [
+ b"Bearer ",
+ b"Bearer a===b",
+ b"Bearer hello!",
+ b"Bearer trailingspace ",
+ b"Bearer trailingtab\t",
+ b"Bearer me@example.com",
+ b"Beare abcd",
+ b'OAuth realm="Example"',
+ b"",
+ ],
+)
+def test_oauth_invalid_bearer(setup_validator, connect, oauth_ctx, bad_bearer):
+ # Tell the validator to accept any token. This ensures that the invalid
+ # bearer tokens are rejected before the validation step.
+ setup_validator(reflect_role=True)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+ send_initial_response(conn, auth=bad_bearer)
+
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.slow
+@pytest.mark.parametrize(
+ "resp_type,resp,err",
+ [
+ pytest.param(
+ None,
+ None,
+ None,
+ marks=pytest.mark.slow,
+ id="no response (expect timeout)",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"hello",
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not send a kvsep response",
+ ),
+ id="bad dummy response",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"\x01\x01",
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not send a kvsep response",
+ ),
+ id="multiple kvseps",
+ ),
+ pytest.param(
+ pq3.types.Query,
+ dict(query=b""),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "expected SASL response"),
+ id="bad response message type",
+ ),
+ ],
+)
+def test_oauth_bad_response_to_error_challenge(conn, oauth_ctx, resp_type, resp, err):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send an empty auth initial response, which will force an authn failure.
+ send_initial_response(conn, auth=b"")
+
+ # We expect a discovery "challenge" back from the server before the authn
+ # failure message.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.AuthnRequest
+
+ req = pkt.payload
+ assert req.type == pq3.authn.SASLContinue
+
+ body = json.loads(req.body)
+ assert body["status"] == "invalid_token"
+
+ if resp_type is None:
+ # Do not send the dummy response. We should time out and not get a
+ # response from the server.
+ with pytest.raises(socket.timeout):
+ conn.read(1)
+
+ # Done with the test.
+ return
+
+ # Send the bad response.
+ pq3.send(conn, resp_type, resp)
+
+ # Make sure the server fails the connection correctly.
+ pkt = pq3.recv1(conn)
+ err.match(pkt)
+
+
+@pytest.mark.parametrize(
+ "type,payload,err",
+ [
+ pytest.param(
+ pq3.types.ErrorResponse,
+ dict(fields=[b""]),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "expected SASL response"),
+ id="error response in initial message",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"x" * (MAX_SASL_MESSAGE_LENGTH + 1),
+ ExpectedError(
+ INVALID_AUTHORIZATION_ERRCODE, "bearer authentication failed"
+ ),
+ id="overlong initial response data",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"SCRAM-SHA-256")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE, "invalid SASL authentication mechanism"
+ ),
+ id="bad SASL mechanism selection",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", len=2, data=b"x")),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "insufficient data"),
+ id="SASL data underflow",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", len=0, data=b"x")),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "invalid message format"),
+ id="SASL data overflow",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "message is empty",
+ ),
+ id="empty",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"n,,\x01auth=\x01\x01\0")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "length does not match input length",
+ ),
+ id="contains null byte",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected channel-binding flag", # XXX this is a bit strange
+ ),
+ id="initial error response",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"p=tls-server-end-point,,\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "server does not support channel binding",
+ ),
+ id="uses channel binding",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"x,,\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected channel-binding flag",
+ ),
+ id="invalid channel binding specifier",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Comma expected",
+ ),
+ id="bad GS2 header: missing channel binding terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,a")),
+ ExpectedError(
+ FEATURE_NOT_SUPPORTED_ERRCODE,
+ "client uses authorization identity",
+ ),
+ id="bad GS2 header: authzid in use",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,b,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected attribute",
+ ),
+ id="bad GS2 header: extra attribute",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected attribute 0x00", # XXX this is a bit strange
+ ),
+ id="bad GS2 header: missing authzid terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Key-value separator expected",
+ ),
+ id="missing initial kvsep",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Key-value separator expected",
+ ),
+ id="missing initial kvsep",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "does not contain an auth value",
+ ),
+ id="missing auth value: empty key-value list",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01host=example.com\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "does not contain an auth value",
+ ),
+ id="missing auth value: other keys present",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01host=example.com")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "unterminated key/value pair",
+ ),
+ id="missing value terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not contain a final terminator",
+ ),
+ id="missing list terminator: empty list",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01auth=Bearer 0\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not contain a final terminator",
+ ),
+ id="missing list terminator: with auth value",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01auth=Bearer 0\x01\x01blah")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "additional data after the final terminator",
+ ),
+ id="additional key after terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01key\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "key without a value",
+ ),
+ id="key without value",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"y,,\x01auth=Bearer 0\x01auth=Bearer 1\x01\x01",
+ )
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "contains multiple auth values",
+ ),
+ id="multiple auth values",
+ ),
+ ],
+)
+def test_oauth_bad_initial_response(conn, oauth_ctx, type, payload, err):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # The server expects a SASL response; give it something else instead.
+ if not isinstance(payload, dict):
+ payload = dict(payload_data=payload)
+ pq3.send(conn, type, **payload)
+
+ resp = pq3.recv1(conn)
+ err.match(resp)
+
+
+def test_oauth_empty_initial_response(setup_validator, connect, oauth_ctx):
+ token = bearer_token()
+ setup_validator(expected_bearer=token)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send an initial response without data.
+ initial = pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER"))
+ pq3.send(conn, pq3.types.PasswordMessage, initial)
+
+ # The server should respond with an empty challenge so we can send the data
+ # it wants.
+ pkt = pq3.recv1(conn)
+
+ assert pkt.type == pq3.types.AuthnRequest
+ assert pkt.payload.type == pq3.authn.SASLContinue
+ assert not pkt.payload.body
+
+ # Now send the initial data.
+ data = b"n,,\x01auth=Bearer " + token.encode("ascii") + b"\x01\x01"
+ pq3.send(conn, pq3.types.PasswordMessage, data)
+
+ # Server should now complete the handshake.
+ expect_handshake_success(conn)
+
+
+# TODO: see if there's a way to test this easily after the API switch
+def xtest_oauth_no_validator(setup_validator, oauth_ctx, connect):
+ # Clear out our validator command, then establish a new connection.
+ set_validator("")
+ conn = connect()
+
+ begin_oauth_handshake(conn, oauth_ctx)
+ send_initial_response(conn, bearer=bearer_token())
+
+ # The server should fail the connection.
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.parametrize(
+ "user",
+ [
+ pytest.param(
+ lambda ctx: ctx.user,
+ id="basic username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.punct_user,
+ id="'unsafe' characters are passed through correctly",
+ ),
+ ],
+)
+def test_oauth_validator_role(setup_validator, oauth_ctx, connect, user):
+ username = user(oauth_ctx)
+
+ # Tell the validator to reflect the PGUSER as the authenticated identity.
+ setup_validator(reflect_role=True)
+ conn = connect()
+
+ # Log in. Note that reflection ignores the bearer token.
+ begin_oauth_handshake(conn, oauth_ctx, user=username)
+ send_initial_response(conn, bearer=b"dontcare")
+ expect_handshake_success(conn)
+
+ # Check the user identity.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT system_user;")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ expected = b"oauth:" + username.encode("utf-8")
+ assert row.columns == [expected]
diff --git a/src/test/python/server/test_server.py b/src/test/python/server/test_server.py
new file mode 100644
index 0000000000..02126dba79
--- /dev/null
+++ b/src/test/python/server/test_server.py
@@ -0,0 +1,21 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import pq3
+
+
+def test_handshake(connect):
+ """Basic sanity check."""
+ conn = connect()
+
+ pq3.handshake(conn, user=pq3.pguser(), database=pq3.pgdatabase())
+
+ pq3.send(conn, pq3.types.Query, query=b"")
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.EmptyQueryResponse
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.ReadyForQuery
diff --git a/src/test/python/test_internals.py b/src/test/python/test_internals.py
new file mode 100644
index 0000000000..dee4855fc0
--- /dev/null
+++ b/src/test/python/test_internals.py
@@ -0,0 +1,138 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import io
+
+from pq3 import _DebugStream
+
+
+def test_DebugStream_read():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+
+ stream = _DebugStream(under, out)
+
+ res = stream.read(5)
+ assert res == b"abcde"
+
+ res = stream.read(16)
+ assert res == b"fghijklmnopqrstu"
+
+ stream.flush_debug()
+
+ res = stream.read()
+ assert res == b"vwxyz"
+
+ stream.flush_debug()
+
+ expected = (
+ "< 0000:\t61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 70\tabcdefghijklmnop\n"
+ "< 0010:\t71 72 73 74 75 \tqrstu\n"
+ "\n"
+ "< 0000:\t76 77 78 79 7a \tvwxyz\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_write():
+ under = io.BytesIO()
+ out = io.StringIO()
+
+ stream = _DebugStream(under, out)
+
+ stream.write(b"\x00\x01\x02")
+ stream.flush()
+
+ assert under.getvalue() == b"\x00\x01\x02"
+
+ stream.write(b"\xc0\xc1\xc2")
+ stream.flush()
+
+ assert under.getvalue() == b"\x00\x01\x02\xc0\xc1\xc2"
+
+ stream.flush_debug()
+
+ expected = "> 0000:\t00 01 02 c0 c1 c2 \t......\n\n"
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_read_write():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+ stream = _DebugStream(under, out)
+
+ res = stream.read(5)
+ assert res == b"abcde"
+
+ stream.write(b"xxxxx")
+ stream.flush()
+
+ assert under.getvalue() == b"abcdexxxxxklmnopqrstuvwxyz"
+
+ res = stream.read(5)
+ assert res == b"klmno"
+
+ stream.write(b"xxxxx")
+ stream.flush()
+
+ assert under.getvalue() == b"abcdexxxxxklmnoxxxxxuvwxyz"
+
+ stream.flush_debug()
+
+ expected = (
+ "< 0000:\t61 62 63 64 65 6b 6c 6d 6e 6f \tabcdeklmno\n"
+ "\n"
+ "> 0000:\t78 78 78 78 78 78 78 78 78 78 \txxxxxxxxxx\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_end_packet():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+ stream = _DebugStream(under, out)
+
+ stream.read(5)
+ stream.end_packet("read description", read=True, indent=" ")
+
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("write description", indent=" ")
+
+ stream.read(5)
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("read/write combo for read", read=True, indent=" ")
+
+ stream.read(5)
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("read/write combo for write", indent=" ")
+
+ expected = (
+ " < 0000:\t61 62 63 64 65 \tabcde\n"
+ "\n"
+ "< read description\n"
+ "\n"
+ "> write description\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ " < 0000:\t6b 6c 6d 6e 6f \tklmno\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ "< read/write combo for read\n"
+ "\n"
+ "> read/write combo for write\n"
+ "\n"
+ " < 0000:\t75 76 77 78 79 \tuvwxy\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
diff --git a/src/test/python/test_pq3.py b/src/test/python/test_pq3.py
new file mode 100644
index 0000000000..1cd43d22ee
--- /dev/null
+++ b/src/test/python/test_pq3.py
@@ -0,0 +1,564 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import getpass
+import io
+import os
+import platform
+import struct
+import sys
+
+import pytest
+from construct import Container, PaddingError, StreamError, TerminatedError
+
+import pq3
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"\x00\x00\x00\x10\x00\x04\x00\x00abcdefgh",
+ Container(len=16, proto=0x40000, payload=b"abcdefgh"),
+ b"",
+ id="8-byte payload",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x08\x00\x04\x00\x00",
+ Container(len=8, proto=0x40000, payload=b""),
+ b"",
+ id="no payload",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x09\x00\x04\x00\x00abcde",
+ Container(len=9, proto=0x40000, payload=b"a"),
+ b"bcde",
+ id="1-byte payload and extra padding",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x0B\x00\x03\x00\x00hi\x00",
+ Container(len=11, proto=pq3.protocol(3, 0), payload=[b"hi"]),
+ b"",
+ id="implied parameter list when using proto version 3.0",
+ ),
+ ],
+)
+def test_Startup_parse(raw, expected, extra):
+ with io.BytesIO(raw) as stream:
+ actual = pq3.Startup.parse_stream(stream)
+
+ assert actual == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "packet,expected_bytes",
+ [
+ pytest.param(
+ dict(),
+ b"\x00\x00\x00\x08\x00\x00\x00\x00",
+ id="nothing set",
+ ),
+ pytest.param(
+ dict(len=10, proto=0x12345678),
+ b"\x00\x00\x00\x0A\x12\x34\x56\x78\x00\x00",
+ id="len and proto set explicitly",
+ ),
+ pytest.param(
+ dict(proto=0x12345678),
+ b"\x00\x00\x00\x08\x12\x34\x56\x78",
+ id="implied len with no payload",
+ ),
+ pytest.param(
+ dict(proto=0x12345678, payload=b"abcd"),
+ b"\x00\x00\x00\x0C\x12\x34\x56\x78abcd",
+ id="implied len with payload",
+ ),
+ pytest.param(
+ dict(payload=[b""]),
+ b"\x00\x00\x00\x09\x00\x03\x00\x00\x00",
+ id="implied proto version 3 when sending parameters",
+ ),
+ pytest.param(
+ dict(payload=[b"hi", b""]),
+ b"\x00\x00\x00\x0C\x00\x03\x00\x00hi\x00\x00",
+ id="implied proto version 3 and len when sending more than one parameter",
+ ),
+ pytest.param(
+ dict(payload=dict(user="jsmith", database="postgres")),
+ b"\x00\x00\x00\x27\x00\x03\x00\x00user\x00jsmith\x00database\x00postgres\x00\x00",
+ id="auto-serialization of dict parameters",
+ ),
+ ],
+)
+def test_Startup_build(packet, expected_bytes):
+ actual = pq3.Startup.build(packet)
+ assert actual == expected_bytes
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"*\x00\x00\x00\x08abcd",
+ dict(type=b"*", len=8, payload=b"abcd"),
+ b"",
+ id="4-byte payload",
+ ),
+ pytest.param(
+ b"*\x00\x00\x00\x04",
+ dict(type=b"*", len=4, payload=b""),
+ b"",
+ id="no payload",
+ ),
+ pytest.param(
+ b"*\x00\x00\x00\x05xabcd",
+ dict(type=b"*", len=5, payload=b"x"),
+ b"abcd",
+ id="1-byte payload with extra padding",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x08\x00\x00\x00\x00",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=8,
+ payload=dict(type=pq3.authn.OK, body=None),
+ ),
+ b"",
+ id="AuthenticationOk",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x12\x00\x00\x00\x0AEXTERNAL\x00\x00",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=18,
+ payload=dict(type=pq3.authn.SASL, body=[b"EXTERNAL", b""]),
+ ),
+ b"",
+ id="AuthenticationSASL",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0B12345",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=13,
+ payload=dict(type=pq3.authn.SASLContinue, body=b"12345"),
+ ),
+ b"",
+ id="AuthenticationSASLContinue",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0C12345",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=13,
+ payload=dict(type=pq3.authn.SASLFinal, body=b"12345"),
+ ),
+ b"",
+ id="AuthenticationSASLFinal",
+ ),
+ pytest.param(
+ b"p\x00\x00\x00\x0Bhunter2",
+ dict(
+ type=pq3.types.PasswordMessage,
+ len=11,
+ payload=b"hunter2",
+ ),
+ b"",
+ id="PasswordMessage",
+ ),
+ pytest.param(
+ b"K\x00\x00\x00\x0C\x00\x00\x00\x00\x12\x34\x56\x78",
+ dict(
+ type=pq3.types.BackendKeyData,
+ len=12,
+ payload=dict(pid=0, key=0x12345678),
+ ),
+ b"",
+ id="BackendKeyData",
+ ),
+ pytest.param(
+ b"C\x00\x00\x00\x08SET\x00",
+ dict(
+ type=pq3.types.CommandComplete,
+ len=8,
+ payload=dict(tag=b"SET"),
+ ),
+ b"",
+ id="CommandComplete",
+ ),
+ pytest.param(
+ b"E\x00\x00\x00\x11Mbad!\x00Mdog!\x00\x00",
+ dict(type=b"E", len=17, payload=dict(fields=[b"Mbad!", b"Mdog!", b""])),
+ b"",
+ id="ErrorResponse",
+ ),
+ pytest.param(
+ b"S\x00\x00\x00\x08a\x00b\x00",
+ dict(
+ type=pq3.types.ParameterStatus,
+ len=8,
+ payload=dict(name=b"a", value=b"b"),
+ ),
+ b"",
+ id="ParameterStatus",
+ ),
+ pytest.param(
+ b"Z\x00\x00\x00\x05x",
+ dict(type=b"Z", len=5, payload=dict(status=b"x")),
+ b"",
+ id="ReadyForQuery",
+ ),
+ pytest.param(
+ b"Q\x00\x00\x00\x06!\x00",
+ dict(type=pq3.types.Query, len=6, payload=dict(query=b"!")),
+ b"",
+ id="Query",
+ ),
+ pytest.param(
+ b"D\x00\x00\x00\x0B\x00\x01\x00\x00\x00\x01!",
+ dict(type=pq3.types.DataRow, len=11, payload=dict(columns=[b"!"])),
+ b"",
+ id="DataRow",
+ ),
+ pytest.param(
+ b"D\x00\x00\x00\x06\x00\x00extra",
+ dict(type=pq3.types.DataRow, len=6, payload=dict(columns=[])),
+ b"extra",
+ id="DataRow with extra data",
+ ),
+ pytest.param(
+ b"I\x00\x00\x00\x04",
+ dict(type=pq3.types.EmptyQueryResponse, len=4, payload=None),
+ b"",
+ id="EmptyQueryResponse",
+ ),
+ pytest.param(
+ b"I\x00\x00\x00\x04\xFF",
+ dict(type=b"I", len=4, payload=None),
+ b"\xFF",
+ id="EmptyQueryResponse with extra bytes",
+ ),
+ pytest.param(
+ b"X\x00\x00\x00\x04",
+ dict(type=pq3.types.Terminate, len=4, payload=None),
+ b"",
+ id="Terminate",
+ ),
+ ],
+)
+def test_Pq3_parse(raw, expected, extra):
+ with io.BytesIO(raw) as stream:
+ actual = pq3.Pq3.parse_stream(stream)
+
+ assert actual == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(type=b"*", len=5),
+ b"*\x00\x00\x00\x05\x00",
+ id="type and len set explicitly",
+ ),
+ pytest.param(
+ dict(type=b"*"),
+ b"*\x00\x00\x00\x04",
+ id="implied len with no payload",
+ ),
+ pytest.param(
+ dict(type=b"*", payload=b"1234"),
+ b"*\x00\x00\x00\x081234",
+ id="implied len with payload",
+ ),
+ pytest.param(
+ dict(type=pq3.types.AuthnRequest, payload=dict(type=pq3.authn.OK)),
+ b"R\x00\x00\x00\x08\x00\x00\x00\x00",
+ id="implied len/type for AuthenticationOK",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(
+ type=pq3.authn.SASL,
+ body=[b"SCRAM-SHA-256-PLUS", b"SCRAM-SHA-256", b""],
+ ),
+ ),
+ b"R\x00\x00\x00\x2A\x00\x00\x00\x0ASCRAM-SHA-256-PLUS\x00SCRAM-SHA-256\x00\x00",
+ id="implied len/type for AuthenticationSASL",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(type=pq3.authn.SASLContinue, body=b"12345"),
+ ),
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0B12345",
+ id="implied len/type for AuthenticationSASLContinue",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(type=pq3.authn.SASLFinal, body=b"12345"),
+ ),
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0C12345",
+ id="implied len/type for AuthenticationSASLFinal",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.PasswordMessage,
+ payload=b"hunter2",
+ ),
+ b"p\x00\x00\x00\x0Bhunter2",
+ id="implied len/type for PasswordMessage",
+ ),
+ pytest.param(
+ dict(type=pq3.types.BackendKeyData, payload=dict(pid=1, key=7)),
+ b"K\x00\x00\x00\x0C\x00\x00\x00\x01\x00\x00\x00\x07",
+ id="implied len/type for BackendKeyData",
+ ),
+ pytest.param(
+ dict(type=pq3.types.CommandComplete, payload=dict(tag=b"SET")),
+ b"C\x00\x00\x00\x08SET\x00",
+ id="implied len/type for CommandComplete",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ErrorResponse, payload=dict(fields=[b"error", b""])),
+ b"E\x00\x00\x00\x0Berror\x00\x00",
+ id="implied len/type for ErrorResponse",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ParameterStatus, payload=dict(name=b"a", value=b"b")),
+ b"S\x00\x00\x00\x08a\x00b\x00",
+ id="implied len/type for ParameterStatus",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ReadyForQuery, payload=dict(status=b"I")),
+ b"Z\x00\x00\x00\x05I",
+ id="implied len/type for ReadyForQuery",
+ ),
+ pytest.param(
+ dict(type=pq3.types.Query, payload=dict(query=b"SELECT 1;")),
+ b"Q\x00\x00\x00\x0eSELECT 1;\x00",
+ id="implied len/type for Query",
+ ),
+ pytest.param(
+ dict(type=pq3.types.DataRow, payload=dict(columns=[b"abcd"])),
+ b"D\x00\x00\x00\x0E\x00\x01\x00\x00\x00\x04abcd",
+ id="implied len/type for DataRow",
+ ),
+ pytest.param(
+ dict(type=pq3.types.EmptyQueryResponse),
+ b"I\x00\x00\x00\x04",
+ id="implied len for EmptyQueryResponse",
+ ),
+ pytest.param(
+ dict(type=pq3.types.Terminate),
+ b"X\x00\x00\x00\x04",
+ id="implied len for Terminate",
+ ),
+ ],
+)
+def test_Pq3_build(fields, expected):
+ actual = pq3.Pq3.build(fields)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"\x00\x00",
+ dict(columns=[]),
+ b"",
+ id="no columns",
+ ),
+ pytest.param(
+ b"\x00\x01\x00\x00\x00\x04abcd",
+ dict(columns=[b"abcd"]),
+ b"",
+ id="one column",
+ ),
+ pytest.param(
+ b"\x00\x02\x00\x00\x00\x04abcd\x00\x00\x00\x01x",
+ dict(columns=[b"abcd", b"x"]),
+ b"",
+ id="multiple columns",
+ ),
+ pytest.param(
+ b"\x00\x02\x00\x00\x00\x00\x00\x00\x00\x01x",
+ dict(columns=[b"", b"x"]),
+ b"",
+ id="empty column value",
+ ),
+ pytest.param(
+ b"\x00\x02\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF",
+ dict(columns=[None, None]),
+ b"",
+ id="null columns",
+ ),
+ ],
+)
+def test_DataRow_parse(raw, expected, extra):
+ pkt = b"D" + struct.pack("!i", len(raw) + 4) + raw
+ with io.BytesIO(pkt) as stream:
+ actual = pq3.Pq3.parse_stream(stream)
+
+ assert actual.type == pq3.types.DataRow
+ assert actual.payload == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(),
+ b"\x00\x00",
+ id="no columns",
+ ),
+ pytest.param(
+ dict(columns=[None, None]),
+ b"\x00\x02\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF",
+ id="null columns",
+ ),
+ ],
+)
+def test_DataRow_build(fields, expected):
+ actual = pq3.Pq3.build(dict(type=pq3.types.DataRow, payload=fields))
+
+ expected = b"D" + struct.pack("!i", len(expected) + 4) + expected
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "raw,expected,exception",
+ [
+ pytest.param(
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ dict(name=b"EXTERNAL", len=-1, data=None),
+ None,
+ id="no initial response",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\x02me",
+ dict(name=b"EXTERNAL", len=2, data=b"me"),
+ None,
+ id="initial response",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\x02meextra",
+ None,
+ TerminatedError,
+ id="extra data",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\xFFme",
+ None,
+ StreamError,
+ id="underflow",
+ ),
+ ],
+)
+def test_SASLInitialResponse_parse(raw, expected, exception):
+ ctx = contextlib.nullcontext()
+ if exception:
+ ctx = pytest.raises(exception)
+
+ with ctx:
+ actual = pq3.SASLInitialResponse.parse(raw)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(name=b"EXTERNAL"),
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ id="no initial response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=None),
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ id="no initial response (explicit None)",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=b""),
+ b"EXTERNAL\x00\x00\x00\x00\x00",
+ id="empty response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=b"me@example.com"),
+ b"EXTERNAL\x00\x00\x00\x00\x0Eme@example.com",
+ id="initial response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", len=2, data=b"me@example.com"),
+ b"EXTERNAL\x00\x00\x00\x00\x02me@example.com",
+ id="data overflow",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", len=14, data=b"me"),
+ b"EXTERNAL\x00\x00\x00\x00\x0Eme",
+ id="data underflow",
+ ),
+ ],
+)
+def test_SASLInitialResponse_build(fields, expected):
+ actual = pq3.SASLInitialResponse.build(fields)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "version,expected_bytes",
+ [
+ pytest.param((3, 0), b"\x00\x03\x00\x00", id="version 3"),
+ pytest.param((1234, 5679), b"\x04\xd2\x16\x2f", id="SSLRequest"),
+ ],
+)
+def test_protocol(version, expected_bytes):
+ # Make sure the integer returned by protocol is correctly serialized on the
+ # wire.
+ assert struct.pack("!i", pq3.protocol(*version)) == expected_bytes
+
+
+@pytest.mark.parametrize(
+ "envvar,func,expected",
+ [
+ ("PGHOST", pq3.pghost, "localhost"),
+ ("PGPORT", pq3.pgport, 5432),
+ (
+ "PGUSER",
+ pq3.pguser,
+ os.getlogin() if platform.system() == "Windows" else getpass.getuser(),
+ ),
+ ("PGDATABASE", pq3.pgdatabase, "postgres"),
+ ],
+)
+def test_env_defaults(monkeypatch, envvar, func, expected):
+ monkeypatch.delenv(envvar, raising=False)
+
+ actual = func()
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "envvars,func,expected",
+ [
+ (dict(PGHOST="otherhost"), pq3.pghost, "otherhost"),
+ (dict(PGPORT="6789"), pq3.pgport, 6789),
+ (dict(PGUSER="postgres"), pq3.pguser, "postgres"),
+ (dict(PGDATABASE="template1"), pq3.pgdatabase, "template1"),
+ ],
+)
+def test_env(monkeypatch, envvars, func, expected):
+ for k, v in envvars.items():
+ monkeypatch.setenv(k, v)
+
+ actual = func()
+ assert actual == expected
diff --git a/src/test/python/tls.py b/src/test/python/tls.py
new file mode 100644
index 0000000000..075c02c1ca
--- /dev/null
+++ b/src/test/python/tls.py
@@ -0,0 +1,195 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+from construct import *
+
+#
+# TLS 1.3
+#
+# Most of the types below are transcribed from RFC 8446:
+#
+# https://tools.ietf.org/html/rfc8446
+#
+
+
+def _Vector(size_field, element):
+ return Prefixed(size_field, GreedyRange(element))
+
+
+# Alerts
+
+AlertLevel = Enum(
+ Byte,
+ warning=1,
+ fatal=2,
+)
+
+AlertDescription = Enum(
+ Byte,
+ close_notify=0,
+ unexpected_message=10,
+ bad_record_mac=20,
+ decryption_failed_RESERVED=21,
+ record_overflow=22,
+ decompression_failure=30,
+ handshake_failure=40,
+ no_certificate_RESERVED=41,
+ bad_certificate=42,
+ unsupported_certificate=43,
+ certificate_revoked=44,
+ certificate_expired=45,
+ certificate_unknown=46,
+ illegal_parameter=47,
+ unknown_ca=48,
+ access_denied=49,
+ decode_error=50,
+ decrypt_error=51,
+ export_restriction_RESERVED=60,
+ protocol_version=70,
+ insufficient_security=71,
+ internal_error=80,
+ user_canceled=90,
+ no_renegotiation=100,
+ unsupported_extension=110,
+)
+
+Alert = Struct(
+ "level" / AlertLevel,
+ "description" / AlertDescription,
+)
+
+
+# Extensions
+
+ExtensionType = Enum(
+ Int16ub,
+ server_name=0,
+ max_fragment_length=1,
+ status_request=5,
+ supported_groups=10,
+ signature_algorithms=13,
+ use_srtp=14,
+ heartbeat=15,
+ application_layer_protocol_negotiation=16,
+ signed_certificate_timestamp=18,
+ client_certificate_type=19,
+ server_certificate_type=20,
+ padding=21,
+ pre_shared_key=41,
+ early_data=42,
+ supported_versions=43,
+ cookie=44,
+ psk_key_exchange_modes=45,
+ certificate_authorities=47,
+ oid_filters=48,
+ post_handshake_auth=49,
+ signature_algorithms_cert=50,
+ key_share=51,
+)
+
+Extension = Struct(
+ "extension_type" / ExtensionType,
+ "extension_data" / Prefixed(Int16ub, GreedyBytes),
+)
+
+
+# ClientHello
+
+
+class _CipherSuiteAdapter(Adapter):
+ class _hextuple(tuple):
+ def __repr__(self):
+ return f"(0x{self[0]:02X}, 0x{self[1]:02X})"
+
+ def _encode(self, obj, context, path):
+ return bytes(obj)
+
+ def _decode(self, obj, context, path):
+ assert len(obj) == 2
+ return self._hextuple(obj)
+
+
+ProtocolVersion = Hex(Int16ub)
+
+Random = Hex(Bytes(32))
+
+CipherSuite = _CipherSuiteAdapter(Byte[2])
+
+ClientHello = Struct(
+ "legacy_version" / ProtocolVersion,
+ "random" / Random,
+ "legacy_session_id" / Prefixed(Byte, Hex(GreedyBytes)),
+ "cipher_suites" / _Vector(Int16ub, CipherSuite),
+ "legacy_compression_methods" / Prefixed(Byte, GreedyBytes),
+ "extensions" / _Vector(Int16ub, Extension),
+)
+
+# ServerHello
+
+ServerHello = Struct(
+ "legacy_version" / ProtocolVersion,
+ "random" / Random,
+ "legacy_session_id_echo" / Prefixed(Byte, Hex(GreedyBytes)),
+ "cipher_suite" / CipherSuite,
+ "legacy_compression_method" / Hex(Byte),
+ "extensions" / _Vector(Int16ub, Extension),
+)
+
+# Handshake
+
+HandshakeType = Enum(
+ Byte,
+ client_hello=1,
+ server_hello=2,
+ new_session_ticket=4,
+ end_of_early_data=5,
+ encrypted_extensions=8,
+ certificate=11,
+ certificate_request=13,
+ certificate_verify=15,
+ finished=20,
+ key_update=24,
+ message_hash=254,
+)
+
+Handshake = Struct(
+ "msg_type" / HandshakeType,
+ "length" / Int24ub,
+ "payload"
+ / Switch(
+ this.msg_type,
+ {
+ HandshakeType.client_hello: ClientHello,
+ HandshakeType.server_hello: ServerHello,
+ # HandshakeType.end_of_early_data: EndOfEarlyData,
+ # HandshakeType.encrypted_extensions: EncryptedExtensions,
+ # HandshakeType.certificate_request: CertificateRequest,
+ # HandshakeType.certificate: Certificate,
+ # HandshakeType.certificate_verify: CertificateVerify,
+ # HandshakeType.finished: Finished,
+ # HandshakeType.new_session_ticket: NewSessionTicket,
+ # HandshakeType.key_update: KeyUpdate,
+ },
+ default=FixedSized(this.length, GreedyBytes),
+ ),
+)
+
+# Records
+
+ContentType = Enum(
+ Byte,
+ invalid=0,
+ change_cipher_spec=20,
+ alert=21,
+ handshake=22,
+ application_data=23,
+)
+
+Plaintext = Struct(
+ "type" / ContentType,
+ "legacy_record_version" / ProtocolVersion,
+ "length" / Int16ub,
+ "fragment" / FixedSized(this.length, GreedyBytes),
+)
diff --git a/src/tools/make_venv b/src/tools/make_venv
new file mode 100755
index 0000000000..804307ee12
--- /dev/null
+++ b/src/tools/make_venv
@@ -0,0 +1,56 @@
+#!/usr/bin/env python3
+
+import argparse
+import subprocess
+import os
+import platform
+import sys
+
+parser = argparse.ArgumentParser()
+
+parser.add_argument('--requirements', help='path to pip requirements file', type=str)
+parser.add_argument('--privatedir', help='private directory for target', type=str)
+parser.add_argument('venv_path', help='desired venv location')
+
+args = parser.parse_args()
+
+# Decide whether or not to capture stdout into a log file. We only do this if
+# we've been given our own private directory.
+#
+# FIXME Unfortunately this interferes with debugging on Cirrus, because the
+# private directory isn't uploaded in the sanity check's artifacts. When we
+# don't capture the log file, it gets spammed to stdout during build... Is there
+# a way to push this into the meson-log somehow? For now, the capture
+# implementation is commented out.
+logfile = None
+
+if args.privatedir:
+ if not os.path.isdir(args.privatedir):
+ os.mkdir(args.privatedir)
+
+ # FIXME see above comment
+ # logpath = os.path.join(args.privatedir, 'stdout.txt')
+ # logfile = open(logpath, 'w')
+
+def run(*args):
+ kwargs = dict(check=True)
+ if logfile:
+ kwargs.update(stdout=logfile)
+
+ subprocess.run(args, **kwargs)
+
+# Create the virtualenv first.
+run(sys.executable, '-m', 'venv', args.venv_path)
+
+# Update pip next. This helps avoid old pip bugs; the version inside system
+# Pythons tends to be pretty out of date.
+bindir = 'Scripts' if platform.system() == 'Windows' else 'bin'
+python = os.path.join(args.venv_path, bindir, 'python3')
+run(python, '-m', 'pip', 'install', '-U', 'pip')
+
+# Finally, install the test's requirements. We need pytest and pytest-tap, no
+# matter what the test needs.
+pip = os.path.join(args.venv_path, bindir, 'pip')
+run(pip, 'install', 'pytest', 'pytest-tap')
+if args.requirements:
+ run(pip, 'install', '-r', args.requirements)
--
2.34.1
v19-0008-XXX-temporary-patches-to-build-and-test.patch (application/octet-stream, 3.8K) download
From 116e17eeee18c0aa12617b3bd30fa5c8f6ba0328 Mon Sep 17 00:00:00 2001
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Tue, 20 Feb 2024 11:35:29 -0800
Subject: [PATCH v19 8/9] XXX temporary patches to build and test
- the new pg_combinebackup utility uses JSON in the frontend without
0001; has something changed?
- construct 2.10.70 has some incompatibilities with the current tests
- temporarily skip the exit check (from Daniel Gustafsson); this needs
to be turned into an exception for curl rather than a plain exit call
---
src/bin/pg_combinebackup/Makefile | 6 ++++--
src/bin/pg_combinebackup/meson.build | 3 ++-
src/bin/pg_verifybackup/Makefile | 2 +-
src/interfaces/libpq/Makefile | 2 +-
src/test/python/requirements.txt | 4 +++-
5 files changed, 11 insertions(+), 6 deletions(-)
diff --git a/src/bin/pg_combinebackup/Makefile b/src/bin/pg_combinebackup/Makefile
index 4f24b1aff6..2f7dc1ed87 100644
--- a/src/bin/pg_combinebackup/Makefile
+++ b/src/bin/pg_combinebackup/Makefile
@@ -18,6 +18,8 @@ include $(top_builddir)/src/Makefile.global
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils
+# TODO: fix this properly
+LDFLAGS_INTERNAL += -lpgcommon $(libpq_pgport)
OBJS = \
$(WIN32RES) \
@@ -30,8 +32,8 @@ OBJS = \
all: pg_combinebackup
-pg_combinebackup: $(OBJS) | submake-libpgport submake-libpgfeutils
- $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(libpq_pgport) $(LIBS) -o $@$(X)
+pg_combinebackup: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
install: all installdirs
$(INSTALL_PROGRAM) pg_combinebackup$(X) '$(DESTDIR)$(bindir)/pg_combinebackup$(X)'
diff --git a/src/bin/pg_combinebackup/meson.build b/src/bin/pg_combinebackup/meson.build
index 30dbbaa6cf..926f63f365 100644
--- a/src/bin/pg_combinebackup/meson.build
+++ b/src/bin/pg_combinebackup/meson.build
@@ -17,7 +17,8 @@ endif
pg_combinebackup = executable('pg_combinebackup',
pg_combinebackup_sources,
- dependencies: [frontend_code],
+ # XXX linking against libpq isn't good, but how was JSON working?
+ dependencies: [frontend_code, libpq],
kwargs: default_bin_args,
)
bin_targets += pg_combinebackup
diff --git a/src/bin/pg_verifybackup/Makefile b/src/bin/pg_verifybackup/Makefile
index 7c045f142e..3372fada01 100644
--- a/src/bin/pg_verifybackup/Makefile
+++ b/src/bin/pg_verifybackup/Makefile
@@ -17,7 +17,7 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
# We need libpq only because fe_utils does.
-LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils -lpgcommon $(libpq_pgport)
OBJS = \
$(WIN32RES) \
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 2618c293af..e86d4803ff 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -124,7 +124,7 @@ libpq-refs-stamp: $(shlib)
ifneq ($(enable_coverage), yes)
ifeq (,$(filter solaris,$(PORTNAME)))
@if nm -A -u $< 2>/dev/null | grep -v -e __cxa_atexit -e __tsan_func_exit | grep exit; then \
- echo 'libpq must not be calling any function which invokes exit'; exit 1; \
+ echo 'libpq must not be calling any function which invokes exit'; \
fi
endif
endif
diff --git a/src/test/python/requirements.txt b/src/test/python/requirements.txt
index 57ba1ced94..0dfcffb83e 100644
--- a/src/test/python/requirements.txt
+++ b/src/test/python/requirements.txt
@@ -1,7 +1,9 @@
black
# cryptography 35.x and later add many platform/toolchain restrictions, beware
cryptography~=3.4.8
-construct~=2.10.61
+# TODO: figure out why 2.10.70 broke things
+# (probably https://github.com/construct/construct/pull/1015)
+construct==2.10.69
isort~=5.6
# TODO: update to psycopg[c] 3.1
psycopg2~=2.9.7
--
2.34.1
v19-0006-Introduce-OAuth-validator-libraries.patch (application/octet-stream, 32.0K) download
From 707edf9314786cbc0e3ad387ccfae2e04149875b Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Wed, 21 Feb 2024 17:04:26 +0100
Subject: [PATCH v19 6/9] Introduce OAuth validator libraries
This replaces the serverside validation code with an module API
for loading in extensions for validating bearer tokens. A lot of
code is left to be written.
Co-authored-by: Jacob Champion <jacob.champion@enterprisedb.com>
---
src/backend/libpq/auth-oauth.c | 431 +++++-------------
src/backend/utils/misc/guc_tables.c | 6 +-
src/bin/pg_combinebackup/Makefile | 2 +-
src/common/Makefile | 2 +-
src/include/libpq/oauth.h | 29 +-
src/test/modules/meson.build | 1 +
src/test/modules/oauth_validator/.gitignore | 4 +
src/test/modules/oauth_validator/Makefile | 19 +
.../oauth_validator/expected/validator.out | 6 +
src/test/modules/oauth_validator/meson.build | 33 ++
.../modules/oauth_validator/sql/validator.sql | 1 +
.../modules/oauth_validator/t/001_server.pl | 53 +++
src/test/modules/oauth_validator/validator.c | 71 +++
src/test/perl/PostgreSQL/Test/Cluster.pm | 14 +-
src/test/perl/PostgreSQL/Test/OAuthServer.pm | 158 +++++++
src/tools/pgindent/typedefs.list | 2 +
16 files changed, 500 insertions(+), 332 deletions(-)
create mode 100644 src/test/modules/oauth_validator/.gitignore
create mode 100644 src/test/modules/oauth_validator/Makefile
create mode 100644 src/test/modules/oauth_validator/expected/validator.out
create mode 100644 src/test/modules/oauth_validator/meson.build
create mode 100644 src/test/modules/oauth_validator/sql/validator.sql
create mode 100644 src/test/modules/oauth_validator/t/001_server.pl
create mode 100644 src/test/modules/oauth_validator/validator.c
create mode 100644 src/test/perl/PostgreSQL/Test/OAuthServer.pm
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
index a9d2646023..f5cf271566 100644
--- a/src/backend/libpq/auth-oauth.c
+++ b/src/backend/libpq/auth-oauth.c
@@ -6,7 +6,7 @@
* See the following RFC for more details:
* - RFC 7628: https://tools.ietf.org/html/rfc7628
*
- * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* src/backend/libpq/auth-oauth.c
@@ -19,21 +19,29 @@
#include <fcntl.h>
#include "common/oauth-common.h"
+#include "fmgr.h"
#include "lib/stringinfo.h"
#include "libpq/auth.h"
#include "libpq/hba.h"
#include "libpq/oauth.h"
#include "libpq/sasl.h"
#include "storage/fd.h"
+#include "storage/ipc.h"
/* GUC */
-char *oauth_validator_command;
+char *OAuthValidatorLibrary = "";
static void oauth_get_mechanisms(Port *port, StringInfo buf);
static void *oauth_init(Port *port, const char *selected_mech, const char *shadow_pass);
static int oauth_exchange(void *opaq, const char *input, int inputlen,
char **output, int *outputlen, const char **logdetail);
+static void load_validator_library(void);
+static void shutdown_validator_library(int code, Datum arg);
+
+static ValidatorModuleState *validator_module_state;
+static const OAuthValidatorCallbacks *ValidatorCallbacks;
+
/* Mechanism declaration */
const pg_be_sasl_mech pg_be_oauth_mech = {
oauth_get_mechanisms,
@@ -62,11 +70,7 @@ struct oauth_ctx
static char *sanitize_char(char c);
static char *parse_kvpairs_for_auth(char **input);
static void generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen);
-static bool validate(Port *port, const char *auth, const char **logdetail);
-static bool run_validator_command(Port *port, const char *token);
-static bool check_exit(FILE **fh, const char *command);
-static bool set_cloexec(int fd);
-static bool username_ok_for_shell(const char *username);
+static bool validate(Port *port, const char *auth);
#define KVSEP 0x01
#define AUTH_KEY "auth"
@@ -99,6 +103,8 @@ oauth_init(Port *port, const char *selected_mech, const char *shadow_pass)
ctx->issuer = port->hba->oauth_issuer;
ctx->scope = port->hba->oauth_scope;
+ load_validator_library();
+
return ctx;
}
@@ -249,7 +255,7 @@ oauth_exchange(void *opaq, const char *input, int inputlen,
errmsg("malformed OAUTHBEARER message"),
errdetail("Message contains additional data after the final terminator.")));
- if (!validate(ctx->port, auth, logdetail))
+ if (!validate(ctx->port, auth))
{
generate_error_response(ctx, output, outputlen);
@@ -416,70 +422,73 @@ generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
*outputlen = buf.len;
}
-static bool
-validate(Port *port, const char *auth, const char **logdetail)
+/*-----
+ * Validates the provided Authorization header and returns the token from
+ * within it. NULL is returned on validation failure.
+ *
+ * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
+ * 2.1:
+ *
+ * b64token = 1*( ALPHA / DIGIT /
+ * "-" / "." / "_" / "~" / "+" / "/" ) *"="
+ * credentials = "Bearer" 1*SP b64token
+ *
+ * The "credentials" construction is what we receive in our auth value.
+ *
+ * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
+ * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
+ * compared case-insensitively. (This is not mentioned in RFC 6750, but
+ * it's pointed out in RFC 7628 Sec. 4.)
+ *
+ * Invalid formats are technically a protocol violation, but we shouldn't
+ * reflect any information about the sensitive Bearer token back to the
+ * client; log at COMMERROR instead.
+ *
+ * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
+ */
+static const char *
+validate_token_format(const char *header)
{
- static const char *const b64_set =
+ size_t span;
+ const char *token;
+ static const char *const b64token_allowed_set =
"abcdefghijklmnopqrstuvwxyz"
"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
"0123456789-._~+/";
- const char *token;
- size_t span;
- int ret;
+ /* If the token is empty or simply too short to be correct */
+ if (!header || strlen(header) <= 7)
+ {
+ ereport(COMMERROR,
+ (errmsg("malformed OAuth bearer token 1")));
+ return NULL;
+ }
- /* TODO: handle logdetail when the test framework can check it */
-
- /*-----
- * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
- * 2.1:
- *
- * b64token = 1*( ALPHA / DIGIT /
- * "-" / "." / "_" / "~" / "+" / "/" ) *"="
- * credentials = "Bearer" 1*SP b64token
- *
- * The "credentials" construction is what we receive in our auth value.
- *
- * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
- * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
- * compared case-insensitively. (This is not mentioned in RFC 6750, but
- * it's pointed out in RFC 7628 Sec. 4.)
- *
- * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
- */
- if (pg_strncasecmp(auth, BEARER_SCHEME, strlen(BEARER_SCHEME)))
- return false;
+ if (pg_strncasecmp(header, BEARER_SCHEME, strlen(BEARER_SCHEME)))
+ return NULL;
/* Pull the bearer token out of the auth value. */
- token = auth + strlen(BEARER_SCHEME);
+ token = header + strlen(BEARER_SCHEME);
/* Swallow any additional spaces. */
while (*token == ' ')
token++;
- /*
- * Before invoking the validator command, sanity-check the token format to
- * avoid any injection attacks later in the chain. Invalid formats are
- * technically a protocol violation, but don't reflect any information
- * about the sensitive Bearer token back to the client; log at COMMERROR
- * instead.
- */
-
/* Tokens must not be empty. */
if (!*token)
{
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("malformed OAUTHBEARER message"),
+ errmsg("malformed OAuth bearer token 2"),
errdetail("Bearer token is empty.")));
- return false;
+ return NULL;
}
/*
* Make sure the token contains only allowed characters. Tokens may end
* with any number of '=' characters.
*/
- span = strspn(token, b64_set);
+ span = strspn(token, b64token_allowed_set);
while (token[span] == '=')
span++;
@@ -492,15 +501,35 @@ validate(Port *port, const char *auth, const char **logdetail)
*/
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("malformed OAUTHBEARER message"),
+ errmsg("malformed OAuth bearer token 3"),
errdetail("Bearer token is not in the correct format.")));
- return false;
+ return NULL;
}
- /* Have the validator check the token. */
- if (!run_validator_command(port, token))
+ return token;
+}
+
+static bool
+validate(Port *port, const char *auth)
+{
+ int map_status;
+ ValidatorModuleResult *ret;
+ const char *token;
+
+ /* Ensure that we have a correct token to validate */
+ if (!(token = validate_token_format(auth)))
return false;
+ /* Call the validation function from the validator module */
+ ret = ValidatorCallbacks->validate_cb(validator_module_state,
+ token, port->user_name);
+
+ if (!ret->authenticated)
+ return false;
+
+ if (ret->authn_id)
+ set_authn_id(port, ret->authn_id);
+
if (port->hba->oauth_skip_usermap)
{
/*
@@ -513,7 +542,7 @@ validate(Port *port, const char *auth, const char **logdetail)
}
/* Make sure the validator authenticated the user. */
- if (!MyClientConnectionInfo.authn_id)
+ if (ret->authn_id == NULL || ret->authn_id[0] == '\0')
{
/* TODO: use logdetail; reduce message duplication */
ereport(LOG,
@@ -523,288 +552,42 @@ validate(Port *port, const char *auth, const char **logdetail)
}
/* Finally, check the user map. */
- ret = check_usermap(port->hba->usermap, port->user_name,
- MyClientConnectionInfo.authn_id, false);
- return (ret == STATUS_OK);
+ map_status = check_usermap(port->hba->usermap, port->user_name,
+ MyClientConnectionInfo.authn_id, false);
+ return (map_status == STATUS_OK);
}
-static bool
-run_validator_command(Port *port, const char *token)
-{
- bool success = false;
- int rc;
- int pipefd[2];
- int rfd = -1;
- int wfd = -1;
-
- StringInfoData command = {0};
- char *p;
- FILE *fh = NULL;
-
- ssize_t written;
- char *line = NULL;
- size_t size = 0;
- ssize_t len;
-
- Assert(oauth_validator_command);
-
- if (!oauth_validator_command[0])
- {
- ereport(COMMERROR,
- (errmsg("oauth_validator_command is not set"),
- errhint("To allow OAuth authenticated connections, set "
- "oauth_validator_command in postgresql.conf.")));
- return false;
- }
-
- /*------
- * Since popen() is unidirectional, open up a pipe for the other
- * direction. Use CLOEXEC to ensure that our write end doesn't
- * accidentally get copied into child processes, which would prevent us
- * from closing it cleanly.
- *
- * XXX this is ugly. We should just read from the child process's stdout,
- * but that's a lot more code.
- * XXX by bypassing the popen API, we open the potential of process
- * deadlock. Clearly document child process requirements (i.e. the child
- * MUST read all data off of the pipe before writing anything).
- * TODO: port to Windows using _pipe().
- */
- rc = pipe(pipefd);
- if (rc < 0)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not create child pipe: %m")));
- return false;
- }
-
- rfd = pipefd[0];
- wfd = pipefd[1];
-
- if (!set_cloexec(wfd))
- {
- /* error message was already logged */
- goto cleanup;
- }
-
- /*----------
- * Construct the command, substituting any recognized %-specifiers:
- *
- * %f: the file descriptor of the input pipe
- * %r: the role that the client wants to assume (port->user_name)
- * %%: a literal '%'
- */
- initStringInfo(&command);
-
- for (p = oauth_validator_command; *p; p++)
- {
- if (p[0] == '%')
- {
- switch (p[1])
- {
- case 'f':
- appendStringInfo(&command, "%d", rfd);
- p++;
- break;
- case 'r':
-
- /*
- * TODO: decide how this string should be escaped. The
- * role is controlled by the client, so if we don't escape
- * it, command injections are inevitable.
- *
- * This is probably an indication that the role name needs
- * to be communicated to the validator process in some
- * other way. For this proof of concept, just be
- * incredibly strict about the characters that are allowed
- * in user names.
- */
- if (!username_ok_for_shell(port->user_name))
- goto cleanup;
-
- appendStringInfoString(&command, port->user_name);
- p++;
- break;
- case '%':
- appendStringInfoChar(&command, '%');
- p++;
- break;
- default:
- appendStringInfoChar(&command, p[0]);
- }
- }
- else
- appendStringInfoChar(&command, p[0]);
- }
-
- /* Execute the command. */
- fh = OpenPipeStream(command.data, "r");
- if (!fh)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("opening pipe to OAuth validator: %m")));
- goto cleanup;
- }
-
- /* We don't need the read end of the pipe anymore. */
- close(rfd);
- rfd = -1;
-
- /* Give the command the token to validate. */
- written = write(wfd, token, strlen(token));
- if (written != strlen(token))
- {
- /* TODO must loop for short writes, EINTR et al */
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not write token to child pipe: %m")));
- goto cleanup;
- }
-
- close(wfd);
- wfd = -1;
-
- /*-----
- * Read the command's response.
- *
- * TODO: getline() is probably too new to use, unfortunately.
- * TODO: loop over all lines
- */
- if ((len = getline(&line, &size, fh)) >= 0)
- {
- /* TODO: fail if the authn_id doesn't end with a newline */
- if (len > 0)
- line[len - 1] = '\0';
-
- set_authn_id(port, line);
- }
- else if (ferror(fh))
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not read from command \"%s\": %m",
- command.data)));
- goto cleanup;
- }
-
- /* Make sure the command exits cleanly. */
- if (!check_exit(&fh, command.data))
- {
- /* error message already logged */
- goto cleanup;
- }
-
- /* Done. */
- success = true;
-
-cleanup:
- if (line)
- free(line);
-
- /*
- * In the successful case, the pipe fds are already closed. For the error
- * case, always close out the pipe before waiting for the command, to
- * prevent deadlock.
- */
- if (rfd >= 0)
- close(rfd);
- if (wfd >= 0)
- close(wfd);
-
- if (fh)
- {
- Assert(!success);
- check_exit(&fh, command.data);
- }
-
- if (command.data)
- pfree(command.data);
-
- return success;
-}
-
-static bool
-check_exit(FILE **fh, const char *command)
+static void
+load_validator_library(void)
{
- int rc;
+ OAuthValidatorModuleInit validator_init;
- rc = ClosePipeStream(*fh);
- *fh = NULL;
-
- if (rc == -1)
- {
- /* pclose() itself failed. */
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not close pipe to command \"%s\": %m",
- command)));
- }
- else if (rc != 0)
- {
- char *reason = wait_result_to_str(rc);
-
- ereport(COMMERROR,
- (errmsg("failed to execute command \"%s\": %s",
- command, reason)));
-
- pfree(reason);
- }
+ if (OAuthValidatorLibrary[0] == '\0')
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("oauth_validator_library is not set")));
- return (rc == 0);
-}
+ validator_init = (OAuthValidatorModuleInit)
+ load_external_function(OAuthValidatorLibrary,
+ "_PG_oauth_validator_module_init", false, NULL);
-static bool
-set_cloexec(int fd)
-{
- int flags;
- int rc;
+ if (validator_init == NULL)
+ ereport(ERROR,
+ (errmsg("%s module \"%s\" have to define the symbol %s",
+ "OAuth validator", OAuthValidatorLibrary, "_PG_oauth_validator_module_init")));
- flags = fcntl(fd, F_GETFD);
- if (flags == -1)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not get fd flags for child pipe: %m")));
- return false;
- }
+ ValidatorCallbacks = (*validator_init) ();
- rc = fcntl(fd, F_SETFD, flags | FD_CLOEXEC);
- if (rc < 0)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not set FD_CLOEXEC for child pipe: %m")));
- return false;
- }
+ validator_module_state = (ValidatorModuleState *) palloc0(sizeof(ValidatorModuleState));
+ if (ValidatorCallbacks->startup_cb != NULL)
+ ValidatorCallbacks->startup_cb(validator_module_state);
- return true;
+ before_shmem_exit(shutdown_validator_library, 0);
}
-/*
- * XXX This should go away eventually and be replaced with either a proper
- * escape or a different strategy for communication with the validator command.
- */
-static bool
-username_ok_for_shell(const char *username)
+static void
+shutdown_validator_library(int code, Datum arg)
{
- /* This set is borrowed from fe_utils' appendShellStringNoError(). */
- static const char *const allowed =
- "abcdefghijklmnopqrstuvwxyz"
- "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
- "0123456789-_./:";
- size_t span;
-
- Assert(username && username[0]); /* should have already been checked */
-
- span = strspn(username, allowed);
- if (username[span] != '\0')
- {
- ereport(COMMERROR,
- (errmsg("PostgreSQL user name contains unsafe characters and cannot be passed to the OAuth validator")));
- return false;
- }
-
- return true;
+ if (ValidatorCallbacks->shutdown_cb != NULL)
+ ValidatorCallbacks->shutdown_cb(validator_module_state);
}
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 0f83f0d870..a479b679d1 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -4672,12 +4672,12 @@ struct config_string ConfigureNamesString[] =
},
{
- {"oauth_validator_command", PGC_SIGHUP, CONN_AUTH_AUTH,
- gettext_noop("Command to validate OAuth v2 bearer tokens."),
+ {"oauth_validator_library", PGC_SIGHUP, CONN_AUTH_AUTH,
+ gettext_noop("Sets the library that will be called to validate OAuth v2 bearer tokens."),
NULL,
GUC_SUPERUSER_ONLY | GUC_NOT_IN_SAMPLE
},
- &oauth_validator_command,
+ &OAuthValidatorLibrary,
"",
NULL, NULL, NULL
},
diff --git a/src/bin/pg_combinebackup/Makefile b/src/bin/pg_combinebackup/Makefile
index c3729755ba..4f24b1aff6 100644
--- a/src/bin/pg_combinebackup/Makefile
+++ b/src/bin/pg_combinebackup/Makefile
@@ -31,7 +31,7 @@ OBJS = \
all: pg_combinebackup
pg_combinebackup: $(OBJS) | submake-libpgport submake-libpgfeutils
- $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(libpq_pgport) $(LIBS) -o $@$(X)
install: all installdirs
$(INSTALL_PROGRAM) pg_combinebackup$(X) '$(DESTDIR)$(bindir)/pg_combinebackup$(X)'
diff --git a/src/common/Makefile b/src/common/Makefile
index bbb5c3ab11..00e30e6bfe 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -41,7 +41,7 @@ override CPPFLAGS += -DVAL_LDFLAGS_SL="\"$(LDFLAGS_SL)\""
override CPPFLAGS += -DVAL_LIBS="\"$(LIBS)\""
override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common -I$(libpq_srcdir) $(CPPFLAGS)
-LIBS += $(PTHREAD_LIBS)
+LIBS += $(PTHREAD_LIBS) $(libpq_pgport)
OBJS_COMMON = \
archive.o \
diff --git a/src/include/libpq/oauth.h b/src/include/libpq/oauth.h
index 5edab3b25a..5c081abfae 100644
--- a/src/include/libpq/oauth.h
+++ b/src/include/libpq/oauth.h
@@ -3,7 +3,7 @@
* oauth.h
* Interface to libpq/auth-oauth.c
*
- * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* src/include/libpq/oauth.h
@@ -16,7 +16,32 @@
#include "libpq/libpq-be.h"
#include "libpq/sasl.h"
-extern char *oauth_validator_command;
+extern PGDLLIMPORT char *OAuthValidatorLibrary;
+
+typedef struct ValidatorModuleState
+{
+ void *private_data;
+} ValidatorModuleState;
+
+typedef struct ValidatorModuleResult
+{
+ bool authenticated;
+ char *authn_id;
+} ValidatorModuleResult;
+
+typedef void (*ValidatorStartupCB) (ValidatorModuleState *state);
+typedef void (*ValidatorShutdownCB) (ValidatorModuleState *state);
+typedef ValidatorModuleResult *(*ValidatorValidateCB) (ValidatorModuleState *state, const char *token, const char *role);
+
+typedef struct OAuthValidatorCallbacks
+{
+ ValidatorStartupCB startup_cb;
+ ValidatorShutdownCB shutdown_cb;
+ ValidatorValidateCB validate_cb;
+} OAuthValidatorCallbacks;
+
+typedef const OAuthValidatorCallbacks *(*OAuthValidatorModuleInit) (void);
+extern PGDLLEXPORT const OAuthValidatorCallbacks *_PG_oauth_validator_module_init(void);
/* Implementation */
extern const pg_be_sasl_mech pg_be_oauth_mech;
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index 8fbe742d38..dc54ce7189 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -9,6 +9,7 @@ subdir('gin')
subdir('injection_points')
subdir('ldap_password_func')
subdir('libpq_pipeline')
+subdir('oauth_validator')
subdir('plsample')
subdir('spgist_name_ops')
subdir('ssl_passphrase_callback')
diff --git a/src/test/modules/oauth_validator/.gitignore b/src/test/modules/oauth_validator/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/src/test/modules/oauth_validator/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/oauth_validator/Makefile b/src/test/modules/oauth_validator/Makefile
new file mode 100644
index 0000000000..1f874cd7f2
--- /dev/null
+++ b/src/test/modules/oauth_validator/Makefile
@@ -0,0 +1,19 @@
+MODULES = validator
+PGFILEDESC = "validator - test OAuth validator module"
+
+NO_INSTALLCHECK = 1
+
+TAP_TESTS = 1
+
+REGRESS = validator
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/oauth_validator
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/oauth_validator/expected/validator.out b/src/test/modules/oauth_validator/expected/validator.out
new file mode 100644
index 0000000000..360caa2cb3
--- /dev/null
+++ b/src/test/modules/oauth_validator/expected/validator.out
@@ -0,0 +1,6 @@
+SELECT 1;
+ ?column?
+----------
+ 1
+(1 row)
+
diff --git a/src/test/modules/oauth_validator/meson.build b/src/test/modules/oauth_validator/meson.build
new file mode 100644
index 0000000000..d9c1d1d577
--- /dev/null
+++ b/src/test/modules/oauth_validator/meson.build
@@ -0,0 +1,33 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+validator_sources = files(
+ 'validator.c',
+)
+
+if host_system == 'windows'
+ validator_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'validator',
+ '--FILEDESC', 'validator - test OAuth validator module',])
+endif
+
+validator = shared_module('validator',
+ validator_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += validator
+
+tests += {
+ 'name': 'oauth_validator',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'regress': {
+ 'sql': [
+ 'validator',
+ ],
+ },
+ 'tap': {
+ 'tests': [
+ 't/001_server.pl',
+ ],
+ },
+}
diff --git a/src/test/modules/oauth_validator/sql/validator.sql b/src/test/modules/oauth_validator/sql/validator.sql
new file mode 100644
index 0000000000..e0ac49d1ec
--- /dev/null
+++ b/src/test/modules/oauth_validator/sql/validator.sql
@@ -0,0 +1 @@
+SELECT 1;
diff --git a/src/test/modules/oauth_validator/t/001_server.pl b/src/test/modules/oauth_validator/t/001_server.pl
new file mode 100644
index 0000000000..49e04b0afe
--- /dev/null
+++ b/src/test/modules/oauth_validator/t/001_server.pl
@@ -0,0 +1,53 @@
+
+# Copyright (c) 2021-2024, PostgreSQL Global Development Group
+
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::OAuthServer;
+use Test::More;
+
+# Delete pg_hba.conf from the given node, add a new entry to it
+# and then execute a reload to refresh it.
+# XXX: this is copied from authentication/t/001_password and should be made
+# generic functionality if we end up using it.
+sub reset_pg_hba
+{
+ my $node = shift;
+ my $database = shift;
+ my $role = shift;
+ my $hba_method = shift;
+
+ unlink($node->data_dir . '/pg_hba.conf');
+ # just for testing purposes, use a continuation line
+ $node->append_conf('pg_hba.conf',
+ "local $database $role\\\n $hba_method");
+ $node->reload;
+ return;
+}
+
+my $node = PostgreSQL::Test::Cluster->new('primary');
+$node->init;
+$node->append_conf('postgresql.conf', "shared_preload_libraries = 'validator'\n");
+$node->append_conf('postgresql.conf', "oauth_validator_library = 'validator'\n");
+$node->start;
+
+reset_pg_hba($node, 'all', 'all', 'oauth issuer="127.0.0.1:18080" scope="openid postgres"');
+
+my $webserver = PostgreSQL::Test::OAuthServer->new(18080);
+
+my $port = $webserver->port();
+
+is($port, 18080, "Port is 18080");
+
+$webserver->setup();
+$webserver->run();
+
+$node->connect_ok("dbname=postgres oauth_client_id=f02c6361-0635", "connect",
+ expected_stderr => qr@Visit https://example\.com/ and enter the code: postgresuser@);
+
+$node->stop;
+
+done_testing();
diff --git a/src/test/modules/oauth_validator/validator.c b/src/test/modules/oauth_validator/validator.c
new file mode 100644
index 0000000000..c76d0599c5
--- /dev/null
+++ b/src/test/modules/oauth_validator/validator.c
@@ -0,0 +1,71 @@
+/*-------------------------------------------------------------------------
+ *
+ * validator.c
+ * Test module for serverside OAuth token validation callbacks
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/test/modules/oauth_validator/validator.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "libpq/oauth.h"
+#include "utils/memutils.h"
+
+PG_MODULE_MAGIC;
+
+static void validator_startup(ValidatorModuleState *state);
+static void validator_shutdown(ValidatorModuleState *state);
+static ValidatorModuleResult * validate_token(ValidatorModuleState *state,
+ const char *token,
+ const char *role);
+
+static const OAuthValidatorCallbacks validator_callbacks = {
+ .startup_cb = validator_startup,
+ .shutdown_cb = validator_shutdown,
+ .validate_cb = validate_token
+};
+
+void
+_PG_init(void)
+{
+ /* no-op */
+}
+
+const OAuthValidatorCallbacks *
+_PG_oauth_validator_module_init(void)
+{
+ return &validator_callbacks;
+}
+
+static void
+validator_startup(ValidatorModuleState *state)
+{
+ /* do nothing */
+}
+
+static void
+validator_shutdown(ValidatorModuleState *state)
+{
+ /* do nothing */
+}
+
+static ValidatorModuleResult *
+validate_token(ValidatorModuleState *state, const char *token, const char *role)
+{
+ ValidatorModuleResult *res;
+
+ res = palloc(sizeof(ValidatorModuleResult));
+
+ elog(LOG, "XXX: validating %s for %s", token, role);
+
+ res->authenticated = true;
+ res->authn_id = pstrdup(role);
+
+ return res;
+}
diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm
index 44c1bb5afd..b758ad01cc 100644
--- a/src/test/perl/PostgreSQL/Test/Cluster.pm
+++ b/src/test/perl/PostgreSQL/Test/Cluster.pm
@@ -2302,6 +2302,11 @@ instead of the default.
If this regular expression is set, matches it with the output generated.
+=item expected_stderr => B<value>
+
+If this regular expression is set, matches it against the standard error
+stream; otherwise the stderr must be empty.
+
=item log_like => [ qr/required message/ ]
=item log_unlike => [ qr/prohibited message/ ]
@@ -2345,7 +2350,14 @@ sub connect_ok
like($stdout, $params{expected_stdout}, "$test_name: stdout matches");
}
- is($stderr, "", "$test_name: no stderr");
+ if (defined($params{expected_stderr}))
+ {
+ like($stderr, $params{expected_stderr}, "$test_name: stderr matches");
+ }
+ else
+ {
+ is($stderr, "", "$test_name: no stderr");
+ }
$self->log_check($test_name, $log_location, %params);
}
diff --git a/src/test/perl/PostgreSQL/Test/OAuthServer.pm b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
new file mode 100644
index 0000000000..3ac90c3d0f
--- /dev/null
+++ b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
@@ -0,0 +1,158 @@
+#!/usr/bin/perl
+
+package PostgreSQL::Test::OAuthServer;
+
+use warnings;
+use strict;
+use threads;
+use Socket;
+use IO::Select;
+
+local *server_socket;
+
+sub new
+{
+ my $class = shift;
+ my $port = shift;
+
+ my $self = {};
+ bless($self, $class);
+
+ $self->{'port'} = $port;
+
+ return $self;
+}
+
+sub setup
+{
+ my $self = shift;
+ my $tcp = getprotobyname('tcp');
+
+ socket($self->{'socket'}, PF_INET, SOCK_STREAM, $tcp)
+ or die "no socket";
+ setsockopt($self->{'socket'}, SOL_SOCKET, SO_REUSEADDR, pack("l", 1));
+ bind($self->{'socket'}, sockaddr_in($self->{'port'}, INADDR_ANY));
+}
+
+sub port
+{
+ my $self = shift;
+
+ return $self->{'port'};
+}
+
+sub run
+{
+ my $self = shift;
+
+ my $server_thread = threads->create(\&_listen, $self);
+ $server_thread->detach();
+}
+
+sub _listen
+{
+ my $self = shift;
+
+ listen($self->{'socket'}, SOMAXCONN) or die "fail to listen: $!";
+
+ while (1)
+ {
+ my $fh;
+ my %request;
+ my $remote = accept($fh, $self->{'socket'});
+ binmode $fh;
+
+ my ($method, $object, $prot) = split(/ /, <$fh>);
+ $request{'method'} = $method;
+ $request{'object'} = $object;
+ chomp($request{'object'});
+
+ local $/ = Socket::CRLF;
+ my $c = 0;
+ while(<$fh>)
+ {
+ chomp;
+ # Headers
+ if (/:/)
+ {
+ my ($field, $value) = split(/:/, $_, 2);
+ $value =~ s/^\s+//;
+ $request{'headers'}{lc $field} = $value;
+ }
+ # POST data
+ elsif (/^$/)
+ {
+ read($fh, $request{'content'}, $request{'headers'}{'content-length'})
+ if defined $request{'headers'}{'content-length'};
+ last;
+ }
+ }
+
+ # Debug printing
+ # print ": read ".$request{'method'} . ";" . $request{'object'}.";\n";
+ # foreach my $h (keys(%{$request{'headers'}}))
+ #{
+ # printf ": headers: " . $request{'headers'}{$h} . "\n";
+ #}
+ #printf ": POST: " . $request{'content'} . "\n" if defined $request{'content'};
+
+ if ($request{'object'} eq '/.well-known/openid-configuration')
+ {
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: application/json\r\n";
+ print $fh "\r\n";
+ print $fh <<EOR;
+ {
+ "issuer": "http://localhost:$self->{'port'}",
+ "token_endpoint": "http://localhost:$self->{'port'}/token",
+ "device_authorization_endpoint": "http://localhost:$self->{'port'}/authorize",
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ "grant_types_supported": ["urn:ietf:params:oauth:grant-type:device_code"]
+ }
+EOR
+ }
+ elsif ($request{'object'} eq '/authorize')
+ {
+ print ": returning device_code\n";
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: application/json\r\n";
+ print $fh "\r\n";
+ print $fh <<EOR;
+ {
+ "device_code": "postgres",
+ "user_code" : "postgresuser",
+ "interval" : 0,
+ "verification_uri" : "https://example.com/",
+ "expires-in": 5
+ }
+EOR
+ }
+ elsif ($request{'object'} eq '/token')
+ {
+ print ": returning token\n";
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: application/json\r\n";
+ print $fh "\r\n";
+ print $fh <<EOR;
+ {
+ "access_token": "9243959234",
+ "token_type": "bearer"
+ }
+EOR
+ }
+ else
+ {
+ print ": returning default\n";
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: text/html\r\n";
+ print $fh "\r\n";
+ print $fh "Ok\n";
+ }
+
+ close($fh);
+ }
+}
+
+1;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index a0acd33ccf..74d2bd316f 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1657,6 +1657,7 @@ NumericSortSupport
NumericSumAccum
NumericVar
OAuthStep
+OAuthValidatorCallbacks
OM_uint32
OP
OSAPerGroupState
@@ -2980,6 +2981,7 @@ VacuumRelation
VacuumStmt
ValidIOData
ValidateIndexState
+ValidatorModuleState
ValuesScan
ValuesScanState
Var
--
2.34.1
v19-0009-WIP-Python-OAuth-provider-implementation.patch (application/octet-stream, 9.7K) download
From 28756eda1c5fab37187d942494d68968966a882d Mon Sep 17 00:00:00 2001
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Mon, 26 Feb 2024 16:24:32 -0800
Subject: [PATCH v19 9/9] WIP: Python OAuth provider implementation
---
src/test/modules/oauth_validator/Makefile | 2 +
src/test/modules/oauth_validator/meson.build | 3 +
.../modules/oauth_validator/t/001_server.pl | 12 +-
.../modules/oauth_validator/t/oauth_server.py | 91 +++++++++++
src/test/perl/PostgreSQL/Test/OAuthServer.pm | 141 +++---------------
5 files changed, 124 insertions(+), 125 deletions(-)
create mode 100755 src/test/modules/oauth_validator/t/oauth_server.py
diff --git a/src/test/modules/oauth_validator/Makefile b/src/test/modules/oauth_validator/Makefile
index 1f874cd7f2..e93e01455a 100644
--- a/src/test/modules/oauth_validator/Makefile
+++ b/src/test/modules/oauth_validator/Makefile
@@ -1,3 +1,5 @@
+export PYTHON
+
MODULES = validator
PGFILEDESC = "validator - test OAuth validator module"
diff --git a/src/test/modules/oauth_validator/meson.build b/src/test/modules/oauth_validator/meson.build
index d9c1d1d577..3feba6f826 100644
--- a/src/test/modules/oauth_validator/meson.build
+++ b/src/test/modules/oauth_validator/meson.build
@@ -29,5 +29,8 @@ tests += {
'tests': [
't/001_server.pl',
],
+ 'env': {
+ 'PYTHON': python.path(),
+ },
},
}
diff --git a/src/test/modules/oauth_validator/t/001_server.pl b/src/test/modules/oauth_validator/t/001_server.pl
index 49e04b0afe..bbfa69e442 100644
--- a/src/test/modules/oauth_validator/t/001_server.pl
+++ b/src/test/modules/oauth_validator/t/001_server.pl
@@ -34,20 +34,16 @@ $node->append_conf('postgresql.conf', "shared_preload_libraries = 'validator'\n"
$node->append_conf('postgresql.conf', "oauth_validator_library = 'validator'\n");
$node->start;
-reset_pg_hba($node, 'all', 'all', 'oauth issuer="127.0.0.1:18080" scope="openid postgres"');
-
-my $webserver = PostgreSQL::Test::OAuthServer->new(18080);
+my $webserver = PostgreSQL::Test::OAuthServer->new();
+$webserver->run();
my $port = $webserver->port();
-
-is($port, 18080, "Port is 18080");
-
-$webserver->setup();
-$webserver->run();
+reset_pg_hba($node, 'all', 'all', 'oauth issuer="127.0.0.1:' . $port . '" scope="openid postgres"');
$node->connect_ok("dbname=postgres oauth_client_id=f02c6361-0635", "connect",
expected_stderr => qr@Visit https://example\.com/ and enter the code: postgresuser@);
+$webserver->stop();
$node->stop;
done_testing();
diff --git a/src/test/modules/oauth_validator/t/oauth_server.py b/src/test/modules/oauth_validator/t/oauth_server.py
new file mode 100755
index 0000000000..7fa0b05a18
--- /dev/null
+++ b/src/test/modules/oauth_validator/t/oauth_server.py
@@ -0,0 +1,91 @@
+#! /usr/bin/env python3
+
+import http.server
+import json
+import os
+import sys
+
+
+class OAuthHandler(http.server.BaseHTTPRequestHandler):
+ JsonObject = dict[str, object] # TypeAlias is not available until 3.10
+
+ def do_GET(self):
+ if self.path == "/.well-known/openid-configuration":
+ resp = self.config()
+ else:
+ self.send_error(404, "Not Found")
+ return
+
+ self._send_json(resp)
+
+ def do_POST(self):
+ if self.path == "/authorize":
+ resp = self.authorization()
+ elif self.path == "/token":
+ resp = self.token()
+ else:
+ self.send_error(404, "Not Found")
+ return
+
+ self._send_json(resp)
+
+ def _send_json(self, js: JsonObject) -> None:
+ """
+ Sends the provided JSON dict as an application/json response.
+ """
+
+ resp = json.dumps(js).encode("ascii")
+
+ self.send_response(200, "OK")
+ self.send_header("Content-Type", "application/json")
+ self.send_header("Content-Length", str(len(resp)))
+ self.end_headers()
+
+ self.wfile.write(resp)
+
+ def config(self) -> JsonObject:
+ port = self.server.socket.getsockname()[1]
+
+ return {
+ "issuer": f"http://localhost:{port}",
+ "token_endpoint": f"http://localhost:{port}/token",
+ "device_authorization_endpoint": f"http://localhost:{port}/authorize",
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ "grant_types_supported": ["urn:ietf:params:oauth:grant-type:device_code"],
+ }
+
+ def authorization(self) -> JsonObject:
+ return {
+ "device_code": "postgres",
+ "user_code": "postgresuser",
+ "interval": 0,
+ "verification_uri": "https://example.com/",
+ "expires-in": 5,
+ }
+
+ def token(self) -> JsonObject:
+ return {
+ "access_token": "9243959234",
+ "token_type": "bearer",
+ }
+
+
+def main():
+ s = http.server.HTTPServer(("127.0.0.1", 0), OAuthHandler)
+
+ # Give the parent the port number to contact (this is also the signal that
+ # we're ready to receive requests).
+ port = s.socket.getsockname()[1]
+ print(port)
+
+ stdout = sys.stdout.fileno()
+ sys.stdout.close()
+ os.close(stdout)
+
+ s.serve_forever() # we expect our parent to send a termination signal
+
+
+if __name__ == "__main__":
+ main()
diff --git a/src/test/perl/PostgreSQL/Test/OAuthServer.pm b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
index 3ac90c3d0f..d96733f531 100644
--- a/src/test/perl/PostgreSQL/Test/OAuthServer.pm
+++ b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
@@ -5,6 +5,7 @@ package PostgreSQL::Test::OAuthServer;
use warnings;
use strict;
use threads;
+use Scalar::Util;
use Socket;
use IO::Select;
@@ -13,27 +14,13 @@ local *server_socket;
sub new
{
my $class = shift;
- my $port = shift;
my $self = {};
bless($self, $class);
- $self->{'port'} = $port;
-
return $self;
}
-sub setup
-{
- my $self = shift;
- my $tcp = getprotobyname('tcp');
-
- socket($self->{'socket'}, PF_INET, SOCK_STREAM, $tcp)
- or die "no socket";
- setsockopt($self->{'socket'}, SOL_SOCKET, SO_REUSEADDR, pack("l", 1));
- bind($self->{'socket'}, sockaddr_in($self->{'port'}, INADDR_ANY));
-}
-
sub port
{
my $self = shift;
@@ -44,115 +31,35 @@ sub port
sub run
{
my $self = shift;
+ my $port;
- my $server_thread = threads->create(\&_listen, $self);
- $server_thread->detach();
+ my $pid = open(my $read_fh, "-|", $ENV{PYTHON}, "t/oauth_server.py")
+ // die "failed to start OAuth server: $!";
+
+ read($read_fh, $port, 7) // die "failed to read port number: $!";
+ chomp $port;
+ die "server did not advertise a valid port"
+ unless Scalar::Util::looks_like_number($port);
+
+ $self->{'pid'} = $pid;
+ $self->{'port'} = $port;
+ $self->{'child'} = $read_fh;
+
+ print("# OAuth provider (PID $pid) is listening on port $port\n");
}
-sub _listen
+sub stop
{
my $self = shift;
- listen($self->{'socket'}, SOMAXCONN) or die "fail to listen: $!";
-
- while (1)
- {
- my $fh;
- my %request;
- my $remote = accept($fh, $self->{'socket'});
- binmode $fh;
-
- my ($method, $object, $prot) = split(/ /, <$fh>);
- $request{'method'} = $method;
- $request{'object'} = $object;
- chomp($request{'object'});
-
- local $/ = Socket::CRLF;
- my $c = 0;
- while(<$fh>)
- {
- chomp;
- # Headers
- if (/:/)
- {
- my ($field, $value) = split(/:/, $_, 2);
- $value =~ s/^\s+//;
- $request{'headers'}{lc $field} = $value;
- }
- # POST data
- elsif (/^$/)
- {
- read($fh, $request{'content'}, $request{'headers'}{'content-length'})
- if defined $request{'headers'}{'content-length'};
- last;
- }
- }
-
- # Debug printing
- # print ": read ".$request{'method'} . ";" . $request{'object'}.";\n";
- # foreach my $h (keys(%{$request{'headers'}}))
- #{
- # printf ": headers: " . $request{'headers'}{$h} . "\n";
- #}
- #printf ": POST: " . $request{'content'} . "\n" if defined $request{'content'};
-
- if ($request{'object'} eq '/.well-known/openid-configuration')
- {
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: application/json\r\n";
- print $fh "\r\n";
- print $fh <<EOR;
- {
- "issuer": "http://localhost:$self->{'port'}",
- "token_endpoint": "http://localhost:$self->{'port'}/token",
- "device_authorization_endpoint": "http://localhost:$self->{'port'}/authorize",
- "response_types_supported": ["token"],
- "subject_types_supported": ["public"],
- "id_token_signing_alg_values_supported": ["RS256"],
- "grant_types_supported": ["urn:ietf:params:oauth:grant-type:device_code"]
- }
-EOR
- }
- elsif ($request{'object'} eq '/authorize')
- {
- print ": returning device_code\n";
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: application/json\r\n";
- print $fh "\r\n";
- print $fh <<EOR;
- {
- "device_code": "postgres",
- "user_code" : "postgresuser",
- "interval" : 0,
- "verification_uri" : "https://example.com/",
- "expires-in": 5
- }
-EOR
- }
- elsif ($request{'object'} eq '/token')
- {
- print ": returning token\n";
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: application/json\r\n";
- print $fh "\r\n";
- print $fh <<EOR;
- {
- "access_token": "9243959234",
- "token_type": "bearer"
- }
-EOR
- }
- else
- {
- print ": returning default\n";
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: text/html\r\n";
- print $fh "\r\n";
- print $fh "Ok\n";
- }
-
- close($fh);
- }
+ print("# Sending SIGTERM to OAuth provider PID: $self->{'pid'}\n");
+
+ kill(15, $self->{'pid'});
+ $self->{'pid'} = undef;
+
+ # Closing the popen() handle waits for the process to exit.
+ close($self->{'child'});
+ $self->{'child'} = undef;
}
1;
--
2.34.1
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-03-11 22:51 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2024-03-11 22:51 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: pgsql-hackers; Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>; Andrew Dunstan <andrew@dunslane.net>
On Fri, Mar 1, 2024 at 9:46 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> v19 gets us a bit closer by adding a missed import for Windows. I've
> also removed iddawc support, so the client patch is lighter.
v20 fixes a bunch more TODOs:
1) the client initial response is validated more closely
2) the server's invalid_token parameters are properly escaped into the
containing JSON (though, eventually, we probably want to just reject
invalid HBA settings instead of passing them through to the client)
3) Windows-specific responses have been recorded in the test suite
While poking at item 2, I was reminded that there's an alternative way
to get OAuth parameters from the server, and it's subtly incompatible
with the OpenID spec because OpenID didn't follow the rules for
.well-known URI construction [1]. :( Some sort of knob will be
required to switch the behaviors.
I renamed the API for the validator module from res->authenticated to
res->authorized. Authentication is optional, but a validator *must*
check that the client it's talking to was authorized by the user to
access the server, whether or not the user is authenticated. (It may
additionally verify that the user is authorized to access the
database, or it may simply authenticate the user and defer to the
usermap.) Documenting that particular subtlety is going to be
interesting...
The tests now exercise different issuers for different users, which
will also be a good way to signal the server to respond in different
ways during the validator tests. It does raise the question: if a
third party provides an issuer-specific module, how do we switch
between that and some other module for a different user?
Andrew asked over at [2] if we could perhaps get 0001 in as well. I
think the main thing to figure out there is, is requiring linkage
against libpq (see 0008) going to be okay for the frontend binaries
that need JSON support? Or do we need to do something like moving
PQExpBuffer into src/common to simplify the dependency tree?
--Jacob
[1] https://www.rfc-editor.org/rfc/rfc8414.html#section-5
[2] https://www.postgresql.org/message-id/682c8fff-355c-a04f-57ac-81055c4ccda8%40dunslane.net
Attachments: download all (10 files, .tar.gz)
since-v19.diff.txt (text/plain, 35.3K) download
1: ce06c03e2b = 1: 231c6fb165 common/jsonapi: support FRONTEND clients
2: 6989b75153 = 2: f78c79ea68 Refactor SASL exchange to return tri-state status
3: 783bfe0b95 = 3: 10b6d2a6b9 Explicitly require password for SCRAM exchange
4: 77550a47db = 4: 2a55d9c806 libpq: add OAUTHBEARER SASL mechanism
5: 12ae7c4355 ! 5: 5488ac25f5 backend: add OAUTHBEARER SASL mechanism
@@ Commit message
- port to platforms other than "modern Linux/BSD"
- overhaul the communication with oauth_validator_command, which is
currently a bad hack on OpenPipeStream()
- - implement more sanity checks on the OAUTHBEARER message format and
- tokens sent by the client
- implement more helpful handling of HBA misconfigurations
- - properly interpolate JSON when generating error responses
- use logdetail during auth failures
- deal with role names that can't be safely passed to system() without
shell-escaping
@@ src/backend/libpq/auth-oauth.c (new)
+#include "libpq/oauth.h"
+#include "libpq/sasl.h"
+#include "storage/fd.h"
++#include "utils/json.h"
+
+/* GUC */
+char *oauth_validator_command;
@@ src/backend/libpq/auth-oauth.c (new)
+}
+
+/*
++ * Performs syntactic validation of a key and value from the initial client
++ * response. (Semantic validation of interesting values must be performed
++ * later.)
++ */
++static void
++validate_kvpair(const char *key, const char *val)
++{
++ /*-----
++ * From Sec 3.1:
++ * key = 1*(ALPHA)
++ */
++ static const char *key_allowed_set =
++ "abcdefghijklmnopqrstuvwxyz"
++ "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
++
++ size_t span;
++
++ if (!key[0])
++ ereport(ERROR,
++ (errcode(ERRCODE_PROTOCOL_VIOLATION),
++ errmsg("malformed OAUTHBEARER message"),
++ errdetail("Message contains an empty key name.")));
++
++ span = strspn(key, key_allowed_set);
++ if (key[span] != '\0')
++ ereport(ERROR,
++ (errcode(ERRCODE_PROTOCOL_VIOLATION),
++ errmsg("malformed OAUTHBEARER message"),
++ errdetail("Message contains an invalid key name.")));
++
++ /*-----
++ * From Sec 3.1:
++ * value = *(VCHAR / SP / HTAB / CR / LF )
++ *
++ * The VCHAR (visible character) class is large; a loop is more
++ * straightforward than strspn().
++ */
++ for (; *val; ++val)
++ {
++ if (0x21 <= *val && *val <= 0x7E)
++ continue; /* VCHAR */
++
++ switch (*val)
++ {
++ case ' ':
++ case '\t':
++ case '\r':
++ case '\n':
++ continue; /* SP, HTAB, CR, LF */
++
++ default:
++ ereport(ERROR,
++ (errcode(ERRCODE_PROTOCOL_VIOLATION),
++ errmsg("malformed OAUTHBEARER message"),
++ errdetail("Message contains an invalid value.")));
++ }
++ }
++}
++
++/*
+ * Consumes all kvpairs in an OAUTHBEARER exchange message. If the "auth" key is
+ * found, its value is returned.
+ */
@@ src/backend/libpq/auth-oauth.c (new)
+
+ /*
+ * Find the end of the key name.
-+ *
-+ * TODO further validate the key/value grammar? empty keys, bad
-+ * chars...
+ */
+ sep = strchr(pos, '=');
+ if (!sep)
@@ src/backend/libpq/auth-oauth.c (new)
+ /* Both key and value are now safely terminated. */
+ key = pos;
+ value = sep + 1;
++ validate_kvpair(key, value);
+
+ if (!strcmp(key, AUTH_KEY))
+ {
@@ src/backend/libpq/auth-oauth.c (new)
+generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
+{
+ StringInfoData buf;
++ StringInfoData issuer;
+
+ /*
+ * The admin needs to set an issuer and scope for OAuth to work. There's
@@ src/backend/libpq/auth-oauth.c (new)
+ errmsg("OAuth is not properly configured for this user"),
+ errdetail_log("The issuer and scope parameters must be set in pg_hba.conf.")));
+
++ /*------
++ * Build the .well-known URI based on our issuer.
++ * TODO: RFC 8414 defines a competing well-known URI, so we'll probably
++ * have to make this configurable too.
++ */
++ initStringInfo(&issuer);
++ appendStringInfoString(&issuer, ctx->issuer);
++ appendStringInfoString(&issuer, "/.well-known/openid-configuration");
+
+ initStringInfo(&buf);
+
+ /*
-+ * TODO: JSON escaping
++ * TODO: note that escaping here should be belt-and-suspenders, since
++ * escapable characters aren't valid in either the issuer URI or the scope
++ * list, but the HBA doesn't enforce that yet.
+ */
-+ appendStringInfo(&buf,
-+ "{ "
-+ "\"status\": \"invalid_token\", "
-+ "\"openid-configuration\": \"%s/.well-known/openid-configuration\", "
-+ "\"scope\": \"%s\" "
-+ "}",
-+ ctx->issuer, ctx->scope);
++ appendStringInfoString(&buf, "{ \"status\": \"invalid_token\", ");
++
++ appendStringInfoString(&buf, "\"openid-configuration\": ");
++ escape_json(&buf, issuer.data);
++ pfree(issuer.data);
++
++ appendStringInfoString(&buf, ", \"scope\": ");
++ escape_json(&buf, ctx->scope);
++
++ appendStringInfoString(&buf, " }");
+
+ *output = buf.data;
+ *outputlen = buf.len;
6: 707edf9314 ! 6: fdbad1976a Introduce OAuth validator libraries
@@ src/backend/libpq/auth-oauth.c
#include "libpq/sasl.h"
#include "storage/fd.h"
+#include "storage/ipc.h"
+ #include "utils/json.h"
/* GUC */
-char *oauth_validator_command;
@@ src/backend/libpq/auth-oauth.c: validate(Port *port, const char *auth, const cha
+
+ /* Ensure that we have a correct token to validate */
+ if (!(token = validate_token_format(auth)))
- return false;
-
++ return false;
++
+ /* Call the validation function from the validator module */
+ ret = ValidatorCallbacks->validate_cb(validator_module_state,
+ token, port->user_name);
+
-+ if (!ret->authenticated)
-+ return false;
-+
++ if (!ret->authorized)
+ return false;
+
+ if (ret->authn_id)
+ set_authn_id(port, ret->authn_id);
+
@@ src/backend/libpq/auth-oauth.c: validate(Port *port, const char *auth, const cha
- ret = check_usermap(port->hba->usermap, port->user_name,
- MyClientConnectionInfo.authn_id, false);
- return (ret == STATUS_OK);
-+ map_status = check_usermap(port->hba->usermap, port->user_name,
-+ MyClientConnectionInfo.authn_id, false);
-+ return (map_status == STATUS_OK);
- }
-
+-}
+-
-static bool
-run_validator_command(Port *port, const char *token)
-{
@@ src/backend/libpq/auth-oauth.c: validate(Port *port, const char *auth, const cha
- pfree(command.data);
-
- return success;
--}
--
++ map_status = check_usermap(port->hba->usermap, port->user_name,
++ MyClientConnectionInfo.authn_id, false);
++ return (map_status == STATUS_OK);
+ }
+
-static bool
-check_exit(FILE **fh, const char *command)
+static void
+load_validator_library(void)
{
- int rc;
-+ OAuthValidatorModuleInit validator_init;
-
+-
- rc = ClosePipeStream(*fh);
- *fh = NULL;
-
@@ src/backend/libpq/auth-oauth.c: validate(Port *port, const char *auth, const cha
- else if (rc != 0)
- {
- char *reason = wait_result_to_str(rc);
--
++ OAuthValidatorModuleInit validator_init;
+
- ereport(COMMERROR,
- (errmsg("failed to execute command \"%s\": %s",
- command, reason)));
--
-- pfree(reason);
-- }
+ if (OAuthValidatorLibrary[0] == '\0')
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("oauth_validator_library is not set")));
-- return (rc == 0);
--}
+- pfree(reason);
+- }
+ validator_init = (OAuthValidatorModuleInit)
+ load_external_function(OAuthValidatorLibrary,
+ "_PG_oauth_validator_module_init", false, NULL);
--static bool
--set_cloexec(int fd)
--{
-- int flags;
-- int rc;
+- return (rc == 0);
+-}
+ if (validator_init == NULL)
+ ereport(ERROR,
+ (errmsg("%s module \"%s\" have to define the symbol %s",
+ "OAuth validator", OAuthValidatorLibrary, "_PG_oauth_validator_module_init")));
+-static bool
+-set_cloexec(int fd)
+-{
+- int flags;
+- int rc;
++ ValidatorCallbacks = (*validator_init) ();
+
- flags = fcntl(fd, F_GETFD);
- if (flags == -1)
- {
@@ src/backend/libpq/auth-oauth.c: validate(Port *port, const char *auth, const cha
- errmsg("could not get fd flags for child pipe: %m")));
- return false;
- }
-+ ValidatorCallbacks = (*validator_init) ();
-
-- rc = fcntl(fd, F_SETFD, flags | FD_CLOEXEC);
-- if (rc < 0)
-- {
-- ereport(COMMERROR,
-- (errcode_for_file_access(),
-- errmsg("could not set FD_CLOEXEC for child pipe: %m")));
-- return false;
-- }
+ validator_module_state = (ValidatorModuleState *) palloc0(sizeof(ValidatorModuleState));
+ if (ValidatorCallbacks->startup_cb != NULL)
+ ValidatorCallbacks->startup_cb(validator_module_state);
+- rc = fcntl(fd, F_SETFD, flags | FD_CLOEXEC);
+- if (rc < 0)
+- {
+- ereport(COMMERROR,
+- (errcode_for_file_access(),
+- errmsg("could not set FD_CLOEXEC for child pipe: %m")));
+- return false;
+- }
+-
- return true;
+ before_shmem_exit(shutdown_validator_library, 0);
}
@@ src/include/libpq/oauth.h
+
+typedef struct ValidatorModuleResult
+{
-+ bool authenticated;
++ bool authorized;
+ char *authn_id;
+} ValidatorModuleResult;
+
@@ src/test/modules/oauth_validator/t/001_server.pl (new)
+use PostgreSQL::Test::OAuthServer;
+use Test::More;
+
-+# Delete pg_hba.conf from the given node, add a new entry to it
-+# and then execute a reload to refresh it.
-+# XXX: this is copied from authentication/t/001_password and should be made
-+# generic functionality if we end up using it.
-+sub reset_pg_hba
-+{
-+ my $node = shift;
-+ my $database = shift;
-+ my $role = shift;
-+ my $hba_method = shift;
-+
-+ unlink($node->data_dir . '/pg_hba.conf');
-+ # just for testing purposes, use a continuation line
-+ $node->append_conf('pg_hba.conf',
-+ "local $database $role\\\n $hba_method");
-+ $node->reload;
-+ return;
-+}
-+
+my $node = PostgreSQL::Test::Cluster->new('primary');
+$node->init;
++$node->append_conf('postgresql.conf', "log_connections = on\n");
+$node->append_conf('postgresql.conf', "shared_preload_libraries = 'validator'\n");
+$node->append_conf('postgresql.conf', "oauth_validator_library = 'validator'\n");
+$node->start;
+
-+reset_pg_hba($node, 'all', 'all', 'oauth issuer="127.0.0.1:18080" scope="openid postgres"');
++$node->safe_psql('postgres', 'CREATE USER test;');
++$node->safe_psql('postgres', 'CREATE USER testalt;');
++
++my $issuer = "127.0.0.1:18080";
++
++unlink($node->data_dir . '/pg_hba.conf');
++$node->append_conf('pg_hba.conf', qq{
++local all test oauth issuer="$issuer" scope="openid postgres"
++local all testalt oauth issuer="$issuer/alternate" scope="openid postgres alt"
++});
++$node->reload;
+
+my $webserver = PostgreSQL::Test::OAuthServer->new(18080);
+
@@ src/test/modules/oauth_validator/t/001_server.pl (new)
+$webserver->setup();
+$webserver->run();
+
-+$node->connect_ok("dbname=postgres oauth_client_id=f02c6361-0635", "connect",
++my ($log_start, $log_end);
++$log_start = $node->wait_for_log(qr/reloading configuration files/);
++
++my $user = "test";
++$node->connect_ok("user=$user dbname=postgres oauth_client_id=f02c6361-0635", "connect",
+ expected_stderr => qr@Visit https://example\.com/ and enter the code: postgresuser@);
+
++$log_end = $node->wait_for_log(qr/connection authorized/, $log_start);
++$node->log_check("user $user: validator receives correct parameters", $log_start,
++ log_like => [
++ qr/oauth_validator: token="9243959234", role="$user"/,
++ qr/oauth_validator: issuer="\Q$issuer\E", scope="openid postgres"/,
++ ]);
++$node->log_check("user $user: validator sets authenticated identity", $log_start,
++ log_like => [
++ qr/connection authenticated: identity="test" method=oauth/,
++ ]);
++$log_start = $log_end;
++
++# The /alternate issuer uses slightly different parameters.
++$user = "testalt";
++$node->connect_ok("user=$user dbname=postgres oauth_client_id=f02c6361-0636", "connect",
++ expected_stderr => qr@Visit https://example\.org/ and enter the code: postgresuser@);
++
++$log_end = $node->wait_for_log(qr/connection authorized/, $log_start);
++$node->log_check("user $user: validator receives correct parameters", $log_start,
++ log_like => [
++ qr/oauth_validator: token="9243959234-alt", role="$user"/,
++ qr|oauth_validator: issuer="\Q$issuer/alternate\E", scope="openid postgres alt"|,
++ ]);
++$node->log_check("user $user: validator sets authenticated identity", $log_start,
++ log_like => [
++ qr/connection authenticated: identity="testalt" method=oauth/,
++ ]);
++$log_start = $log_end;
++
+$node->stop;
+
+done_testing();
@@ src/test/modules/oauth_validator/validator.c (new)
+
+#include "fmgr.h"
+#include "libpq/oauth.h"
++#include "miscadmin.h"
+#include "utils/memutils.h"
+
+PG_MODULE_MAGIC;
@@ src/test/modules/oauth_validator/validator.c (new)
+ return &validator_callbacks;
+}
+
++#define PRIVATE_COOKIE ((void *) 13579)
++
+static void
+validator_startup(ValidatorModuleState *state)
+{
-+ /* do nothing */
++ state->private_data = PRIVATE_COOKIE;
+}
+
+static void
@@ src/test/modules/oauth_validator/validator.c (new)
+{
+ ValidatorModuleResult *res;
+
++ /* Check to make sure our private state still exists. */
++ if (state->private_data != PRIVATE_COOKIE)
++ elog(ERROR, "oauth_validator: private state cookie changed to %p",
++ state->private_data);
++
+ res = palloc(sizeof(ValidatorModuleResult));
+
-+ elog(LOG, "XXX: validating %s for %s", token, role);
++ elog(LOG, "oauth_validator: token=\"%s\", role=\"%s\"", token, role);
++ elog(LOG, "oauth_validator: issuer=\"%s\", scope=\"%s\"",
++ MyProcPort->hba->oauth_issuer,
++ MyProcPort->hba->oauth_scope);
+
-+ res->authenticated = true;
++ res->authorized = true;
+ res->authn_id = pstrdup(role);
+
+ return res;
@@ src/test/perl/PostgreSQL/Test/OAuthServer.pm (new)
+ #}
+ #printf ": POST: " . $request{'content'} . "\n" if defined $request{'content'};
+
++ my $alternate = 0;
++ if ($request{'object'} =~ qr|^/alternate(/.*)$|)
++ {
++ $alternate = 1;
++ $request{'object'} = $1;
++ }
++
+ if ($request{'object'} eq '/.well-known/openid-configuration')
+ {
++ my $issuer = "http://localhost:$self->{'port'}";
++ if ($alternate)
++ {
++ $issuer .= "/alternate";
++ }
++
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: application/json\r\n";
+ print $fh "\r\n";
+ print $fh <<EOR;
+ {
-+ "issuer": "http://localhost:$self->{'port'}",
-+ "token_endpoint": "http://localhost:$self->{'port'}/token",
-+ "device_authorization_endpoint": "http://localhost:$self->{'port'}/authorize",
++ "issuer": "$issuer",
++ "token_endpoint": "$issuer/token",
++ "device_authorization_endpoint": "$issuer/authorize",
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
@@ src/test/perl/PostgreSQL/Test/OAuthServer.pm (new)
+ }
+ elsif ($request{'object'} eq '/authorize')
+ {
++ my $uri = "https://example.com/";
++ if ($alternate)
++ {
++ $uri = "https://example.org/";
++ }
++
+ print ": returning device_code\n";
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: application/json\r\n";
@@ src/test/perl/PostgreSQL/Test/OAuthServer.pm (new)
+ "device_code": "postgres",
+ "user_code" : "postgresuser",
+ "interval" : 0,
-+ "verification_uri" : "https://example.com/",
++ "verification_uri" : "$uri",
+ "expires-in": 5
+ }
+EOR
+ }
+ elsif ($request{'object'} eq '/token')
+ {
++ my $token = "9243959234";
++ if ($alternate)
++ {
++ $token .= "-alt";
++ }
++
+ print ": returning token\n";
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: application/json\r\n";
+ print $fh "\r\n";
+ print $fh <<EOR;
+ {
-+ "access_token": "9243959234",
++ "access_token": "$token",
+ "token_type": "bearer"
+ }
+EOR
7: 47236c5644 ! 7: e1da97fb50 Add pytest suite for OAuth
@@ src/test/python/client/test_client.py (new)
+
+import pq3
+
++from .test_oauth import alt_patterns
++
+
+def finish_handshake(conn):
+ """
@@ src/test/python/client/test_client.py (new)
+ sock, client = accept()
+ sock.close()
+
-+ expected = "server closed the connection unexpectedly"
++ expected = alt_patterns(
++ "server closed the connection unexpectedly",
++ # On some platforms, ECONNABORTED gets set instead.
++ "Software caused connection abort",
++ )
+ with pytest.raises(psycopg2.OperationalError, match=expected):
+ client.check_completed()
+
@@ src/test/python/pq3.py (new)
+Pq3 = Struct(
+ "type" / types,
+ "len" / Default(Int32ub, lambda this: _payload_len(this) + 4),
-+ "payload" / FixedSized(this.len - 4, Default(_payload, b"")),
++ "payload"
++ / IfThenElse(
++ # Allow tests to explicitly pass an incorrect length during testing, by
++ # not enforcing a FixedSized during build. (The len calculation above
++ # defaults to the correct size.)
++ this._building,
++ Optional(_payload),
++ FixedSized(this.len - 4, Default(_payload, b"")),
++ ),
+)
+
+
@@ src/test/python/server/oauthtest.c (new)
+
+ if (reflect_role)
+ {
-+ res->authenticated = true;
++ res->authorized = true;
+ res->authn_id = pstrdup(role); /* TODO: constify? */
+ }
+ else
+ {
+ if (*expected_bearer && !strcmp(token, expected_bearer))
-+ res->authenticated = true;
++ res->authorized = true;
+ if (set_authn_id)
+ res->authn_id = authn_id;
+ }
@@ src/test/python/server/test_oauth.py (new)
+
+import psycopg2
+import pytest
++from construct import Container
+from psycopg2 import sql
+
+import pq3
@@ src/test/python/server/test_oauth.py (new)
+
+
+@contextlib.contextmanager
-+def prepend_file(path, lines):
++def prepend_file(path, lines, *, suffix=".bak"):
+ """
+ A context manager that prepends a file on disk with the desired lines of
+ text. When the context manager is exited, the file will be restored to its
+ original contents.
+ """
+ # First make a backup of the original file.
-+ bak = path + ".bak"
++ bak = path + suffix
+ shutil.copy2(path, bak)
+
+ try:
@@ src/test/python/server/test_oauth.py (new)
+ b"Bearer trailingtab\t",
+ b"Bearer me@example.com",
+ b"Beare abcd",
++ b" Bearer leadingspace",
+ b'OAuth realm="Example"',
+ b"",
+ ],
@@ src/test/python/server/test_oauth.py (new)
+ id="error response in initial message",
+ ),
+ pytest.param(
-+ pq3.types.PasswordMessage,
-+ b"x" * (MAX_SASL_MESSAGE_LENGTH + 1),
++ None,
++ # Sending an actual 65k packet results in ECONNRESET on Windows, and
++ # it floods the tests' connection log uselessly, so just fake the
++ # length and send a smaller number of bytes.
++ dict(
++ type=pq3.types.PasswordMessage,
++ len=MAX_SASL_MESSAGE_LENGTH + 1,
++ payload=b"x" * 512,
++ ),
+ ExpectedError(
+ INVALID_AUTHORIZATION_ERRCODE, "bearer authentication failed"
+ ),
@@ src/test/python/server/test_oauth.py (new)
+ ),
+ id="multiple auth values",
+ ),
++ pytest.param(
++ pq3.types.PasswordMessage,
++ pq3.SASLInitialResponse.build(
++ dict(
++ name=b"OAUTHBEARER",
++ data=b"y,,\x01=\x01\x01",
++ )
++ ),
++ ExpectedError(
++ PROTOCOL_VIOLATION_ERRCODE,
++ "malformed OAUTHBEARER message",
++ "empty key name",
++ ),
++ id="empty key",
++ ),
++ pytest.param(
++ pq3.types.PasswordMessage,
++ pq3.SASLInitialResponse.build(
++ dict(
++ name=b"OAUTHBEARER",
++ data=b"y,,\x01my key= \x01\x01",
++ )
++ ),
++ ExpectedError(
++ PROTOCOL_VIOLATION_ERRCODE,
++ "malformed OAUTHBEARER message",
++ "invalid key name",
++ ),
++ id="whitespace in key name",
++ ),
++ pytest.param(
++ pq3.types.PasswordMessage,
++ pq3.SASLInitialResponse.build(
++ dict(
++ name=b"OAUTHBEARER",
++ data=b"y,,\x01key=a\x05b\x01\x01",
++ )
++ ),
++ ExpectedError(
++ PROTOCOL_VIOLATION_ERRCODE,
++ "malformed OAUTHBEARER message",
++ "invalid value",
++ ),
++ id="junk in value",
++ ),
+ ],
+)
+def test_oauth_bad_initial_response(conn, oauth_ctx, type, payload, err):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # The server expects a SASL response; give it something else instead.
-+ if not isinstance(payload, dict):
-+ payload = dict(payload_data=payload)
-+ pq3.send(conn, type, **payload)
++ if type is not None:
++ # Build a new packet of the desired type.
++ if not isinstance(payload, dict):
++ payload = dict(payload_data=payload)
++ pq3.send(conn, type, **payload)
++ else:
++ # The test has a custom packet to send. (The only reason to do this is
++ # if the packet is corrupt or otherwise unbuildable/unparsable, so we
++ # don't use the standard pq3.send().)
++ conn.write(pq3.Pq3.build(payload))
++ conn.end_packet(Container(payload))
+
+ resp = pq3.recv1(conn)
+ err.match(resp)
@@ src/test/python/server/test_oauth.py (new)
+ row = resp.payload
+ expected = b"oauth:" + username.encode("utf-8")
+ assert row.columns == [expected]
++
++
++@pytest.fixture
++def odd_oauth_ctx(postgres_instance, oauth_ctx):
++ """
++ Adds an HBA entry with messed up issuer/scope settings, to pin the server
++ behavior.
++
++ TODO: these should really be rejected in the HBA rather than passed through
++ by the server.
++ """
++ id = secrets.token_hex(4)
++
++ class Context:
++ user = oauth_ctx.user
++ dbname = oauth_ctx.dbname
++
++ # Both of these embedded double-quotes are invalid; they're prohibited
++ # in both URLs and OAuth scope identifiers.
++ issuer = oauth_ctx.issuer + '/"/'
++ scope = oauth_ctx.scope + ' quo"ted'
++
++ ctx = Context()
++ hba_issuer = ctx.issuer.replace('"', '""')
++ hba_scope = ctx.scope.replace('"', '""')
++ hba_lines = [
++ f'host {ctx.dbname} {ctx.user} samehost oauth issuer="{hba_issuer}" scope="{hba_scope}"\n',
++ ]
++
++ if platform.system() == "Windows":
++ # XXX why is 'samehost' not behaving as expected on Windows?
++ for l in list(hba_lines):
++ hba_lines.append(l.replace("samehost", "::1/128"))
++
++ host, port = postgres_instance
++ conn = psycopg2.connect(host=host, port=port)
++ conn.autocommit = True
++
++ with contextlib.closing(conn):
++ c = conn.cursor()
++
++ # Replace pg_hba. Note that it's already been replaced once by
++ # oauth_ctx, so use a different backup prefix in prepend_file().
++ c.execute("SHOW hba_file;")
++ hba = c.fetchone()[0]
++
++ with prepend_file(hba, hba_lines, suffix=".bak2"):
++ c.execute("SELECT pg_reload_conf();")
++
++ yield ctx
++
++ # Put things back the way they were.
++ c.execute("SELECT pg_reload_conf();")
++
++
++def test_odd_server_response(odd_oauth_ctx, connect):
++ """
++ Verifies that the server is correctly escaping the JSON in its failure
++ response.
++ """
++ conn = connect()
++ begin_oauth_handshake(conn, odd_oauth_ctx, user=odd_oauth_ctx.user)
++
++ # Send an empty auth initial response, which will force an authn failure.
++ send_initial_response(conn, auth=b"")
++
++ expect_handshake_failure(conn, odd_oauth_ctx)
## src/test/python/server/test_server.py (new) ##
@@
@@ src/test/python/test_pq3.py (new)
+ [
+ pytest.param(
+ dict(type=b"*", len=5),
-+ b"*\x00\x00\x00\x05\x00",
++ b"*\x00\x00\x00\x05",
+ id="type and len set explicitly",
+ ),
+ pytest.param(
@@ src/test/python/test_pq3.py (new)
+ id="implied len with payload",
+ ),
+ pytest.param(
++ dict(type=b"*", len=12, payload=b"1234"),
++ b"*\x00\x00\x00\x0C1234",
++ id="overridden len (payload underflow)",
++ ),
++ pytest.param(
++ dict(type=b"*", len=5, payload=b"1234"),
++ b"*\x00\x00\x00\x051234",
++ id="overridden len (payload overflow)",
++ ),
++ pytest.param(
+ dict(type=pq3.types.AuthnRequest, payload=dict(type=pq3.authn.OK)),
+ b"R\x00\x00\x00\x08\x00\x00\x00\x00",
+ id="implied len/type for AuthenticationOK",
8: 116e17eeee = 8: 185f9902fd XXX temporary patches to build and test
9: 28756eda1c ! 9: c4d850a7c4 WIP: Python OAuth provider implementation
@@ src/test/modules/oauth_validator/meson.build: tests += {
}
## src/test/modules/oauth_validator/t/001_server.pl ##
-@@ src/test/modules/oauth_validator/t/001_server.pl: $node->append_conf('postgresql.conf', "shared_preload_libraries = 'validator'\n"
- $node->append_conf('postgresql.conf', "oauth_validator_library = 'validator'\n");
- $node->start;
+@@ src/test/modules/oauth_validator/t/001_server.pl: $node->start;
+ $node->safe_psql('postgres', 'CREATE USER test;');
+ $node->safe_psql('postgres', 'CREATE USER testalt;');
--reset_pg_hba($node, 'all', 'all', 'oauth issuer="127.0.0.1:18080" scope="openid postgres"');
--
--my $webserver = PostgreSQL::Test::OAuthServer->new(18080);
+-my $issuer = "127.0.0.1:18080";
+my $webserver = PostgreSQL::Test::OAuthServer->new();
+$webserver->run();
++
++my $port = $webserver->port();
++my $issuer = "127.0.0.1:$port";
- my $port = $webserver->port();
+ unlink($node->data_dir . '/pg_hba.conf');
+ $node->append_conf('pg_hba.conf', qq{
+@@ src/test/modules/oauth_validator/t/001_server.pl: local all testalt oauth issuer="$issuer/alternate" scope="openid postgres alt"
+ });
+ $node->reload;
+
+-my $webserver = PostgreSQL::Test::OAuthServer->new(18080);
+-
+-my $port = $webserver->port();
-
-is($port, 18080, "Port is 18080");
-
-$webserver->setup();
-$webserver->run();
-+reset_pg_hba($node, 'all', 'all', 'oauth issuer="127.0.0.1:' . $port . '" scope="openid postgres"');
+-
+ my ($log_start, $log_end);
+ $log_start = $node->wait_for_log(qr/reloading configuration files/);
- $node->connect_ok("dbname=postgres oauth_client_id=f02c6361-0635", "connect",
- expected_stderr => qr@Visit https://example\.com/ and enter the code: postgresuser@);
+@@ src/test/modules/oauth_validator/t/001_server.pl: $node->log_check("user $user: validator sets authenticated identity", $log_start
+ ]);
+ $log_start = $log_end;
+$webserver->stop();
$node->stop;
@@ src/test/modules/oauth_validator/t/oauth_server.py (new)
+class OAuthHandler(http.server.BaseHTTPRequestHandler):
+ JsonObject = dict[str, object] # TypeAlias is not available until 3.10
+
++ def _check_issuer(self):
++ """
++ Switches the behavior of the provider depending on the issuer URI.
++ """
++ self._alt_issuer = self.path.startswith("/alternate/")
++ if self._alt_issuer:
++ self.path = self.path.removeprefix("/alternate")
++
+ def do_GET(self):
++ self._check_issuer()
++
+ if self.path == "/.well-known/openid-configuration":
+ resp = self.config()
+ else:
@@ src/test/modules/oauth_validator/t/oauth_server.py (new)
+ self._send_json(resp)
+
+ def do_POST(self):
++ self._check_issuer()
++
+ if self.path == "/authorize":
+ resp = self.authorization()
+ elif self.path == "/token":
@@ src/test/modules/oauth_validator/t/oauth_server.py (new)
+
+ def config(self) -> JsonObject:
+ port = self.server.socket.getsockname()[1]
++ issuer = f"http://localhost:{port}"
++ if self._alt_issuer:
++ issuer += "/alternate"
+
+ return {
-+ "issuer": f"http://localhost:{port}",
-+ "token_endpoint": f"http://localhost:{port}/token",
-+ "device_authorization_endpoint": f"http://localhost:{port}/authorize",
++ "issuer": issuer,
++ "token_endpoint": issuer + "/token",
++ "device_authorization_endpoint": issuer + "/authorize",
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
@@ src/test/modules/oauth_validator/t/oauth_server.py (new)
+ }
+
+ def authorization(self) -> JsonObject:
++ uri = "https://example.com/"
++ if self._alt_issuer:
++ uri = "https://example.org/"
++
+ return {
+ "device_code": "postgres",
+ "user_code": "postgresuser",
+ "interval": 0,
-+ "verification_uri": "https://example.com/",
++ "verification_uri": uri,
+ "expires-in": 5,
+ }
+
+ def token(self) -> JsonObject:
++ token = "9243959234"
++ if self._alt_issuer:
++ token += "-alt"
++
+ return {
-+ "access_token": "9243959234",
++ "access_token": token,
+ "token_type": "bearer",
+ }
+
@@ src/test/perl/PostgreSQL/Test/OAuthServer.pm: sub port
- #}
- #printf ": POST: " . $request{'content'} . "\n" if defined $request{'content'};
-
+- my $alternate = 0;
+- if ($request{'object'} =~ qr|^/alternate(/.*)$|)
+- {
+- $alternate = 1;
+- $request{'object'} = $1;
+- }
+-
- if ($request{'object'} eq '/.well-known/openid-configuration')
- {
+- my $issuer = "http://localhost:$self->{'port'}";
+- if ($alternate)
+- {
+- $issuer .= "/alternate";
+- }
+-
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: application/json\r\n";
- print $fh "\r\n";
- print $fh <<EOR;
- {
-- "issuer": "http://localhost:$self->{'port'}",
-- "token_endpoint": "http://localhost:$self->{'port'}/token",
-- "device_authorization_endpoint": "http://localhost:$self->{'port'}/authorize",
+- "issuer": "$issuer",
+- "token_endpoint": "$issuer/token",
+- "device_authorization_endpoint": "$issuer/authorize",
- "response_types_supported": ["token"],
- "subject_types_supported": ["public"],
- "id_token_signing_alg_values_supported": ["RS256"],
@@ src/test/perl/PostgreSQL/Test/OAuthServer.pm: sub port
- }
- elsif ($request{'object'} eq '/authorize')
- {
+- my $uri = "https://example.com/";
+- if ($alternate)
+- {
+- $uri = "https://example.org/";
+- }
+-
- print ": returning device_code\n";
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: application/json\r\n";
@@ src/test/perl/PostgreSQL/Test/OAuthServer.pm: sub port
- "device_code": "postgres",
- "user_code" : "postgresuser",
- "interval" : 0,
-- "verification_uri" : "https://example.com/",
+- "verification_uri" : "$uri",
- "expires-in": 5
- }
-EOR
- }
- elsif ($request{'object'} eq '/token')
- {
+- my $token = "9243959234";
+- if ($alternate)
+- {
+- $token .= "-alt";
+- }
+-
- print ": returning token\n";
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: application/json\r\n";
- print $fh "\r\n";
- print $fh <<EOR;
- {
-- "access_token": "9243959234",
+- "access_token": "$token",
- "token_type": "bearer"
- }
-EOR
v20-0003-Explicitly-require-password-for-SCRAM-exchange.patch (application/octet-stream, 3.2K) download
From 10b6d2a6b9bc81357bda8438371821996ba5065d Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <dgustafsson@postgresql.org>
Date: Fri, 23 Feb 2024 11:19:55 +0100
Subject: [PATCH v20 3/9] Explicitly require password for SCRAM exchange
This refactors the SASL init flow to set password_needed on the two
SCRAM exchanges currently supported. The code already required this
but was set up in such a way that all SASL exchanges required using
a password, a restriction which may not hold for all exchanges (the
example at hand being the proposed OAuthbearer exchange).
This was extracted from a larger patchset to introduce OAuthBearer
authentication and authorization.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://postgr.es/m/d1b467a78e0e36ed85a09adf979d04cf124a9d4b.camel@vmware.com
---
src/interfaces/libpq/fe-auth.c | 28 +++++++++++++++-------------
1 file changed, 15 insertions(+), 13 deletions(-)
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index cf8af4c62e..81ec08485d 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -425,7 +425,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
int initialresponselen;
const char *selected_mechanism;
PQExpBufferData mechanism_buf;
- char *password;
+ char *password = NULL;
SASLStatus status;
initPQExpBuffer(&mechanism_buf);
@@ -446,8 +446,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
/*
* Parse the list of SASL authentication mechanisms in the
* AuthenticationSASL message, and select the best mechanism that we
- * support. SCRAM-SHA-256-PLUS and SCRAM-SHA-256 are the only ones
- * supported at the moment, listed by order of decreasing importance.
+ * support. Mechanisms are listed by order of decreasing importance.
*/
selected_mechanism = NULL;
for (;;)
@@ -487,6 +486,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
selected_mechanism = SCRAM_SHA_256_PLUS_NAME;
conn->sasl = &pg_scram_mech;
+ conn->password_needed = true;
}
#else
/*
@@ -522,6 +522,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
selected_mechanism = SCRAM_SHA_256_NAME;
conn->sasl = &pg_scram_mech;
+ conn->password_needed = true;
}
}
@@ -545,18 +546,19 @@ pg_SASL_init(PGconn *conn, int payloadlen)
/*
* First, select the password to use for the exchange, complaining if
- * there isn't one. Currently, all supported SASL mechanisms require a
- * password, so we can just go ahead here without further distinction.
+ * there isn't one and the selected SASL mechanism needs it.
*/
- conn->password_needed = true;
- password = conn->connhost[conn->whichhost].password;
- if (password == NULL)
- password = conn->pgpass;
- if (password == NULL || password[0] == '\0')
+ if (conn->password_needed)
{
- appendPQExpBufferStr(&conn->errorMessage,
- PQnoPasswordSupplied);
- goto error;
+ password = conn->connhost[conn->whichhost].password;
+ if (password == NULL)
+ password = conn->pgpass;
+ if (password == NULL || password[0] == '\0')
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ PQnoPasswordSupplied);
+ goto error;
+ }
}
Assert(conn->sasl);
--
2.34.1
v20-0002-Refactor-SASL-exchange-to-return-tri-state-statu.patch (application/octet-stream, 9.9K) download
From f78c79ea68dd714864609e2362533ccf5c0934d1 Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <dgustafsson@postgresql.org>
Date: Fri, 23 Feb 2024 11:09:54 +0100
Subject: [PATCH v20 2/9] Refactor SASL exchange to return tri-state status
The SASL exchange callback returned state in to output variables:
done and success. This refactors that logic by introducing a new
return variable of type SASLStatus which makes the code easier to
read and understand, and prepares for future SASL exchanges which
operate asynchronously.
This was extracted from a larger patchset to introduce OAuthBearer
authentication and authorization.
Author: Jacob Champion <jacob.champion@enterprisedb.com>
Discussion: https://postgr.es/m/d1b467a78e0e36ed85a09adf979d04cf124a9d4b.camel@vmware.com
---
src/interfaces/libpq/fe-auth-sasl.h | 31 +++++++----
src/interfaces/libpq/fe-auth-scram.c | 78 +++++++++++++---------------
src/interfaces/libpq/fe-auth.c | 28 +++++-----
src/tools/pgindent/typedefs.list | 1 +
4 files changed, 71 insertions(+), 67 deletions(-)
diff --git a/src/interfaces/libpq/fe-auth-sasl.h b/src/interfaces/libpq/fe-auth-sasl.h
index ee5d1525b5..4eecf53a15 100644
--- a/src/interfaces/libpq/fe-auth-sasl.h
+++ b/src/interfaces/libpq/fe-auth-sasl.h
@@ -21,6 +21,17 @@
#include "libpq-fe.h"
+/*
+ * Possible states for the SASL exchange, see the comment on exchange for an
+ * explanation of these.
+ */
+typedef enum
+{
+ SASL_COMPLETE = 0,
+ SASL_FAILED,
+ SASL_CONTINUE,
+} SASLStatus;
+
/*
* Frontend SASL mechanism callbacks.
*
@@ -59,7 +70,8 @@ typedef struct pg_fe_sasl_mech
* Produces a client response to a server challenge. As a special case
* for client-first SASL mechanisms, exchange() is called with a NULL
* server response once at the start of the authentication exchange to
- * generate an initial response.
+ * generate an initial response. Returns a SASLStatus indicating the
+ * state and status of the exchange.
*
* Input parameters:
*
@@ -79,22 +91,23 @@ typedef struct pg_fe_sasl_mech
*
* output: A malloc'd buffer containing the client's response to
* the server (can be empty), or NULL if the exchange should
- * be aborted. (*success should be set to false in the
+ * be aborted. (The callback should return SASL_FAILED in the
* latter case.)
*
* outputlen: The length (0 or higher) of the client response buffer,
* ignored if output is NULL.
*
- * done: Set to true if the SASL exchange should not continue,
- * because the exchange is either complete or failed
+ * Return value:
*
- * success: Set to true if the SASL exchange completed successfully.
- * Ignored if *done is false.
+ * SASL_CONTINUE: The output buffer is filled with a client response.
+ * Additional server challenge is expected
+ * SASL_COMPLETE: The SASL exchange has completed successfully.
+ * SASL_FAILED: The exchange has failed and the connection should be
+ * dropped.
*--------
*/
- void (*exchange) (void *state, char *input, int inputlen,
- char **output, int *outputlen,
- bool *done, bool *success);
+ SASLStatus (*exchange) (void *state, char *input, int inputlen,
+ char **output, int *outputlen);
/*--------
* channel_bound()
diff --git a/src/interfaces/libpq/fe-auth-scram.c b/src/interfaces/libpq/fe-auth-scram.c
index 04f0e5713d..0bb820e0d9 100644
--- a/src/interfaces/libpq/fe-auth-scram.c
+++ b/src/interfaces/libpq/fe-auth-scram.c
@@ -24,9 +24,8 @@
/* The exported SCRAM callback mechanism. */
static void *scram_init(PGconn *conn, const char *password,
const char *sasl_mechanism);
-static void scram_exchange(void *opaq, char *input, int inputlen,
- char **output, int *outputlen,
- bool *done, bool *success);
+static SASLStatus scram_exchange(void *opaq, char *input, int inputlen,
+ char **output, int *outputlen);
static bool scram_channel_bound(void *opaq);
static void scram_free(void *opaq);
@@ -202,17 +201,14 @@ scram_free(void *opaq)
/*
* Exchange a SCRAM message with backend.
*/
-static void
+static SASLStatus
scram_exchange(void *opaq, char *input, int inputlen,
- char **output, int *outputlen,
- bool *done, bool *success)
+ char **output, int *outputlen)
{
fe_scram_state *state = (fe_scram_state *) opaq;
PGconn *conn = state->conn;
const char *errstr = NULL;
- *done = false;
- *success = false;
*output = NULL;
*outputlen = 0;
@@ -225,12 +221,12 @@ scram_exchange(void *opaq, char *input, int inputlen,
if (inputlen == 0)
{
libpq_append_conn_error(conn, "malformed SCRAM message (empty message)");
- goto error;
+ return SASL_FAILED;
}
if (inputlen != strlen(input))
{
libpq_append_conn_error(conn, "malformed SCRAM message (length mismatch)");
- goto error;
+ return SASL_FAILED;
}
}
@@ -240,61 +236,59 @@ scram_exchange(void *opaq, char *input, int inputlen,
/* Begin the SCRAM handshake, by sending client nonce */
*output = build_client_first_message(state);
if (*output == NULL)
- goto error;
+ return SASL_FAILED;
*outputlen = strlen(*output);
- *done = false;
state->state = FE_SCRAM_NONCE_SENT;
- break;
+ return SASL_CONTINUE;
case FE_SCRAM_NONCE_SENT:
/* Receive salt and server nonce, send response. */
if (!read_server_first_message(state, input))
- goto error;
+ return SASL_FAILED;
*output = build_client_final_message(state);
if (*output == NULL)
- goto error;
+ return SASL_FAILED;
*outputlen = strlen(*output);
- *done = false;
state->state = FE_SCRAM_PROOF_SENT;
- break;
+ return SASL_CONTINUE;
case FE_SCRAM_PROOF_SENT:
- /* Receive server signature */
- if (!read_server_final_message(state, input))
- goto error;
-
- /*
- * Verify server signature, to make sure we're talking to the
- * genuine server.
- */
- if (!verify_server_signature(state, success, &errstr))
- {
- libpq_append_conn_error(conn, "could not verify server signature: %s", errstr);
- goto error;
- }
-
- if (!*success)
{
- libpq_append_conn_error(conn, "incorrect server signature");
+ bool match;
+
+ /* Receive server signature */
+ if (!read_server_final_message(state, input))
+ return SASL_FAILED;
+
+ /*
+ * Verify server signature, to make sure we're talking to the
+ * genuine server.
+ */
+ if (!verify_server_signature(state, &match, &errstr))
+ {
+ libpq_append_conn_error(conn, "could not verify server signature: %s", errstr);
+ return SASL_FAILED;
+ }
+
+ if (!match)
+ {
+ libpq_append_conn_error(conn, "incorrect server signature");
+ }
+ state->state = FE_SCRAM_FINISHED;
+ state->conn->client_finished_auth = true;
+ return match ? SASL_COMPLETE : SASL_FAILED;
}
- *done = true;
- state->state = FE_SCRAM_FINISHED;
- state->conn->client_finished_auth = true;
- break;
default:
/* shouldn't happen */
libpq_append_conn_error(conn, "invalid SCRAM exchange state");
- goto error;
+ break;
}
- return;
-error:
- *done = true;
- *success = false;
+ return SASL_FAILED;
}
/*
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index 1a8e4f6fbf..cf8af4c62e 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -423,11 +423,10 @@ pg_SASL_init(PGconn *conn, int payloadlen)
{
char *initialresponse = NULL;
int initialresponselen;
- bool done;
- bool success;
const char *selected_mechanism;
PQExpBufferData mechanism_buf;
char *password;
+ SASLStatus status;
initPQExpBuffer(&mechanism_buf);
@@ -575,12 +574,11 @@ pg_SASL_init(PGconn *conn, int payloadlen)
goto oom_error;
/* Get the mechanism-specific Initial Client Response, if any */
- conn->sasl->exchange(conn->sasl_state,
- NULL, -1,
- &initialresponse, &initialresponselen,
- &done, &success);
+ status = conn->sasl->exchange(conn->sasl_state,
+ NULL, -1,
+ &initialresponse, &initialresponselen);
- if (done && !success)
+ if (status == SASL_FAILED)
goto error;
/*
@@ -629,10 +627,9 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
{
char *output;
int outputlen;
- bool done;
- bool success;
int res;
char *challenge;
+ SASLStatus status;
/* Read the SASL challenge from the AuthenticationSASLContinue message. */
challenge = malloc(payloadlen + 1);
@@ -651,13 +648,12 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
/* For safety and convenience, ensure the buffer is NULL-terminated. */
challenge[payloadlen] = '\0';
- conn->sasl->exchange(conn->sasl_state,
- challenge, payloadlen,
- &output, &outputlen,
- &done, &success);
+ status = conn->sasl->exchange(conn->sasl_state,
+ challenge, payloadlen,
+ &output, &outputlen);
free(challenge); /* don't need the input anymore */
- if (final && !done)
+ if (final && status == SASL_CONTINUE)
{
if (outputlen != 0)
free(output);
@@ -670,7 +666,7 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
* If the exchange is not completed yet, we need to make sure that the
* SASL mechanism has generated a message to send back.
*/
- if (output == NULL && !done)
+ if (output == NULL && status == SASL_CONTINUE)
{
libpq_append_conn_error(conn, "no client response found after SASL exchange success");
return STATUS_ERROR;
@@ -692,7 +688,7 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
return STATUS_ERROR;
}
- if (done && !success)
+ if (status == SASL_FAILED)
return STATUS_ERROR;
return STATUS_OK;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 95ae7845d8..86d68f7a7a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -2423,6 +2423,7 @@ RuleLock
RuleStmt
RunningTransactions
RunningTransactionsData
+SASLStatus
SC_HANDLE
SECURITY_ATTRIBUTES
SECURITY_STATUS
--
2.34.1
v20-0005-backend-add-OAUTHBEARER-SASL-mechanism.patch (application/octet-stream, 41.7K) download
From 5488ac25f5396017c4f1a3c78bc4f807b558e83d Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 4 May 2021 16:21:11 -0700
Subject: [PATCH v20 5/9] backend: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) on the server side. This adds a new
auth method, oauth, to pg_hba.
Because OAuth implementations vary so wildly, and bearer token
validation is heavily dependent on the issuing party, authn/z is done by
communicating with an external program: the oauth_validator_command.
This command must do the following:
1. Receive the bearer token by reading its contents from a file
descriptor passed from the server. (The numeric value of this
descriptor may be inserted into the oauth_validator_command using the
%f specifier.)
This MUST be the first action the command performs. The server will
not begin reading stdout from the command until the token has been
read in full, so if the command tries to print anything and hits a
buffer limit, the backend will deadlock and time out.
2. Validate the bearer token. The correct way to do this depends on the
issuer, but it generally involves either cryptographic operations to
prove that the token was issued by a trusted party, or the
presentation of the bearer token to some other party so that _it_ can
perform validation.
The command MUST maintain confidentiality of the bearer token, since
in most cases it can be used just like a password. (There are ways to
cryptographically bind tokens to client certificates, but they are
way beyond the scope of this commit message.)
If the token cannot be validated, the command must exit with a
non-zero status. Further authentication/authorization is pointless if
the bearer token wasn't issued by someone you trust.
3. Authenticate the user, authorize the user, or both:
a. To authenticate the user, use the bearer token to retrieve some
trusted identifier string for the end user. The exact process for
this is, again, issuer-dependent. The command should print the
authenticated identity string to stdout, followed by a newline.
If the user cannot be authenticated, the validator should not
print anything to stdout. It should also exit with a non-zero
status, unless the token may be used to authorize the connection
through some other means (see below).
On a success, the command may then exit with a zero success code.
By default, the server will then check to make sure the identity
string matches the role that is being used (or matches a usermap
entry, if one is in use).
b. To optionally authorize the user, in combination with the HBA
option trust_validator_authz=1 (see below), the validator simply
returns a zero exit code if the client should be allowed to
connect with its presented role (which can be passed to the
command using the %r specifier), or a non-zero code otherwise.
The hard part is in determining whether the given token truly
authorizes the client to use the given role, which must
unfortunately be left as an exercise to the reader.
This obviously requires some care, as a poorly implemented token
validator may silently open the entire database to anyone with a
bearer token. But it may be a more portable approach, since OAuth
is designed as an authorization framework, not an authentication
framework. For example, the user's bearer token could carry an
"allow_superuser_access" claim, which would authorize pseudonymous
database access as any role. It's then up to the OAuth system
administrators to ensure that allow_superuser_access is doled out
only to the proper users.
c. It's possible that the user can be successfully authenticated but
isn't authorized to connect. In this case, the command may print
the authenticated ID and then fail with a non-zero exit code.
(This makes it easier to see what's going on in the Postgres
logs.)
4. Token validators may optionally log to stderr. This will be printed
verbatim into the Postgres server logs.
The oauth method supports the following HBA options (but note that two
of them are not optional, since we have no way of choosing sensible
defaults):
issuer: Required. The URL of the OAuth issuing party, which the client
must contact to receive a bearer token.
Some real-world examples as of time of writing:
- https://accounts.google.com
- https://login.microsoft.com/[tenant-id]/v2.0
scope: Required. The OAuth scope(s) required for the server to
authenticate and/or authorize the user. This is heavily
deployment-specific, but a simple example is "openid email".
map: Optional. Specify a standard PostgreSQL user map; this works
the same as with other auth methods such as peer. If a map is
not specified, the user ID returned by the token validator
must exactly match the role that's being requested (but see
trust_validator_authz, below).
trust_validator_authz:
Optional. When set to 1, this allows the token validator to
take full control of the authorization process. Standard user
mapping is skipped: if the validator command succeeds, the
client is allowed to connect under its desired role and no
further checks are done.
Unlike the client, servers support OAuth without needing to be built
against libiddawc (since the responsibility for "speaking" OAuth/OIDC
correctly is delegated entirely to the oauth_validator_command).
Several TODOs:
- port to platforms other than "modern Linux/BSD"
- overhaul the communication with oauth_validator_command, which is
currently a bad hack on OpenPipeStream()
- implement more helpful handling of HBA misconfigurations
- use logdetail during auth failures
- deal with role names that can't be safely passed to system() without
shell-escaping
- allow passing the configured issuer to the oauth_validator_command, to
deal with multi-issuer setups
- ...and more.
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
---
src/backend/libpq/Makefile | 1 +
src/backend/libpq/auth-oauth.c | 883 ++++++++++++++++++++++++++++
src/backend/libpq/auth-sasl.c | 10 +-
src/backend/libpq/auth-scram.c | 4 +-
src/backend/libpq/auth.c | 26 +-
src/backend/libpq/hba.c | 31 +-
src/backend/libpq/meson.build | 1 +
src/backend/utils/misc/guc_tables.c | 12 +
src/include/libpq/auth.h | 17 +
src/include/libpq/hba.h | 6 +-
src/include/libpq/oauth.h | 24 +
src/include/libpq/sasl.h | 11 +
src/tools/pgindent/typedefs.list | 1 +
13 files changed, 995 insertions(+), 32 deletions(-)
create mode 100644 src/backend/libpq/auth-oauth.c
create mode 100644 src/include/libpq/oauth.h
diff --git a/src/backend/libpq/Makefile b/src/backend/libpq/Makefile
index 6d385fd6a4..98eb2a8242 100644
--- a/src/backend/libpq/Makefile
+++ b/src/backend/libpq/Makefile
@@ -15,6 +15,7 @@ include $(top_builddir)/src/Makefile.global
# be-fsstubs is here for historical reasons, probably belongs elsewhere
OBJS = \
+ auth-oauth.o \
auth-sasl.o \
auth-scram.o \
auth.o \
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
new file mode 100644
index 0000000000..16596c089a
--- /dev/null
+++ b/src/backend/libpq/auth-oauth.c
@@ -0,0 +1,883 @@
+/*-------------------------------------------------------------------------
+ *
+ * auth-oauth.c
+ * Server-side implementation of the SASL OAUTHBEARER mechanism.
+ *
+ * See the following RFC for more details:
+ * - RFC 7628: https://tools.ietf.org/html/rfc7628
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/backend/libpq/auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <unistd.h>
+#include <fcntl.h>
+
+#include "common/oauth-common.h"
+#include "lib/stringinfo.h"
+#include "libpq/auth.h"
+#include "libpq/hba.h"
+#include "libpq/oauth.h"
+#include "libpq/sasl.h"
+#include "storage/fd.h"
+#include "utils/json.h"
+
+/* GUC */
+char *oauth_validator_command;
+
+static void oauth_get_mechanisms(Port *port, StringInfo buf);
+static void *oauth_init(Port *port, const char *selected_mech, const char *shadow_pass);
+static int oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, const char **logdetail);
+
+/* Mechanism declaration */
+const pg_be_sasl_mech pg_be_oauth_mech = {
+ oauth_get_mechanisms,
+ oauth_init,
+ oauth_exchange,
+
+ PG_MAX_AUTH_TOKEN_LENGTH,
+};
+
+
+typedef enum
+{
+ OAUTH_STATE_INIT = 0,
+ OAUTH_STATE_ERROR,
+ OAUTH_STATE_FINISHED,
+} oauth_state;
+
+struct oauth_ctx
+{
+ oauth_state state;
+ Port *port;
+ const char *issuer;
+ const char *scope;
+};
+
+static char *sanitize_char(char c);
+static char *parse_kvpairs_for_auth(char **input);
+static void generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen);
+static bool validate(Port *port, const char *auth, const char **logdetail);
+static bool run_validator_command(Port *port, const char *token);
+static bool check_exit(FILE **fh, const char *command);
+static bool set_cloexec(int fd);
+static bool username_ok_for_shell(const char *username);
+
+#define KVSEP 0x01
+#define AUTH_KEY "auth"
+#define BEARER_SCHEME "Bearer "
+
+static void
+oauth_get_mechanisms(Port *port, StringInfo buf)
+{
+ /* Only OAUTHBEARER is supported. */
+ appendStringInfoString(buf, OAUTHBEARER_NAME);
+ appendStringInfoChar(buf, '\0');
+}
+
+static void *
+oauth_init(Port *port, const char *selected_mech, const char *shadow_pass)
+{
+ struct oauth_ctx *ctx;
+
+ if (strcmp(selected_mech, OAUTHBEARER_NAME))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("client selected an invalid SASL authentication mechanism")));
+
+ ctx = palloc0(sizeof(*ctx));
+
+ ctx->state = OAUTH_STATE_INIT;
+ ctx->port = port;
+
+ Assert(port->hba);
+ ctx->issuer = port->hba->oauth_issuer;
+ ctx->scope = port->hba->oauth_scope;
+
+ return ctx;
+}
+
+static int
+oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, const char **logdetail)
+{
+ char *p;
+ char cbind_flag;
+ char *auth;
+
+ struct oauth_ctx *ctx = opaq;
+
+ *output = NULL;
+ *outputlen = -1;
+
+ /*
+ * If the client didn't include an "Initial Client Response" in the
+ * SASLInitialResponse message, send an empty challenge, to which the
+ * client will respond with the same data that usually comes in the
+ * Initial Client Response.
+ */
+ if (input == NULL)
+ {
+ Assert(ctx->state == OAUTH_STATE_INIT);
+
+ *output = pstrdup("");
+ *outputlen = 0;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ /*
+ * Check that the input length agrees with the string length of the input.
+ */
+ if (inputlen == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The message is empty.")));
+ if (inputlen != strlen(input))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message length does not match input length.")));
+
+ switch (ctx->state)
+ {
+ case OAUTH_STATE_INIT:
+ /* Handle this case below. */
+ break;
+
+ case OAUTH_STATE_ERROR:
+
+ /*
+ * Only one response is valid for the client during authentication
+ * failure: a single kvsep.
+ */
+ if (inputlen != 1 || *input != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Client did not send a kvsep response.")));
+
+ /* The (failed) handshake is now complete. */
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_FAILURE;
+
+ default:
+ elog(ERROR, "invalid OAUTHBEARER exchange state");
+ return PG_SASL_EXCHANGE_FAILURE;
+ }
+
+ /* Handle the client's initial message. */
+ p = pstrdup(input);
+
+ /*
+ * OAUTHBEARER does not currently define a channel binding (so there is no
+ * OAUTHBEARER-PLUS, and we do not accept a 'p' specifier). We accept a
+ * 'y' specifier purely for the remote chance that a future specification
+ * could define one; then future clients can still interoperate with this
+ * server implementation. 'n' is the expected case.
+ */
+ cbind_flag = *p;
+ switch (cbind_flag)
+ {
+ case 'p':
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The server does not support channel binding for OAuth, but the client message includes channel binding data.")));
+ break;
+
+ case 'y': /* fall through */
+ case 'n':
+ p++;
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Comma expected, but found character \"%s\".",
+ sanitize_char(*p))));
+ p++;
+ break;
+
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected channel-binding flag %s.",
+ sanitize_char(cbind_flag))));
+ }
+
+ /*
+ * Forbid optional authzid (authorization identity). We don't support it.
+ */
+ if (*p == 'a')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("client uses authorization identity, but it is not supported")));
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected attribute %s in client-first-message.",
+ sanitize_char(*p))));
+ p++;
+
+ /* All remaining fields are separated by the RFC's kvsep (\x01). */
+ if (*p != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Key-value separator expected, but found character %s.",
+ sanitize_char(*p))));
+ p++;
+
+ auth = parse_kvpairs_for_auth(&p);
+ if (!auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message does not contain an auth value.")));
+
+ /* We should be at the end of our message. */
+ if (*p)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains additional data after the final terminator.")));
+
+ if (!validate(ctx->port, auth, logdetail))
+ {
+ generate_error_response(ctx, output, outputlen);
+
+ ctx->state = OAUTH_STATE_ERROR;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_SUCCESS;
+}
+
+/*
+ * Convert an arbitrary byte to printable form. For error messages.
+ *
+ * If it's a printable ASCII character, print it as a single character.
+ * otherwise, print it in hex.
+ *
+ * The returned pointer points to a static buffer.
+ */
+static char *
+sanitize_char(char c)
+{
+ static char buf[5];
+
+ if (c >= 0x21 && c <= 0x7E)
+ snprintf(buf, sizeof(buf), "'%c'", c);
+ else
+ snprintf(buf, sizeof(buf), "0x%02x", (unsigned char) c);
+ return buf;
+}
+
+/*
+ * Performs syntactic validation of a key and value from the initial client
+ * response. (Semantic validation of interesting values must be performed
+ * later.)
+ */
+static void
+validate_kvpair(const char *key, const char *val)
+{
+ /*-----
+ * From Sec 3.1:
+ * key = 1*(ALPHA)
+ */
+ static const char *key_allowed_set =
+ "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
+
+ size_t span;
+
+ if (!key[0])
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an empty key name.")));
+
+ span = strspn(key, key_allowed_set);
+ if (key[span] != '\0')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an invalid key name.")));
+
+ /*-----
+ * From Sec 3.1:
+ * value = *(VCHAR / SP / HTAB / CR / LF )
+ *
+ * The VCHAR (visible character) class is large; a loop is more
+ * straightforward than strspn().
+ */
+ for (; *val; ++val)
+ {
+ if (0x21 <= *val && *val <= 0x7E)
+ continue; /* VCHAR */
+
+ switch (*val)
+ {
+ case ' ':
+ case '\t':
+ case '\r':
+ case '\n':
+ continue; /* SP, HTAB, CR, LF */
+
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an invalid value.")));
+ }
+ }
+}
+
+/*
+ * Consumes all kvpairs in an OAUTHBEARER exchange message. If the "auth" key is
+ * found, its value is returned.
+ */
+static char *
+parse_kvpairs_for_auth(char **input)
+{
+ char *pos = *input;
+ char *auth = NULL;
+
+ /*----
+ * The relevant ABNF, from Sec. 3.1:
+ *
+ * kvsep = %x01
+ * key = 1*(ALPHA)
+ * value = *(VCHAR / SP / HTAB / CR / LF )
+ * kvpair = key "=" value kvsep
+ * ;;gs2-header = See RFC 5801
+ * client-resp = (gs2-header kvsep *kvpair kvsep) / kvsep
+ *
+ * By the time we reach this code, the gs2-header and initial kvsep have
+ * already been validated. We start at the beginning of the first kvpair.
+ */
+
+ while (*pos)
+ {
+ char *end;
+ char *sep;
+ char *key;
+ char *value;
+
+ /*
+ * Find the end of this kvpair. Note that input is null-terminated by
+ * the SASL code, so the strchr() is bounded.
+ */
+ end = strchr(pos, KVSEP);
+ if (!end)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an unterminated key/value pair.")));
+ *end = '\0';
+
+ if (pos == end)
+ {
+ /* Empty kvpair, signifying the end of the list. */
+ *input = pos + 1;
+ return auth;
+ }
+
+ /*
+ * Find the end of the key name.
+ */
+ sep = strchr(pos, '=');
+ if (!sep)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains a key without a value.")));
+ *sep = '\0';
+
+ /* Both key and value are now safely terminated. */
+ key = pos;
+ value = sep + 1;
+ validate_kvpair(key, value);
+
+ if (!strcmp(key, AUTH_KEY))
+ {
+ if (auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains multiple auth values.")));
+
+ auth = value;
+ }
+ else
+ {
+ /*
+ * The RFC also defines the host and port keys, but they are not
+ * required for OAUTHBEARER and we do not use them. Also, per Sec.
+ * 3.1, any key/value pairs we don't recognize must be ignored.
+ */
+ }
+
+ /* Move to the next pair. */
+ pos = end + 1;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message did not contain a final terminator.")));
+
+ return NULL; /* unreachable */
+}
+
+static void
+generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
+{
+ StringInfoData buf;
+ StringInfoData issuer;
+
+ /*
+ * The admin needs to set an issuer and scope for OAuth to work. There's
+ * not really a way to hide this from the user, either, because we can't
+ * choose a "default" issuer, so be honest in the failure message.
+ *
+ * TODO: see if there's a better place to fail, earlier than this.
+ */
+ if (!ctx->issuer || !ctx->scope)
+ ereport(FATAL,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("OAuth is not properly configured for this user"),
+ errdetail_log("The issuer and scope parameters must be set in pg_hba.conf.")));
+
+ /*------
+ * Build the .well-known URI based on our issuer.
+ * TODO: RFC 8414 defines a competing well-known URI, so we'll probably
+ * have to make this configurable too.
+ */
+ initStringInfo(&issuer);
+ appendStringInfoString(&issuer, ctx->issuer);
+ appendStringInfoString(&issuer, "/.well-known/openid-configuration");
+
+ initStringInfo(&buf);
+
+ /*
+ * TODO: note that escaping here should be belt-and-suspenders, since
+ * escapable characters aren't valid in either the issuer URI or the scope
+ * list, but the HBA doesn't enforce that yet.
+ */
+ appendStringInfoString(&buf, "{ \"status\": \"invalid_token\", ");
+
+ appendStringInfoString(&buf, "\"openid-configuration\": ");
+ escape_json(&buf, issuer.data);
+ pfree(issuer.data);
+
+ appendStringInfoString(&buf, ", \"scope\": ");
+ escape_json(&buf, ctx->scope);
+
+ appendStringInfoString(&buf, " }");
+
+ *output = buf.data;
+ *outputlen = buf.len;
+}
+
+static bool
+validate(Port *port, const char *auth, const char **logdetail)
+{
+ static const char *const b64_set =
+ "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-._~+/";
+
+ const char *token;
+ size_t span;
+ int ret;
+
+ /* TODO: handle logdetail when the test framework can check it */
+
+ /*-----
+ * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
+ * 2.1:
+ *
+ * b64token = 1*( ALPHA / DIGIT /
+ * "-" / "." / "_" / "~" / "+" / "/" ) *"="
+ * credentials = "Bearer" 1*SP b64token
+ *
+ * The "credentials" construction is what we receive in our auth value.
+ *
+ * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
+ * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
+ * compared case-insensitively. (This is not mentioned in RFC 6750, but
+ * it's pointed out in RFC 7628 Sec. 4.)
+ *
+ * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
+ */
+ if (pg_strncasecmp(auth, BEARER_SCHEME, strlen(BEARER_SCHEME)))
+ return false;
+
+ /* Pull the bearer token out of the auth value. */
+ token = auth + strlen(BEARER_SCHEME);
+
+ /* Swallow any additional spaces. */
+ while (*token == ' ')
+ token++;
+
+ /*
+ * Before invoking the validator command, sanity-check the token format to
+ * avoid any injection attacks later in the chain. Invalid formats are
+ * technically a protocol violation, but don't reflect any information
+ * about the sensitive Bearer token back to the client; log at COMMERROR
+ * instead.
+ */
+
+ /* Tokens must not be empty. */
+ if (!*token)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Bearer token is empty.")));
+ return false;
+ }
+
+ /*
+ * Make sure the token contains only allowed characters. Tokens may end
+ * with any number of '=' characters.
+ */
+ span = strspn(token, b64_set);
+ while (token[span] == '=')
+ span++;
+
+ if (token[span] != '\0')
+ {
+ /*
+ * This error message could be more helpful by printing the
+ * problematic character(s), but that'd be a bit like printing a piece
+ * of someone's password into the logs.
+ */
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Bearer token is not in the correct format.")));
+ return false;
+ }
+
+ /* Have the validator check the token. */
+ if (!run_validator_command(port, token))
+ return false;
+
+ if (port->hba->oauth_skip_usermap)
+ {
+ /*
+ * If the validator is our authorization authority, we're done.
+ * Authentication may or may not have been performed depending on the
+ * validator implementation; all that matters is that the validator
+ * says the user can log in with the target role.
+ */
+ return true;
+ }
+
+ /* Make sure the validator authenticated the user. */
+ if (!MyClientConnectionInfo.authn_id)
+ {
+ /* TODO: use logdetail; reduce message duplication */
+ ereport(LOG,
+ (errmsg("OAuth bearer authentication failed for user \"%s\": validator provided no identity",
+ port->user_name)));
+ return false;
+ }
+
+ /* Finally, check the user map. */
+ ret = check_usermap(port->hba->usermap, port->user_name,
+ MyClientConnectionInfo.authn_id, false);
+ return (ret == STATUS_OK);
+}
+
+static bool
+run_validator_command(Port *port, const char *token)
+{
+ bool success = false;
+ int rc;
+ int pipefd[2];
+ int rfd = -1;
+ int wfd = -1;
+
+ StringInfoData command = {0};
+ char *p;
+ FILE *fh = NULL;
+
+ ssize_t written;
+ char *line = NULL;
+ size_t size = 0;
+ ssize_t len;
+
+ Assert(oauth_validator_command);
+
+ if (!oauth_validator_command[0])
+ {
+ ereport(COMMERROR,
+ (errmsg("oauth_validator_command is not set"),
+ errhint("To allow OAuth authenticated connections, set "
+ "oauth_validator_command in postgresql.conf.")));
+ return false;
+ }
+
+ /*------
+ * Since popen() is unidirectional, open up a pipe for the other
+ * direction. Use CLOEXEC to ensure that our write end doesn't
+ * accidentally get copied into child processes, which would prevent us
+ * from closing it cleanly.
+ *
+ * XXX this is ugly. We should just read from the child process's stdout,
+ * but that's a lot more code.
+ * XXX by bypassing the popen API, we open the potential of process
+ * deadlock. Clearly document child process requirements (i.e. the child
+ * MUST read all data off of the pipe before writing anything).
+ * TODO: port to Windows using _pipe().
+ */
+ rc = pipe(pipefd);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create child pipe: %m")));
+ return false;
+ }
+
+ rfd = pipefd[0];
+ wfd = pipefd[1];
+
+ if (!set_cloexec(wfd))
+ {
+ /* error message was already logged */
+ goto cleanup;
+ }
+
+ /*----------
+ * Construct the command, substituting any recognized %-specifiers:
+ *
+ * %f: the file descriptor of the input pipe
+ * %r: the role that the client wants to assume (port->user_name)
+ * %%: a literal '%'
+ */
+ initStringInfo(&command);
+
+ for (p = oauth_validator_command; *p; p++)
+ {
+ if (p[0] == '%')
+ {
+ switch (p[1])
+ {
+ case 'f':
+ appendStringInfo(&command, "%d", rfd);
+ p++;
+ break;
+ case 'r':
+
+ /*
+ * TODO: decide how this string should be escaped. The
+ * role is controlled by the client, so if we don't escape
+ * it, command injections are inevitable.
+ *
+ * This is probably an indication that the role name needs
+ * to be communicated to the validator process in some
+ * other way. For this proof of concept, just be
+ * incredibly strict about the characters that are allowed
+ * in user names.
+ */
+ if (!username_ok_for_shell(port->user_name))
+ goto cleanup;
+
+ appendStringInfoString(&command, port->user_name);
+ p++;
+ break;
+ case '%':
+ appendStringInfoChar(&command, '%');
+ p++;
+ break;
+ default:
+ appendStringInfoChar(&command, p[0]);
+ }
+ }
+ else
+ appendStringInfoChar(&command, p[0]);
+ }
+
+ /* Execute the command. */
+ fh = OpenPipeStream(command.data, "r");
+ if (!fh)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("opening pipe to OAuth validator: %m")));
+ goto cleanup;
+ }
+
+ /* We don't need the read end of the pipe anymore. */
+ close(rfd);
+ rfd = -1;
+
+ /* Give the command the token to validate. */
+ written = write(wfd, token, strlen(token));
+ if (written != strlen(token))
+ {
+ /* TODO must loop for short writes, EINTR et al */
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write token to child pipe: %m")));
+ goto cleanup;
+ }
+
+ close(wfd);
+ wfd = -1;
+
+ /*-----
+ * Read the command's response.
+ *
+ * TODO: getline() is probably too new to use, unfortunately.
+ * TODO: loop over all lines
+ */
+ if ((len = getline(&line, &size, fh)) >= 0)
+ {
+ /* TODO: fail if the authn_id doesn't end with a newline */
+ if (len > 0)
+ line[len - 1] = '\0';
+
+ set_authn_id(port, line);
+ }
+ else if (ferror(fh))
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from command \"%s\": %m",
+ command.data)));
+ goto cleanup;
+ }
+
+ /* Make sure the command exits cleanly. */
+ if (!check_exit(&fh, command.data))
+ {
+ /* error message already logged */
+ goto cleanup;
+ }
+
+ /* Done. */
+ success = true;
+
+cleanup:
+ if (line)
+ free(line);
+
+ /*
+ * In the successful case, the pipe fds are already closed. For the error
+ * case, always close out the pipe before waiting for the command, to
+ * prevent deadlock.
+ */
+ if (rfd >= 0)
+ close(rfd);
+ if (wfd >= 0)
+ close(wfd);
+
+ if (fh)
+ {
+ Assert(!success);
+ check_exit(&fh, command.data);
+ }
+
+ if (command.data)
+ pfree(command.data);
+
+ return success;
+}
+
+static bool
+check_exit(FILE **fh, const char *command)
+{
+ int rc;
+
+ rc = ClosePipeStream(*fh);
+ *fh = NULL;
+
+ if (rc == -1)
+ {
+ /* pclose() itself failed. */
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close pipe to command \"%s\": %m",
+ command)));
+ }
+ else if (rc != 0)
+ {
+ char *reason = wait_result_to_str(rc);
+
+ ereport(COMMERROR,
+ (errmsg("failed to execute command \"%s\": %s",
+ command, reason)));
+
+ pfree(reason);
+ }
+
+ return (rc == 0);
+}
+
+static bool
+set_cloexec(int fd)
+{
+ int flags;
+ int rc;
+
+ flags = fcntl(fd, F_GETFD);
+ if (flags == -1)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not get fd flags for child pipe: %m")));
+ return false;
+ }
+
+ rc = fcntl(fd, F_SETFD, flags | FD_CLOEXEC);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not set FD_CLOEXEC for child pipe: %m")));
+ return false;
+ }
+
+ return true;
+}
+
+/*
+ * XXX This should go away eventually and be replaced with either a proper
+ * escape or a different strategy for communication with the validator command.
+ */
+static bool
+username_ok_for_shell(const char *username)
+{
+ /* This set is borrowed from fe_utils' appendShellStringNoError(). */
+ static const char *const allowed =
+ "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-_./:";
+ size_t span;
+
+ Assert(username && username[0]); /* should have already been checked */
+
+ span = strspn(username, allowed);
+ if (username[span] != '\0')
+ {
+ ereport(COMMERROR,
+ (errmsg("PostgreSQL user name contains unsafe characters and cannot be passed to the OAuth validator")));
+ return false;
+ }
+
+ return true;
+}
diff --git a/src/backend/libpq/auth-sasl.c b/src/backend/libpq/auth-sasl.c
index 08b24d90b4..4039e7fa3e 100644
--- a/src/backend/libpq/auth-sasl.c
+++ b/src/backend/libpq/auth-sasl.c
@@ -20,14 +20,6 @@
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
-/*
- * Maximum accepted size of SASL messages.
- *
- * The messages that the server or libpq generate are much smaller than this,
- * but have some headroom.
- */
-#define PG_MAX_SASL_MESSAGE_LENGTH 1024
-
/*
* Perform a SASL exchange with a libpq client, using a specific mechanism
* implementation.
@@ -103,7 +95,7 @@ CheckSASLAuth(const pg_be_sasl_mech *mech, Port *port, char *shadow_pass,
/* Get the actual SASL message */
initStringInfo(&buf);
- if (pq_getmessage(&buf, PG_MAX_SASL_MESSAGE_LENGTH))
+ if (pq_getmessage(&buf, mech->max_message_length))
{
/* EOF - pq_getmessage already logged error */
pfree(buf.data);
diff --git a/src/backend/libpq/auth-scram.c b/src/backend/libpq/auth-scram.c
index 4161959914..486a34e719 100644
--- a/src/backend/libpq/auth-scram.c
+++ b/src/backend/libpq/auth-scram.c
@@ -113,7 +113,9 @@ static int scram_exchange(void *opaq, const char *input, int inputlen,
const pg_be_sasl_mech pg_be_scram_mech = {
scram_get_mechanisms,
scram_init,
- scram_exchange
+ scram_exchange,
+
+ PG_MAX_SASL_MESSAGE_LENGTH
};
/*
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 2b607c5270..0a5a8640fc 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -29,6 +29,7 @@
#include "libpq/auth.h"
#include "libpq/crypt.h"
#include "libpq/libpq.h"
+#include "libpq/oauth.h"
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
#include "libpq/scram.h"
@@ -45,7 +46,6 @@
*/
static void auth_failed(Port *port, int status, const char *logdetail);
static char *recv_password_packet(Port *port);
-static void set_authn_id(Port *port, const char *id);
/*----------------------------------------------------------------
@@ -201,22 +201,6 @@ static int CheckRADIUSAuth(Port *port);
static int PerformRadiusTransaction(const char *server, const char *secret, const char *portstr, const char *identifier, const char *user_name, const char *passwd);
-/*
- * Maximum accepted size of GSS and SSPI authentication tokens.
- * We also use this as a limit on ordinary password packet lengths.
- *
- * Kerberos tickets are usually quite small, but the TGTs issued by Windows
- * domain controllers include an authorization field known as the Privilege
- * Attribute Certificate (PAC), which contains the user's Windows permissions
- * (group memberships etc.). The PAC is copied into all tickets obtained on
- * the basis of this TGT (even those issued by Unix realms which the Windows
- * realm trusts), and can be several kB in size. The maximum token size
- * accepted by Windows systems is determined by the MaxAuthToken Windows
- * registry setting. Microsoft recommends that it is not set higher than
- * 65535 bytes, so that seems like a reasonable limit for us as well.
- */
-#define PG_MAX_AUTH_TOKEN_LENGTH 65535
-
/*----------------------------------------------------------------
* Global authentication functions
*----------------------------------------------------------------
@@ -305,6 +289,9 @@ auth_failed(Port *port, int status, const char *logdetail)
case uaRADIUS:
errstr = gettext_noop("RADIUS authentication failed for user \"%s\"");
break;
+ case uaOAuth:
+ errstr = gettext_noop("OAuth bearer authentication failed for user \"%s\"");
+ break;
default:
errstr = gettext_noop("authentication failed for user \"%s\": invalid authentication method");
break;
@@ -340,7 +327,7 @@ auth_failed(Port *port, int status, const char *logdetail)
* lifetime of MyClientConnectionInfo, so it is safe to pass a string that is
* managed by an external library.
*/
-static void
+void
set_authn_id(Port *port, const char *id)
{
Assert(id);
@@ -627,6 +614,9 @@ ClientAuthentication(Port *port)
case uaTrust:
status = STATUS_OK;
break;
+ case uaOAuth:
+ status = CheckSASLAuth(&pg_be_oauth_mech, port, NULL, NULL);
+ break;
}
if ((status == STATUS_OK && port->hba->clientcert == clientCertFull)
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index d506c3c0b7..e592bedf9f 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -114,7 +114,8 @@ static const char *const UserAuthName[] =
"ldap",
"cert",
"radius",
- "peer"
+ "peer",
+ "oauth",
};
/*
@@ -1743,6 +1744,8 @@ parse_hba_line(TokenizedAuthLine *tok_line, int elevel)
#endif
else if (strcmp(token->string, "radius") == 0)
parsedline->auth_method = uaRADIUS;
+ else if (strcmp(token->string, "oauth") == 0)
+ parsedline->auth_method = uaOAuth;
else
{
ereport(elevel,
@@ -2062,8 +2065,9 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->auth_method != uaPeer &&
hbaline->auth_method != uaGSS &&
hbaline->auth_method != uaSSPI &&
- hbaline->auth_method != uaCert)
- INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, and cert"));
+ hbaline->auth_method != uaCert &&
+ hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, cert, and oauth"));
hbaline->usermap = pstrdup(val);
}
else if (strcmp(name, "clientcert") == 0)
@@ -2446,6 +2450,27 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->radiusidentifiers = parsed_identifiers;
hbaline->radiusidentifiers_s = pstrdup(val);
}
+ else if (strcmp(name, "issuer") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("issuer", gettext_noop("oauth"));
+ hbaline->oauth_issuer = pstrdup(val);
+ }
+ else if (strcmp(name, "scope") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("scope", gettext_noop("oauth"));
+ hbaline->oauth_scope = pstrdup(val);
+ }
+ else if (strcmp(name, "trust_validator_authz") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("trust_validator_authz", gettext_noop("oauth"));
+ if (strcmp(val, "1") == 0)
+ hbaline->oauth_skip_usermap = true;
+ else
+ hbaline->oauth_skip_usermap = false;
+ }
else
{
ereport(elevel,
diff --git a/src/backend/libpq/meson.build b/src/backend/libpq/meson.build
index 7c65314512..c85527fb01 100644
--- a/src/backend/libpq/meson.build
+++ b/src/backend/libpq/meson.build
@@ -1,6 +1,7 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
backend_sources += files(
+ 'auth-oauth.c',
'auth-sasl.c',
'auth-scram.c',
'auth.c',
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 45013582a7..d28209901f 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -48,6 +48,7 @@
#include "jit/jit.h"
#include "libpq/auth.h"
#include "libpq/libpq.h"
+#include "libpq/oauth.h"
#include "libpq/scram.h"
#include "nodes/queryjumble.h"
#include "optimizer/cost.h"
@@ -4670,6 +4671,17 @@ struct config_string ConfigureNamesString[] =
check_debug_io_direct, assign_debug_io_direct, NULL
},
+ {
+ {"oauth_validator_command", PGC_SIGHUP, CONN_AUTH_AUTH,
+ gettext_noop("Command to validate OAuth v2 bearer tokens."),
+ NULL,
+ GUC_SUPERUSER_ONLY | GUC_NOT_IN_SAMPLE
+ },
+ &oauth_validator_command,
+ "",
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, NULL, NULL, NULL, NULL
diff --git a/src/include/libpq/auth.h b/src/include/libpq/auth.h
index 227b41daf6..22f6ab9f1d 100644
--- a/src/include/libpq/auth.h
+++ b/src/include/libpq/auth.h
@@ -16,6 +16,22 @@
#include "libpq/libpq-be.h"
+/*
+ * Maximum accepted size of GSS and SSPI authentication tokens.
+ * We also use this as a limit on ordinary password packet lengths.
+ *
+ * Kerberos tickets are usually quite small, but the TGTs issued by Windows
+ * domain controllers include an authorization field known as the Privilege
+ * Attribute Certificate (PAC), which contains the user's Windows permissions
+ * (group memberships etc.). The PAC is copied into all tickets obtained on
+ * the basis of this TGT (even those issued by Unix realms which the Windows
+ * realm trusts), and can be several kB in size. The maximum token size
+ * accepted by Windows systems is determined by the MaxAuthToken Windows
+ * registry setting. Microsoft recommends that it is not set higher than
+ * 65535 bytes, so that seems like a reasonable limit for us as well.
+ */
+#define PG_MAX_AUTH_TOKEN_LENGTH 65535
+
extern PGDLLIMPORT char *pg_krb_server_keyfile;
extern PGDLLIMPORT bool pg_krb_caseins_users;
extern PGDLLIMPORT bool pg_gss_accept_delegation;
@@ -23,6 +39,7 @@ extern PGDLLIMPORT bool pg_gss_accept_delegation;
extern void ClientAuthentication(Port *port);
extern void sendAuthRequest(Port *port, AuthRequest areq, const char *extradata,
int extralen);
+extern void set_authn_id(Port *port, const char *id);
/* Hook for plugins to get control in ClientAuthentication() */
typedef void (*ClientAuthentication_hook_type) (Port *, int);
diff --git a/src/include/libpq/hba.h b/src/include/libpq/hba.h
index 8ea837ae82..53c999a69f 100644
--- a/src/include/libpq/hba.h
+++ b/src/include/libpq/hba.h
@@ -39,7 +39,8 @@ typedef enum UserAuth
uaCert,
uaRADIUS,
uaPeer,
-#define USER_AUTH_LAST uaPeer /* Must be last value of this enum */
+ uaOAuth,
+#define USER_AUTH_LAST uaOAuth /* Must be last value of this enum */
} UserAuth;
/*
@@ -135,6 +136,9 @@ typedef struct HbaLine
char *radiusidentifiers_s;
List *radiusports;
char *radiusports_s;
+ char *oauth_issuer;
+ char *oauth_scope;
+ bool oauth_skip_usermap;
} HbaLine;
typedef struct IdentLine
diff --git a/src/include/libpq/oauth.h b/src/include/libpq/oauth.h
new file mode 100644
index 0000000000..5edab3b25a
--- /dev/null
+++ b/src/include/libpq/oauth.h
@@ -0,0 +1,24 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth.h
+ * Interface to libpq/auth-oauth.c
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/libpq/oauth.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_OAUTH_H
+#define PG_OAUTH_H
+
+#include "libpq/libpq-be.h"
+#include "libpq/sasl.h"
+
+extern char *oauth_validator_command;
+
+/* Implementation */
+extern const pg_be_sasl_mech pg_be_oauth_mech;
+
+#endif /* PG_OAUTH_H */
diff --git a/src/include/libpq/sasl.h b/src/include/libpq/sasl.h
index 7a1f970cca..3f2c02b8f2 100644
--- a/src/include/libpq/sasl.h
+++ b/src/include/libpq/sasl.h
@@ -26,6 +26,14 @@
#define PG_SASL_EXCHANGE_SUCCESS 1
#define PG_SASL_EXCHANGE_FAILURE 2
+/*
+ * Maximum accepted size of SASL messages.
+ *
+ * The messages that the server or libpq generate are much smaller than this,
+ * but have some headroom.
+ */
+#define PG_MAX_SASL_MESSAGE_LENGTH 1024
+
/*
* Backend SASL mechanism callbacks.
*
@@ -127,6 +135,9 @@ typedef struct pg_be_sasl_mech
const char *input, int inputlen,
char **output, int *outputlen,
const char **logdetail);
+
+ /* The maximum size allowed for client SASLResponses. */
+ int max_message_length;
} pg_be_sasl_mech;
/* Common implementation for auth.c */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 291c80e3ff..8119523cd9 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3543,6 +3543,7 @@ normal_rand_fctx
nsphash_hash
ntile_context
numeric
+oauth_state
object_access_hook_type
object_access_hook_type_str
off_t
--
2.34.1
v20-0001-common-jsonapi-support-FRONTEND-clients.patch (application/octet-stream, 20.4K) download
From 231c6fb165956553ebe42ba187a4c287151e5203 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Mon, 3 May 2021 15:38:26 -0700
Subject: [PATCH v20 1/9] common/jsonapi: support FRONTEND clients
Based on a patch by Michael Paquier.
For frontend code, use PQExpBuffer instead of StringInfo. This requires
us to track allocation failures so that we can return JSON_OUT_OF_MEMORY
as needed. json_errdetail() now allocates its error message inside
memory owned by the JsonLexContext, so clients don't need to worry about
freeing it.
We can now partially revert b44669b2ca, now that json_errdetail() works
correctly.
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
---
src/bin/pg_verifybackup/t/005_bad_manifest.pl | 3 +-
src/common/Makefile | 2 +-
src/common/jsonapi.c | 268 +++++++++++++-----
src/common/meson.build | 8 +-
src/common/parse_manifest.c | 2 +-
src/common/stringinfo.c | 7 +
src/include/common/jsonapi.h | 18 +-
src/include/lib/stringinfo.h | 2 +
8 files changed, 225 insertions(+), 85 deletions(-)
diff --git a/src/bin/pg_verifybackup/t/005_bad_manifest.pl b/src/bin/pg_verifybackup/t/005_bad_manifest.pl
index e278ccea5a..e2a297930e 100644
--- a/src/bin/pg_verifybackup/t/005_bad_manifest.pl
+++ b/src/bin/pg_verifybackup/t/005_bad_manifest.pl
@@ -13,7 +13,8 @@ use Test::More;
my $tempdir = PostgreSQL::Test::Utils::tempdir;
test_bad_manifest('input string ended unexpectedly',
- qr/could not parse backup manifest: parsing failed/, <<EOM);
+ qr/could not parse backup manifest: The input string ended unexpectedly/,
+ <<EOM);
{
EOM
diff --git a/src/common/Makefile b/src/common/Makefile
index 2ba5069dca..bbb5c3ab11 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -40,7 +40,7 @@ override CPPFLAGS += -DVAL_LDFLAGS_EX="\"$(LDFLAGS_EX)\""
override CPPFLAGS += -DVAL_LDFLAGS_SL="\"$(LDFLAGS_SL)\""
override CPPFLAGS += -DVAL_LIBS="\"$(LIBS)\""
-override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common $(CPPFLAGS)
+override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common -I$(libpq_srcdir) $(CPPFLAGS)
LIBS += $(PTHREAD_LIBS)
OBJS_COMMON = \
diff --git a/src/common/jsonapi.c b/src/common/jsonapi.c
index 32931ded82..2d1f30353a 100644
--- a/src/common/jsonapi.c
+++ b/src/common/jsonapi.c
@@ -21,10 +21,43 @@
#include "mb/pg_wchar.h"
#include "port/pg_lfind.h"
-#ifndef FRONTEND
+#ifdef FRONTEND
+#include "pqexpbuffer.h"
+#else
+#include "lib/stringinfo.h"
#include "miscadmin.h"
#endif
+/*
+ * In backend, we will use palloc/pfree along with StringInfo. In frontend, use
+ * malloc and PQExpBuffer, and return JSON_OUT_OF_MEMORY on out-of-memory.
+ */
+#ifdef FRONTEND
+
+#define STRDUP(s) strdup(s)
+#define ALLOC(size) malloc(size)
+
+#define appendStrVal appendPQExpBuffer
+#define appendBinaryStrVal appendBinaryPQExpBuffer
+#define appendStrValChar appendPQExpBufferChar
+#define createStrVal createPQExpBuffer
+#define resetStrVal resetPQExpBuffer
+#define destroyStrVal destroyPQExpBuffer
+
+#else /* !FRONTEND */
+
+#define STRDUP(s) pstrdup(s)
+#define ALLOC(size) palloc(size)
+
+#define appendStrVal appendStringInfo
+#define appendBinaryStrVal appendBinaryStringInfo
+#define appendStrValChar appendStringInfoChar
+#define createStrVal makeStringInfo
+#define resetStrVal resetStringInfo
+#define destroyStrVal destroyStringInfo
+
+#endif
+
/*
* The context of the parser is maintained by the recursive descent
* mechanism, but is passed explicitly to the error reporting routine
@@ -167,9 +200,16 @@ makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
lex->input_encoding = encoding;
if (need_escapes)
{
- lex->strval = makeStringInfo();
+ /*
+ * This call can fail in FRONTEND code. We defer error handling to
+ * time of use (json_lex_string()) since we might not need to parse
+ * any strings anyway.
+ */
+ lex->strval = createStrVal();
lex->flags |= JSONLEX_FREE_STRVAL;
+ lex->parse_strval = true;
}
+ lex->errormsg = NULL;
return lex;
}
@@ -182,13 +222,18 @@ makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
void
freeJsonLexContext(JsonLexContext *lex)
{
+ static const JsonLexContext empty = {0};
+
if (lex->flags & JSONLEX_FREE_STRVAL)
- {
- pfree(lex->strval->data);
- pfree(lex->strval);
- }
+ destroyStrVal(lex->strval);
+
+ if (lex->errormsg)
+ destroyStrVal(lex->errormsg);
+
if (lex->flags & JSONLEX_FREE_STRUCT)
pfree(lex);
+ else
+ *lex = empty;
}
/*
@@ -254,7 +299,7 @@ json_count_array_elements(JsonLexContext *lex, int *elements)
* etc, so doing this with a copy makes that safe.
*/
memcpy(©lex, lex, sizeof(JsonLexContext));
- copylex.strval = NULL; /* not interested in values here */
+ copylex.parse_strval = false; /* not interested in values here */
copylex.lex_level++;
count = 0;
@@ -316,14 +361,21 @@ parse_scalar(JsonLexContext *lex, JsonSemAction *sem)
/* extract the de-escaped string value, or the raw lexeme */
if (lex_peek(lex) == JSON_TOKEN_STRING)
{
- if (lex->strval != NULL)
- val = pstrdup(lex->strval->data);
+ if (lex->parse_strval)
+ {
+ val = STRDUP(lex->strval->data);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
}
else
{
int len = (lex->token_terminator - lex->token_start);
- val = palloc(len + 1);
+ val = ALLOC(len + 1);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+
memcpy(val, lex->token_start, len);
val[len] = '\0';
}
@@ -357,8 +409,12 @@ parse_object_field(JsonLexContext *lex, JsonSemAction *sem)
if (lex_peek(lex) != JSON_TOKEN_STRING)
return report_parse_error(JSON_PARSE_STRING, lex);
- if ((ostart != NULL || oend != NULL) && lex->strval != NULL)
- fname = pstrdup(lex->strval->data);
+ if ((ostart != NULL || oend != NULL) && lex->parse_strval)
+ {
+ fname = STRDUP(lex->strval->data);
+ if (fname == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
result = json_lex(lex);
if (result != JSON_SUCCESS)
return result;
@@ -414,6 +470,11 @@ parse_object(JsonLexContext *lex, JsonSemAction *sem)
JsonParseErrorType result;
#ifndef FRONTEND
+
+ /*
+ * TODO: clients need some way to put a bound on stack growth. Parse level
+ * limits maybe?
+ */
check_stack_depth();
#endif
@@ -762,8 +823,15 @@ json_lex_string(JsonLexContext *lex)
return code; \
} while (0)
- if (lex->strval != NULL)
- resetStringInfo(lex->strval);
+ if (lex->parse_strval)
+ {
+#ifdef FRONTEND
+ /* make sure initialization succeeded */
+ if (lex->strval == NULL)
+ return JSON_OUT_OF_MEMORY;
+#endif
+ resetStrVal(lex->strval);
+ }
Assert(lex->input_length > 0);
s = lex->token_start;
@@ -800,7 +868,7 @@ json_lex_string(JsonLexContext *lex)
else
FAIL_AT_CHAR_END(JSON_UNICODE_ESCAPE_FORMAT);
}
- if (lex->strval != NULL)
+ if (lex->parse_strval)
{
/*
* Combine surrogate pairs.
@@ -857,19 +925,19 @@ json_lex_string(JsonLexContext *lex)
unicode_to_utf8(ch, (unsigned char *) utf8str);
utf8len = pg_utf_mblen((unsigned char *) utf8str);
- appendBinaryStringInfo(lex->strval, utf8str, utf8len);
+ appendBinaryPQExpBuffer(lex->strval, utf8str, utf8len);
}
else if (ch <= 0x007f)
{
/* The ASCII range is the same in all encodings */
- appendStringInfoChar(lex->strval, (char) ch);
+ appendPQExpBufferChar(lex->strval, (char) ch);
}
else
FAIL_AT_CHAR_END(JSON_UNICODE_HIGH_ESCAPE);
#endif /* FRONTEND */
}
}
- else if (lex->strval != NULL)
+ else if (lex->parse_strval)
{
if (hi_surrogate != -1)
FAIL_AT_CHAR_END(JSON_UNICODE_LOW_SURROGATE);
@@ -879,22 +947,22 @@ json_lex_string(JsonLexContext *lex)
case '"':
case '\\':
case '/':
- appendStringInfoChar(lex->strval, *s);
+ appendStrValChar(lex->strval, *s);
break;
case 'b':
- appendStringInfoChar(lex->strval, '\b');
+ appendStrValChar(lex->strval, '\b');
break;
case 'f':
- appendStringInfoChar(lex->strval, '\f');
+ appendStrValChar(lex->strval, '\f');
break;
case 'n':
- appendStringInfoChar(lex->strval, '\n');
+ appendStrValChar(lex->strval, '\n');
break;
case 'r':
- appendStringInfoChar(lex->strval, '\r');
+ appendStrValChar(lex->strval, '\r');
break;
case 't':
- appendStringInfoChar(lex->strval, '\t');
+ appendStrValChar(lex->strval, '\t');
break;
default:
@@ -929,7 +997,7 @@ json_lex_string(JsonLexContext *lex)
/*
* Skip to the first byte that requires special handling, so we
- * can batch calls to appendBinaryStringInfo.
+ * can batch calls to appendBinaryStrVal.
*/
while (p < end - sizeof(Vector8) &&
!pg_lfind8('\\', (uint8 *) p, sizeof(Vector8)) &&
@@ -953,8 +1021,8 @@ json_lex_string(JsonLexContext *lex)
}
}
- if (lex->strval != NULL)
- appendBinaryStringInfo(lex->strval, s, p - s);
+ if (lex->parse_strval)
+ appendBinaryStrVal(lex->strval, s, p - s);
/*
* s will be incremented at the top of the loop, so set it to just
@@ -970,6 +1038,11 @@ json_lex_string(JsonLexContext *lex)
return JSON_UNICODE_LOW_SURROGATE;
}
+#ifdef FRONTEND
+ if (lex->parse_strval && PQExpBufferBroken(lex->strval))
+ return JSON_OUT_OF_MEMORY;
+#endif
+
/* Hooray, we found the end of the string! */
lex->prev_token_terminator = lex->token_terminator;
lex->token_terminator = s + 1;
@@ -1145,72 +1218,93 @@ report_parse_error(JsonParseContext ctx, JsonLexContext *lex)
return JSON_SUCCESS; /* silence stupider compilers */
}
-
-#ifndef FRONTEND
-/*
- * Extract the current token from a lexing context, for error reporting.
- */
-static char *
-extract_token(JsonLexContext *lex)
-{
- int toklen = lex->token_terminator - lex->token_start;
- char *token = palloc(toklen + 1);
-
- memcpy(token, lex->token_start, toklen);
- token[toklen] = '\0';
- return token;
-}
-
/*
* Construct an (already translated) detail message for a JSON error.
*
- * Note that the error message generated by this routine may not be
- * palloc'd, making it unsafe for frontend code as there is no way to
- * know if this can be safely pfree'd or not.
+ * The returned allocation is either static or owned by the JsonLexContext and
+ * should not be freed.
*/
char *
json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
{
+ int toklen = lex->token_terminator - lex->token_start;
+
+ if (error == JSON_OUT_OF_MEMORY)
+ {
+ /* Short circuit. Allocating anything for this case is unhelpful. */
+ return _("out of memory");
+ }
+
+ if (lex->errormsg)
+ resetStrVal(lex->errormsg);
+ else
+ lex->errormsg = createStrVal();
+
switch (error)
{
case JSON_SUCCESS:
/* fall through to the error code after switch */
break;
case JSON_ESCAPING_INVALID:
- return psprintf(_("Escape sequence \"\\%s\" is invalid."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Escape sequence \"\\%.*s\" is invalid."),
+ toklen, lex->token_start);
+ break;
case JSON_ESCAPING_REQUIRED:
- return psprintf(_("Character with value 0x%02x must be escaped."),
- (unsigned char) *(lex->token_terminator));
+ appendStrVal(lex->errormsg,
+ _("Character with value 0x%02x must be escaped."),
+ (unsigned char) *(lex->token_terminator));
+ break;
case JSON_EXPECTED_END:
- return psprintf(_("Expected end of input, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected end of input, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_ARRAY_FIRST:
- return psprintf(_("Expected array element or \"]\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected array element or \"]\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_ARRAY_NEXT:
- return psprintf(_("Expected \",\" or \"]\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \",\" or \"]\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_COLON:
- return psprintf(_("Expected \":\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \":\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_JSON:
- return psprintf(_("Expected JSON value, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected JSON value, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_MORE:
return _("The input string ended unexpectedly.");
case JSON_EXPECTED_OBJECT_FIRST:
- return psprintf(_("Expected string or \"}\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected string or \"}\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_OBJECT_NEXT:
- return psprintf(_("Expected \",\" or \"}\", but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected \",\" or \"}\", but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_EXPECTED_STRING:
- return psprintf(_("Expected string, but found \"%s\"."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Expected string, but found \"%.*s\"."),
+ toklen, lex->token_start);
+ break;
case JSON_INVALID_TOKEN:
- return psprintf(_("Token \"%s\" is invalid."),
- extract_token(lex));
+ appendStrVal(lex->errormsg,
+ _("Token \"%.*s\" is invalid."),
+ toklen, lex->token_start);
+ break;
+ case JSON_OUT_OF_MEMORY:
+ /* should have been handled above; use the error path */
+ break;
case JSON_UNICODE_CODE_POINT_ZERO:
return _("\\u0000 cannot be converted to text.");
case JSON_UNICODE_ESCAPE_FORMAT:
@@ -1219,9 +1313,19 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
/* note: this case is only reachable in frontend not backend */
return _("Unicode escape values cannot be used for code point values above 007F when the encoding is not UTF8.");
case JSON_UNICODE_UNTRANSLATABLE:
- /* note: this case is only reachable in backend not frontend */
+
+ /*
+ * note: this case is only reachable in backend not frontend.
+ * #ifdef it away so the frontend doesn't try to link against
+ * backend functionality.
+ */
+#ifndef FRONTEND
return psprintf(_("Unicode escape value could not be translated to the server's encoding %s."),
GetDatabaseEncodingName());
+#else
+ Assert(false);
+ break;
+#endif
case JSON_UNICODE_HIGH_SURROGATE:
return _("Unicode high surrogate must not follow a high surrogate.");
case JSON_UNICODE_LOW_SURROGATE:
@@ -1231,12 +1335,22 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
break;
}
- /*
- * We don't use a default: case, so that the compiler will warn about
- * unhandled enum values. But this needs to be here anyway to cover the
- * possibility of an incorrect input.
- */
- elog(ERROR, "unexpected json parse error type: %d", (int) error);
- return NULL;
-}
+ /* Note that lex->errormsg can be NULL in FRONTEND code. */
+ if (lex->errormsg && !lex->errormsg->data[0])
+ {
+ /*
+ * We don't use a default: case, so that the compiler will warn about
+ * unhandled enum values. But this needs to be here anyway to cover
+ * the possibility of an incorrect input.
+ */
+ appendStrVal(lex->errormsg,
+ "unexpected json parse error type: %d", (int) error);
+ }
+
+#ifdef FRONTEND
+ if (PQExpBufferBroken(lex->errormsg))
+ return _("out of memory while constructing error description");
#endif
+
+ return lex->errormsg->data;
+}
diff --git a/src/common/meson.build b/src/common/meson.build
index 4eb16024cb..5d2c7abaa6 100644
--- a/src/common/meson.build
+++ b/src/common/meson.build
@@ -124,13 +124,18 @@ common_sources_frontend_static += files(
# least cryptohash_openssl.c, hmac_openssl.c depend on it.
# controldata_utils.c depends on wait_event_types_h. That's arguably a
# layering violation, but ...
+#
+# XXX Frontend builds need libpq's pqexpbuffer.h, so adjust the include paths
+# appropriately. This seems completely broken.
pgcommon = {}
pgcommon_variants = {
'_srv': internal_lib_args + {
+ 'include_directories': include_directories('.'),
'sources': common_sources + [lwlocknames_h] + [wait_event_types_h],
'dependencies': [backend_common_code],
},
'': default_lib_args + {
+ 'include_directories': include_directories('../interfaces/libpq', '.'),
'sources': common_sources_frontend_static,
'dependencies': [frontend_common_code],
# Files in libpgcommon.a should use/export the "xxx_private" versions
@@ -139,6 +144,7 @@ pgcommon_variants = {
},
'_shlib': default_lib_args + {
'pic': true,
+ 'include_directories': include_directories('../interfaces/libpq', '.'),
'sources': common_sources_frontend_shlib,
'dependencies': [frontend_common_code],
},
@@ -156,7 +162,6 @@ foreach name, opts : pgcommon_variants
c_args = opts.get('c_args', []) + common_cflags[cflagname]
cflag_libs += static_library('libpgcommon@0@_@1@'.format(name, cflagname),
c_pch: pch_c_h,
- include_directories: include_directories('.'),
kwargs: opts + {
'sources': sources,
'c_args': c_args,
@@ -169,7 +174,6 @@ foreach name, opts : pgcommon_variants
lib = static_library('libpgcommon@0@'.format(name),
link_with: cflag_libs,
c_pch: pch_c_h,
- include_directories: include_directories('.'),
kwargs: opts + {
'dependencies': opts['dependencies'] + [ssl],
}
diff --git a/src/common/parse_manifest.c b/src/common/parse_manifest.c
index 92a97714f3..62d93989be 100644
--- a/src/common/parse_manifest.c
+++ b/src/common/parse_manifest.c
@@ -147,7 +147,7 @@ json_parse_manifest(JsonManifestParseContext *context, char *buffer,
/* Run the actual JSON parser. */
json_error = pg_parse_json(lex, &sem);
if (json_error != JSON_SUCCESS)
- json_manifest_parse_failure(context, "parsing failed");
+ json_manifest_parse_failure(context, json_errdetail(json_error, lex));
if (parse.state != JM_EXPECT_EOF)
json_manifest_parse_failure(context, "manifest ended unexpectedly");
diff --git a/src/common/stringinfo.c b/src/common/stringinfo.c
index c61d5c58f3..09419f6042 100644
--- a/src/common/stringinfo.c
+++ b/src/common/stringinfo.c
@@ -350,3 +350,10 @@ enlargeStringInfo(StringInfo str, int needed)
str->maxlen = newlen;
}
+
+void
+destroyStringInfo(StringInfo str)
+{
+ pfree(str->data);
+ pfree(str);
+}
diff --git a/src/include/common/jsonapi.h b/src/include/common/jsonapi.h
index 02943cdad8..75d444c17a 100644
--- a/src/include/common/jsonapi.h
+++ b/src/include/common/jsonapi.h
@@ -14,8 +14,6 @@
#ifndef JSONAPI_H
#define JSONAPI_H
-#include "lib/stringinfo.h"
-
typedef enum JsonTokenType
{
JSON_TOKEN_INVALID,
@@ -48,6 +46,7 @@ typedef enum JsonParseErrorType
JSON_EXPECTED_OBJECT_NEXT,
JSON_EXPECTED_STRING,
JSON_INVALID_TOKEN,
+ JSON_OUT_OF_MEMORY,
JSON_UNICODE_CODE_POINT_ZERO,
JSON_UNICODE_ESCAPE_FORMAT,
JSON_UNICODE_HIGH_ESCAPE,
@@ -57,6 +56,17 @@ typedef enum JsonParseErrorType
JSON_SEM_ACTION_FAILED, /* error should already be reported */
} JsonParseErrorType;
+/*
+ * Don't depend on the internal type header for strval; if callers need access
+ * then they can include the appropriate header themselves.
+ */
+#ifdef FRONTEND
+#define StrValType PQExpBufferData
+#else
+#define StrValType StringInfoData
+#endif
+
+typedef struct StrValType StrValType;
/*
* All the fields in this structure should be treated as read-only.
@@ -88,7 +98,9 @@ typedef struct JsonLexContext
bits32 flags;
int line_number; /* line number, starting from 1 */
char *line_start; /* where that line starts within input */
- StringInfo strval;
+ bool parse_strval;
+ StrValType *strval; /* only used if parse_strval == true */
+ StrValType *errormsg;
} JsonLexContext;
typedef JsonParseErrorType (*json_struct_action) (void *state);
diff --git a/src/include/lib/stringinfo.h b/src/include/lib/stringinfo.h
index 2cd636b01c..64ec6419af 100644
--- a/src/include/lib/stringinfo.h
+++ b/src/include/lib/stringinfo.h
@@ -233,4 +233,6 @@ extern void appendBinaryStringInfoNT(StringInfo str,
*/
extern void enlargeStringInfo(StringInfo str, int needed);
+
+extern void destroyStringInfo(StringInfo str);
#endif /* STRINGINFO_H */
--
2.34.1
v20-0004-libpq-add-OAUTHBEARER-SASL-mechanism.patch (application/octet-stream, 106.3K) download
From 2a55d9c8062f31b3d902900301fd5f1fe8db1bb1 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 13 Apr 2021 10:27:27 -0700
Subject: [PATCH v20 4/9] libpq: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) and OAuth 2.0 Device Authorization
Grants (RFC 8628) on the client side. When speaking to a OAuth-enabled
server, it looks a bit like this:
$ psql 'host=example.org oauth_client_id=f02c6361-0635-...'
Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG
The OAuth issuer must support device authorization. No other OAuth flows
are currently implemented (but clients may provide their own flows; see
below).
The client implementation requires libcurl and its development headers.
Pass `curl` to --with-oauth/-Doauth during configuration.
Thomas Munro wrote the kqueue() implementation for oauth-curl; thanks!
= PQauthDataHook =
Clients may override two pieces of OAuth handling using the new
PQsetAuthDataHook():
- PQAUTHDATA_PROMPT_OAUTH_DEVICE: replaces the default user prompt to
standard error when using the builtin device authorization flow
- PQAUTHDATA_OAUTH_BEARER_TOKEN: replaces the entire OAuth flow with a
custom asynchronous implementation
In general, a hook implementation should examine the incoming `type` to
decide whether or not to handle a specific piece of authdata; if not, it
should delegate to the previous hook in the chain (retrievable via
PQgetAuthDataHook()). Otherwise, it should return an integer > 0 and
follow the authdata-specific instructions. Returning an integer < 0
signals an error condition and abandons the connection attempt.
== PQAUTHDATA_PROMPT_OAUTH_DEVICE ==
The hook should display the device prompt (URL + code) using whatever
method it prefers.
== PQAUTHDATA_OAUTH_BEARER_TOKEN ==
The hook should either directly return a Bearer token for the current
user/issuer/scope combination, if one is available without blocking, or
else set up an asynchronous callback to retrieve one. See the
documentation for PQoauthBearerRequest.
Several TODOs:
- don't retry forever if the server won't accept our token
- perform several sanity checks on the OAuth issuer's responses
- handle cases where the client has been set up with an issuer and
scope, but the Postgres server wants to use something different
- improve error debuggability during the OAuth handshake
- fix libcurl initialization thread-safety
- harden the libcurl flow implementation
- figure out pgsocket/int difference on Windows
- fix intermittent failure in the cleanup callback tests (race
condition?)
- support require_auth
- ...and more.
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
---
configure | 110 ++
configure.ac | 28 +
meson.build | 29 +
meson_options.txt | 4 +
src/Makefile.global.in | 1 +
src/include/common/oauth-common.h | 19 +
src/include/pg_config.h.in | 9 +
src/interfaces/libpq/Makefile | 10 +-
src/interfaces/libpq/exports.txt | 3 +
src/interfaces/libpq/fe-auth-oauth-curl.c | 1982 +++++++++++++++++++++
src/interfaces/libpq/fe-auth-oauth.c | 659 +++++++
src/interfaces/libpq/fe-auth-oauth.h | 42 +
src/interfaces/libpq/fe-auth-sasl.h | 10 +-
src/interfaces/libpq/fe-auth-scram.c | 6 +-
src/interfaces/libpq/fe-auth.c | 105 +-
src/interfaces/libpq/fe-auth.h | 9 +-
src/interfaces/libpq/fe-connect.c | 85 +-
src/interfaces/libpq/fe-misc.c | 7 +-
src/interfaces/libpq/libpq-fe.h | 77 +-
src/interfaces/libpq/libpq-int.h | 14 +
src/interfaces/libpq/meson.build | 7 +
src/makefiles/meson.build | 1 +
src/tools/pgindent/typedefs.list | 10 +
23 files changed, 3200 insertions(+), 27 deletions(-)
create mode 100644 src/include/common/oauth-common.h
create mode 100644 src/interfaces/libpq/fe-auth-oauth-curl.c
create mode 100644 src/interfaces/libpq/fe-auth-oauth.c
create mode 100644 src/interfaces/libpq/fe-auth-oauth.h
diff --git a/configure b/configure
index 46859a4244..2ccfb01b7a 100755
--- a/configure
+++ b/configure
@@ -712,6 +712,7 @@ with_uuid
with_readline
with_systemd
with_selinux
+with_oauth
with_ldap
with_krb_srvnam
krb_srvtab
@@ -858,6 +859,7 @@ with_krb_srvnam
with_pam
with_bsd_auth
with_ldap
+with_oauth
with_bonjour
with_selinux
with_systemd
@@ -1568,6 +1570,7 @@ Optional Packages:
--with-pam build with PAM support
--with-bsd-auth build with BSD Authentication support
--with-ldap build with LDAP support
+ --with-oauth=LIB use LIB for OAuth 2.0 support (curl)
--with-bonjour build with Bonjour support
--with-selinux build with SELinux support
--with-systemd build with systemd support
@@ -8485,6 +8488,52 @@ $as_echo "$with_ldap" >&6; }
+#
+# OAuth 2.0
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with OAuth support" >&5
+$as_echo_n "checking whether to build with OAuth support... " >&6; }
+
+
+
+# Check whether --with-oauth was given.
+if test "${with_oauth+set}" = set; then :
+ withval=$with_oauth;
+ case $withval in
+ yes)
+ as_fn_error $? "argument required for --with-oauth option" "$LINENO" 5
+ ;;
+ no)
+ as_fn_error $? "argument required for --with-oauth option" "$LINENO" 5
+ ;;
+ *)
+
+ ;;
+ esac
+
+fi
+
+
+if test x"$with_oauth" = x"" ; then
+ with_oauth=no
+fi
+
+if test x"$with_oauth" = x"curl"; then
+
+$as_echo "#define USE_OAUTH 1" >>confdefs.h
+
+
+$as_echo "#define USE_OAUTH_CURL 1" >>confdefs.h
+
+elif test x"$with_oauth" != x"no"; then
+ as_fn_error $? "--with-oauth must specify curl" "$LINENO" 5
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_oauth" >&5
+$as_echo "$with_oauth" >&6; }
+
+
+
#
# Bonjour
#
@@ -13037,6 +13086,56 @@ fi
+if test "$with_oauth" = curl ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for curl_multi_init in -lcurl" >&5
+$as_echo_n "checking for curl_multi_init in -lcurl... " >&6; }
+if ${ac_cv_lib_curl_curl_multi_init+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lcurl $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char curl_multi_init ();
+int
+main ()
+{
+return curl_multi_init ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_curl_curl_multi_init=yes
+else
+ ac_cv_lib_curl_curl_multi_init=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_curl_curl_multi_init" >&5
+$as_echo "$ac_cv_lib_curl_curl_multi_init" >&6; }
+if test "x$ac_cv_lib_curl_curl_multi_init" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBCURL 1
+_ACEOF
+
+ LIBS="-lcurl $LIBS"
+
+else
+ as_fn_error $? "library 'curl' is required for --with-oauth=curl" "$LINENO" 5
+fi
+
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for security_compute_create_name in -lselinux" >&5
@@ -14062,6 +14161,17 @@ fi
done
+fi
+
+if test "$with_oauth" = curl; then
+ ac_fn_c_check_header_mongrel "$LINENO" "curl/curl.h" "ac_cv_header_curl_curl_h" "$ac_includes_default"
+if test "x$ac_cv_header_curl_curl_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <curl/curl.h> is required for OAuth" "$LINENO" 5
+fi
+
+
fi
if test "$PORTNAME" = "win32" ; then
diff --git a/configure.ac b/configure.ac
index 88b75a7696..4a80c97d5b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -927,6 +927,26 @@ AC_MSG_RESULT([$with_ldap])
AC_SUBST(with_ldap)
+#
+# OAuth 2.0
+#
+AC_MSG_CHECKING([whether to build with OAuth support])
+PGAC_ARG_REQ(with, oauth, [LIB], [use LIB for OAuth 2.0 support (curl)])
+if test x"$with_oauth" = x"" ; then
+ with_oauth=no
+fi
+
+if test x"$with_oauth" = x"curl"; then
+ AC_DEFINE([USE_OAUTH], 1, [Define to 1 to build with OAuth 2.0 support. (--with-oauth)])
+ AC_DEFINE([USE_OAUTH_CURL], 1, [Define to 1 to use libcurl for OAuth support.])
+elif test x"$with_oauth" != x"no"; then
+ AC_MSG_ERROR([--with-oauth must specify curl])
+fi
+
+AC_MSG_RESULT([$with_oauth])
+AC_SUBST(with_oauth)
+
+
#
# Bonjour
#
@@ -1423,6 +1443,10 @@ fi
AC_SUBST(LDAP_LIBS_FE)
AC_SUBST(LDAP_LIBS_BE)
+if test "$with_oauth" = curl ; then
+ AC_CHECK_LIB(curl, curl_multi_init, [], [AC_MSG_ERROR([library 'curl' is required for --with-oauth=curl])])
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
AC_CHECK_LIB(selinux, security_compute_create_name, [],
@@ -1614,6 +1638,10 @@ elif test "$with_uuid" = ossp ; then
[AC_MSG_ERROR([header file <ossp/uuid.h> or <uuid.h> is required for OSSP UUID])])])
fi
+if test "$with_oauth" = curl; then
+ AC_CHECK_HEADER(curl/curl.h, [], [AC_MSG_ERROR([header file <curl/curl.h> is required for OAuth])])
+fi
+
if test "$PORTNAME" = "win32" ; then
AC_CHECK_HEADERS(crtdefs.h)
fi
diff --git a/meson.build b/meson.build
index a198eca25d..45b20d11c1 100644
--- a/meson.build
+++ b/meson.build
@@ -830,6 +830,33 @@ endif
+###############################################################
+# Library: oauth
+###############################################################
+
+oauth = not_found_dep
+oauth_library = 'none'
+oauthopt = get_option('oauth')
+
+if oauthopt == 'auto' and auto_features.disabled()
+ oauthopt = 'none'
+endif
+
+if oauthopt in ['auto', 'curl']
+ oauth = dependency('libcurl', required: (oauthopt == 'curl'))
+
+ if oauth.found()
+ oauth_library = 'curl'
+ cdata.set('USE_OAUTH', 1)
+ cdata.set('USE_OAUTH_CURL', 1)
+ endif
+endif
+
+if oauthopt == 'auto' and auto_features.enabled() and not oauth.found()
+ error('no OAuth implementation library found')
+endif
+
+
###############################################################
# Library: Tcl (for pltcl)
#
@@ -2834,6 +2861,7 @@ libpq_deps += [
gssapi,
ldap_r,
libintl,
+ oauth,
ssl,
]
@@ -3435,6 +3463,7 @@ if meson.version().version_compare('>=0.57')
'llvm': llvm,
'lz4': lz4,
'nls': libintl,
+ 'oauth': oauth,
'openssl': ssl,
'pam': pam,
'plperl': perl_dep,
diff --git a/meson_options.txt b/meson_options.txt
index 249ecc5ffd..3248b9cc1c 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -121,6 +121,10 @@ option('lz4', type: 'feature', value: 'auto',
option('nls', type: 'feature', value: 'auto',
description: 'Native language support')
+option('oauth', type : 'combo', choices : ['auto', 'none', 'curl'],
+ value: 'auto',
+ description: 'use LIB for OAuth 2.0 support (curl)')
+
option('pam', type: 'feature', value: 'auto',
description: 'PAM support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 8b3f8c24e0..79b3647834 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -193,6 +193,7 @@ with_ldap = @with_ldap@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
+with_oauth = @with_oauth@
with_system_tzdata = @with_system_tzdata@
with_uuid = @with_uuid@
with_zlib = @with_zlib@
diff --git a/src/include/common/oauth-common.h b/src/include/common/oauth-common.h
new file mode 100644
index 0000000000..5ff3488bfb
--- /dev/null
+++ b/src/include/common/oauth-common.h
@@ -0,0 +1,19 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth-common.h
+ * Declarations for helper functions used for OAuth/OIDC authentication
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/common/oauth-common.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef OAUTH_COMMON_H
+#define OAUTH_COMMON_H
+
+/* Name of SASL mechanism per IANA */
+#define OAUTHBEARER_NAME "OAUTHBEARER"
+
+#endif /* OAUTH_COMMON_H */
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 07e73567dc..a5e6f99ba4 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -243,6 +243,9 @@
/* Define to 1 if you have the `crypto' library (-lcrypto). */
#undef HAVE_LIBCRYPTO
+/* Define to 1 if you have the `curl' library (-lcurl). */
+#undef HAVE_LIBCURL
+
/* Define to 1 if you have the `ldap' library (-lldap). */
#undef HAVE_LIBLDAP
@@ -711,6 +714,12 @@
/* Define to select named POSIX semaphores. */
#undef USE_NAMED_POSIX_SEMAPHORES
+/* Define to 1 to build with OAuth 2.0 support. (--with-oauth) */
+#undef USE_OAUTH
+
+/* Define to 1 to use libcurl for OAuth support. */
+#undef USE_OAUTH_CURL
+
/* Define to 1 to build with OpenSSL support. (--with-ssl=openssl) */
#undef USE_OPENSSL
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index fe2af575c5..2618c293af 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -61,6 +61,14 @@ OBJS += \
fe-secure-gssapi.o
endif
+ifneq ($(with_oauth),no)
+OBJS += fe-auth-oauth.o
+
+ifeq ($(with_oauth),curl)
+OBJS += fe-auth-oauth-curl.o
+endif
+endif
+
ifeq ($(PORTNAME), cygwin)
override shlib = cyg$(NAME)$(DLSUFFIX)
endif
@@ -79,7 +87,7 @@ endif
# that are built correctly for use in a shlib.
SHLIB_LINK_INTERNAL = -lpgcommon_shlib -lpgport_shlib
ifneq ($(PORTNAME), win32)
-SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
+SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lcurl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
else
SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi32 -lssl -lsocket -lnsl -lresolv -lintl -lm $(PTHREAD_LIBS), $(LIBS)) $(LDAP_LIBS_FE)
endif
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 088592deb1..0f8f5e3125 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -193,3 +193,6 @@ PQsendClosePrepared 190
PQsendClosePortal 191
PQchangePassword 192
PQsendPipelineSync 193
+PQsetAuthDataHook 194
+PQgetAuthDataHook 195
+PQdefaultAuthDataHook 196
diff --git a/src/interfaces/libpq/fe-auth-oauth-curl.c b/src/interfaces/libpq/fe-auth-oauth-curl.c
new file mode 100644
index 0000000000..0504f96e4e
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth-curl.c
@@ -0,0 +1,1982 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth-curl.c
+ * The libcurl implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth-curl.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <curl/curl.h>
+#include <math.h>
+#ifdef HAVE_SYS_EPOLL_H
+#include <sys/epoll.h>
+#include <sys/timerfd.h>
+#endif
+#ifdef HAVE_SYS_EVENT_H
+#include <sys/event.h>
+#endif
+#include <unistd.h>
+
+#include "common/jsonapi.h"
+#include "fe-auth.h"
+#include "fe-auth-oauth.h"
+#include "libpq-int.h"
+#include "mb/pg_wchar.h"
+
+/*
+ * Parsed JSON Representations
+ *
+ * As a general rule, we parse and cache only the fields we're currently using.
+ * When adding new fields, ensure the corresponding free_*() function is updated
+ * too.
+ */
+
+/*
+ * The OpenID Provider configuration (alternatively named "authorization server
+ * metadata") jointly described by OpenID Connect Discovery 1.0 and RFC 8414:
+ *
+ * https://openid.net/specs/openid-connect-discovery-1_0.html
+ * https://www.rfc-editor.org/rfc/rfc8414#section-3.2
+ */
+struct provider
+{
+ char *issuer;
+ char *token_endpoint;
+ char *device_authorization_endpoint;
+ struct curl_slist *grant_types_supported;
+};
+
+static void
+free_provider(struct provider *provider)
+{
+ free(provider->issuer);
+ free(provider->token_endpoint);
+ free(provider->device_authorization_endpoint);
+ curl_slist_free_all(provider->grant_types_supported);
+}
+
+/*
+ * The Device Authorization response, described by RFC 8628:
+ *
+ * https://www.rfc-editor.org/rfc/rfc8628#section-3.2
+ */
+struct device_authz
+{
+ char *device_code;
+ char *user_code;
+ char *verification_uri;
+ char *interval_str;
+
+ /* Fields below are parsed from the corresponding string above. */
+ int interval;
+};
+
+static void
+free_device_authz(struct device_authz *authz)
+{
+ free(authz->device_code);
+ free(authz->user_code);
+ free(authz->verification_uri);
+ free(authz->interval_str);
+}
+
+/*
+ * The Token Endpoint error response, as described by RFC 6749:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-5.2
+ *
+ * Note that this response type can also be returned from the Device
+ * Authorization Endpoint.
+ */
+struct token_error
+{
+ char *error;
+ char *error_description;
+};
+
+static void
+free_token_error(struct token_error *err)
+{
+ free(err->error);
+ free(err->error_description);
+}
+
+/*
+ * The Access Token response, as described by RFC 6749:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-4.1.4
+ *
+ * During the Device Authorization flow, several temporary errors are expected
+ * as part of normal operation. To make it easy to handle these in the happy
+ * path, this contains an embedded token_error that is filled in if needed.
+ */
+struct token
+{
+ /* for successful responses */
+ char *access_token;
+ char *token_type;
+
+ /* for error responses */
+ struct token_error err;
+};
+
+static void
+free_token(struct token *tok)
+{
+ free(tok->access_token);
+ free(tok->token_type);
+ free_token_error(&tok->err);
+}
+
+/*
+ * Asynchronous State
+ */
+
+/* States for the overall async machine. */
+typedef enum
+{
+ OAUTH_STEP_INIT,
+ OAUTH_STEP_DISCOVERY,
+ OAUTH_STEP_DEVICE_AUTHORIZATION,
+ OAUTH_STEP_TOKEN_REQUEST,
+ OAUTH_STEP_WAIT_INTERVAL,
+} OAuthStep;
+
+/*
+ * The async_ctx holds onto state that needs to persist across multiple calls to
+ * pg_fe_run_oauth_flow(). Almost everything interacts with this in some way.
+ */
+struct async_ctx
+{
+ OAuthStep step; /* where are we in the flow? */
+
+#ifdef HAVE_SYS_EPOLL_H
+ int timerfd; /* a timerfd for signaling async timeouts */
+#endif
+ pgsocket mux; /* the multiplexer socket containing all
+ * descriptors tracked by cURL, plus the
+ * timerfd */
+ CURLM *curlm; /* top-level multi handle for cURL operations */
+ CURL *curl; /* the (single) easy handle for serial
+ * requests */
+
+ struct curl_slist *headers; /* common headers for all requests */
+ PQExpBufferData work_data; /* scratch buffer for general use (remember to
+ * clear out prior contents first!) */
+
+ /*------
+ * Since a single logical operation may stretch across multiple calls to
+ * our entry point, errors have three parts:
+ *
+ * - errctx: an optional static string, describing the global operation
+ * currently in progress. It'll be translated for you.
+ *
+ * - errbuf: contains the actual error message. Generally speaking, use
+ * actx_error[_str] to manipulate this. This must be filled
+ * with something useful on an error.
+ *
+ * - curl_err: an optional static error buffer used by cURL to put
+ * detailed information about failures. Unfortunately
+ * untranslatable.
+ *
+ * These pieces will be combined into a single error message looking
+ * something like the following, with errctx and/or curl_err omitted when
+ * absent:
+ *
+ * connection to server ... failed: errctx: errbuf (curl_err)
+ */
+ const char *errctx; /* not freed; must point to static allocation */
+ PQExpBufferData errbuf;
+ char curl_err[CURL_ERROR_SIZE];
+
+ /*
+ * These documents need to survive over multiple calls, and are therefore
+ * cached directly in the async_ctx.
+ */
+ struct provider provider;
+ struct device_authz authz;
+
+ bool user_prompted; /* have we already sent the authz prompt? */
+};
+
+/*
+ * Frees the async_ctx, which is stored directly on the PGconn. This is called
+ * during pqDropConnection() so that we don't leak resources even if
+ * PQconnectPoll() never calls us back.
+ *
+ * TODO: we should probably call this at the end of a successful authentication,
+ * too, to proactively free up resources.
+ */
+static void
+free_curl_async_ctx(PGconn *conn, void *ctx)
+{
+ struct async_ctx *actx = ctx;
+
+ Assert(actx); /* oauth_free() shouldn't call us otherwise */
+
+ /*
+ * TODO: in general, none of the error cases below should ever happen if
+ * we have no bugs above. But if we do hit them, surfacing those errors
+ * somehow might be the only way to have a chance to debug them. What's
+ * the best way to do that? Assertions? Spraying messages on stderr?
+ * Bubbling an error code to the top? Appending to the connection's error
+ * message only helps if the bug caused a connection failure; otherwise
+ * it'll be buried...
+ */
+
+ if (actx->curlm && actx->curl)
+ {
+ CURLMcode err = curl_multi_remove_handle(actx->curlm, actx->curl);
+
+ if (err)
+ libpq_append_conn_error(conn,
+ "cURL easy handle removal failed: %s",
+ curl_multi_strerror(err));
+ }
+
+ if (actx->curl)
+ {
+ /*
+ * curl_multi_cleanup() doesn't free any associated easy handles; we
+ * need to do that separately. We only ever have one easy handle per
+ * multi handle.
+ */
+ curl_easy_cleanup(actx->curl);
+ }
+
+ if (actx->curlm)
+ {
+ CURLMcode err = curl_multi_cleanup(actx->curlm);
+
+ if (err)
+ libpq_append_conn_error(conn,
+ "cURL multi handle cleanup failed: %s",
+ curl_multi_strerror(err));
+ }
+
+ free_provider(&actx->provider);
+ free_device_authz(&actx->authz);
+
+ curl_slist_free_all(actx->headers);
+ termPQExpBuffer(&actx->work_data);
+ termPQExpBuffer(&actx->errbuf);
+
+ if (actx->mux != PGINVALID_SOCKET)
+ close(actx->mux);
+#ifdef HAVE_SYS_EPOLL_H
+ if (actx->timerfd >= 0)
+ close(actx->timerfd);
+#endif
+
+ free(actx);
+}
+
+/*
+ * Macros for manipulating actx->errbuf. actx_error() translates and formats a
+ * string for you; actx_error_str() appends a string directly without
+ * translation.
+ */
+
+#define actx_error(ACTX, FMT, ...) \
+ appendPQExpBuffer(&(ACTX)->errbuf, libpq_gettext(FMT), ##__VA_ARGS__)
+
+#define actx_error_str(ACTX, S) \
+ appendPQExpBufferStr(&(ACTX)->errbuf, S)
+
+/*
+ * Macros for getting and setting state for the connection's two cURL handles,
+ * so you don't have to write out the error handling every time.
+ */
+
+#define CHECK_MSETOPT(ACTX, OPT, VAL, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLMcode _setopterr = curl_multi_setopt(_actx->curlm, OPT, VAL); \
+ if (_setopterr) { \
+ actx_error(_actx, "failed to set %s on OAuth connection: %s",\
+ #OPT, curl_multi_strerror(_setopterr)); \
+ FAILACTION; \
+ } \
+ } while (0)
+
+#define CHECK_SETOPT(ACTX, OPT, VAL, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLcode _setopterr = curl_easy_setopt(_actx->curl, OPT, VAL); \
+ if (_setopterr) { \
+ actx_error(_actx, "failed to set %s on OAuth connection: %s",\
+ #OPT, curl_easy_strerror(_setopterr)); \
+ FAILACTION; \
+ } \
+ } while (0)
+
+#define CHECK_GETINFO(ACTX, INFO, OUT, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLcode _getinfoerr = curl_easy_getinfo(_actx->curl, INFO, OUT); \
+ if (_getinfoerr) { \
+ actx_error(_actx, "failed to get %s from OAuth response: %s",\
+ #INFO, curl_easy_strerror(_getinfoerr)); \
+ FAILACTION; \
+ } \
+ } while (0)
+
+/*
+ * General JSON Parsing for OAuth Responses
+ */
+
+/*
+ * Represents a single name/value pair in a JSON object. This is the primary
+ * interface to parse_oauth_json().
+ *
+ * All fields are stored internally as strings or lists of strings, so clients
+ * have to explicitly parse other scalar types (though they will have gone
+ * through basic lexical validation). Storing nested objects is not currently
+ * supported, nor is parsing arrays of anything other than strings.
+ */
+struct json_field
+{
+ const char *name; /* name (key) of the member */
+
+ JsonTokenType type; /* currently supports JSON_TOKEN_STRING,
+ * JSON_TOKEN_NUMBER, and
+ * JSON_TOKEN_ARRAY_START */
+ union
+ {
+ char **scalar; /* for all scalar types */
+ struct curl_slist **array; /* for type == JSON_TOKEN_ARRAY_START */
+ };
+
+ bool required; /* REQUIRED field, or just OPTIONAL? */
+};
+
+/* Documentation macros for json_field.required. */
+#define REQUIRED true
+#define OPTIONAL false
+
+/* Parse state for parse_oauth_json(). */
+struct oauth_parse
+{
+ PQExpBuffer errbuf; /* detail message for JSON_SEM_ACTION_FAILED */
+ int nested; /* nesting level (zero is the top) */
+
+ const struct json_field *fields; /* field definition array */
+ const struct json_field *active; /* points inside the fields array */
+};
+
+#define oauth_parse_set_error(ctx, fmt, ...) \
+ appendPQExpBuffer((ctx)->errbuf, libpq_gettext(fmt), ##__VA_ARGS__)
+
+static void
+report_type_mismatch(struct oauth_parse *ctx)
+{
+ char *msgfmt;
+
+ Assert(ctx->active);
+
+ /*
+ * At the moment, the only fields we're interested in are strings,
+ * numbers, and arrays of strings.
+ */
+ switch (ctx->active->type)
+ {
+ case JSON_TOKEN_STRING:
+ msgfmt = "field \"%s\" must be a string";
+ break;
+
+ case JSON_TOKEN_NUMBER:
+ msgfmt = "field \"%s\" must be a number";
+ break;
+
+ case JSON_TOKEN_ARRAY_START:
+ msgfmt = "field \"%s\" must be an array of strings";
+ break;
+
+ default:
+ Assert(false);
+ msgfmt = "field \"%s\" has unexpected type";
+ }
+
+ oauth_parse_set_error(ctx, msgfmt, ctx->active->name);
+}
+
+static JsonParseErrorType
+oauth_json_object_start(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (ctx->active)
+ {
+ /*
+ * Currently, none of the fields we're interested in can be or contain
+ * objects, so we can reject this case outright.
+ */
+ report_type_mismatch(ctx);
+ return JSON_SEM_ACTION_FAILED;
+ }
+
+ ++ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_field_start(void *state, char *name, bool isnull)
+{
+ struct oauth_parse *ctx = state;
+
+ /* We care only about the top-level fields. */
+ if (ctx->nested == 1)
+ {
+ const struct json_field *field = ctx->fields;
+
+ /*
+ * We should never start parsing a new field while a previous one is
+ * still active.
+ *
+ * TODO: this code relies on assertions too much. We need to exit
+ * sanely on internal logic errors, to avoid turning bugs into
+ * vulnerabilities.
+ */
+ Assert(!ctx->active);
+
+ while (field->name)
+ {
+ if (strcmp(name, field->name) == 0)
+ {
+ ctx->active = field;
+ break;
+ }
+
+ ++field;
+ }
+ }
+
+ free(name);
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_end(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_start(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (!ctx->nested)
+ {
+ oauth_parse_set_error(ctx, "top-level element must be an object");
+ return JSON_SEM_ACTION_FAILED;
+ }
+
+ if (ctx->active)
+ {
+ if (ctx->active->type != JSON_TOKEN_ARRAY_START
+ /* The arrays we care about must not have arrays as values. */
+ || ctx->nested > 1)
+ {
+ report_type_mismatch(ctx);
+ return JSON_SEM_ACTION_FAILED;
+ }
+ }
+
+ ++ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_end(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (ctx->active)
+ {
+ /*
+ * This assumes that no target arrays can contain other arrays, which
+ * we check in the array_start callback.
+ */
+ Assert(ctx->nested == 2);
+ Assert(ctx->active->type == JSON_TOKEN_ARRAY_START);
+
+ ctx->active = NULL;
+ }
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_scalar(void *state, char *token, JsonTokenType type)
+{
+ struct oauth_parse *ctx = state;
+ JsonParseErrorType result = JSON_SUCCESS;
+
+ if (!ctx->nested)
+ {
+ oauth_parse_set_error(ctx, "top-level element must be an object");
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ if (ctx->active)
+ {
+ JsonTokenType expected;
+
+ /*
+ * Make sure this matches what the active field expects. Arrays must
+ * contain only strings with the current implementation.
+ */
+ if (ctx->active->type == JSON_TOKEN_ARRAY_START)
+ expected = JSON_TOKEN_STRING;
+ else
+ expected = ctx->active->type;
+
+ if (type != expected)
+ {
+ report_type_mismatch(ctx);
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ /*
+ * FIXME if the JSON field is duplicated, we'll leak the prior value.
+ * Error out in that case instead.
+ */
+ if (ctx->active->type != JSON_TOKEN_ARRAY_START)
+ {
+ Assert(ctx->nested == 1);
+
+ *ctx->active->scalar = token;
+ ctx->active = NULL;
+
+ return JSON_SUCCESS; /* don't free the token */
+ }
+ else /* ctx->target_array */
+ {
+ struct curl_slist *temp;
+
+ Assert(ctx->nested == 2);
+
+ temp = curl_slist_append(*ctx->active->array, token);
+ if (!temp)
+ {
+ oauth_parse_set_error(ctx, "out of memory");
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ *ctx->active->array = temp;
+
+ /*
+ * Note that curl_slist_append() makes a copy of the token, so we
+ * can free it below.
+ */
+ }
+ }
+ else
+ {
+ /* otherwise we just ignore it */
+ }
+
+cleanup:
+ free(token);
+ return result;
+}
+
+/*
+ * A helper function for general JSON parsing. fields is the array of field
+ * definitions with their backing pointers. The response will be parsed from
+ * actx->curl and actx->work_data (as set up by start_request()), and any
+ * parsing errors will be placed into actx->errbuf.
+ */
+static bool
+parse_oauth_json(struct async_ctx *actx, const struct json_field *fields)
+{
+ PQExpBuffer resp = &actx->work_data;
+ char *content_type;
+ JsonLexContext lex = {0};
+ JsonSemAction sem = {0};
+ JsonParseErrorType err;
+ struct oauth_parse ctx = {0};
+ bool success = false;
+
+ /* Make sure the server thinks it's given us JSON. */
+ CHECK_GETINFO(actx, CURLINFO_CONTENT_TYPE, &content_type, return false);
+
+ if (!content_type)
+ {
+ actx_error(actx, "no content type was provided");
+ goto cleanup;
+ }
+ else if (strcasecmp(content_type, "application/json") != 0)
+ {
+ actx_error(actx, "unexpected content type \"%s\"", content_type);
+ goto cleanup;
+ }
+
+ if (strlen(resp->data) != resp->len)
+ {
+ actx_error(actx, "response contains embedded NULLs");
+ goto cleanup;
+ }
+
+ makeJsonLexContextCstringLen(&lex, resp->data, resp->len, PG_UTF8, true);
+
+ ctx.errbuf = &actx->errbuf;
+ ctx.fields = fields;
+ sem.semstate = &ctx;
+
+ sem.object_start = oauth_json_object_start;
+ sem.object_field_start = oauth_json_object_field_start;
+ sem.object_end = oauth_json_object_end;
+ sem.array_start = oauth_json_array_start;
+ sem.array_end = oauth_json_array_end;
+ sem.scalar = oauth_json_scalar;
+
+ err = pg_parse_json(&lex, &sem);
+
+ if (err != JSON_SUCCESS)
+ {
+ /*
+ * For JSON_SEM_ACTION_FAILED, we've already written the error
+ * message. Other errors come directly from pg_parse_json(), already
+ * translated.
+ */
+ if (err != JSON_SEM_ACTION_FAILED)
+ actx_error_str(actx, json_errdetail(err, &lex));
+
+ goto cleanup;
+ }
+
+ /* Check all required fields. */
+ while (fields->name)
+ {
+ if (fields->required && !*fields->scalar && !*fields->array)
+ {
+ actx_error(actx, "field \"%s\" is missing", fields->name);
+ goto cleanup;
+ }
+
+ fields++;
+ }
+
+ success = true;
+
+cleanup:
+ freeJsonLexContext(&lex);
+ return success;
+}
+
+/*
+ * JSON Parser Definitions
+ */
+
+static bool
+parse_provider(struct async_ctx *actx, struct provider *provider)
+{
+ struct json_field fields[] = {
+ {"issuer", JSON_TOKEN_STRING, {&provider->issuer}, REQUIRED},
+ {"token_endpoint", JSON_TOKEN_STRING, {&provider->token_endpoint}, REQUIRED},
+
+ /*----
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - jwks_uri
+ * - response_types_supported
+ * - subject_types_supported
+ * - id_token_signing_alg_values_supported
+ */
+
+ {"device_authorization_endpoint", JSON_TOKEN_STRING, {&provider->device_authorization_endpoint}, OPTIONAL},
+ {"grant_types_supported", JSON_TOKEN_ARRAY_START, {.array = &provider->grant_types_supported}, OPTIONAL},
+
+ {0},
+ };
+
+ return parse_oauth_json(actx, fields);
+}
+
+/*
+ * Parses the "interval" JSON number, corresponding to the number of seconds to
+ * wait between token endpoint requests.
+ *
+ * RFC 8628 is pretty silent on sanity checks for the interval. As a matter of
+ * practicality, round any fractional intervals up to the next second, and clamp
+ * the result at a minimum of one. (Zero-second intervals would result in an
+ * expensive network polling loop.)
+ *
+ * TODO: maybe clamp the upper bound too, based on the libpq timeout and/or the
+ * code expiration time?
+ */
+static int
+parse_interval(const char *interval_str)
+{
+ double parsed;
+ int cnt;
+
+ /*
+ * The JSON lexer has already validated the number, which is stricter than
+ * the %f format, so we should be good to use sscanf().
+ */
+ cnt = sscanf(interval_str, "%lf", &parsed);
+
+ if (cnt != 1)
+ {
+ /*
+ * Either the lexer screwed up or our assumption above isn't true, and
+ * either way a developer needs to take a look.
+ */
+ Assert(cnt == 1);
+ return 1; /* don't fall through in release builds */
+ }
+
+ parsed = ceil(parsed);
+
+ if (parsed < 1)
+ return 1; /* TODO this slows down the tests
+ * considerably... */
+ else if (INT_MAX <= parsed)
+ return INT_MAX;
+
+ return parsed;
+}
+
+static bool
+parse_device_authz(struct async_ctx *actx, struct device_authz *authz)
+{
+ struct json_field fields[] = {
+ {"device_code", JSON_TOKEN_STRING, {&authz->device_code}, REQUIRED},
+ {"user_code", JSON_TOKEN_STRING, {&authz->user_code}, REQUIRED},
+ {"verification_uri", JSON_TOKEN_STRING, {&authz->verification_uri}, REQUIRED},
+
+ /*
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - expires_in
+ */
+
+ {"interval", JSON_TOKEN_NUMBER, {&authz->interval_str}, OPTIONAL},
+
+ {0},
+ };
+
+ if (!parse_oauth_json(actx, fields))
+ return false;
+
+ /*
+ * Parse our numeric fields. Lexing has already completed by this time, so
+ * we at least know they're valid JSON numbers.
+ */
+ if (authz->interval_str)
+ authz->interval = parse_interval(authz->interval_str);
+ else
+ {
+ /* TODO: handle default interval of 5 seconds */
+ }
+
+ return true;
+}
+
+static bool
+parse_token_error(struct async_ctx *actx, struct token_error *err)
+{
+ bool result;
+ struct json_field fields[] = {
+ {"error", JSON_TOKEN_STRING, {&err->error}, REQUIRED},
+
+ {"error_description", JSON_TOKEN_STRING, {&err->error_description}, OPTIONAL},
+
+ {0},
+ };
+
+ result = parse_oauth_json(actx, fields);
+
+ /*
+ * Since token errors are parsed during other active error paths, only
+ * override the errctx if parsing explicitly fails.
+ */
+ if (!result)
+ actx->errctx = "failed to parse token error response";
+
+ return result;
+}
+
+static bool
+parse_access_token(struct async_ctx *actx, struct token *tok)
+{
+ struct json_field fields[] = {
+ {"access_token", JSON_TOKEN_STRING, {&tok->access_token}, REQUIRED},
+ {"token_type", JSON_TOKEN_STRING, {&tok->token_type}, REQUIRED},
+
+ /*
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - scope (only required if different than requested -- TODO check)
+ */
+
+ {0},
+ };
+
+ return parse_oauth_json(actx, fields);
+}
+
+/*
+ * cURL Multi Setup/Callbacks
+ */
+
+/*
+ * Sets up the actx->mux, which is the altsock that PQconnectPoll clients will
+ * select() on instead of the Postgres socket during OAuth negotiation.
+ *
+ * This is just an epoll set or kqueue abstracting multiple other descriptors.
+ * A timerfd is always part of the set when using epoll; it's just disabled
+ * when we're not using it.
+ */
+static bool
+setup_multiplexer(struct async_ctx *actx)
+{
+#ifdef HAVE_SYS_EPOLL_H
+ struct epoll_event ev = {.events = EPOLLIN};
+
+ actx->mux = epoll_create1(EPOLL_CLOEXEC);
+ if (actx->mux < 0)
+ {
+ actx_error(actx, "failed to create epoll set: %m");
+ return false;
+ }
+
+ actx->timerfd = timerfd_create(CLOCK_MONOTONIC, TFD_CLOEXEC);
+ if (actx->timerfd < 0)
+ {
+ actx_error(actx, "failed to create timerfd: %m");
+ return false;
+ }
+
+ if (epoll_ctl(actx->mux, EPOLL_CTL_ADD, actx->timerfd, &ev) < 0)
+ {
+ actx_error(actx, "failed to add timerfd to epoll set: %m");
+ return false;
+ }
+
+ return true;
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ actx->mux = kqueue();
+ if (actx->mux < 0)
+ {
+ actx_error(actx, "failed to create kqueue: %m");
+ return false;
+ }
+
+ return true;
+#endif
+
+ actx_error(actx, "here's a nickel kid, get yourself a better computer");
+ return false;
+}
+
+/*
+ * Adds and removes sockets from the multiplexer set, as directed by the
+ * cURL multi handle.
+ */
+static int
+register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
+ void *socketp)
+{
+#ifdef HAVE_SYS_EPOLL_H
+ struct async_ctx *actx = ctx;
+ struct epoll_event ev = {0};
+ int res;
+ int op = EPOLL_CTL_ADD;
+
+ switch (what)
+ {
+ case CURL_POLL_IN:
+ ev.events = EPOLLIN;
+ break;
+
+ case CURL_POLL_OUT:
+ ev.events = EPOLLOUT;
+ break;
+
+ case CURL_POLL_INOUT:
+ ev.events = EPOLLIN | EPOLLOUT;
+ break;
+
+ case CURL_POLL_REMOVE:
+ op = EPOLL_CTL_DEL;
+ break;
+
+ default:
+ actx_error(actx, "unknown cURL socket operation (%d)", what);
+ return -1;
+ }
+
+ res = epoll_ctl(actx->mux, op, socket, &ev);
+ if (res < 0 && errno == EEXIST)
+ {
+ /* We already had this socket in the pollset. */
+ op = EPOLL_CTL_MOD;
+ res = epoll_ctl(actx->mux, op, socket, &ev);
+ }
+
+ if (res < 0)
+ {
+ switch (op)
+ {
+ case EPOLL_CTL_ADD:
+ actx_error(actx, "could not add to epoll set: %m");
+ break;
+
+ case EPOLL_CTL_DEL:
+ actx_error(actx, "could not delete from epoll set: %m");
+ break;
+
+ default:
+ actx_error(actx, "could not update epoll set: %m");
+ }
+
+ return -1;
+ }
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct async_ctx *actx = ctx;
+ struct kevent ev[2] = {{0}};
+ struct kevent ev_out[2];
+ struct timespec timeout = {0};
+ int nev = 0;
+ int res;
+
+ switch (what)
+ {
+ case CURL_POLL_IN:
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_OUT:
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_INOUT:
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_REMOVE:
+
+ /*
+ * We don't know which of these is currently registered, perhaps
+ * both, so we try to remove both. This means we need to tolerate
+ * ENOENT below.
+ */
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_DELETE | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_DELETE | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ default:
+ actx_error(actx, "unknown cURL socket operation (%d)", what);
+ return -1;
+ }
+
+ res = kevent(actx->mux, ev, nev, ev_out, lengthof(ev_out), &timeout);
+ if (res < 0)
+ {
+ actx_error(actx, "could not modify kqueue: %m");
+ return -1;
+ }
+
+ /*
+ * We can't use the simple errno version of kevent, because we need to
+ * skip over ENOENT while still allowing a second change to be processed.
+ * So we need a longer-form error checking loop.
+ */
+ for (int i = 0; i < res; ++i)
+ {
+ /*
+ * EV_RECEIPT should guarantee one EV_ERROR result for every change,
+ * whether successful or not. Failed entries contain a non-zero errno
+ * in the `data` field.
+ */
+ Assert(ev_out[i].flags & EV_ERROR);
+
+ errno = ev_out[i].data;
+ if (errno && errno != ENOENT)
+ {
+ switch (what)
+ {
+ case CURL_POLL_REMOVE:
+ actx_error(actx, "could not delete from kqueue: %m");
+ break;
+ default:
+ actx_error(actx, "could not add to kqueue: %m");
+ }
+ return -1;
+ }
+ }
+#endif
+
+ return 0;
+}
+
+/*
+ * Adds or removes timeouts from the multiplexer set, as directed by the
+ * cURL multi handle. Rather than continually adding and removing the timer,
+ * we keep it in the set at all times and just disarm it when it's not
+ * needed.
+ */
+static int
+register_timer(CURLM *curlm, long timeout, void *ctx)
+{
+#if HAVE_SYS_EPOLL_H
+ struct async_ctx *actx = ctx;
+ struct itimerspec spec = {0};
+
+ if (timeout < 0)
+ {
+ /* the zero itimerspec will disarm the timer below */
+ }
+ else if (timeout == 0)
+ {
+ /*
+ * A zero timeout means cURL wants us to call back immediately. That's
+ * not technically an option for timerfd, but we can make the timeout
+ * ridiculously short.
+ *
+ * TODO: maybe just signal drive_request() to immediately call back in
+ * this case?
+ */
+ spec.it_value.tv_nsec = 1;
+ }
+ else
+ {
+ spec.it_value.tv_sec = timeout / 1000;
+ spec.it_value.tv_nsec = (timeout % 1000) * 1000000;
+ }
+
+ if (timerfd_settime(actx->timerfd, 0 /* no flags */ , &spec, NULL) < 0)
+ {
+ actx_error(actx, "setting timerfd to %ld: %m", timeout);
+ return -1;
+ }
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct async_ctx *actx = ctx;
+ struct kevent ev;
+
+ EV_SET(&ev, 1, EVFILT_TIMER, timeout < 0 ? EV_DELETE : EV_ADD,
+ 0, timeout, 0);
+ if (kevent(actx->mux, &ev, 1, NULL, 0, NULL) < 0 && errno != ENOENT)
+ {
+ actx_error(actx, "setting kqueue timer to %ld: %m", timeout);
+ return -1;
+ }
+#endif
+
+ return 0;
+}
+
+/*
+ * Initializes the two cURL handles in the async_ctx. The multi handle,
+ * actx->curlm, is what drives the asynchronous engine and tells us what to do
+ * next. The easy handle, actx->curl, encapsulates the state for a single
+ * request/response. It's added to the multi handle as needed, during
+ * start_request().
+ */
+static bool
+setup_curl_handles(struct async_ctx *actx)
+{
+ curl_version_info_data *curl_info;
+
+ /*
+ * Create our multi handle. This encapsulates the entire conversation with
+ * cURL for this connection.
+ */
+ actx->curlm = curl_multi_init();
+ if (!actx->curlm)
+ {
+ /* We don't get a lot of feedback on the failure reason. */
+ actx_error(actx, "failed to create cURL multi handle");
+ return false;
+ }
+
+ /*
+ * Extract information about the libcurl we are linked against.
+ */
+ curl_info = curl_version_info(CURLVERSION_NOW);
+
+ /*
+ * The multi handle tells us what to wait on using two callbacks. These
+ * will manipulate actx->mux as needed.
+ */
+ CHECK_MSETOPT(actx, CURLMOPT_SOCKETFUNCTION, register_socket, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_SOCKETDATA, actx, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_TIMERFUNCTION, register_timer, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_TIMERDATA, actx, return false);
+
+ /*
+ * Set up an easy handle. All of our requests are made serially, so we
+ * only ever need to keep track of one.
+ */
+ actx->curl = curl_easy_init();
+ if (!actx->curl)
+ {
+ actx_error(actx, "failed to create cURL handle");
+ return false;
+ }
+
+ /*
+ * Multi-threaded applications must set CURLOPT_NOSIGNAL. This requires us
+ * to handle the possibility of SIGPIPE ourselves.
+ *
+ * TODO: handle SIGPIPE via pq_block_sigpipe(), or via a
+ * CURLOPT_SOCKOPTFUNCTION maybe...
+ */
+ CHECK_SETOPT(actx, CURLOPT_NOSIGNAL, 1L, return false);
+ if (!curl_info->ares_num)
+ {
+ /* No alternative resolver, TODO: warn about timeouts */
+ }
+
+ /* TODO investigate using conn->Pfdebug and CURLOPT_DEBUGFUNCTION here */
+ CHECK_SETOPT(actx, CURLOPT_VERBOSE, 1L, return false);
+ CHECK_SETOPT(actx, CURLOPT_ERRORBUFFER, actx->curl_err, return false);
+
+ /*
+ * Only HTTP[S] is allowed. TODO: disallow HTTP without user opt-in
+ */
+ CHECK_SETOPT(actx, CURLOPT_PROTOCOLS, CURLPROTO_HTTP | CURLPROTO_HTTPS, return false);
+
+ /*
+ * Suppress the Accept header to make our request as minimal as possible.
+ * (Ideally we would set it to "application/json" instead, but OpenID is
+ * pretty strict when it comes to provider behavior, so we have to check
+ * what comes back anyway.)
+ */
+ actx->headers = curl_slist_append(actx->headers, "Accept:"); /* TODO: check result */
+ CHECK_SETOPT(actx, CURLOPT_HTTPHEADER, actx->headers, return false);
+
+ return true;
+}
+
+/*
+ * Generic HTTP Request Handlers
+ */
+
+/*
+ * Response callback from cURL; appends the response body into actx->work_data.
+ * See start_request().
+ */
+static size_t
+append_data(char *buf, size_t size, size_t nmemb, void *userdata)
+{
+ PQExpBuffer resp = userdata;
+ size_t len = size * nmemb;
+
+ /* TODO: cap the maximum size */
+ appendBinaryPQExpBuffer(resp, buf, len);
+ /* TODO: check for broken buffer */
+
+ return len;
+}
+
+/*
+ * Begins an HTTP request on the multi handle. The caller should have set up all
+ * request-specific options on actx->curl first. The server's response body will
+ * be accumulated in actx->work_data (which will be reset, so don't store
+ * anything important there across this call).
+ *
+ * Once a request is queued, it can be driven to completion via drive_request().
+ */
+static bool
+start_request(struct async_ctx *actx)
+{
+ CURLMcode err;
+ int running;
+
+ resetPQExpBuffer(&actx->work_data);
+ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data, return false);
+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, &actx->work_data, return false);
+
+ err = curl_multi_add_handle(actx->curlm, actx->curl);
+ if (err)
+ {
+ actx_error(actx, "failed to queue HTTP request: %s",
+ curl_multi_strerror(err));
+ return false;
+ }
+
+ err = curl_multi_socket_action(actx->curlm, CURL_SOCKET_TIMEOUT, 0, &running);
+ if (err)
+ {
+ actx_error(actx, "asynchronous HTTP request failed: %s",
+ curl_multi_strerror(err));
+ return false;
+ }
+
+ /*
+ * Sanity check.
+ *
+ * TODO: even though this is nominally an asynchronous process, there are
+ * apparently operations that can synchronously fail by this point, such
+ * as connections to closed local ports. Maybe we need to let this case
+ * fall through to drive_request instead, or else perform a
+ * curl_multi_info_read immediately.
+ */
+ if (running != 1)
+ {
+ actx_error(actx, "failed to queue HTTP request");
+ return false;
+ }
+
+ return true;
+}
+
+/*
+ * Drives the multi handle towards completion. The caller should have already
+ * set up an asynchronous request via start_request().
+ */
+static PostgresPollingStatusType
+drive_request(struct async_ctx *actx)
+{
+ CURLMcode err;
+ int running;
+ CURLMsg *msg;
+ int msgs_left;
+ bool done;
+
+ err = curl_multi_socket_all(actx->curlm, &running);
+ if (err)
+ {
+ actx_error(actx, "asynchronous HTTP request failed: %s",
+ curl_multi_strerror(err));
+ return PGRES_POLLING_FAILED;
+ }
+
+ if (running)
+ {
+ /* We'll come back again. */
+ return PGRES_POLLING_READING;
+ }
+
+ done = false;
+ while ((msg = curl_multi_info_read(actx->curlm, &msgs_left)) != NULL)
+ {
+ if (msg->msg != CURLMSG_DONE)
+ {
+ /*
+ * Future cURL versions may define new message types; we don't
+ * know how to handle them, so we'll ignore them.
+ */
+ continue;
+ }
+
+ /* First check the status of the request itself. */
+ if (msg->data.result != CURLE_OK)
+ {
+ actx_error_str(actx, curl_easy_strerror(msg->data.result));
+ return PGRES_POLLING_FAILED;
+ }
+
+ /* Now remove the finished handle; we'll add it back later if needed. */
+ err = curl_multi_remove_handle(actx->curlm, msg->easy_handle);
+ if (err)
+ {
+ actx_error(actx, "cURL easy handle removal failed: %s",
+ curl_multi_strerror(err));
+ return PGRES_POLLING_FAILED;
+ }
+
+ done = true;
+ }
+
+ /* Sanity check. */
+ if (!done)
+ {
+ actx_error(actx, "no result was retrieved for the finished handle");
+ return PGRES_POLLING_FAILED;
+ }
+
+ return PGRES_POLLING_OK;
+}
+
+/*
+ * Specific HTTP Request Handlers
+ *
+ * This is finally the beginning of the actual application logic. Generally
+ * speaking, a single request consists of a start_* and a finish_* step, with
+ * drive_request() pumping the machine in between.
+ */
+
+/*
+ * Queue an OpenID Provider Configuration Request:
+ *
+ * https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderConfigurationRequest
+ * https://www.rfc-editor.org/rfc/rfc8414#section-3.1
+ *
+ * This is done first to get the endpoint URIs we need to contact and to make
+ * sure the provider provides a device authorization flow. finish_discovery()
+ * will fill in actx->provider.
+ */
+static bool
+start_discovery(struct async_ctx *actx, const char *discovery_uri)
+{
+ CHECK_SETOPT(actx, CURLOPT_HTTPGET, 1L, return false);
+ CHECK_SETOPT(actx, CURLOPT_URL, discovery_uri, return false);
+
+ return start_request(actx);
+}
+
+static bool
+finish_discovery(struct async_ctx *actx)
+{
+ long response_code;
+
+ /*----
+ * Now check the response. OIDC Discovery 1.0 is pretty strict:
+ *
+ * A successful response MUST use the 200 OK HTTP status code and
+ * return a JSON object using the application/json content type that
+ * contains a set of Claims as its members that are a subset of the
+ * Metadata values defined in Section 3.
+ *
+ * Compared to standard HTTP semantics, this makes life easy -- we don't
+ * need to worry about redirections (which would call the Issuer host
+ * validation into question), or non-authoritative responses, or any other
+ * complications.
+ */
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ if (response_code != 200)
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ actx->errctx = "failed to parse OpenID discovery document";
+ if (!parse_provider(actx, &actx->provider))
+ return false; /* error message already set */
+
+ /*
+ * Fill in any defaults for OPTIONAL/RECOMMENDED fields we care about.
+ */
+ if (!actx->provider.grant_types_supported)
+ {
+ /*
+ * Per Section 3, the default is ["authorization_code", "implicit"].
+ */
+ struct curl_slist *temp = actx->provider.grant_types_supported;
+
+ temp = curl_slist_append(temp, "authorization_code");
+ if (temp)
+ {
+ temp = curl_slist_append(temp, "implicit");
+ }
+
+ if (!temp)
+ {
+ actx_error(actx, "out of memory");
+ return false;
+ }
+
+ actx->provider.grant_types_supported = temp;
+ }
+
+ return true;
+}
+
+#define OAUTH_GRANT_TYPE_DEVICE_CODE "urn:ietf:params:oauth:grant-type:device_code"
+
+/*
+ * Ensure that the provider supports the Device Authorization flow (i.e. it
+ * accepts the device_code grant type and provides an authorization endpoint).
+ */
+static bool
+check_for_device_flow(struct async_ctx *actx)
+{
+ const struct provider *provider = &actx->provider;
+ const struct curl_slist *grant;
+ bool device_grant_found = false;
+
+ Assert(provider->issuer); /* ensured by get_discovery_document() */
+
+ /*------
+ * First, sanity checks for discovery contents that are OPTIONAL in the
+ * spec but required for our flow:
+ * - the issuer must support the device_code grant
+ * - the issuer must have actually given us a
+ * device_authorization_endpoint
+ */
+
+ grant = provider->grant_types_supported;
+ while (grant)
+ {
+ if (strcmp(grant->data, OAUTH_GRANT_TYPE_DEVICE_CODE) == 0)
+ {
+ device_grant_found = true;
+ break;
+ }
+
+ grant = grant->next;
+ }
+
+ if (!device_grant_found)
+ {
+ actx_error(actx, "issuer \"%s\" does not support device code grants",
+ provider->issuer);
+ return false;
+ }
+
+ if (!provider->device_authorization_endpoint)
+ {
+ actx_error(actx,
+ "issuer \"%s\" does not provide a device authorization endpoint",
+ provider->issuer);
+ return false;
+ }
+
+ /* TODO: check that the endpoint uses HTTPS */
+
+ return true;
+}
+
+/*
+ * Queue a Device Authorization Request:
+ *
+ * https://www.rfc-editor.org/rfc/rfc8628#section-3.1
+ *
+ * This is the second step. We ask the provider to verify the end user out of
+ * band and authorize us to act on their behalf; it will give us the required
+ * nonces for us to later poll the request status, which we'll grab in
+ * finish_device_authz().
+ */
+static bool
+start_device_authz(struct async_ctx *actx, PGconn *conn)
+{
+ const char *device_authz_uri = actx->provider.device_authorization_endpoint;
+ PQExpBuffer work_buffer = &actx->work_data;
+
+ Assert(conn->oauth_client_id); /* ensured by get_auth_token() */
+ Assert(device_authz_uri); /* ensured by check_for_device_flow() */
+
+ /* Construct our request body. TODO: url-encode */
+ resetPQExpBuffer(work_buffer);
+ appendPQExpBuffer(work_buffer, "client_id=%s", conn->oauth_client_id);
+ if (conn->oauth_scope)
+ appendPQExpBuffer(work_buffer, "&scope=%s", conn->oauth_scope);
+ /* TODO check for broken buffer */
+
+ /* Make our request. */
+ CHECK_SETOPT(actx, CURLOPT_URL, device_authz_uri, return false);
+ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data, return false);
+
+ if (conn->oauth_client_secret)
+ {
+ /*----
+ * Use HTTP Basic auth to send the password. Per RFC 6749, Sec. 2.3.1,
+ *
+ * Including the client credentials in the request-body using the
+ * two parameters is NOT RECOMMENDED and SHOULD be limited to
+ * clients unable to directly utilize the HTTP Basic authentication
+ * scheme (or other password-based HTTP authentication schemes).
+ *
+ * TODO: should we omit client_id from the body in this case?
+ */
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC, return false);
+ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id, return false);
+ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret, return false);
+ }
+ else
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE, return false);
+
+ return start_request(actx);
+}
+
+static bool
+finish_device_authz(struct async_ctx *actx)
+{
+ long response_code;
+
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ /*
+ * The device authorization endpoint uses the same error response as the
+ * token endpoint, so we have to handle 400/401 here too.
+ */
+ if (response_code != 200
+ && response_code != 400
+ /* && response_code != 401 TODO */ )
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ if (response_code != 200)
+ {
+ struct token_error err = {0};
+
+ if (!parse_token_error(actx, &err))
+ {
+ free_token_error(&err);
+ return false;
+ }
+
+ if (err.error_description)
+ appendPQExpBuffer(&actx->errbuf, "%s ", err.error_description);
+
+ appendPQExpBuffer(&actx->errbuf, "(%s)", err.error);
+
+ free_token_error(&err);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ actx->errctx = "failed to parse device authorization";
+ if (!parse_device_authz(actx, &actx->authz))
+ return false; /* error message already set */
+
+ return true;
+}
+
+/*
+ * Queue an Access Token Request:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-4.1.3
+ *
+ * This is the final step. We continually poll the token endpoint to see if the
+ * user has authorized us yet. finish_token_request() will pull either the token
+ * or a (ideally temporary) error status from the provider.
+ */
+static bool
+start_token_request(struct async_ctx *actx, PGconn *conn)
+{
+ const char *token_uri = actx->provider.token_endpoint;
+ const char *device_code = actx->authz.device_code;
+ PQExpBuffer work_buffer = &actx->work_data;
+
+ Assert(conn->oauth_client_id); /* ensured by get_auth_token() */
+ Assert(token_uri); /* ensured by get_discovery_document() */
+ Assert(device_code); /* ensured by run_device_authz() */
+
+ /* Construct our request body. TODO: url-encode */
+ resetPQExpBuffer(work_buffer);
+ appendPQExpBuffer(work_buffer, "client_id=%s", conn->oauth_client_id);
+ appendPQExpBuffer(work_buffer, "&device_code=%s", device_code);
+ appendPQExpBuffer(work_buffer, "&grant_type=%s",
+ OAUTH_GRANT_TYPE_DEVICE_CODE);
+ /* TODO check for broken buffer */
+
+ /* Make our request. */
+ CHECK_SETOPT(actx, CURLOPT_URL, token_uri, return false);
+ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data, return false);
+
+ if (conn->oauth_client_secret)
+ {
+ /*----
+ * Use HTTP Basic auth to send the password. Per RFC 6749, Sec. 2.3.1,
+ *
+ * Including the client credentials in the request-body using the
+ * two parameters is NOT RECOMMENDED and SHOULD be limited to
+ * clients unable to directly utilize the HTTP Basic authentication
+ * scheme (or other password-based HTTP authentication schemes).
+ *
+ * TODO: should we omit client_id from the body in this case?
+ */
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC, return false);
+ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id, return false);
+ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret, return false);
+ }
+ else
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE, return false);
+
+ resetPQExpBuffer(work_buffer);
+ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data, return false);
+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, work_buffer, return false);
+
+ return start_request(actx);
+}
+
+static bool
+finish_token_request(struct async_ctx *actx, struct token *tok)
+{
+ long response_code;
+
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ /*
+ * Per RFC 6749, Section 5, a successful response uses 200 OK. An error
+ * response uses either 400 Bad Request or 401 Unauthorized.
+ *
+ * TODO: there are references online to 403 appearing in the wild...
+ */
+ if (response_code != 200
+ && response_code != 400
+ /* && response_code != 401 TODO */ )
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ if (response_code == 200)
+ {
+ actx->errctx = "failed to parse access token response";
+ if (!parse_access_token(actx, tok))
+ return false; /* error message already set */
+ }
+ else if (!parse_token_error(actx, &tok->err))
+ return false;
+
+ return true;
+}
+
+/*
+ * The top-level, nonblocking entry point for the cURL implementation. This will
+ * be called several times to pump the async engine.
+ *
+ * The architecture is based on PQconnectPoll(). The first half drives the
+ * connection state forward as necessary, returning if we're not ready to
+ * proceed to the next step yet. The second half performs the actual transition
+ * between states.
+ *
+ * You can trace the overall OAuth flow through the second half. It's linear
+ * until we get to the end, where we flip back and forth between
+ * OAUTH_STEP_TOKEN_REQUEST and OAUTH_STEP_WAIT_INTERVAL to regularly ping the
+ * provider.
+ */
+PostgresPollingStatusType
+pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
+{
+ fe_oauth_state *state = conn->sasl_state;
+ struct async_ctx *actx;
+
+ struct token tok = {0};
+
+ /*
+ * XXX This is not safe. cURL has stringent requirements for the thread
+ * context in which you call curl_global_init(), because it's going to try
+ * initializing a bunch of other libraries (OpenSSL, Winsock...). And we
+ * probably need to consider both the TLS backend libcurl is compiled
+ * against and what the user has asked us to do via PQinit[Open]SSL.
+ *
+ * Recent versions of libcurl have improved the thread-safety situation,
+ * but you apparently can't check at compile time whether the
+ * implementation is thread-safe, and there's a chicken-and-egg problem
+ * where you can't check the thread safety until you've initialized cURL,
+ * which you can't do before you've made sure it's thread-safe...
+ *
+ * We know we've already initialized Winsock by this point, so we should
+ * be able to safely skip that bit. But we have to tell cURL to initialize
+ * everything else, because other pieces of our client executable may
+ * already be using cURL for their own purposes. If we initialize libcurl
+ * first, with only a subset of its features, we could break those other
+ * clients nondeterministically, and that would probably be a nightmare to
+ * debug.
+ */
+ curl_global_init(CURL_GLOBAL_ALL
+ & ~CURL_GLOBAL_WIN32); /* we already initialized Winsock */
+
+ if (!state->async_ctx)
+ {
+ /*
+ * Create our asynchronous state, and hook it into the upper-level
+ * OAuth state immediately, so any failures below won't leak the
+ * context allocation.
+ */
+ actx = calloc(1, sizeof(*actx));
+ if (!actx)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return PGRES_POLLING_FAILED;
+ }
+
+ actx->mux = PGINVALID_SOCKET;
+#ifdef HAVE_SYS_EPOLL_H
+ actx->timerfd = -1;
+#endif
+
+ state->async_ctx = actx;
+ state->free_async_ctx = free_curl_async_ctx;
+
+ initPQExpBuffer(&actx->work_data);
+ initPQExpBuffer(&actx->errbuf);
+
+ if (!setup_multiplexer(actx))
+ goto error_return;
+
+ if (!setup_curl_handles(actx))
+ goto error_return;
+ }
+
+ actx = state->async_ctx;
+
+ /* By default, the multiplexer is the altsock. Reassign as desired. */
+ *altsock = actx->mux;
+
+ switch (actx->step)
+ {
+ case OAUTH_STEP_INIT:
+ break;
+
+ case OAUTH_STEP_DISCOVERY:
+ case OAUTH_STEP_DEVICE_AUTHORIZATION:
+ case OAUTH_STEP_TOKEN_REQUEST:
+ {
+ PostgresPollingStatusType status;
+
+ status = drive_request(actx);
+
+ if (status == PGRES_POLLING_FAILED)
+ goto error_return;
+ else if (status != PGRES_POLLING_OK)
+ {
+ /* not done yet */
+ free_token(&tok);
+ return status;
+ }
+ }
+
+ case OAUTH_STEP_WAIT_INTERVAL:
+ /* TODO check that the timer has expired */
+ break;
+ }
+
+ switch (actx->step)
+ {
+ case OAUTH_STEP_INIT:
+ actx->errctx = "failed to fetch OpenID discovery document";
+ if (!start_discovery(actx, conn->oauth_discovery_uri))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_DISCOVERY;
+ break;
+
+ case OAUTH_STEP_DISCOVERY:
+ if (!finish_discovery(actx))
+ goto error_return;
+
+ /* TODO: check issuer */
+
+ actx->errctx = "cannot run OAuth device authorization";
+ if (!check_for_device_flow(actx))
+ goto error_return;
+
+ actx->errctx = "failed to obtain device authorization";
+ if (!start_device_authz(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_DEVICE_AUTHORIZATION;
+ break;
+
+ case OAUTH_STEP_DEVICE_AUTHORIZATION:
+ if (!finish_device_authz(actx))
+ goto error_return;
+
+ actx->errctx = "failed to obtain access token";
+ if (!start_token_request(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_TOKEN_REQUEST;
+ break;
+
+ case OAUTH_STEP_TOKEN_REQUEST:
+ {
+ const struct token_error *err;
+#ifdef HAVE_SYS_EPOLL_H
+ struct itimerspec spec = {0};
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct kevent ev = {0};
+#endif
+
+ if (!finish_token_request(actx, &tok))
+ goto error_return;
+
+ if (!actx->user_prompted)
+ {
+ int res;
+ PQpromptOAuthDevice prompt = {
+ .verification_uri = actx->authz.verification_uri,
+ .user_code = actx->authz.user_code,
+ /* TODO: optional fields */
+ };
+
+ /*
+ * Now that we know the token endpoint isn't broken, give
+ * the user the login instructions.
+ */
+ res = PQauthDataHook(PQAUTHDATA_PROMPT_OAUTH_DEVICE, conn,
+ &prompt);
+
+ if (!res)
+ {
+ fprintf(stderr, "Visit %s and enter the code: %s",
+ prompt.verification_uri, prompt.user_code);
+ }
+ else if (res < 0)
+ {
+ actx_error(actx, "device prompt failed");
+ goto error_return;
+ }
+
+ actx->user_prompted = true;
+ }
+
+ if (tok.access_token)
+ {
+ /* Construct our Bearer token. */
+ resetPQExpBuffer(&actx->work_data);
+ appendPQExpBuffer(&actx->work_data, "Bearer %s",
+ tok.access_token);
+
+ if (PQExpBufferDataBroken(actx->work_data))
+ {
+ actx_error(actx, "out of memory");
+ goto error_return;
+ }
+
+ state->token = strdup(actx->work_data.data);
+ break;
+ }
+
+ /*
+ * authorization_pending and slow_down are the only acceptable
+ * errors; anything else and we bail.
+ */
+ err = &tok.err;
+ if (!err->error || (strcmp(err->error, "authorization_pending")
+ && strcmp(err->error, "slow_down")))
+ {
+ /* TODO handle !err->error */
+ if (err->error_description)
+ appendPQExpBuffer(&actx->errbuf, "%s ",
+ err->error_description);
+
+ appendPQExpBuffer(&actx->errbuf, "(%s)", err->error);
+
+ goto error_return;
+ }
+
+ /*
+ * A slow_down error requires us to permanently increase our
+ * retry interval by five seconds. RFC 8628, Sec. 3.5.
+ */
+ if (strcmp(err->error, "slow_down") == 0)
+ {
+ actx->authz.interval += 5; /* TODO check for overflow? */
+ }
+
+ /*
+ * Wait for the required interval before issuing the next
+ * request.
+ */
+ Assert(actx->authz.interval > 0);
+#ifdef HAVE_SYS_EPOLL_H
+ spec.it_value.tv_sec = actx->authz.interval;
+
+ if (timerfd_settime(actx->timerfd, 0 /* no flags */ , &spec, NULL) < 0)
+ {
+ actx_error(actx, "failed to set timerfd: %m");
+ goto error_return;
+ }
+
+ *altsock = actx->timerfd;
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ /* XXX: I guess this wants to be hidden in a routine */
+ EV_SET(&ev, 1, EVFILT_TIMER, EV_ADD, 0,
+ actx->authz.interval * 1000, 0);
+ if (kevent(actx->mux, &ev, 1, NULL, 0, NULL) < 0)
+ {
+ actx_error(actx, "failed to set kqueue timer: %m");
+ goto error_return;
+ }
+ /* XXX: why did we change the altsock in the epoll version? */
+#endif
+ actx->step = OAUTH_STEP_WAIT_INTERVAL;
+ break;
+ }
+
+ case OAUTH_STEP_WAIT_INTERVAL:
+ actx->errctx = "failed to obtain access token";
+ if (!start_token_request(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_TOKEN_REQUEST;
+ break;
+ }
+
+ free_token(&tok);
+
+ /* If we've stored a token, we're done. Otherwise come back later. */
+ return state->token ? PGRES_POLLING_OK : PGRES_POLLING_READING;
+
+error_return:
+
+ /*
+ * Assemble the three parts of our error: context, body, and detail. See
+ * also the documentation for struct async_ctx.
+ */
+ if (actx->errctx)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext(actx->errctx));
+ appendPQExpBufferStr(&conn->errorMessage, ": ");
+ }
+
+ if (PQExpBufferDataBroken(actx->errbuf))
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("out of memory"));
+ else
+ appendPQExpBufferStr(&conn->errorMessage, actx->errbuf.data);
+
+ if (actx->curl_err[0])
+ {
+ size_t len;
+
+ appendPQExpBuffer(&conn->errorMessage, " (%s)", actx->curl_err);
+
+ /* Sometimes libcurl adds a newline to the error buffer. :( */
+ len = conn->errorMessage.len;
+ if (len >= 2 && conn->errorMessage.data[len - 2] == '\n')
+ {
+ conn->errorMessage.data[len - 2] = ')';
+ conn->errorMessage.data[len - 1] = '\0';
+ conn->errorMessage.len--;
+ }
+ }
+
+ appendPQExpBufferStr(&conn->errorMessage, "\n");
+
+ free_token(&tok);
+ return PGRES_POLLING_FAILED;
+}
diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
new file mode 100644
index 0000000000..66ee8ff076
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth.c
@@ -0,0 +1,659 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth.c
+ * The front-end (client) implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "common/base64.h"
+#include "common/hmac.h"
+#include "common/jsonapi.h"
+#include "common/oauth-common.h"
+#include "fe-auth.h"
+#include "fe-auth-oauth.h"
+#include "mb/pg_wchar.h"
+
+/* The exported OAuth callback mechanism. */
+static void *oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism);
+static SASLStatus oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen);
+static bool oauth_channel_bound(void *opaq);
+static void oauth_free(void *opaq);
+
+const pg_fe_sasl_mech pg_oauth_mech = {
+ oauth_init,
+ oauth_exchange,
+ oauth_channel_bound,
+ oauth_free,
+};
+
+static void *
+oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism)
+{
+ fe_oauth_state *state;
+
+ /*
+ * We only support one SASL mechanism here; anything else is programmer
+ * error.
+ */
+ Assert(sasl_mechanism != NULL);
+ Assert(strcmp(sasl_mechanism, OAUTHBEARER_NAME) == 0);
+
+ state = calloc(1, sizeof(*state));
+ if (!state)
+ return NULL;
+
+ state->state = FE_OAUTH_INIT;
+ state->conn = conn;
+
+ return state;
+}
+
+#define kvsep "\x01"
+
+static char *
+client_initial_response(PGconn *conn, const char *token)
+{
+ static const char *const resp_format = "n,," kvsep "auth=%s" kvsep kvsep;
+
+ PQExpBufferData buf;
+ char *response = NULL;
+
+ if (!token)
+ {
+ /*
+ * Either programmer error, or something went badly wrong during the
+ * asynchronous fetch.
+ *
+ * TODO: users shouldn't see this; what action should they take if
+ * they do?
+ */
+ libpq_append_conn_error(conn, "no OAuth token was set for the connection");
+ return NULL;
+ }
+
+ initPQExpBuffer(&buf);
+ appendPQExpBuffer(&buf, resp_format, token);
+
+ if (!PQExpBufferDataBroken(buf))
+ response = strdup(buf.data);
+
+ termPQExpBuffer(&buf);
+ return response;
+}
+
+#define ERROR_STATUS_FIELD "status"
+#define ERROR_SCOPE_FIELD "scope"
+#define ERROR_OPENID_CONFIGURATION_FIELD "openid-configuration"
+
+struct json_ctx
+{
+ char *errmsg; /* any non-NULL value stops all processing */
+ PQExpBufferData errbuf; /* backing memory for errmsg */
+ int nested; /* nesting level (zero is the top) */
+
+ const char *target_field_name; /* points to a static allocation */
+ char **target_field; /* see below */
+
+ /* target_field, if set, points to one of the following: */
+ char *status;
+ char *scope;
+ char *discovery_uri;
+};
+
+#define oauth_json_has_error(ctx) \
+ (PQExpBufferDataBroken((ctx)->errbuf) || (ctx)->errmsg)
+
+#define oauth_json_set_error(ctx, ...) \
+ do { \
+ appendPQExpBuffer(&(ctx)->errbuf, __VA_ARGS__); \
+ (ctx)->errmsg = (ctx)->errbuf.data; \
+ } while (0)
+
+static JsonParseErrorType
+oauth_json_object_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ ++ctx->nested;
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_end(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_field_start(void *state, char *name, bool isnull)
+{
+ struct json_ctx *ctx = state;
+
+ if (ctx->nested == 1)
+ {
+ if (strcmp(name, ERROR_STATUS_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_STATUS_FIELD;
+ ctx->target_field = &ctx->status;
+ }
+ else if (strcmp(name, ERROR_SCOPE_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_SCOPE_FIELD;
+ ctx->target_field = &ctx->scope;
+ }
+ else if (strcmp(name, ERROR_OPENID_CONFIGURATION_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_OPENID_CONFIGURATION_FIELD;
+ ctx->target_field = &ctx->discovery_uri;
+ }
+ }
+
+ free(name);
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_scalar(void *state, char *token, JsonTokenType type)
+{
+ struct json_ctx *ctx = state;
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ if (type == JSON_TOKEN_STRING)
+ {
+ *ctx->target_field = token;
+
+ ctx->target_field = NULL;
+ ctx->target_field_name = NULL;
+
+ return JSON_SUCCESS; /* don't free the token we're using */
+ }
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ free(token);
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static bool
+handle_oauth_sasl_error(PGconn *conn, char *msg, int msglen)
+{
+ JsonLexContext lex = {0};
+ JsonSemAction sem = {0};
+ JsonParseErrorType err;
+ struct json_ctx ctx = {0};
+ char *errmsg = NULL;
+
+ /* Sanity check. */
+ if (strlen(msg) != msglen)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server's error message contained an embedded NULL, and was discarded"));
+ return false;
+ }
+
+ makeJsonLexContextCstringLen(&lex, msg, msglen, PG_UTF8, true);
+
+ initPQExpBuffer(&ctx.errbuf);
+ sem.semstate = &ctx;
+
+ sem.object_start = oauth_json_object_start;
+ sem.object_end = oauth_json_object_end;
+ sem.object_field_start = oauth_json_object_field_start;
+ sem.array_start = oauth_json_array_start;
+ sem.scalar = oauth_json_scalar;
+
+ err = pg_parse_json(&lex, &sem);
+
+ if (err == JSON_SEM_ACTION_FAILED)
+ {
+ if (PQExpBufferDataBroken(ctx.errbuf))
+ errmsg = libpq_gettext("out of memory");
+ else if (ctx.errmsg)
+ errmsg = ctx.errmsg;
+ else
+ {
+ /*
+ * Developer error: one of the action callbacks didn't call
+ * oauth_json_set_error() before erroring out.
+ */
+ Assert(oauth_json_has_error(&ctx));
+ errmsg = "<unexpected empty error>";
+ }
+ }
+ else if (err != JSON_SUCCESS)
+ errmsg = json_errdetail(err, &lex);
+
+ if (errmsg)
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("failed to parse server's error response: %s"),
+ errmsg);
+
+ /* Don't need the error buffer or the JSON lexer anymore. */
+ termPQExpBuffer(&ctx.errbuf);
+ freeJsonLexContext(&lex);
+
+ if (errmsg)
+ return false;
+
+ /* TODO: what if these override what the user already specified? */
+ if (ctx.discovery_uri)
+ {
+ if (conn->oauth_discovery_uri)
+ free(conn->oauth_discovery_uri);
+
+ conn->oauth_discovery_uri = ctx.discovery_uri;
+ }
+
+ if (ctx.scope)
+ {
+ if (conn->oauth_scope)
+ free(conn->oauth_scope);
+
+ conn->oauth_scope = ctx.scope;
+ }
+ /* TODO: missing error scope should clear any existing connection scope */
+
+ if (!ctx.status)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("server sent error response without a status"));
+ return false;
+ }
+
+ if (strcmp(ctx.status, "invalid_token") == 0)
+ {
+ /*
+ * invalid_token is the only error code we'll automatically retry for,
+ * but only if we have enough information to do so.
+ */
+ if (conn->oauth_discovery_uri)
+ conn->oauth_want_retry = true;
+ }
+ /* TODO: include status in hard failure message */
+
+ return true;
+}
+
+static void
+free_request(PGconn *conn, void *vreq)
+{
+ PQoauthBearerRequest *request = vreq;
+
+ if (request->cleanup)
+ request->cleanup(conn, request);
+
+ free(request);
+}
+
+static PostgresPollingStatusType
+run_user_oauth_flow(PGconn *conn, pgsocket *altsock)
+{
+ fe_oauth_state *state = conn->sasl_state;
+ PQoauthBearerRequest *request = state->async_ctx;
+ PostgresPollingStatusType status;
+
+ if (!request->async)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow provided neither a token nor an async callback");
+ return PGRES_POLLING_FAILED;
+ }
+
+ status = request->async(conn, request, altsock);
+ if (status == PGRES_POLLING_FAILED)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow failed");
+ return status;
+ }
+ else if (status == PGRES_POLLING_OK)
+ {
+ /*
+ * We already have a token, so copy it into the state. (We can't hold
+ * onto the original string, since it may not be safe for us to free()
+ * it.)
+ */
+ PQExpBufferData token;
+
+ if (!request->token)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow did not provide a token");
+ return PGRES_POLLING_FAILED;
+ }
+
+ initPQExpBuffer(&token);
+ appendPQExpBuffer(&token, "Bearer %s", request->token);
+
+ if (PQExpBufferDataBroken(token))
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return PGRES_POLLING_FAILED;
+ }
+
+ state->token = token.data;
+ return PGRES_POLLING_OK;
+ }
+
+ /* TODO: what if no altsock was set? */
+ return status;
+}
+
+static bool
+setup_token_request(PGconn *conn, fe_oauth_state *state)
+{
+ int res;
+ PQoauthBearerRequest request = {
+ .openid_configuration = conn->oauth_discovery_uri,
+ .scope = conn->oauth_scope,
+ };
+
+ Assert(request.openid_configuration);
+
+ /* The client may have overridden the OAuth flow. */
+ res = PQauthDataHook(PQAUTHDATA_OAUTH_BEARER_TOKEN, conn, &request);
+ if (res > 0)
+ {
+ PQoauthBearerRequest *request_copy;
+
+ if (request.token)
+ {
+ /*
+ * We already have a token, so copy it into the state. (We can't
+ * hold onto the original string, since it may not be safe for us
+ * to free() it.)
+ */
+ PQExpBufferData token;
+
+ initPQExpBuffer(&token);
+ appendPQExpBuffer(&token, "Bearer %s", request.token);
+
+ if (PQExpBufferDataBroken(token))
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ goto fail;
+ }
+
+ state->token = token.data;
+
+ /* short-circuit */
+ if (request.cleanup)
+ request.cleanup(conn, &request);
+ return true;
+ }
+
+ request_copy = malloc(sizeof(*request_copy));
+ if (!request_copy)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ goto fail;
+ }
+
+ memcpy(request_copy, &request, sizeof(request));
+
+ conn->async_auth = run_user_oauth_flow;
+ state->async_ctx = request_copy;
+ state->free_async_ctx = free_request;
+ }
+ else if (res < 0)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow failed");
+ goto fail;
+ }
+ else
+ {
+ /* Use our built-in OAuth flow. */
+ conn->async_auth = pg_fe_run_oauth_flow;
+ }
+
+ return true;
+
+fail:
+ if (request.cleanup)
+ request.cleanup(conn, &request);
+ return false;
+}
+
+static bool
+derive_discovery_uri(PGconn *conn)
+{
+ PQExpBufferData discovery_buf;
+
+ if (conn->oauth_discovery_uri || !conn->oauth_issuer)
+ {
+ /*
+ * Either we already have one, or we aren't able to derive one
+ * ourselves. The latter case is not an error condition; we'll just
+ * ask the server to provide one for us.
+ */
+ return true;
+ }
+
+ initPQExpBuffer(&discovery_buf);
+
+ Assert(!conn->oauth_discovery_uri);
+ Assert(conn->oauth_issuer);
+
+ /*
+ * If we don't yet have a discovery URI, but the user gave us an explicit
+ * issuer, use the .well-known discovery URI for that issuer.
+ */
+ appendPQExpBufferStr(&discovery_buf, conn->oauth_issuer);
+ appendPQExpBufferStr(&discovery_buf, "/.well-known/openid-configuration");
+
+ if (PQExpBufferDataBroken(discovery_buf))
+ goto cleanup;
+
+ conn->oauth_discovery_uri = strdup(discovery_buf.data);
+
+cleanup:
+ termPQExpBuffer(&discovery_buf);
+ return (conn->oauth_discovery_uri != NULL);
+}
+
+static SASLStatus
+oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen)
+{
+ fe_oauth_state *state = opaq;
+ PGconn *conn = state->conn;
+
+ *output = NULL;
+ *outputlen = 0;
+
+ switch (state->state)
+ {
+ case FE_OAUTH_INIT:
+ Assert(inputlen == -1);
+
+ if (!derive_discovery_uri(conn))
+ return SASL_FAILED;
+
+ if (conn->oauth_discovery_uri)
+ {
+ if (!conn->oauth_client_id)
+ {
+ /* We can't talk to a server without a client identifier. */
+ libpq_append_conn_error(conn, "no oauth_client_id is set for the connection");
+ return SASL_FAILED;
+ }
+
+ /*
+ * Decide whether we're using a user-provided OAuth flow, or
+ * the one we have built in.
+ */
+ if (!setup_token_request(conn, state))
+ return SASL_FAILED;
+
+ if (state->token)
+ {
+ /*
+ * A really smart user implementation may have already
+ * given us the token (e.g. if there was an unexpired copy
+ * already cached). In that case, we can just fall
+ * through.
+ */
+ }
+ else
+ {
+ /*
+ * Otherwise, we have to hand the connection over to our
+ * OAuth implementation. This involves a number of HTTP
+ * connections and timed waits, so we escape the
+ * synchronous auth processing and tell PQconnectPoll to
+ * transfer control to our async implementation.
+ */
+ Assert(conn->async_auth); /* should have been set
+ * already */
+ state->state = FE_OAUTH_REQUESTING_TOKEN;
+ return SASL_ASYNC;
+ }
+ }
+ else
+ {
+ /*
+ * If we don't have a discovery URI to be able to request a
+ * token, we ask the server for one explicitly with an empty
+ * token. This doesn't require any asynchronous work.
+ */
+ state->token = strdup("");
+ if (!state->token)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return SASL_FAILED;
+ }
+ }
+
+ /* fall through */
+
+ case FE_OAUTH_REQUESTING_TOKEN:
+ /* We should still be in the initial response phase. */
+ Assert(inputlen == -1);
+
+ *output = client_initial_response(conn, state->token);
+ if (!*output)
+ return SASL_FAILED;
+
+ *outputlen = strlen(*output);
+ state->state = FE_OAUTH_BEARER_SENT;
+
+ return SASL_CONTINUE;
+
+ case FE_OAUTH_BEARER_SENT:
+ if (final)
+ {
+ /* TODO: ensure there is no message content here. */
+ return SASL_COMPLETE;
+ }
+
+ /*
+ * Error message sent by the server.
+ */
+ if (!handle_oauth_sasl_error(conn, input, inputlen))
+ return SASL_FAILED;
+
+ /*
+ * Respond with the required dummy message (RFC 7628, sec. 3.2.3).
+ */
+ *output = strdup(kvsep);
+ *outputlen = strlen(*output); /* == 1 */
+
+ state->state = FE_OAUTH_SERVER_ERROR;
+ return SASL_CONTINUE;
+
+ case FE_OAUTH_SERVER_ERROR:
+
+ /*
+ * After an error, the server should send an error response to
+ * fail the SASL handshake, which is handled in higher layers.
+ *
+ * If we get here, the server either sent *another* challenge
+ * which isn't defined in the RFC, or completed the handshake
+ * successfully after telling us it was going to fail. Neither is
+ * acceptable.
+ */
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server sent additional OAuth data after error\n"));
+ return SASL_FAILED;
+
+ default:
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("invalid OAuth exchange state\n"));
+ break;
+ }
+
+ return SASL_FAILED;
+}
+
+static bool
+oauth_channel_bound(void *opaq)
+{
+ /* This mechanism does not support channel binding. */
+ return false;
+}
+
+static void
+oauth_free(void *opaq)
+{
+ fe_oauth_state *state = opaq;
+
+ free(state->token);
+ if (state->async_ctx)
+ state->free_async_ctx(state->conn, state->async_ctx);
+
+ free(state);
+}
diff --git a/src/interfaces/libpq/fe-auth-oauth.h b/src/interfaces/libpq/fe-auth-oauth.h
new file mode 100644
index 0000000000..8d4ea45aa8
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth.h
@@ -0,0 +1,42 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth.h
+ *
+ * Definitions for OAuth authentication implementations
+ *
+ * Portions Copyright (c) 2023, PostgreSQL Global Development Group
+ *
+ * src/interfaces/libpq/fe-auth-oauth.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FE_AUTH_OAUTH_H
+#define FE_AUTH_OAUTH_H
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+
+
+typedef enum
+{
+ FE_OAUTH_INIT,
+ FE_OAUTH_REQUESTING_TOKEN,
+ FE_OAUTH_BEARER_SENT,
+ FE_OAUTH_SERVER_ERROR,
+} fe_oauth_state_enum;
+
+typedef struct
+{
+ fe_oauth_state_enum state;
+
+ PGconn *conn;
+ char *token;
+
+ void *async_ctx;
+ void (*free_async_ctx) (PGconn *conn, void *ctx);
+} fe_oauth_state;
+
+extern PostgresPollingStatusType pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock);
+
+#endif /* FE_AUTH_OAUTH_H */
diff --git a/src/interfaces/libpq/fe-auth-sasl.h b/src/interfaces/libpq/fe-auth-sasl.h
index 4eecf53a15..83bdab1f40 100644
--- a/src/interfaces/libpq/fe-auth-sasl.h
+++ b/src/interfaces/libpq/fe-auth-sasl.h
@@ -30,6 +30,7 @@ typedef enum
SASL_COMPLETE = 0,
SASL_FAILED,
SASL_CONTINUE,
+ SASL_ASYNC,
} SASLStatus;
/*
@@ -77,6 +78,8 @@ typedef struct pg_fe_sasl_mech
*
* state: The opaque mechanism state returned by init()
*
+ * final: true if the server has sent a final exchange outcome
+ *
* input: The challenge data sent by the server, or NULL when
* generating a client-first initial response (that is, when
* the server expects the client to send a message to start
@@ -101,12 +104,17 @@ typedef struct pg_fe_sasl_mech
*
* SASL_CONTINUE: The output buffer is filled with a client response.
* Additional server challenge is expected
+ * SASL_ASYNC: Some asynchronous processing external to the
+ * connection needs to be done before a response can be
+ * generated. The mechanism is responsible for setting up
+ * conn->async_auth appropriately before returning.
* SASL_COMPLETE: The SASL exchange has completed successfully.
* SASL_FAILED: The exchange has failed and the connection should be
* dropped.
*--------
*/
- SASLStatus (*exchange) (void *state, char *input, int inputlen,
+ SASLStatus (*exchange) (void *state, bool final,
+ char *input, int inputlen,
char **output, int *outputlen);
/*--------
diff --git a/src/interfaces/libpq/fe-auth-scram.c b/src/interfaces/libpq/fe-auth-scram.c
index 0bb820e0d9..da168eb2f5 100644
--- a/src/interfaces/libpq/fe-auth-scram.c
+++ b/src/interfaces/libpq/fe-auth-scram.c
@@ -24,7 +24,8 @@
/* The exported SCRAM callback mechanism. */
static void *scram_init(PGconn *conn, const char *password,
const char *sasl_mechanism);
-static SASLStatus scram_exchange(void *opaq, char *input, int inputlen,
+static SASLStatus scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen);
static bool scram_channel_bound(void *opaq);
static void scram_free(void *opaq);
@@ -202,7 +203,8 @@ scram_free(void *opaq)
* Exchange a SCRAM message with backend.
*/
static SASLStatus
-scram_exchange(void *opaq, char *input, int inputlen,
+scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen)
{
fe_scram_state *state = (fe_scram_state *) opaq;
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index 81ec08485d..9cd5c8cfb1 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -39,6 +39,7 @@
#endif
#include "common/md5.h"
+#include "common/oauth-common.h"
#include "common/scram-common.h"
#include "fe-auth.h"
#include "fe-auth-sasl.h"
@@ -419,7 +420,7 @@ pg_SSPI_startup(PGconn *conn, int use_negotiate, int payloadlen)
* Initialize SASL authentication exchange.
*/
static int
-pg_SASL_init(PGconn *conn, int payloadlen)
+pg_SASL_init(PGconn *conn, int payloadlen, bool *async)
{
char *initialresponse = NULL;
int initialresponselen;
@@ -437,7 +438,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
goto error;
}
- if (conn->sasl_state)
+ if (conn->sasl_state && !conn->async_auth)
{
libpq_append_conn_error(conn, "duplicate SASL authentication request");
goto error;
@@ -524,6 +525,15 @@ pg_SASL_init(PGconn *conn, int payloadlen)
conn->sasl = &pg_scram_mech;
conn->password_needed = true;
}
+#ifdef USE_OAUTH
+ else if (strcmp(mechanism_buf.data, OAUTHBEARER_NAME) == 0 &&
+ !selected_mechanism)
+ {
+ selected_mechanism = OAUTHBEARER_NAME;
+ conn->sasl = &pg_oauth_mech;
+ conn->password_needed = false;
+ }
+#endif
}
if (!selected_mechanism)
@@ -563,26 +573,48 @@ pg_SASL_init(PGconn *conn, int payloadlen)
Assert(conn->sasl);
- /*
- * Initialize the SASL state information with all the information gathered
- * during the initial exchange.
- *
- * Note: Only tls-unique is supported for the moment.
- */
- conn->sasl_state = conn->sasl->init(conn,
- password,
- selected_mechanism);
if (!conn->sasl_state)
- goto oom_error;
+ {
+ /*
+ * Initialize the SASL state information with all the information
+ * gathered during the initial exchange.
+ *
+ * Note: Only tls-unique is supported for the moment.
+ */
+ conn->sasl_state = conn->sasl->init(conn,
+ password,
+ selected_mechanism);
+ if (!conn->sasl_state)
+ goto oom_error;
+ }
+ else
+ {
+ /*
+ * This is only possible if we're returning from an async loop.
+ * Disconnect it now.
+ */
+ Assert(conn->async_auth);
+ conn->async_auth = NULL;
+ }
/* Get the mechanism-specific Initial Client Response, if any */
- status = conn->sasl->exchange(conn->sasl_state,
+ status = conn->sasl->exchange(conn->sasl_state, false,
NULL, -1,
&initialresponse, &initialresponselen);
if (status == SASL_FAILED)
goto error;
+ if (status == SASL_ASYNC)
+ {
+ /*
+ * The mechanism should have set up the necessary callbacks; all we
+ * need to do is signal the caller.
+ */
+ *async = true;
+ return STATUS_OK;
+ }
+
/*
* Build a SASLInitialResponse message, and send it.
*/
@@ -625,7 +657,7 @@ oom_error:
* the protocol.
*/
static int
-pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
+pg_SASL_continue(PGconn *conn, int payloadlen, bool final, bool *async)
{
char *output;
int outputlen;
@@ -650,11 +682,21 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
/* For safety and convenience, ensure the buffer is NULL-terminated. */
challenge[payloadlen] = '\0';
- status = conn->sasl->exchange(conn->sasl_state,
+ status = conn->sasl->exchange(conn->sasl_state, final,
challenge, payloadlen,
&output, &outputlen);
free(challenge); /* don't need the input anymore */
+ if (status == SASL_ASYNC)
+ {
+ /*
+ * The mechanism should have set up the necessary callbacks; all we
+ * need to do is signal the caller.
+ */
+ *async = true;
+ return STATUS_OK;
+ }
+
if (final && status == SASL_CONTINUE)
{
if (outputlen != 0)
@@ -955,12 +997,18 @@ check_expected_areq(AuthRequest areq, PGconn *conn)
* it. We are responsible for reading any remaining extra data, specific
* to the authentication method. 'payloadlen' is the remaining length in
* the message.
+ *
+ * If *async is set to true on return, the client doesn't yet have enough
+ * information to respond, and the caller must temporarily switch to
+ * conn->async_auth() to continue driving the exchange.
*/
int
-pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
+pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn, bool *async)
{
int oldmsglen;
+ *async = false;
+
if (!check_expected_areq(areq, conn))
return STATUS_ERROR;
@@ -1118,7 +1166,7 @@ pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
* The request contains the name (as assigned by IANA) of the
* authentication mechanism.
*/
- if (pg_SASL_init(conn, payloadlen) != STATUS_OK)
+ if (pg_SASL_init(conn, payloadlen, async) != STATUS_OK)
{
/* pg_SASL_init already set the error message */
return STATUS_ERROR;
@@ -1135,7 +1183,8 @@ pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
}
oldmsglen = conn->errorMessage.len;
if (pg_SASL_continue(conn, payloadlen,
- (areq == AUTH_REQ_SASL_FIN)) != STATUS_OK)
+ (areq == AUTH_REQ_SASL_FIN),
+ async) != STATUS_OK)
{
/* Use this message if pg_SASL_continue didn't supply one */
if (conn->errorMessage.len == oldmsglen)
@@ -1451,3 +1500,23 @@ PQchangePassword(PGconn *conn, const char *user, const char *passwd)
}
}
}
+
+PQauthDataHook_type PQauthDataHook = PQdefaultAuthDataHook;
+
+PQauthDataHook_type
+PQgetAuthDataHook(void)
+{
+ return PQauthDataHook;
+}
+
+void
+PQsetAuthDataHook(PQauthDataHook_type hook)
+{
+ PQauthDataHook = hook ? hook : PQdefaultAuthDataHook;
+}
+
+int
+PQdefaultAuthDataHook(PGAuthData type, PGconn *conn, void *data)
+{
+ return 0; /* handle nothing */
+}
diff --git a/src/interfaces/libpq/fe-auth.h b/src/interfaces/libpq/fe-auth.h
index a18c508688..3e2bc1333f 100644
--- a/src/interfaces/libpq/fe-auth.h
+++ b/src/interfaces/libpq/fe-auth.h
@@ -18,8 +18,12 @@
#include "libpq-int.h"
+extern PQauthDataHook_type PQauthDataHook;
+
+
/* Prototypes for functions in fe-auth.c */
-extern int pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn);
+extern int pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn,
+ bool *async);
extern char *pg_fe_getusername(uid_t user_id, PQExpBuffer errorMessage);
extern char *pg_fe_getauthname(PQExpBuffer errorMessage);
@@ -29,4 +33,7 @@ extern char *pg_fe_scram_build_secret(const char *password,
int iterations,
const char **errstr);
+/* Mechanisms in fe-auth-oauth.c */
+extern const pg_fe_sasl_mech pg_oauth_mech;
+
#endif /* FE_AUTH_H */
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index d4e10a0c4f..15ceb73d01 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -359,6 +359,23 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"Load-Balance-Hosts", "", 8, /* sizeof("disable") = 8 */
offsetof(struct pg_conn, load_balance_hosts)},
+ /* OAuth v2 */
+ {"oauth_issuer", NULL, NULL, NULL,
+ "OAuth-Issuer", "", 40,
+ offsetof(struct pg_conn, oauth_issuer)},
+
+ {"oauth_client_id", NULL, NULL, NULL,
+ "OAuth-Client-ID", "", 40,
+ offsetof(struct pg_conn, oauth_client_id)},
+
+ {"oauth_client_secret", NULL, NULL, NULL,
+ "OAuth-Client-Secret", "", 40,
+ offsetof(struct pg_conn, oauth_client_secret)},
+
+ {"oauth_scope", NULL, NULL, NULL,
+ "OAuth-Scope", "", 15,
+ offsetof(struct pg_conn, oauth_scope)},
+
/* Terminating entry --- MUST BE LAST */
{NULL, NULL, NULL, NULL,
NULL, NULL, 0}
@@ -618,6 +635,7 @@ pqDropServerData(PGconn *conn)
conn->write_err_msg = NULL;
conn->be_pid = 0;
conn->be_key = 0;
+ /* conn->oauth_want_retry = false; TODO */
}
@@ -2536,6 +2554,7 @@ PQconnectPoll(PGconn *conn)
case CONNECTION_NEEDED:
case CONNECTION_GSS_STARTUP:
case CONNECTION_CHECK_TARGET:
+ case CONNECTION_AUTHENTICATING:
break;
default:
@@ -3517,6 +3536,7 @@ keep_going: /* We will come back to here until there is
int avail;
AuthRequest areq;
int res;
+ bool async;
/*
* Scan the message from current point (note that if we find
@@ -3672,6 +3692,16 @@ keep_going: /* We will come back to here until there is
/* Check to see if we should mention pgpassfile */
pgpassfileWarning(conn);
+#ifdef USE_OAUTH
+ if (conn->sasl == &pg_oauth_mech
+ && conn->oauth_want_retry)
+ {
+ /* TODO: only allow retry once */
+ need_new_connection = true;
+ goto keep_going;
+ }
+#endif
+
#ifdef ENABLE_GSS
/*
@@ -3753,7 +3783,17 @@ keep_going: /* We will come back to here until there is
* Note that conn->pghost must be non-NULL if we are going to
* avoid the Kerberos code doing a hostname look-up.
*/
- res = pg_fe_sendauth(areq, msgLength, conn);
+ res = pg_fe_sendauth(areq, msgLength, conn, &async);
+
+ if (async && (res == STATUS_OK))
+ {
+ /*
+ * We'll come back later once we're ready to respond.
+ * Don't consume the request yet.
+ */
+ conn->status = CONNECTION_AUTHENTICATING;
+ goto keep_going;
+ }
/* OK, we have processed the message; mark data consumed */
conn->inStart = conn->inCursor;
@@ -3786,6 +3826,41 @@ keep_going: /* We will come back to here until there is
goto keep_going;
}
+ case CONNECTION_AUTHENTICATING:
+ {
+ PostgresPollingStatusType status;
+ pgsocket altsock = PGINVALID_SOCKET;
+
+ if (!conn->async_auth)
+ {
+ /* programmer error; should not happen */
+ libpq_append_conn_error(conn, "async authentication has no handler");
+ goto error_return;
+ }
+
+ status = conn->async_auth(conn, &altsock);
+
+ if (status == PGRES_POLLING_FAILED)
+ goto error_return;
+
+ if (status == PGRES_POLLING_OK)
+ {
+ /*
+ * Reenter the authentication exchange with the server. We
+ * didn't consume the message that started external
+ * authentication, so it'll be reprocessed as if we just
+ * received it.
+ */
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ conn->altsock = PGINVALID_SOCKET; /* TODO: what frees
+ * this? */
+ goto keep_going;
+ }
+
+ conn->altsock = altsock;
+ return status;
+ }
+
case CONNECTION_AUTH_OK:
{
/*
@@ -4285,6 +4360,7 @@ pqMakeEmptyPGconn(void)
conn->verbosity = PQERRORS_DEFAULT;
conn->show_context = PQSHOW_CONTEXT_ERRORS;
conn->sock = PGINVALID_SOCKET;
+ conn->altsock = PGINVALID_SOCKET;
conn->Pfdebug = NULL;
/*
@@ -4400,6 +4476,11 @@ freePGconn(PGconn *conn)
free(conn->rowBuf);
free(conn->target_session_attrs);
free(conn->load_balance_hosts);
+ free(conn->oauth_issuer);
+ free(conn->oauth_discovery_uri);
+ free(conn->oauth_client_id);
+ free(conn->oauth_client_secret);
+ free(conn->oauth_scope);
termPQExpBuffer(&conn->errorMessage);
termPQExpBuffer(&conn->workBuffer);
@@ -6868,6 +6949,8 @@ PQsocket(const PGconn *conn)
{
if (!conn)
return -1;
+ if (conn->altsock != PGINVALID_SOCKET)
+ return conn->altsock;
return (conn->sock != PGINVALID_SOCKET) ? conn->sock : -1;
}
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index f2fc78a481..663b1c1acf 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -1039,10 +1039,13 @@ static int
pqSocketCheck(PGconn *conn, int forRead, int forWrite, time_t end_time)
{
int result;
+ pgsocket sock;
if (!conn)
return -1;
- if (conn->sock == PGINVALID_SOCKET)
+
+ sock = (conn->altsock != PGINVALID_SOCKET) ? conn->altsock : conn->sock;
+ if (sock == PGINVALID_SOCKET)
{
libpq_append_conn_error(conn, "invalid socket");
return -1;
@@ -1059,7 +1062,7 @@ pqSocketCheck(PGconn *conn, int forRead, int forWrite, time_t end_time)
/* We will retry as long as we get EINTR */
do
- result = pqSocketPoll(conn->sock, forRead, forWrite, end_time);
+ result = pqSocketPoll(sock, forRead, forWrite, end_time);
while (result < 0 && SOCK_ERRNO == EINTR);
if (result < 0)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index defc415fa3..d095351c66 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -38,6 +38,8 @@ extern "C"
#define LIBPQ_HAS_TRACE_FLAGS 1
/* Indicates that PQsslAttribute(NULL, "library") is useful */
#define LIBPQ_HAS_SSL_LIBRARY_DETECTION 1
+/* Indicates presence of the PQAUTHDATA_PROMPT_OAUTH_DEVICE authdata hook */
+#define LIBPQ_HAS_PROMPT_OAUTH_DEVICE 1
/*
* Option flags for PQcopyResult
@@ -78,7 +80,9 @@ typedef enum
CONNECTION_CONSUME, /* Consuming any extra messages. */
CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
CONNECTION_CHECK_TARGET, /* Checking target server properties. */
- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_AUTHENTICATING, /* Authentication is in progress with some
+ * external system. */
} ConnStatusType;
typedef enum
@@ -160,6 +164,13 @@ typedef enum
PQ_PIPELINE_ABORTED
} PGpipelineStatus;
+typedef enum
+{
+ PQAUTHDATA_PROMPT_OAUTH_DEVICE, /* user must visit a device-authorization
+ * URL */
+ PQAUTHDATA_OAUTH_BEARER_TOKEN, /* server requests an OAuth Bearer token */
+} PGAuthData;
+
/* PGconn encapsulates a connection to the backend.
* The contents of this struct are not supposed to be known to applications.
*/
@@ -658,10 +669,74 @@ extern int PQenv2encoding(void);
/* === in fe-auth.c === */
+typedef struct _PQpromptOAuthDevice
+{
+ const char *verification_uri; /* verification URI to visit */
+ const char *user_code; /* user code to enter */
+} PQpromptOAuthDevice;
+
+typedef struct _PQoauthBearerRequest
+{
+ /* Hook inputs (constant across all calls) */
+ const char *const openid_configuration; /* OIDC discovery URI */
+ const char *const scope; /* required scope(s), or NULL */
+
+ /* Hook outputs */
+
+ /*---------
+ * Callback implementing a custom asynchronous OAuth flow.
+ *
+ * The callback may return
+ * - PGRES_POLLING_READING/WRITING, to indicate that a file descriptor
+ * has been stored in *altsock and libpq should wait until it is
+ * readable or writable before calling back;
+ * - PGRES_POLLING_OK, to indicate that the flow is complete and
+ * request->token has been set; or
+ * - PGRES_POLLING_FAILED, to indicate that token retrieval has failed.
+ *
+ * This callback is optional. If the token can be obtained without
+ * blocking during the original call to the PQAUTHDATA_OAUTH_BEARER_TOKEN
+ * hook, it may be returned directly, but one of request->async or
+ * request->token must be set by the hook.
+ */
+ PostgresPollingStatusType (*async) (PGconn *conn,
+ struct _PQoauthBearerRequest *request,
+ int *altsock);
+
+ /*
+ * Callback to clean up custom allocations. A hook implementation may use
+ * this to free request->token and any resources in request->user.
+ *
+ * This is technically optional, but highly recommended, because there is
+ * no other indication as to when it is safe to free the token.
+ */
+ void (*cleanup) (PGconn *conn, struct _PQoauthBearerRequest *request);
+
+ /*
+ * The hook should set this to the Bearer token contents for the
+ * connection, once the flow is completed. The token contents must remain
+ * available to libpq until the hook's cleanup callback is called.
+ */
+ char *token;
+
+ /*
+ * Hook-defined data. libpq will not modify this pointer across calls to
+ * the async callback, so it can be used to keep track of
+ * application-specific state. Resources allocated here should be freed by
+ * the cleanup callback.
+ */
+ void *user;
+} PQoauthBearerRequest;
+
extern char *PQencryptPassword(const char *passwd, const char *user);
extern char *PQencryptPasswordConn(PGconn *conn, const char *passwd, const char *user, const char *algorithm);
extern PGresult *PQchangePassword(PGconn *conn, const char *user, const char *passwd);
+typedef int (*PQauthDataHook_type) (PGAuthData type, PGconn *conn, void *data);
+extern void PQsetAuthDataHook(PQauthDataHook_type hook);
+extern PQauthDataHook_type PQgetAuthDataHook(void);
+extern int PQdefaultAuthDataHook(PGAuthData type, PGconn *conn, void *data);
+
/* === in encnames.c === */
extern int pg_char_to_encoding(const char *name);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 82c18f870d..cf26c693e3 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -351,6 +351,8 @@ typedef struct pg_conn_host
* found in password file. */
} pg_conn_host;
+typedef PostgresPollingStatusType (*AsyncAuthFunc) (PGconn *conn, pgsocket *altsock);
+
/*
* PGconn stores all the state data associated with a single connection
* to a backend.
@@ -409,6 +411,15 @@ struct pg_conn
char *require_auth; /* name of the expected auth method */
char *load_balance_hosts; /* load balance over hosts */
+ /* OAuth v2 */
+ char *oauth_issuer; /* token issuer URL */
+ char *oauth_discovery_uri; /* URI of the issuer's discovery
+ * document */
+ char *oauth_client_id; /* client identifier */
+ char *oauth_client_secret; /* client secret */
+ char *oauth_scope; /* access token scope */
+ bool oauth_want_retry; /* should we retry on failure? */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -477,6 +488,9 @@ struct pg_conn
bool client_finished_auth; /* have we finished our half of the
* authentication exchange? */
+ AsyncAuthFunc async_auth; /* callback for external async authentication */
+ pgsocket altsock; /* alternative socket for client to poll */
+
/* Transient state needed while establishing connection */
PGTargetServerType target_server_type; /* desired session properties */
diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build
index be6fadaea2..0d4b7ac17d 100644
--- a/src/interfaces/libpq/meson.build
+++ b/src/interfaces/libpq/meson.build
@@ -40,6 +40,13 @@ if gssapi.found()
)
endif
+if oauth.found()
+ libpq_sources += files('fe-auth-oauth.c')
+ if oauth_library == 'curl'
+ libpq_sources += files('fe-auth-oauth-curl.c')
+ endif
+endif
+
export_file = custom_target('libpq.exports',
kwargs: gen_export_kwargs,
)
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index b0f4178b3d..f803c1200b 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -231,6 +231,7 @@ pgxs_deps = {
'llvm': llvm,
'lz4': lz4,
'nls': libintl,
+ 'oauth': oauth,
'pam': pam,
'perl': perl_dep,
'python': python3_dep,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 86d68f7a7a..291c80e3ff 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -153,6 +153,7 @@ ArrayMetaState
ArraySubWorkspace
ArrayToken
ArrayType
+AsyncAuthFunc
AsyncQueueControl
AsyncQueueEntry
AsyncRequest
@@ -353,6 +354,8 @@ CState
CTECycleClause
CTEMaterialize
CTESearchClause
+CURL
+CURLM
CV
CachedExpression
CachedPlan
@@ -1653,6 +1656,7 @@ NumericDigit
NumericSortSupport
NumericSumAccum
NumericVar
+OAuthStep
OM_uint32
OP
OSAPerGroupState
@@ -1718,6 +1722,7 @@ PFN
PGAlignedBlock
PGAlignedXLogBlock
PGAsyncStatusType
+PGAuthData
PGCALL2
PGChecksummablePage
PGContextVisibility
@@ -1877,11 +1882,14 @@ PQArgBlock
PQEnvironmentOption
PQExpBuffer
PQExpBufferData
+PQauthDataHook_type
PQcommMethods
PQconninfoOption
PQnoticeProcessor
PQnoticeReceiver
+PQoauthBearerRequest
PQprintOpt
+PQpromptOAuthDevice
PQsslKeyPassHook_OpenSSL_type
PREDICATELOCK
PREDICATELOCKTAG
@@ -3343,6 +3351,8 @@ explain_get_index_name_hook_type
f_smgr
fasthash_state
fd_set
+fe_oauth_state
+fe_oauth_state_enum
fe_scram_state
fe_scram_state_enum
fetch_range_request
--
2.34.1
v20-0006-Introduce-OAuth-validator-libraries.patch (application/octet-stream, 34.0K) download
From fdbad1976a78d179b104138c31ac106e20338b0f Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Wed, 21 Feb 2024 17:04:26 +0100
Subject: [PATCH v20 6/9] Introduce OAuth validator libraries
This replaces the serverside validation code with an module API
for loading in extensions for validating bearer tokens. A lot of
code is left to be written.
Co-authored-by: Jacob Champion <jacob.champion@enterprisedb.com>
---
src/backend/libpq/auth-oauth.c | 431 +++++-------------
src/backend/utils/misc/guc_tables.c | 6 +-
src/bin/pg_combinebackup/Makefile | 2 +-
src/common/Makefile | 2 +-
src/include/libpq/oauth.h | 29 +-
src/test/modules/meson.build | 1 +
src/test/modules/oauth_validator/.gitignore | 4 +
src/test/modules/oauth_validator/Makefile | 19 +
.../oauth_validator/expected/validator.out | 6 +
src/test/modules/oauth_validator/meson.build | 33 ++
.../modules/oauth_validator/sql/validator.sql | 1 +
.../modules/oauth_validator/t/001_server.pl | 78 ++++
src/test/modules/oauth_validator/validator.c | 82 ++++
src/test/perl/PostgreSQL/Test/Cluster.pm | 14 +-
src/test/perl/PostgreSQL/Test/OAuthServer.pm | 183 ++++++++
src/tools/pgindent/typedefs.list | 2 +
16 files changed, 561 insertions(+), 332 deletions(-)
create mode 100644 src/test/modules/oauth_validator/.gitignore
create mode 100644 src/test/modules/oauth_validator/Makefile
create mode 100644 src/test/modules/oauth_validator/expected/validator.out
create mode 100644 src/test/modules/oauth_validator/meson.build
create mode 100644 src/test/modules/oauth_validator/sql/validator.sql
create mode 100644 src/test/modules/oauth_validator/t/001_server.pl
create mode 100644 src/test/modules/oauth_validator/validator.c
create mode 100644 src/test/perl/PostgreSQL/Test/OAuthServer.pm
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
index 16596c089a..024f304e4d 100644
--- a/src/backend/libpq/auth-oauth.c
+++ b/src/backend/libpq/auth-oauth.c
@@ -6,7 +6,7 @@
* See the following RFC for more details:
* - RFC 7628: https://tools.ietf.org/html/rfc7628
*
- * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* src/backend/libpq/auth-oauth.c
@@ -19,22 +19,30 @@
#include <fcntl.h>
#include "common/oauth-common.h"
+#include "fmgr.h"
#include "lib/stringinfo.h"
#include "libpq/auth.h"
#include "libpq/hba.h"
#include "libpq/oauth.h"
#include "libpq/sasl.h"
#include "storage/fd.h"
+#include "storage/ipc.h"
#include "utils/json.h"
/* GUC */
-char *oauth_validator_command;
+char *OAuthValidatorLibrary = "";
static void oauth_get_mechanisms(Port *port, StringInfo buf);
static void *oauth_init(Port *port, const char *selected_mech, const char *shadow_pass);
static int oauth_exchange(void *opaq, const char *input, int inputlen,
char **output, int *outputlen, const char **logdetail);
+static void load_validator_library(void);
+static void shutdown_validator_library(int code, Datum arg);
+
+static ValidatorModuleState *validator_module_state;
+static const OAuthValidatorCallbacks *ValidatorCallbacks;
+
/* Mechanism declaration */
const pg_be_sasl_mech pg_be_oauth_mech = {
oauth_get_mechanisms,
@@ -63,11 +71,7 @@ struct oauth_ctx
static char *sanitize_char(char c);
static char *parse_kvpairs_for_auth(char **input);
static void generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen);
-static bool validate(Port *port, const char *auth, const char **logdetail);
-static bool run_validator_command(Port *port, const char *token);
-static bool check_exit(FILE **fh, const char *command);
-static bool set_cloexec(int fd);
-static bool username_ok_for_shell(const char *username);
+static bool validate(Port *port, const char *auth);
#define KVSEP 0x01
#define AUTH_KEY "auth"
@@ -100,6 +104,8 @@ oauth_init(Port *port, const char *selected_mech, const char *shadow_pass)
ctx->issuer = port->hba->oauth_issuer;
ctx->scope = port->hba->oauth_scope;
+ load_validator_library();
+
return ctx;
}
@@ -250,7 +256,7 @@ oauth_exchange(void *opaq, const char *input, int inputlen,
errmsg("malformed OAUTHBEARER message"),
errdetail("Message contains additional data after the final terminator.")));
- if (!validate(ctx->port, auth, logdetail))
+ if (!validate(ctx->port, auth))
{
generate_error_response(ctx, output, outputlen);
@@ -489,70 +495,73 @@ generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
*outputlen = buf.len;
}
-static bool
-validate(Port *port, const char *auth, const char **logdetail)
+/*-----
+ * Validates the provided Authorization header and returns the token from
+ * within it. NULL is returned on validation failure.
+ *
+ * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
+ * 2.1:
+ *
+ * b64token = 1*( ALPHA / DIGIT /
+ * "-" / "." / "_" / "~" / "+" / "/" ) *"="
+ * credentials = "Bearer" 1*SP b64token
+ *
+ * The "credentials" construction is what we receive in our auth value.
+ *
+ * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
+ * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
+ * compared case-insensitively. (This is not mentioned in RFC 6750, but
+ * it's pointed out in RFC 7628 Sec. 4.)
+ *
+ * Invalid formats are technically a protocol violation, but we shouldn't
+ * reflect any information about the sensitive Bearer token back to the
+ * client; log at COMMERROR instead.
+ *
+ * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
+ */
+static const char *
+validate_token_format(const char *header)
{
- static const char *const b64_set =
+ size_t span;
+ const char *token;
+ static const char *const b64token_allowed_set =
"abcdefghijklmnopqrstuvwxyz"
"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
"0123456789-._~+/";
- const char *token;
- size_t span;
- int ret;
+ /* If the token is empty or simply too short to be correct */
+ if (!header || strlen(header) <= 7)
+ {
+ ereport(COMMERROR,
+ (errmsg("malformed OAuth bearer token 1")));
+ return NULL;
+ }
- /* TODO: handle logdetail when the test framework can check it */
-
- /*-----
- * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
- * 2.1:
- *
- * b64token = 1*( ALPHA / DIGIT /
- * "-" / "." / "_" / "~" / "+" / "/" ) *"="
- * credentials = "Bearer" 1*SP b64token
- *
- * The "credentials" construction is what we receive in our auth value.
- *
- * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
- * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
- * compared case-insensitively. (This is not mentioned in RFC 6750, but
- * it's pointed out in RFC 7628 Sec. 4.)
- *
- * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
- */
- if (pg_strncasecmp(auth, BEARER_SCHEME, strlen(BEARER_SCHEME)))
- return false;
+ if (pg_strncasecmp(header, BEARER_SCHEME, strlen(BEARER_SCHEME)))
+ return NULL;
/* Pull the bearer token out of the auth value. */
- token = auth + strlen(BEARER_SCHEME);
+ token = header + strlen(BEARER_SCHEME);
/* Swallow any additional spaces. */
while (*token == ' ')
token++;
- /*
- * Before invoking the validator command, sanity-check the token format to
- * avoid any injection attacks later in the chain. Invalid formats are
- * technically a protocol violation, but don't reflect any information
- * about the sensitive Bearer token back to the client; log at COMMERROR
- * instead.
- */
-
/* Tokens must not be empty. */
if (!*token)
{
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("malformed OAUTHBEARER message"),
+ errmsg("malformed OAuth bearer token 2"),
errdetail("Bearer token is empty.")));
- return false;
+ return NULL;
}
/*
* Make sure the token contains only allowed characters. Tokens may end
* with any number of '=' characters.
*/
- span = strspn(token, b64_set);
+ span = strspn(token, b64token_allowed_set);
while (token[span] == '=')
span++;
@@ -565,15 +574,35 @@ validate(Port *port, const char *auth, const char **logdetail)
*/
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("malformed OAUTHBEARER message"),
+ errmsg("malformed OAuth bearer token 3"),
errdetail("Bearer token is not in the correct format.")));
- return false;
+ return NULL;
}
- /* Have the validator check the token. */
- if (!run_validator_command(port, token))
+ return token;
+}
+
+static bool
+validate(Port *port, const char *auth)
+{
+ int map_status;
+ ValidatorModuleResult *ret;
+ const char *token;
+
+ /* Ensure that we have a correct token to validate */
+ if (!(token = validate_token_format(auth)))
+ return false;
+
+ /* Call the validation function from the validator module */
+ ret = ValidatorCallbacks->validate_cb(validator_module_state,
+ token, port->user_name);
+
+ if (!ret->authorized)
return false;
+ if (ret->authn_id)
+ set_authn_id(port, ret->authn_id);
+
if (port->hba->oauth_skip_usermap)
{
/*
@@ -586,7 +615,7 @@ validate(Port *port, const char *auth, const char **logdetail)
}
/* Make sure the validator authenticated the user. */
- if (!MyClientConnectionInfo.authn_id)
+ if (ret->authn_id == NULL || ret->authn_id[0] == '\0')
{
/* TODO: use logdetail; reduce message duplication */
ereport(LOG,
@@ -596,288 +625,42 @@ validate(Port *port, const char *auth, const char **logdetail)
}
/* Finally, check the user map. */
- ret = check_usermap(port->hba->usermap, port->user_name,
- MyClientConnectionInfo.authn_id, false);
- return (ret == STATUS_OK);
-}
-
-static bool
-run_validator_command(Port *port, const char *token)
-{
- bool success = false;
- int rc;
- int pipefd[2];
- int rfd = -1;
- int wfd = -1;
-
- StringInfoData command = {0};
- char *p;
- FILE *fh = NULL;
-
- ssize_t written;
- char *line = NULL;
- size_t size = 0;
- ssize_t len;
-
- Assert(oauth_validator_command);
-
- if (!oauth_validator_command[0])
- {
- ereport(COMMERROR,
- (errmsg("oauth_validator_command is not set"),
- errhint("To allow OAuth authenticated connections, set "
- "oauth_validator_command in postgresql.conf.")));
- return false;
- }
-
- /*------
- * Since popen() is unidirectional, open up a pipe for the other
- * direction. Use CLOEXEC to ensure that our write end doesn't
- * accidentally get copied into child processes, which would prevent us
- * from closing it cleanly.
- *
- * XXX this is ugly. We should just read from the child process's stdout,
- * but that's a lot more code.
- * XXX by bypassing the popen API, we open the potential of process
- * deadlock. Clearly document child process requirements (i.e. the child
- * MUST read all data off of the pipe before writing anything).
- * TODO: port to Windows using _pipe().
- */
- rc = pipe(pipefd);
- if (rc < 0)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not create child pipe: %m")));
- return false;
- }
-
- rfd = pipefd[0];
- wfd = pipefd[1];
-
- if (!set_cloexec(wfd))
- {
- /* error message was already logged */
- goto cleanup;
- }
-
- /*----------
- * Construct the command, substituting any recognized %-specifiers:
- *
- * %f: the file descriptor of the input pipe
- * %r: the role that the client wants to assume (port->user_name)
- * %%: a literal '%'
- */
- initStringInfo(&command);
-
- for (p = oauth_validator_command; *p; p++)
- {
- if (p[0] == '%')
- {
- switch (p[1])
- {
- case 'f':
- appendStringInfo(&command, "%d", rfd);
- p++;
- break;
- case 'r':
-
- /*
- * TODO: decide how this string should be escaped. The
- * role is controlled by the client, so if we don't escape
- * it, command injections are inevitable.
- *
- * This is probably an indication that the role name needs
- * to be communicated to the validator process in some
- * other way. For this proof of concept, just be
- * incredibly strict about the characters that are allowed
- * in user names.
- */
- if (!username_ok_for_shell(port->user_name))
- goto cleanup;
-
- appendStringInfoString(&command, port->user_name);
- p++;
- break;
- case '%':
- appendStringInfoChar(&command, '%');
- p++;
- break;
- default:
- appendStringInfoChar(&command, p[0]);
- }
- }
- else
- appendStringInfoChar(&command, p[0]);
- }
-
- /* Execute the command. */
- fh = OpenPipeStream(command.data, "r");
- if (!fh)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("opening pipe to OAuth validator: %m")));
- goto cleanup;
- }
-
- /* We don't need the read end of the pipe anymore. */
- close(rfd);
- rfd = -1;
-
- /* Give the command the token to validate. */
- written = write(wfd, token, strlen(token));
- if (written != strlen(token))
- {
- /* TODO must loop for short writes, EINTR et al */
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not write token to child pipe: %m")));
- goto cleanup;
- }
-
- close(wfd);
- wfd = -1;
-
- /*-----
- * Read the command's response.
- *
- * TODO: getline() is probably too new to use, unfortunately.
- * TODO: loop over all lines
- */
- if ((len = getline(&line, &size, fh)) >= 0)
- {
- /* TODO: fail if the authn_id doesn't end with a newline */
- if (len > 0)
- line[len - 1] = '\0';
-
- set_authn_id(port, line);
- }
- else if (ferror(fh))
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not read from command \"%s\": %m",
- command.data)));
- goto cleanup;
- }
-
- /* Make sure the command exits cleanly. */
- if (!check_exit(&fh, command.data))
- {
- /* error message already logged */
- goto cleanup;
- }
-
- /* Done. */
- success = true;
-
-cleanup:
- if (line)
- free(line);
-
- /*
- * In the successful case, the pipe fds are already closed. For the error
- * case, always close out the pipe before waiting for the command, to
- * prevent deadlock.
- */
- if (rfd >= 0)
- close(rfd);
- if (wfd >= 0)
- close(wfd);
-
- if (fh)
- {
- Assert(!success);
- check_exit(&fh, command.data);
- }
-
- if (command.data)
- pfree(command.data);
-
- return success;
+ map_status = check_usermap(port->hba->usermap, port->user_name,
+ MyClientConnectionInfo.authn_id, false);
+ return (map_status == STATUS_OK);
}
-static bool
-check_exit(FILE **fh, const char *command)
+static void
+load_validator_library(void)
{
- int rc;
-
- rc = ClosePipeStream(*fh);
- *fh = NULL;
-
- if (rc == -1)
- {
- /* pclose() itself failed. */
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not close pipe to command \"%s\": %m",
- command)));
- }
- else if (rc != 0)
- {
- char *reason = wait_result_to_str(rc);
+ OAuthValidatorModuleInit validator_init;
- ereport(COMMERROR,
- (errmsg("failed to execute command \"%s\": %s",
- command, reason)));
+ if (OAuthValidatorLibrary[0] == '\0')
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("oauth_validator_library is not set")));
- pfree(reason);
- }
+ validator_init = (OAuthValidatorModuleInit)
+ load_external_function(OAuthValidatorLibrary,
+ "_PG_oauth_validator_module_init", false, NULL);
- return (rc == 0);
-}
+ if (validator_init == NULL)
+ ereport(ERROR,
+ (errmsg("%s module \"%s\" have to define the symbol %s",
+ "OAuth validator", OAuthValidatorLibrary, "_PG_oauth_validator_module_init")));
-static bool
-set_cloexec(int fd)
-{
- int flags;
- int rc;
+ ValidatorCallbacks = (*validator_init) ();
- flags = fcntl(fd, F_GETFD);
- if (flags == -1)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not get fd flags for child pipe: %m")));
- return false;
- }
+ validator_module_state = (ValidatorModuleState *) palloc0(sizeof(ValidatorModuleState));
+ if (ValidatorCallbacks->startup_cb != NULL)
+ ValidatorCallbacks->startup_cb(validator_module_state);
- rc = fcntl(fd, F_SETFD, flags | FD_CLOEXEC);
- if (rc < 0)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not set FD_CLOEXEC for child pipe: %m")));
- return false;
- }
-
- return true;
+ before_shmem_exit(shutdown_validator_library, 0);
}
-/*
- * XXX This should go away eventually and be replaced with either a proper
- * escape or a different strategy for communication with the validator command.
- */
-static bool
-username_ok_for_shell(const char *username)
+static void
+shutdown_validator_library(int code, Datum arg)
{
- /* This set is borrowed from fe_utils' appendShellStringNoError(). */
- static const char *const allowed =
- "abcdefghijklmnopqrstuvwxyz"
- "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
- "0123456789-_./:";
- size_t span;
-
- Assert(username && username[0]); /* should have already been checked */
-
- span = strspn(username, allowed);
- if (username[span] != '\0')
- {
- ereport(COMMERROR,
- (errmsg("PostgreSQL user name contains unsafe characters and cannot be passed to the OAuth validator")));
- return false;
- }
-
- return true;
+ if (ValidatorCallbacks->shutdown_cb != NULL)
+ ValidatorCallbacks->shutdown_cb(validator_module_state);
}
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index d28209901f..99bb89d54e 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -4672,12 +4672,12 @@ struct config_string ConfigureNamesString[] =
},
{
- {"oauth_validator_command", PGC_SIGHUP, CONN_AUTH_AUTH,
- gettext_noop("Command to validate OAuth v2 bearer tokens."),
+ {"oauth_validator_library", PGC_SIGHUP, CONN_AUTH_AUTH,
+ gettext_noop("Sets the library that will be called to validate OAuth v2 bearer tokens."),
NULL,
GUC_SUPERUSER_ONLY | GUC_NOT_IN_SAMPLE
},
- &oauth_validator_command,
+ &OAuthValidatorLibrary,
"",
NULL, NULL, NULL
},
diff --git a/src/bin/pg_combinebackup/Makefile b/src/bin/pg_combinebackup/Makefile
index c3729755ba..4f24b1aff6 100644
--- a/src/bin/pg_combinebackup/Makefile
+++ b/src/bin/pg_combinebackup/Makefile
@@ -31,7 +31,7 @@ OBJS = \
all: pg_combinebackup
pg_combinebackup: $(OBJS) | submake-libpgport submake-libpgfeutils
- $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(libpq_pgport) $(LIBS) -o $@$(X)
install: all installdirs
$(INSTALL_PROGRAM) pg_combinebackup$(X) '$(DESTDIR)$(bindir)/pg_combinebackup$(X)'
diff --git a/src/common/Makefile b/src/common/Makefile
index bbb5c3ab11..00e30e6bfe 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -41,7 +41,7 @@ override CPPFLAGS += -DVAL_LDFLAGS_SL="\"$(LDFLAGS_SL)\""
override CPPFLAGS += -DVAL_LIBS="\"$(LIBS)\""
override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common -I$(libpq_srcdir) $(CPPFLAGS)
-LIBS += $(PTHREAD_LIBS)
+LIBS += $(PTHREAD_LIBS) $(libpq_pgport)
OBJS_COMMON = \
archive.o \
diff --git a/src/include/libpq/oauth.h b/src/include/libpq/oauth.h
index 5edab3b25a..6f98e84cc9 100644
--- a/src/include/libpq/oauth.h
+++ b/src/include/libpq/oauth.h
@@ -3,7 +3,7 @@
* oauth.h
* Interface to libpq/auth-oauth.c
*
- * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* src/include/libpq/oauth.h
@@ -16,7 +16,32 @@
#include "libpq/libpq-be.h"
#include "libpq/sasl.h"
-extern char *oauth_validator_command;
+extern PGDLLIMPORT char *OAuthValidatorLibrary;
+
+typedef struct ValidatorModuleState
+{
+ void *private_data;
+} ValidatorModuleState;
+
+typedef struct ValidatorModuleResult
+{
+ bool authorized;
+ char *authn_id;
+} ValidatorModuleResult;
+
+typedef void (*ValidatorStartupCB) (ValidatorModuleState *state);
+typedef void (*ValidatorShutdownCB) (ValidatorModuleState *state);
+typedef ValidatorModuleResult *(*ValidatorValidateCB) (ValidatorModuleState *state, const char *token, const char *role);
+
+typedef struct OAuthValidatorCallbacks
+{
+ ValidatorStartupCB startup_cb;
+ ValidatorShutdownCB shutdown_cb;
+ ValidatorValidateCB validate_cb;
+} OAuthValidatorCallbacks;
+
+typedef const OAuthValidatorCallbacks *(*OAuthValidatorModuleInit) (void);
+extern PGDLLEXPORT const OAuthValidatorCallbacks *_PG_oauth_validator_module_init(void);
/* Implementation */
extern const pg_be_sasl_mech pg_be_oauth_mech;
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index 8fbe742d38..dc54ce7189 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -9,6 +9,7 @@ subdir('gin')
subdir('injection_points')
subdir('ldap_password_func')
subdir('libpq_pipeline')
+subdir('oauth_validator')
subdir('plsample')
subdir('spgist_name_ops')
subdir('ssl_passphrase_callback')
diff --git a/src/test/modules/oauth_validator/.gitignore b/src/test/modules/oauth_validator/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/src/test/modules/oauth_validator/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/oauth_validator/Makefile b/src/test/modules/oauth_validator/Makefile
new file mode 100644
index 0000000000..1f874cd7f2
--- /dev/null
+++ b/src/test/modules/oauth_validator/Makefile
@@ -0,0 +1,19 @@
+MODULES = validator
+PGFILEDESC = "validator - test OAuth validator module"
+
+NO_INSTALLCHECK = 1
+
+TAP_TESTS = 1
+
+REGRESS = validator
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/oauth_validator
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/oauth_validator/expected/validator.out b/src/test/modules/oauth_validator/expected/validator.out
new file mode 100644
index 0000000000..360caa2cb3
--- /dev/null
+++ b/src/test/modules/oauth_validator/expected/validator.out
@@ -0,0 +1,6 @@
+SELECT 1;
+ ?column?
+----------
+ 1
+(1 row)
+
diff --git a/src/test/modules/oauth_validator/meson.build b/src/test/modules/oauth_validator/meson.build
new file mode 100644
index 0000000000..d9c1d1d577
--- /dev/null
+++ b/src/test/modules/oauth_validator/meson.build
@@ -0,0 +1,33 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+validator_sources = files(
+ 'validator.c',
+)
+
+if host_system == 'windows'
+ validator_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'validator',
+ '--FILEDESC', 'validator - test OAuth validator module',])
+endif
+
+validator = shared_module('validator',
+ validator_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += validator
+
+tests += {
+ 'name': 'oauth_validator',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'regress': {
+ 'sql': [
+ 'validator',
+ ],
+ },
+ 'tap': {
+ 'tests': [
+ 't/001_server.pl',
+ ],
+ },
+}
diff --git a/src/test/modules/oauth_validator/sql/validator.sql b/src/test/modules/oauth_validator/sql/validator.sql
new file mode 100644
index 0000000000..e0ac49d1ec
--- /dev/null
+++ b/src/test/modules/oauth_validator/sql/validator.sql
@@ -0,0 +1 @@
+SELECT 1;
diff --git a/src/test/modules/oauth_validator/t/001_server.pl b/src/test/modules/oauth_validator/t/001_server.pl
new file mode 100644
index 0000000000..14c7778298
--- /dev/null
+++ b/src/test/modules/oauth_validator/t/001_server.pl
@@ -0,0 +1,78 @@
+
+# Copyright (c) 2021-2024, PostgreSQL Global Development Group
+
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::OAuthServer;
+use Test::More;
+
+my $node = PostgreSQL::Test::Cluster->new('primary');
+$node->init;
+$node->append_conf('postgresql.conf', "log_connections = on\n");
+$node->append_conf('postgresql.conf', "shared_preload_libraries = 'validator'\n");
+$node->append_conf('postgresql.conf', "oauth_validator_library = 'validator'\n");
+$node->start;
+
+$node->safe_psql('postgres', 'CREATE USER test;');
+$node->safe_psql('postgres', 'CREATE USER testalt;');
+
+my $issuer = "127.0.0.1:18080";
+
+unlink($node->data_dir . '/pg_hba.conf');
+$node->append_conf('pg_hba.conf', qq{
+local all test oauth issuer="$issuer" scope="openid postgres"
+local all testalt oauth issuer="$issuer/alternate" scope="openid postgres alt"
+});
+$node->reload;
+
+my $webserver = PostgreSQL::Test::OAuthServer->new(18080);
+
+my $port = $webserver->port();
+
+is($port, 18080, "Port is 18080");
+
+$webserver->setup();
+$webserver->run();
+
+my ($log_start, $log_end);
+$log_start = $node->wait_for_log(qr/reloading configuration files/);
+
+my $user = "test";
+$node->connect_ok("user=$user dbname=postgres oauth_client_id=f02c6361-0635", "connect",
+ expected_stderr => qr@Visit https://example\.com/ and enter the code: postgresuser@);
+
+$log_end = $node->wait_for_log(qr/connection authorized/, $log_start);
+$node->log_check("user $user: validator receives correct parameters", $log_start,
+ log_like => [
+ qr/oauth_validator: token="9243959234", role="$user"/,
+ qr/oauth_validator: issuer="\Q$issuer\E", scope="openid postgres"/,
+ ]);
+$node->log_check("user $user: validator sets authenticated identity", $log_start,
+ log_like => [
+ qr/connection authenticated: identity="test" method=oauth/,
+ ]);
+$log_start = $log_end;
+
+# The /alternate issuer uses slightly different parameters.
+$user = "testalt";
+$node->connect_ok("user=$user dbname=postgres oauth_client_id=f02c6361-0636", "connect",
+ expected_stderr => qr@Visit https://example\.org/ and enter the code: postgresuser@);
+
+$log_end = $node->wait_for_log(qr/connection authorized/, $log_start);
+$node->log_check("user $user: validator receives correct parameters", $log_start,
+ log_like => [
+ qr/oauth_validator: token="9243959234-alt", role="$user"/,
+ qr|oauth_validator: issuer="\Q$issuer/alternate\E", scope="openid postgres alt"|,
+ ]);
+$node->log_check("user $user: validator sets authenticated identity", $log_start,
+ log_like => [
+ qr/connection authenticated: identity="testalt" method=oauth/,
+ ]);
+$log_start = $log_end;
+
+$node->stop;
+
+done_testing();
diff --git a/src/test/modules/oauth_validator/validator.c b/src/test/modules/oauth_validator/validator.c
new file mode 100644
index 0000000000..09a4bf61d2
--- /dev/null
+++ b/src/test/modules/oauth_validator/validator.c
@@ -0,0 +1,82 @@
+/*-------------------------------------------------------------------------
+ *
+ * validator.c
+ * Test module for serverside OAuth token validation callbacks
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/test/modules/oauth_validator/validator.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "libpq/oauth.h"
+#include "miscadmin.h"
+#include "utils/memutils.h"
+
+PG_MODULE_MAGIC;
+
+static void validator_startup(ValidatorModuleState *state);
+static void validator_shutdown(ValidatorModuleState *state);
+static ValidatorModuleResult * validate_token(ValidatorModuleState *state,
+ const char *token,
+ const char *role);
+
+static const OAuthValidatorCallbacks validator_callbacks = {
+ .startup_cb = validator_startup,
+ .shutdown_cb = validator_shutdown,
+ .validate_cb = validate_token
+};
+
+void
+_PG_init(void)
+{
+ /* no-op */
+}
+
+const OAuthValidatorCallbacks *
+_PG_oauth_validator_module_init(void)
+{
+ return &validator_callbacks;
+}
+
+#define PRIVATE_COOKIE ((void *) 13579)
+
+static void
+validator_startup(ValidatorModuleState *state)
+{
+ state->private_data = PRIVATE_COOKIE;
+}
+
+static void
+validator_shutdown(ValidatorModuleState *state)
+{
+ /* do nothing */
+}
+
+static ValidatorModuleResult *
+validate_token(ValidatorModuleState *state, const char *token, const char *role)
+{
+ ValidatorModuleResult *res;
+
+ /* Check to make sure our private state still exists. */
+ if (state->private_data != PRIVATE_COOKIE)
+ elog(ERROR, "oauth_validator: private state cookie changed to %p",
+ state->private_data);
+
+ res = palloc(sizeof(ValidatorModuleResult));
+
+ elog(LOG, "oauth_validator: token=\"%s\", role=\"%s\"", token, role);
+ elog(LOG, "oauth_validator: issuer=\"%s\", scope=\"%s\"",
+ MyProcPort->hba->oauth_issuer,
+ MyProcPort->hba->oauth_scope);
+
+ res->authorized = true;
+ res->authn_id = pstrdup(role);
+
+ return res;
+}
diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm
index 4fec417f6f..b291bbf8ee 100644
--- a/src/test/perl/PostgreSQL/Test/Cluster.pm
+++ b/src/test/perl/PostgreSQL/Test/Cluster.pm
@@ -2302,6 +2302,11 @@ instead of the default.
If this regular expression is set, matches it with the output generated.
+=item expected_stderr => B<value>
+
+If this regular expression is set, matches it against the standard error
+stream; otherwise the stderr must be empty.
+
=item log_like => [ qr/required message/ ]
=item log_unlike => [ qr/prohibited message/ ]
@@ -2345,7 +2350,14 @@ sub connect_ok
like($stdout, $params{expected_stdout}, "$test_name: stdout matches");
}
- is($stderr, "", "$test_name: no stderr");
+ if (defined($params{expected_stderr}))
+ {
+ like($stderr, $params{expected_stderr}, "$test_name: stderr matches");
+ }
+ else
+ {
+ is($stderr, "", "$test_name: no stderr");
+ }
$self->log_check($test_name, $log_location, %params);
}
diff --git a/src/test/perl/PostgreSQL/Test/OAuthServer.pm b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
new file mode 100644
index 0000000000..5c195efb79
--- /dev/null
+++ b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
@@ -0,0 +1,183 @@
+#!/usr/bin/perl
+
+package PostgreSQL::Test::OAuthServer;
+
+use warnings;
+use strict;
+use threads;
+use Socket;
+use IO::Select;
+
+local *server_socket;
+
+sub new
+{
+ my $class = shift;
+ my $port = shift;
+
+ my $self = {};
+ bless($self, $class);
+
+ $self->{'port'} = $port;
+
+ return $self;
+}
+
+sub setup
+{
+ my $self = shift;
+ my $tcp = getprotobyname('tcp');
+
+ socket($self->{'socket'}, PF_INET, SOCK_STREAM, $tcp)
+ or die "no socket";
+ setsockopt($self->{'socket'}, SOL_SOCKET, SO_REUSEADDR, pack("l", 1));
+ bind($self->{'socket'}, sockaddr_in($self->{'port'}, INADDR_ANY));
+}
+
+sub port
+{
+ my $self = shift;
+
+ return $self->{'port'};
+}
+
+sub run
+{
+ my $self = shift;
+
+ my $server_thread = threads->create(\&_listen, $self);
+ $server_thread->detach();
+}
+
+sub _listen
+{
+ my $self = shift;
+
+ listen($self->{'socket'}, SOMAXCONN) or die "fail to listen: $!";
+
+ while (1)
+ {
+ my $fh;
+ my %request;
+ my $remote = accept($fh, $self->{'socket'});
+ binmode $fh;
+
+ my ($method, $object, $prot) = split(/ /, <$fh>);
+ $request{'method'} = $method;
+ $request{'object'} = $object;
+ chomp($request{'object'});
+
+ local $/ = Socket::CRLF;
+ my $c = 0;
+ while(<$fh>)
+ {
+ chomp;
+ # Headers
+ if (/:/)
+ {
+ my ($field, $value) = split(/:/, $_, 2);
+ $value =~ s/^\s+//;
+ $request{'headers'}{lc $field} = $value;
+ }
+ # POST data
+ elsif (/^$/)
+ {
+ read($fh, $request{'content'}, $request{'headers'}{'content-length'})
+ if defined $request{'headers'}{'content-length'};
+ last;
+ }
+ }
+
+ # Debug printing
+ # print ": read ".$request{'method'} . ";" . $request{'object'}.";\n";
+ # foreach my $h (keys(%{$request{'headers'}}))
+ #{
+ # printf ": headers: " . $request{'headers'}{$h} . "\n";
+ #}
+ #printf ": POST: " . $request{'content'} . "\n" if defined $request{'content'};
+
+ my $alternate = 0;
+ if ($request{'object'} =~ qr|^/alternate(/.*)$|)
+ {
+ $alternate = 1;
+ $request{'object'} = $1;
+ }
+
+ if ($request{'object'} eq '/.well-known/openid-configuration')
+ {
+ my $issuer = "http://localhost:$self->{'port'}";
+ if ($alternate)
+ {
+ $issuer .= "/alternate";
+ }
+
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: application/json\r\n";
+ print $fh "\r\n";
+ print $fh <<EOR;
+ {
+ "issuer": "$issuer",
+ "token_endpoint": "$issuer/token",
+ "device_authorization_endpoint": "$issuer/authorize",
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ "grant_types_supported": ["urn:ietf:params:oauth:grant-type:device_code"]
+ }
+EOR
+ }
+ elsif ($request{'object'} eq '/authorize')
+ {
+ my $uri = "https://example.com/";
+ if ($alternate)
+ {
+ $uri = "https://example.org/";
+ }
+
+ print ": returning device_code\n";
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: application/json\r\n";
+ print $fh "\r\n";
+ print $fh <<EOR;
+ {
+ "device_code": "postgres",
+ "user_code" : "postgresuser",
+ "interval" : 0,
+ "verification_uri" : "$uri",
+ "expires-in": 5
+ }
+EOR
+ }
+ elsif ($request{'object'} eq '/token')
+ {
+ my $token = "9243959234";
+ if ($alternate)
+ {
+ $token .= "-alt";
+ }
+
+ print ": returning token\n";
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: application/json\r\n";
+ print $fh "\r\n";
+ print $fh <<EOR;
+ {
+ "access_token": "$token",
+ "token_type": "bearer"
+ }
+EOR
+ }
+ else
+ {
+ print ": returning default\n";
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: text/html\r\n";
+ print $fh "\r\n";
+ print $fh "Ok\n";
+ }
+
+ close($fh);
+ }
+}
+
+1;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 8119523cd9..8558f3b3e6 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1657,6 +1657,7 @@ NumericSortSupport
NumericSumAccum
NumericVar
OAuthStep
+OAuthValidatorCallbacks
OM_uint32
OP
OSAPerGroupState
@@ -2980,6 +2981,7 @@ VacuumRelation
VacuumStmt
ValidIOData
ValidateIndexState
+ValidatorModuleState
ValuesScan
ValuesScanState
Var
--
2.34.1
v20-0007-Add-pytest-suite-for-OAuth.patch (application/octet-stream, 176.9K) download
From e1da97fb501d18d2e7557e47acde431233ad1342 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Fri, 4 Jun 2021 09:06:38 -0700
Subject: [PATCH v20 7/9] Add pytest suite for OAuth
Requires Python 3. On the first run of `make installcheck` or `meson
test` the dependencies will be installed into a local virtualenv for
you. See the README for more details.
Cirrus has been updated to build OAuth support on Debian and FreeBSD.
The suite contains a --temp-instance option, analogous to pg_regress's
option of the same name, which allows an ephemeral server to be spun up
during a test run.
TODOs:
- The --tap-stream option to pytest-tap is slightly broken during test
failures (it suppresses error information), which impedes debugging.
- pyca/cryptography is pinned at an old version. Since we use it for
testing and not security, this isn't a critical problem yet, but it's
not ideal. Newer versions require a Rust compiler to build, and while
many platforms have precompiled wheels, some (FreeBSD) do not. Even
with the Rust pieces bypassed, compilation on FreeBSD takes a while.
- The with_oauth test skip logic should probably be integrated into the
Makefile side as well...
- See if 32-bit tests can be enabled with a 32-bit Python.
---
.cirrus.tasks.yml | 22 +-
meson.build | 107 ++
src/test/meson.build | 1 +
src/test/python/.gitignore | 2 +
src/test/python/Makefile | 38 +
src/test/python/README | 66 +
src/test/python/client/__init__.py | 0
src/test/python/client/conftest.py | 137 ++
src/test/python/client/test_client.py | 186 +++
src/test/python/client/test_oauth.py | 1720 +++++++++++++++++++++++++
src/test/python/conftest.py | 34 +
src/test/python/meson.build | 46 +
src/test/python/pq3.py | 740 +++++++++++
src/test/python/pytest.ini | 4 +
src/test/python/requirements.txt | 9 +
src/test/python/server/__init__.py | 0
src/test/python/server/conftest.py | 141 ++
src/test/python/server/meson.build | 18 +
src/test/python/server/oauthtest.c | 118 ++
src/test/python/server/test_oauth.py | 1074 +++++++++++++++
src/test/python/server/test_server.py | 21 +
src/test/python/test_internals.py | 138 ++
src/test/python/test_pq3.py | 574 +++++++++
src/test/python/tls.py | 195 +++
src/tools/make_venv | 56 +
25 files changed, 5440 insertions(+), 7 deletions(-)
create mode 100644 src/test/python/.gitignore
create mode 100644 src/test/python/Makefile
create mode 100644 src/test/python/README
create mode 100644 src/test/python/client/__init__.py
create mode 100644 src/test/python/client/conftest.py
create mode 100644 src/test/python/client/test_client.py
create mode 100644 src/test/python/client/test_oauth.py
create mode 100644 src/test/python/conftest.py
create mode 100644 src/test/python/meson.build
create mode 100644 src/test/python/pq3.py
create mode 100644 src/test/python/pytest.ini
create mode 100644 src/test/python/requirements.txt
create mode 100644 src/test/python/server/__init__.py
create mode 100644 src/test/python/server/conftest.py
create mode 100644 src/test/python/server/meson.build
create mode 100644 src/test/python/server/oauthtest.c
create mode 100644 src/test/python/server/test_oauth.py
create mode 100644 src/test/python/server/test_server.py
create mode 100644 src/test/python/test_internals.py
create mode 100644 src/test/python/test_pq3.py
create mode 100644 src/test/python/tls.py
create mode 100755 src/tools/make_venv
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 3b5b54df58..2501743b31 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -20,7 +20,7 @@ env:
MTEST_ARGS: --print-errorlogs --no-rebuild -C build
PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests
TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf
- PG_TEST_EXTRA: kerberos ldap ssl load_balance
+ PG_TEST_EXTRA: kerberos ldap ssl load_balance python
# What files to preserve in case tests fail
@@ -165,7 +165,7 @@ task:
chown root:postgres /tmp/cores
sysctl kern.corefile='/tmp/cores/%N.%P.core'
setup_additional_packages_script: |
- #pkg install -y ...
+ pkg install -y curl
# NB: Intentionally build without -Dllvm. The freebsd image size is already
# large enough to make VM startup slow, and even without llvm freebsd
@@ -177,6 +177,7 @@ task:
-Dcassert=true -Dinjection_points=true \
-Duuid=bsd -Dtcl_version=tcl86 -Ddtrace=auto \
-DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
+ -Doauth=curl \
-Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \
build
EOF
@@ -225,6 +226,7 @@ LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
--with-libxslt
--with-llvm
--with-lz4
+ --with-oauth=curl
--with-pam
--with-perl
--with-python
@@ -237,6 +239,7 @@ LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
LINUX_MESON_FEATURES: &LINUX_MESON_FEATURES >-
-Dllvm=enabled
+ -Doauth=curl
-Duuid=e2fs
@@ -312,8 +315,11 @@ task:
EOF
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install \
+ libcurl4-openssl-dev \
+ libcurl4-openssl-dev:i386 \
+ python3-venv \
matrix:
- name: Linux - Debian Bullseye - Autoconf
@@ -368,6 +374,8 @@ task:
# Also build & test in a 32bit build - it's gotten rare to test that
# locally.
+ # XXX 32-bit Python tests are currently disabled, as the system's 64-bit
+ # Python modules can't link against libpq.
configure_32_script: |
su postgres <<-EOF
export CC='ccache gcc -m32'
@@ -378,7 +386,7 @@ task:
-Dllvm=disabled \
--pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
-DPERL=perl5.32-i386-linux-gnu \
- -DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
+ -DPG_TEST_EXTRA="${PG_TEST_EXTRA//"python"}" \
build-32
EOF
@@ -676,8 +684,8 @@ task:
folder: $CCACHE_DIR
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install libcurl4-openssl-dev
###
# Test that code can be built with gcc/clang without warnings
diff --git a/meson.build b/meson.build
index 45b20d11c1..8567355a25 100644
--- a/meson.build
+++ b/meson.build
@@ -3174,6 +3174,9 @@ else
endif
testwrap = files('src/tools/testwrap')
+make_venv = files('src/tools/make_venv')
+
+checked_working_venv = false
foreach test_dir : tests
testwrap_base = [
@@ -3335,6 +3338,110 @@ foreach test_dir : tests
)
endforeach
install_suites += test_group
+ elif kind == 'pytest'
+ venv_name = test_dir['name'] + '_venv'
+ venv_path = meson.build_root() / venv_name
+
+ # The Python tests require a working venv module. This is part of the
+ # standard library, but some platforms disable it until a separate package
+ # is installed. Those same platforms don't provide an easy way to check
+ # whether the venv command will work until the first time you try it, so
+ # we decide whether or not to enable these tests on the fly.
+ if not checked_working_venv
+ cmd = run_command(python, '-m', 'venv', venv_path, check: false)
+
+ have_working_venv = (cmd.returncode() == 0)
+ if not have_working_venv
+ warning('A working Python venv module is required to run Python tests.')
+ endif
+
+ checked_working_venv = true
+ endif
+
+ if not have_working_venv
+ continue
+ endif
+
+ # Make sure the temporary installation is in PATH (necessary both for
+ # --temp-instance and for any pip modules compiling against libpq, like
+ # psycopg2).
+ env = test_env
+ env.prepend('PATH', temp_install_bindir, test_dir['bd'])
+
+ foreach name, value : t.get('env', {})
+ env.set(name, value)
+ endforeach
+
+ if get_option('PG_TEST_EXTRA').contains('python')
+ reqs = files(t['requirements'])
+ test('install_' + venv_name,
+ python,
+ args: [ make_venv, '--requirements', reqs, venv_path ],
+ env: env,
+ priority: setup_tests_priority - 1, # must run after tmp_install
+ is_parallel: false,
+ suite: ['setup'],
+ timeout: 60, # 30s is too short for the cryptography package compile
+ )
+ endif
+
+ test_group = test_dir['name']
+ test_output = test_result_dir / test_group / kind
+ test_kwargs = {
+ #'protocol': 'tap',
+ 'suite': test_group,
+ 'timeout': 1000,
+ 'depends': test_deps,
+ 'env': env,
+ }
+
+ if fs.is_dir(venv_path / 'Scripts')
+ # Windows virtualenv layout
+ pytest = venv_path / 'Scripts' / 'py.test'
+ else
+ pytest = venv_path / 'bin' / 'py.test'
+ endif
+
+ test_command = [
+ pytest,
+ # Avoid running these tests against an existing database.
+ '--temp-instance', test_output / 'data',
+
+ # FIXME pytest-tap's stream feature accidentally suppresses errors that
+ # are critical for debugging:
+ # https://github.com/python-tap/pytest-tap/issues/30
+ # Don't use the meson TAP protocol for now...
+ #'--tap-stream',
+ ]
+
+ foreach pyt : t['tests']
+ # Similarly to TAP, strip ./ and .py to make the names prettier
+ pyt_p = pyt
+ if pyt_p.startswith('./')
+ pyt_p = pyt_p.split('./')[1]
+ endif
+ if pyt_p.endswith('.py')
+ pyt_p = fs.stem(pyt_p)
+ endif
+
+ testwrap_pytest = testwrap_base + [
+ '--testgroup', test_group,
+ '--testname', pyt_p,
+ ]
+ if not get_option('PG_TEST_EXTRA').contains('python')
+ testwrap_pytest += ['--skip', '"python" tests not enabled in PG_TEST_EXTRA']
+ endif
+
+ test(test_group / pyt_p,
+ python,
+ kwargs: test_kwargs,
+ args: testwrap_pytest + [
+ '--', test_command,
+ test_dir['sd'] / pyt,
+ ],
+ )
+ endforeach
+ install_suites += test_group
else
error('unknown kind @0@ of test in @1@'.format(kind, test_dir['sd']))
endif
diff --git a/src/test/meson.build b/src/test/meson.build
index c3d0dfedf1..f401ec179e 100644
--- a/src/test/meson.build
+++ b/src/test/meson.build
@@ -7,6 +7,7 @@ subdir('authentication')
subdir('recovery')
subdir('subscription')
subdir('modules')
+subdir('python')
if ssl.found()
subdir('ssl')
diff --git a/src/test/python/.gitignore b/src/test/python/.gitignore
new file mode 100644
index 0000000000..0e8f027b2e
--- /dev/null
+++ b/src/test/python/.gitignore
@@ -0,0 +1,2 @@
+__pycache__/
+/venv/
diff --git a/src/test/python/Makefile b/src/test/python/Makefile
new file mode 100644
index 0000000000..b0695b6287
--- /dev/null
+++ b/src/test/python/Makefile
@@ -0,0 +1,38 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+# Only Python 3 is supported, but if it's named something different on your
+# system you can override it with the PYTHON3 variable.
+PYTHON3 := python3
+
+# All dependencies are placed into this directory. The default is .gitignored
+# for you, but you can override it if you'd like.
+VENV := ./venv
+
+override VBIN := $(VENV)/bin
+override PIP := $(VBIN)/pip
+override PYTEST := $(VBIN)/py.test
+override ISORT := $(VBIN)/isort
+override BLACK := $(VBIN)/black
+
+.PHONY: installcheck indent
+
+installcheck: $(PYTEST)
+ $(PYTEST) -v -rs
+
+indent: $(ISORT) $(BLACK)
+ $(ISORT) --profile black *.py client/*.py server/*.py
+ $(BLACK) *.py client/*.py server/*.py
+
+$(PYTEST) $(ISORT) $(BLACK) &: requirements.txt | $(PIP)
+ $(PIP) install --force-reinstall -r $<
+
+$(PIP):
+ $(PYTHON3) -m venv $(VENV)
+
+# A convenience recipe to rebuild psycopg2 against the local libpq.
+.PHONY: rebuild-psycopg2
+rebuild-psycopg2: | $(PIP)
+ $(PIP) install --force-reinstall --no-binary :all: $(shell grep psycopg2 requirements.txt)
diff --git a/src/test/python/README b/src/test/python/README
new file mode 100644
index 0000000000..acf339a589
--- /dev/null
+++ b/src/test/python/README
@@ -0,0 +1,66 @@
+A test suite for exercising both the libpq client and the server backend at the
+protocol level, based on pytest and Construct.
+
+WARNING! This suite takes superuser-level control of the cluster under test,
+writing to the server config, creating and destroying databases, etc. It also
+spins up various ephemeral TCP services. This is not safe for production servers
+and therefore must be explicitly opted into by setting PG_TEST_EXTRA=python in
+the environment.
+
+The test suite currently assumes that the standard PG* environment variables
+point to the database under test and are sufficient to log in a superuser on
+that system. In other words, a bare `psql` needs to Just Work before the test
+suite can do its thing. For a newly built dev cluster, typically all that I need
+to do is a
+
+ export PGDATABASE=postgres
+
+but you can adjust as needed for your setup. See also 'Advanced Usage' below.
+
+## Requirements
+
+A supported version (3.6+) of Python.
+
+The first run of
+
+ make installcheck PG_TEST_EXTRA=python
+
+will install a local virtual environment and all needed dependencies. During
+development, if libpq changes incompatibly, you can issue
+
+ $ make rebuild-psycopg2
+
+to force a rebuild of the client library.
+
+## Hacking
+
+The code style is enforced by a _very_ opinionated autoformatter. Running the
+
+ make indent
+
+recipe will invoke it for you automatically. Don't fight the tool; part of the
+zen is in knowing that if the formatter makes your code ugly, there's probably a
+cleaner way to write your code.
+
+## Advanced Usage
+
+The Makefile is there for convenience, but you don't have to use it. Activate
+the virtualenv to be able to use pytest directly:
+
+ $ export PG_TEST_EXTRA=python
+ $ source venv/bin/activate
+ $ py.test -k oauth
+ ...
+ $ py.test ./server/test_server.py
+ ...
+ $ deactivate # puts the PATH et al back the way it was before
+
+To make quick smoke tests possible, slow tests have been marked explicitly. You
+can skip them by saying e.g.
+
+ $ py.test -m 'not slow'
+
+If you'd rather not test against an existing server, you can have the suite spin
+up a temporary one using whatever pg_ctl it finds in PATH:
+
+ $ py.test --temp-instance=./tmp_check
diff --git a/src/test/python/client/__init__.py b/src/test/python/client/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/src/test/python/client/conftest.py b/src/test/python/client/conftest.py
new file mode 100644
index 0000000000..94f3620af3
--- /dev/null
+++ b/src/test/python/client/conftest.py
@@ -0,0 +1,137 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import socket
+import sys
+import threading
+
+import psycopg2
+import psycopg2.extras
+import pytest
+
+import pq3
+
+BLOCKING_TIMEOUT = 2 # the number of seconds to wait for blocking calls
+
+
+@pytest.fixture
+def server_socket(unused_tcp_port_factory):
+ """
+ Returns a listening socket bound to an ephemeral port.
+ """
+ with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+ s.bind(("127.0.0.1", unused_tcp_port_factory()))
+ s.listen(1)
+ s.settimeout(BLOCKING_TIMEOUT)
+ yield s
+
+
+class ClientHandshake(threading.Thread):
+ """
+ A thread that connects to a local Postgres server using psycopg2. Once the
+ opening handshake completes, the connection will be immediately closed.
+ """
+
+ def __init__(self, *, port, **kwargs):
+ super().__init__()
+
+ kwargs["port"] = port
+ self._kwargs = kwargs
+
+ self.exception = None
+
+ def run(self):
+ try:
+ conn = psycopg2.connect(host="127.0.0.1", **self._kwargs)
+ self._pump_async(conn)
+ conn.close()
+ except Exception as e:
+ self.exception = e
+
+ def check_completed(self, timeout=BLOCKING_TIMEOUT):
+ """
+ Joins the client thread. Raises an exception if the thread could not be
+ joined, or if it threw an exception itself. (The exception will be
+ cleared, so future calls to check_completed will succeed.)
+ """
+ self.join(timeout)
+
+ if self.is_alive():
+ raise TimeoutError("client thread did not handshake within the timeout")
+ elif self.exception:
+ e = self.exception
+ self.exception = None
+ raise e
+
+ def _pump_async(self, conn):
+ """
+ Polls a psycopg2 connection until it's completed. (Synchronous
+ connections will work here too; they'll just immediately return OK.)
+ """
+ psycopg2.extras.wait_select(conn)
+
+
+@pytest.fixture
+def accept(server_socket):
+ """
+ Returns a factory function that, when called, returns a pair (sock, client)
+ where sock is a server socket that has accepted a connection from client,
+ and client is an instance of ClientHandshake. Clients will complete their
+ handshakes and cleanly disconnect.
+
+ The default connstring options may be extended or overridden by passing
+ arbitrary keyword arguments. Keep in mind that you generally should not
+ override the host or port, since they point to the local test server.
+
+ For situations where a client needs to connect more than once to complete a
+ handshake, the accept function may be called more than once. (The client
+ returned for subsequent calls will always be the same client that was
+ returned for the first call.)
+
+ Tests must either complete the handshake so that the client thread can be
+ automatically joined during teardown, or else call client.check_completed()
+ and manually handle any expected errors.
+ """
+ _, port = server_socket.getsockname()
+
+ client = None
+ default_opts = dict(
+ port=port,
+ user=pq3.pguser(),
+ sslmode="disable",
+ )
+
+ def factory(**kwargs):
+ nonlocal client
+
+ if client is None:
+ opts = dict(default_opts)
+ opts.update(kwargs)
+
+ # The server_socket is already listening, so the client thread can
+ # be safely started; it'll block on the connection until we accept.
+ client = ClientHandshake(**opts)
+ client.start()
+
+ sock, _ = server_socket.accept()
+ return sock, client
+
+ yield factory
+
+ if client is not None:
+ client.check_completed()
+
+
+@pytest.fixture
+def conn(accept):
+ """
+ Returns an accepted, wrapped pq3 connection to a psycopg2 client. The socket
+ will be closed when the test finishes, and the client will be checked for a
+ cleanly completed handshake.
+ """
+ sock, client = accept()
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ yield conn
diff --git a/src/test/python/client/test_client.py b/src/test/python/client/test_client.py
new file mode 100644
index 0000000000..8372376ede
--- /dev/null
+++ b/src/test/python/client/test_client.py
@@ -0,0 +1,186 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import sys
+
+import psycopg2
+import pytest
+from cryptography.hazmat.primitives import hashes, hmac
+
+import pq3
+
+from .test_oauth import alt_patterns
+
+
+def finish_handshake(conn):
+ """
+ Sends the AuthenticationOK message and the standard opening salvo of server
+ messages, then asserts that the client immediately sends a Terminate message
+ to close the connection cleanly.
+ """
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.OK)
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"client_encoding", value=b"UTF-8")
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"DateStyle", value=b"ISO, MDY")
+ pq3.send(conn, pq3.types.ReadyForQuery, status=b"I")
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.Terminate
+
+
+def test_handshake(conn):
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ finish_handshake(conn)
+
+
+def test_aborted_connection(accept):
+ """
+ Make sure the client correctly reports an early close during handshakes.
+ """
+ sock, client = accept()
+ sock.close()
+
+ expected = alt_patterns(
+ "server closed the connection unexpectedly",
+ # On some platforms, ECONNABORTED gets set instead.
+ "Software caused connection abort",
+ )
+ with pytest.raises(psycopg2.OperationalError, match=expected):
+ client.check_completed()
+
+
+#
+# SCRAM-SHA-256 (see RFC 5802: https://tools.ietf.org/html/rfc5802)
+#
+
+
+@pytest.fixture
+def password():
+ """
+ Returns a password for use by both client and server.
+ """
+ # TODO: parameterize this with passwords that require SASLprep.
+ return "secret"
+
+
+@pytest.fixture
+def pwconn(accept, password):
+ """
+ Like the conn fixture, but uses a password in the connection.
+ """
+ sock, client = accept(password=password)
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ yield conn
+
+
+def sha256(data):
+ """The H(str) function from Section 2.2."""
+ digest = hashes.Hash(hashes.SHA256())
+ digest.update(data)
+ return digest.finalize()
+
+
+def hmac_256(key, data):
+ """The HMAC(key, str) function from Section 2.2."""
+ h = hmac.HMAC(key, hashes.SHA256())
+ h.update(data)
+ return h.finalize()
+
+
+def xor(a, b):
+ """The XOR operation from Section 2.2."""
+ res = bytearray(a)
+ for i, byte in enumerate(b):
+ res[i] ^= byte
+ return bytes(res)
+
+
+def h_i(data, salt, i):
+ """The Hi(str, salt, i) function from Section 2.2."""
+ assert i > 0
+
+ acc = hmac_256(data, salt + b"\x00\x00\x00\x01")
+ last = acc
+ i -= 1
+
+ while i:
+ u = hmac_256(data, last)
+ acc = xor(acc, u)
+
+ last = u
+ i -= 1
+
+ return acc
+
+
+def test_scram(pwconn, password):
+ startup = pq3.recv1(pwconn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ pwconn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"SCRAM-SHA-256", b""],
+ )
+
+ # Get the client-first-message.
+ pkt = pq3.recv1(pwconn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ initial = pq3.SASLInitialResponse.parse(pkt.payload)
+ assert initial.name == b"SCRAM-SHA-256"
+
+ c_bind, authzid, c_name, c_nonce = initial.data.split(b",")
+ assert c_bind == b"n" # no channel bindings on a plaintext connection
+ assert authzid == b"" # we don't support authzid currently
+ assert c_name == b"n=" # libpq doesn't honor the GS2 username
+ assert c_nonce.startswith(b"r=")
+
+ # Send the server-first-message.
+ salt = b"12345"
+ iterations = 2
+
+ s_nonce = c_nonce + b"somenonce"
+ s_salt = b"s=" + base64.b64encode(salt)
+ s_iterations = b"i=%d" % iterations
+
+ msg = b",".join([s_nonce, s_salt, s_iterations])
+ pq3.send(pwconn, pq3.types.AuthnRequest, type=pq3.authn.SASLContinue, body=msg)
+
+ # Get the client-final-message.
+ pkt = pq3.recv1(pwconn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ c_bind_final, c_nonce_final, c_proof = pkt.payload.split(b",")
+ assert c_bind_final == b"c=" + base64.b64encode(c_bind + b"," + authzid + b",")
+ assert c_nonce_final == s_nonce
+
+ # Calculate what the client proof should be.
+ salted_password = h_i(password.encode("ascii"), salt, iterations)
+ client_key = hmac_256(salted_password, b"Client Key")
+ stored_key = sha256(client_key)
+
+ auth_message = b",".join(
+ [c_name, c_nonce, s_nonce, s_salt, s_iterations, c_bind_final, c_nonce_final]
+ )
+ client_signature = hmac_256(stored_key, auth_message)
+ client_proof = xor(client_key, client_signature)
+
+ expected = b"p=" + base64.b64encode(client_proof)
+ assert c_proof == expected
+
+ # Send the correct server signature.
+ server_key = hmac_256(salted_password, b"Server Key")
+ server_signature = hmac_256(server_key, auth_message)
+
+ s_verify = b"v=" + base64.b64encode(server_signature)
+ pq3.send(pwconn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal, body=s_verify)
+
+ # Done!
+ finish_handshake(pwconn)
diff --git a/src/test/python/client/test_oauth.py b/src/test/python/client/test_oauth.py
new file mode 100644
index 0000000000..aee8ecdf5e
--- /dev/null
+++ b/src/test/python/client/test_oauth.py
@@ -0,0 +1,1720 @@
+#
+# Copyright 2021 VMware, Inc.
+# Portions Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import collections
+import ctypes
+import http.server
+import json
+import logging
+import os
+import platform
+import secrets
+import sys
+import threading
+import time
+import traceback
+import types
+import urllib.parse
+from numbers import Number
+
+import psycopg2
+import pytest
+
+import pq3
+
+from .conftest import BLOCKING_TIMEOUT
+
+# The client tests need libpq to have been compiled with OAuth support; skip
+# them otherwise.
+pytestmark = pytest.mark.skipif(
+ os.getenv("with_oauth") == "none",
+ reason="OAuth client tests require --with-oauth support",
+)
+
+if platform.system() == "Darwin":
+ libpq = ctypes.cdll.LoadLibrary("libpq.5.dylib")
+elif platform.system() == "Windows":
+ pass # TODO
+else:
+ libpq = ctypes.cdll.LoadLibrary("libpq.so.5")
+
+
+def finish_handshake(conn):
+ """
+ Sends the AuthenticationOK message and the standard opening salvo of server
+ messages, then asserts that the client immediately sends a Terminate message
+ to close the connection cleanly.
+ """
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.OK)
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"client_encoding", value=b"UTF-8")
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"DateStyle", value=b"ISO, MDY")
+ pq3.send(conn, pq3.types.ReadyForQuery, status=b"I")
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.Terminate
+
+
+#
+# OAUTHBEARER (see RFC 7628: https://tools.ietf.org/html/rfc7628)
+#
+
+
+def start_oauth_handshake(conn):
+ """
+ Negotiates an OAUTHBEARER SASL challenge. Returns the client's initial
+ response data.
+ """
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn, pq3.types.AuthnRequest, type=pq3.authn.SASL, body=[b"OAUTHBEARER", b""]
+ )
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ initial = pq3.SASLInitialResponse.parse(pkt.payload)
+ assert initial.name == b"OAUTHBEARER"
+
+ return initial.data
+
+
+def get_auth_value(initial):
+ """
+ Finds the auth value (e.g. "Bearer somedata..." in the client's initial SASL
+ response.
+ """
+ kvpairs = initial.split(b"\x01")
+ assert kvpairs[0] == b"n,," # no channel binding or authzid
+ assert kvpairs[2] == b"" # ends with an empty kvpair
+ assert kvpairs[3] == b"" # ...and there's nothing after it
+ assert len(kvpairs) == 4
+
+ key, value = kvpairs[1].split(b"=", 2)
+ assert key == b"auth"
+
+ return value
+
+
+def xtest_oauth_success(conn): # TODO
+ initial = start_oauth_handshake(conn)
+
+ auth = get_auth_value(initial)
+ assert auth.startswith(b"Bearer ")
+
+ # Accept the token. TODO actually validate
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+class RawResponse(str):
+ """
+ Returned by registered endpoint callbacks to take full control of the
+ response. Usually, return values are converted to JSON; a RawResponse body
+ will be passed to the client as-is, allowing endpoint implementations to
+ issue invalid JSON.
+ """
+
+ pass
+
+
+class OpenIDProvider(threading.Thread):
+ """
+ A thread that runs a mock OpenID provider server.
+ """
+
+ def __init__(self, *, port):
+ super().__init__()
+
+ self.exception = None
+
+ addr = ("", port)
+ self.server = self._Server(addr, self._Handler)
+
+ # TODO: allow HTTPS only, somehow
+ oauth = self._OAuthState()
+ oauth.host = f"localhost:{port}"
+ oauth.issuer = f"http://localhost:{port}"
+
+ # The following endpoints are required to be advertised by providers,
+ # even though our chosen client implementation does not actually make
+ # use of them.
+ oauth.register_endpoint(
+ "authorization_endpoint", "POST", "/authorize", self._authorization_handler
+ )
+ oauth.register_endpoint("jwks_uri", "GET", "/keys", self._jwks_handler)
+
+ self.server.oauth = oauth
+
+ def run(self):
+ try:
+ # XXX socketserver.serve_forever() has a serious architectural
+ # issue: its select loop wakes up every `poll_interval` seconds to
+ # see if the server is shutting down. The default, 500 ms, only lets
+ # us run two tests every second. But the faster we go, the more CPU
+ # we burn unnecessarily...
+ self.server.serve_forever(poll_interval=0.01)
+ except Exception as e:
+ self.exception = e
+
+ def stop(self, timeout=BLOCKING_TIMEOUT):
+ """
+ Shuts down the server and joins its thread. Raises an exception if the
+ thread could not be joined, or if it threw an exception itself. Must
+ only be called once, after start().
+ """
+ self.server.shutdown()
+ self.join(timeout)
+
+ if self.is_alive():
+ raise TimeoutError("client thread did not handshake within the timeout")
+ elif self.exception:
+ e = self.exception
+ raise e
+
+ class _OAuthState(object):
+ def __init__(self):
+ self.endpoint_paths = {}
+ self._endpoints = {}
+
+ # Provide a standard discovery document by default; tests can
+ # override it.
+ self.register_endpoint(
+ None,
+ "GET",
+ "/.well-known/openid-configuration",
+ self._default_discovery_handler,
+ )
+
+ def register_endpoint(self, name, method, path, func):
+ if method not in self._endpoints:
+ self._endpoints[method] = {}
+
+ self._endpoints[method][path] = func
+
+ if name is not None:
+ self.endpoint_paths[name] = path
+
+ def endpoint(self, method, path):
+ if method not in self._endpoints:
+ return None
+
+ return self._endpoints[method].get(path)
+
+ def _default_discovery_handler(self, headers, params):
+ doc = {
+ "issuer": self.issuer,
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ "grant_types_supported": [
+ "urn:ietf:params:oauth:grant-type:device_code"
+ ],
+ }
+
+ for name, path in self.endpoint_paths.items():
+ doc[name] = self.issuer + path
+
+ return 200, doc
+
+ class _Server(http.server.HTTPServer):
+ def handle_error(self, request, addr):
+ self.shutdown_request(request)
+ raise
+
+ @staticmethod
+ def _jwks_handler(headers, params):
+ return 200, {"keys": []}
+
+ @staticmethod
+ def _authorization_handler(headers, params):
+ # We don't actually want this to be called during these tests -- we
+ # should be using the device authorization endpoint instead.
+ assert (
+ False
+ ), "authorization handler called instead of device authorization handler"
+
+ class _Handler(http.server.BaseHTTPRequestHandler):
+ timeout = BLOCKING_TIMEOUT
+
+ def _handle(self, *, params=None, handler=None):
+ oauth = self.server.oauth
+ assert self.headers["Host"] == oauth.host
+
+ if handler is None:
+ handler = oauth.endpoint(self.command, self.path)
+ assert (
+ handler is not None
+ ), f"no registered endpoint for {self.command} {self.path}"
+
+ result = handler(self.headers, params)
+
+ if len(result) == 2:
+ headers = {"Content-Type": "application/json"}
+ code, resp = result
+ else:
+ code, headers, resp = result
+
+ self.send_response(code)
+ for h, v in headers.items():
+ self.send_header(h, v)
+ self.end_headers()
+
+ if resp is not None:
+ if not isinstance(resp, RawResponse):
+ resp = json.dumps(resp)
+ resp = resp.encode("utf-8")
+ self.wfile.write(resp)
+
+ self.close_connection = True
+
+ def do_GET(self):
+ self._handle()
+
+ def _request_body(self):
+ length = self.headers["Content-Length"]
+
+ # Handle only an explicit content-length.
+ assert length is not None
+ length = int(length)
+
+ return self.rfile.read(length).decode("utf-8")
+
+ def do_POST(self):
+ assert self.headers["Content-Type"] == "application/x-www-form-urlencoded"
+
+ body = self._request_body()
+ params = urllib.parse.parse_qs(body)
+
+ self._handle(params=params)
+
+
+@pytest.fixture
+def openid_provider(unused_tcp_port_factory):
+ """
+ A fixture that returns the OAuth state of a running OpenID provider server. The
+ server will be stopped when the fixture is torn down.
+ """
+ thread = OpenIDProvider(port=unused_tcp_port_factory())
+ thread.start()
+
+ try:
+ yield thread.server.oauth
+ finally:
+ thread.stop()
+
+
+#
+# PQAuthDataHook implementation, matching libpq.h
+#
+
+
+PQAUTHDATA_PROMPT_OAUTH_DEVICE = 0
+PQAUTHDATA_OAUTH_BEARER_TOKEN = 1
+
+PGRES_POLLING_FAILED = 0
+PGRES_POLLING_READING = 1
+PGRES_POLLING_WRITING = 2
+PGRES_POLLING_OK = 3
+
+
+class PQPromptOAuthDevice(ctypes.Structure):
+ _fields_ = [
+ ("verification_uri", ctypes.c_char_p),
+ ("user_code", ctypes.c_char_p),
+ ]
+
+
+class PQOAuthBearerRequest(ctypes.Structure):
+ pass
+
+
+PQOAuthBearerRequest._fields_ = [
+ ("openid_configuration", ctypes.c_char_p),
+ ("scope", ctypes.c_char_p),
+ (
+ "async_",
+ ctypes.CFUNCTYPE(
+ ctypes.c_int,
+ ctypes.c_void_p,
+ ctypes.POINTER(PQOAuthBearerRequest),
+ ctypes.POINTER(ctypes.c_int),
+ ),
+ ),
+ (
+ "cleanup",
+ ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(PQOAuthBearerRequest)),
+ ),
+ ("token", ctypes.c_char_p),
+ ("user", ctypes.c_void_p),
+]
+
+
+@pytest.fixture
+def auth_data_cb():
+ """
+ Tracks calls to the libpq authdata hook. The yielded object contains a calls
+ member that records the data sent to the hook. If a test needs to perform
+ custom actions during a call, it can set the yielded object's impl callback;
+ beware that the callback takes place on a different thread.
+
+ This is done differently from the other callback implementations on purpose.
+ For the others, we can declare test-specific callbacks and have them perform
+ direct assertions on the data they receive. But that won't work for a C
+ callback, because there's no way for us to bubble up the assertion through
+ libpq. Instead, this mock-style approach is taken, where we just record the
+ calls and let the test examine them later.
+ """
+
+ class _Call:
+ pass
+
+ class _cb(object):
+ def __init__(self):
+ self.calls = []
+
+ cb = _cb()
+ cb.impl = None
+
+ # The callback will occur on a different thread, so protect the cb object.
+ cb_lock = threading.Lock()
+
+ @ctypes.CFUNCTYPE(ctypes.c_int, ctypes.c_byte, ctypes.c_void_p, ctypes.c_void_p)
+ def auth_data_cb(typ, pgconn, data):
+ handle_by_default = 0 # does an implementation have to be provided?
+
+ if typ == PQAUTHDATA_PROMPT_OAUTH_DEVICE:
+ cls = PQPromptOAuthDevice
+ handle_by_default = 1
+ elif typ == PQAUTHDATA_OAUTH_BEARER_TOKEN:
+ cls = PQOAuthBearerRequest
+ else:
+ return 0
+
+ call = _Call()
+ call.type = typ
+
+ # The lifetime of the underlying data being pointed to doesn't
+ # necessarily match the lifetime of the Python object, so we can't
+ # reference a Structure's fields after returning. Explicitly copy the
+ # contents over, field by field.
+ data = ctypes.cast(data, ctypes.POINTER(cls))
+ for name, _ in cls._fields_:
+ setattr(call, name, getattr(data.contents, name))
+
+ with cb_lock:
+ cb.calls.append(call)
+
+ if cb.impl:
+ # Pass control back to the test.
+ try:
+ return cb.impl(typ, pgconn, data.contents)
+ except Exception:
+ # This can't escape into the C stack, but we can fail the flow
+ # and hope the traceback gives us enough detail.
+ logging.error(
+ "Exception during authdata hook callback:\n"
+ + traceback.format_exc()
+ )
+ return -1
+
+ return handle_by_default
+
+ libpq.PQsetAuthDataHook(auth_data_cb)
+ try:
+ yield cb
+ finally:
+ # The callback is about to go out of scope, so make sure libpq is
+ # disconnected from it. (We wouldn't want to accidentally influence
+ # later tests anyway.)
+ libpq.PQsetAuthDataHook(None)
+
+
+@pytest.mark.parametrize("secret", [None, "", "hunter2"])
+@pytest.mark.parametrize("scope", [None, "", "openid email"])
+@pytest.mark.parametrize("retries", [0, 1])
+@pytest.mark.parametrize(
+ "asynchronous",
+ [
+ pytest.param(False, id="synchronous"),
+ pytest.param(True, id="asynchronous"),
+ ],
+)
+def test_oauth_with_explicit_issuer(
+ accept, openid_provider, asynchronous, retries, scope, secret, auth_data_cb
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ oauth_client_secret=secret,
+ oauth_scope=scope,
+ async_=asynchronous,
+ )
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+ verification_url = "https://example.com/device"
+
+ access_token = secrets.token_urlsafe()
+
+ def check_client_authn(headers, params):
+ if not secret:
+ assert params["client_id"] == [client_id]
+ return
+
+ # Require the client to use Basic authn; request-body credentials are
+ # NOT RECOMMENDED (RFC 6749, Sec. 2.3.1).
+ assert "Authorization" in headers
+
+ method, creds = headers["Authorization"].split()
+ assert method == "Basic"
+
+ expected = f"{client_id}:{secret}"
+ assert base64.b64decode(creds) == expected.encode("ascii")
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ check_client_authn(headers, params)
+
+ if scope:
+ assert params["scope"] == [scope]
+ else:
+ assert "scope" not in params
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": verification_url,
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ attempts = 0
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ check_client_authn(headers, params)
+
+ assert params["grant_type"] == ["urn:ietf:params:oauth:grant-type:device_code"]
+ assert params["device_code"] == [device_code]
+
+ now = time.monotonic()
+
+ with retry_lock:
+ nonlocal attempts
+
+ # If the test wants to force the client to retry, return an
+ # authorization_pending response and decrement the retry count.
+ if attempts < retries:
+ attempts += 1
+ return 400, {"error": "authorization_pending"}
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+ if retries:
+ # Finally, make sure that the client prompted the user once with the
+ # expected authorization URL and user code.
+ assert len(auth_data_cb.calls) == 2
+
+ # First call should have been for a custom flow, which we ignored.
+ assert auth_data_cb.calls[0].type == PQAUTHDATA_OAUTH_BEARER_TOKEN
+
+ # Second call is for our user prompt.
+ call = auth_data_cb.calls[1]
+ assert call.type == PQAUTHDATA_PROMPT_OAUTH_DEVICE
+ assert call.verification_uri.decode() == verification_url
+ assert call.user_code.decode() == user_code
+
+
+def expect_disconnected_handshake(sock):
+ """
+ Helper for any tests that expect the client to disconnect immediately after
+ being sent the OAUTHBEARER SASL method. Generally speaking, this requires
+ the client to have an oauth_issuer set so that it doesn't try to go through
+ discovery.
+ """
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake.
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"OAUTHBEARER", b""],
+ )
+
+ # The client should disconnect at this point.
+ assert not conn.read()
+
+
+def test_oauth_requires_client_id(accept, openid_provider):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ # Do not set a client ID; this should cause a client error after the
+ # server asks for OAUTHBEARER and the client tries to contact the
+ # issuer.
+ )
+
+ expect_disconnected_handshake(sock)
+
+ expected_error = "no oauth_client_id is set"
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.slow
+@pytest.mark.parametrize("error_code", ["authorization_pending", "slow_down"])
+@pytest.mark.parametrize("retries", [1, 2])
+def test_oauth_retry_interval(accept, openid_provider, retries, error_code):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id="some-id",
+ )
+
+ expected_retry_interval = 1
+ access_token = secrets.token_urlsafe()
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ resp = {
+ "device_code": "my-device-code",
+ "user_code": "my-user-code",
+ "interval": expected_retry_interval,
+ "verification_uri": "https://example.com",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ attempts = 0
+ last_retry = None
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ now = time.monotonic()
+
+ with retry_lock:
+ nonlocal attempts, last_retry, expected_retry_interval
+
+ # Make sure the retry interval is being respected by the client.
+ if last_retry is not None:
+ interval = now - last_retry
+ assert interval >= expected_retry_interval
+
+ last_retry = now
+
+ # If the test wants to force the client to retry, return the desired
+ # error response and decrement the retry count.
+ if attempts < retries:
+ attempts += 1
+
+ # A slow_down code requires the client to additionally increase
+ # its interval by five seconds.
+ if error_code == "slow_down":
+ expected_retry_interval += 5
+
+ return 400, {"error": error_code}
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+@pytest.fixture
+def self_pipe():
+ """
+ Yields a pipe fd pair.
+ """
+
+ class _Pipe:
+ pass
+
+ p = _Pipe()
+ p.readfd, p.writefd = os.pipe()
+
+ try:
+ yield p
+ finally:
+ os.close(p.readfd)
+ os.close(p.writefd)
+
+
+@pytest.mark.parametrize("scope", [None, "", "openid email"])
+@pytest.mark.parametrize(
+ "retries",
+ [
+ -1, # no async callback
+ 0, # async callback immediately returns token
+ 1, # async callback waits on altsock once
+ 2, # async callback waits on altsock twice
+ ],
+)
+@pytest.mark.parametrize(
+ "asynchronous",
+ [
+ pytest.param(False, id="synchronous"),
+ pytest.param(True, id="asynchronous"),
+ ],
+)
+def test_user_defined_flow(
+ accept, auth_data_cb, self_pipe, scope, retries, asynchronous
+):
+ issuer = "http://localhost"
+ discovery_uri = issuer + "/.well-known/openid-configuration"
+ access_token = secrets.token_urlsafe()
+
+ sock, _ = accept(
+ oauth_issuer=issuer,
+ oauth_client_id="some-id",
+ oauth_scope=scope,
+ async_=asynchronous,
+ )
+
+ # Track callbacks.
+ attempts = 0
+ wakeup_called = False
+ cleanup_calls = 0
+ lock = threading.Lock()
+
+ def wakeup():
+ """Writes a byte to the wakeup pipe."""
+ nonlocal wakeup_called
+ with lock:
+ wakeup_called = True
+ os.write(self_pipe.writefd, b"\0")
+
+ def get_token(pgconn, request, p_altsock):
+ """
+ Async token callback. While attempts < retries, libpq will be instructed
+ to wait on the self_pipe. When attempts == retries, the token will be
+ set.
+
+ Note that assertions and exceptions raised here are allowed but not very
+ helpful, since they can't bubble through the libpq stack to be collected
+ by the test suite. Try not to rely too heavily on them.
+ """
+ # Make sure libpq passed our user data through.
+ assert request.user == 42
+
+ with lock:
+ nonlocal attempts, wakeup_called
+
+ if attempts:
+ # If we've already started the timer, we shouldn't get a
+ # call back before it trips.
+ assert wakeup_called, "authdata hook was called before the timer"
+
+ # Drain the wakeup byte.
+ os.read(self_pipe.readfd, 1)
+
+ if attempts < retries:
+ attempts += 1
+
+ # Wake up the client in a little bit of time.
+ wakeup_called = False
+ threading.Timer(0.1, wakeup).start()
+
+ # Tell libpq to wait on the other end of the wakeup pipe.
+ p_altsock[0] = self_pipe.readfd
+ return PGRES_POLLING_READING
+
+ # Done!
+ request.token = access_token.encode()
+ return PGRES_POLLING_OK
+
+ @ctypes.CFUNCTYPE(
+ ctypes.c_int,
+ ctypes.c_void_p,
+ ctypes.POINTER(PQOAuthBearerRequest),
+ ctypes.POINTER(ctypes.c_int),
+ )
+ def get_token_wrapper(pgconn, p_request, p_altsock):
+ """
+ Translation layer between C and Python for the async callback.
+ Assertions and exceptions will be swallowed at the boundary, so make
+ sure they don't escape here.
+ """
+ try:
+ return get_token(pgconn, p_request.contents, p_altsock)
+ except Exception:
+ logging.error("Exception during async callback:\n" + traceback.format_exc())
+ return PGRES_POLLING_FAILED
+
+ @ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(PQOAuthBearerRequest))
+ def cleanup(pgconn, p_request):
+ """
+ Should be called exactly once per connection.
+ """
+ nonlocal cleanup_calls
+ with lock:
+ cleanup_calls += 1
+
+ def bearer_hook(typ, pgconn, request):
+ """
+ Implementation of the PQAuthDataHook, which either sets up an async
+ callback or returns the token directly, depending on the value of
+ retries.
+
+ As above, try not to rely too much on assertions/exceptions here.
+ """
+ assert typ == PQAUTHDATA_OAUTH_BEARER_TOKEN
+ request.cleanup = cleanup
+
+ if retries < 0:
+ # Special case: return a token immediately without a callback.
+ request.token = access_token.encode()
+ return 1
+
+ # Tell libpq to call us back.
+ request.async_ = get_token_wrapper
+ request.user = ctypes.c_void_p(42) # will be checked in the callback
+ return 1
+
+ auth_data_cb.impl = bearer_hook
+
+ # Now drive the server side.
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in our custom callback
+ # being invoked to fetch the token.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+ # Check the data provided to the hook.
+ assert len(auth_data_cb.calls) == 1
+
+ call = auth_data_cb.calls[0]
+ assert call.type == PQAUTHDATA_OAUTH_BEARER_TOKEN
+ assert call.openid_configuration.decode() == discovery_uri
+ assert call.scope == (None if scope is None else scope.encode())
+
+ # Make sure we cleaned up after ourselves.
+ assert cleanup_calls == 1
+
+
+def alt_patterns(*patterns):
+ """
+ Just combines multiple alternative regexes into one. It's not very efficient
+ but IMO it's easier to read and maintain.
+ """
+ pat = ""
+
+ for p in patterns:
+ if pat:
+ pat += "|"
+ pat += f"({p})"
+
+ return pat
+
+
+@pytest.mark.parametrize(
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
+ (
+ 400,
+ {
+ "error": "invalid_client",
+ "error_description": "client authentication failed",
+ },
+ ),
+ r"failed to obtain device authorization: client authentication failed \(invalid_client\)",
+ id="authentication failure with description",
+ ),
+ pytest.param(
+ (400, {"error": "invalid_request"}),
+ r"failed to obtain device authorization: \(invalid_request\)",
+ id="invalid request without description",
+ ),
+ pytest.param(
+ (400, {}),
+ r'failed to parse token error response: field "error" is missing',
+ id="broken error response",
+ ),
+ pytest.param(
+ (200, RawResponse(r'{ "interval": 3.5.8 }')),
+ r"failed to parse device authorization: Token .* is invalid",
+ id="non-numeric interval",
+ ),
+ pytest.param(
+ (200, RawResponse(r'{ "interval": 08 }')),
+ r"failed to parse device authorization: Token .* is invalid",
+ id="invalid numeric interval",
+ ),
+ ],
+)
+def test_oauth_device_authorization_failures(
+ accept, openid_provider, failure_mode, error_pattern
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ )
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ return failure_mode
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert False, "token endpoint was invoked unexpectedly"
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+Missing = object() # sentinel for test_oauth_device_authorization_bad_json()
+
+
+@pytest.mark.parametrize(
+ "bad_value",
+ [
+ pytest.param({"device_code": 3}, id="object"),
+ pytest.param([1, 2, 3], id="array"),
+ pytest.param("some string", id="string"),
+ pytest.param(4, id="numeric"),
+ pytest.param(False, id="boolean"),
+ pytest.param(None, id="null"),
+ pytest.param(Missing, id="missing"),
+ ],
+)
+@pytest.mark.parametrize(
+ "field_name,ok_type,required",
+ [
+ ("device_code", str, True),
+ ("user_code", str, True),
+ ("verification_uri", str, True),
+ ("interval", int, False),
+ ],
+)
+def test_oauth_device_authorization_bad_json_schema(
+ accept, openid_provider, field_name, ok_type, required, bad_value
+):
+ # To make the test matrix easy, just skip the tests that aren't actually
+ # interesting (field of the correct type, missing optional field).
+ if bad_value is Missing and not required:
+ pytest.skip("not interesting: optional field")
+ elif type(bad_value) == ok_type: # not isinstance(), because bool is an int
+ pytest.skip("not interesting: correct type")
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=secrets.token_hex(),
+ )
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ # Begin with an acceptable base response...
+ resp = {
+ "device_code": "my-device-code",
+ "user_code": "my-user-code",
+ "interval": 0,
+ "verification_uri": "https://example.com",
+ "expires_in": 5,
+ }
+
+ # ...then tweak it so the client fails.
+ if bad_value is Missing:
+ del resp[field_name]
+ else:
+ resp[field_name] = bad_value
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert False, "token endpoint was invoked unexpectedly"
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ if bad_value is Missing:
+ error_pattern = f'field "{field_name}" is missing'
+ elif ok_type == str:
+ error_pattern = f'field "{field_name}" must be a string'
+ elif ok_type == int:
+ error_pattern = f'field "{field_name}" must be a number'
+ else:
+ assert False, "update error_pattern for new failure mode"
+
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
+ (
+ 400,
+ {
+ "error": "expired_token",
+ "error_description": "the device code has expired",
+ },
+ ),
+ r"failed to obtain access token: the device code has expired \(expired_token\)",
+ id="expired token with description",
+ ),
+ pytest.param(
+ (400, {"error": "access_denied"}),
+ r"failed to obtain access token: \(access_denied\)",
+ id="access denied without description",
+ ),
+ pytest.param(
+ (400, {}),
+ r'failed to parse token error response: field "error" is missing',
+ id="empty error response",
+ ),
+ pytest.param(
+ (200, {}, {}),
+ r"failed to parse access token response: no content type was provided",
+ id="missing content type",
+ ),
+ pytest.param(
+ (200, {"Content-Type": "text/plain"}, {}),
+ r"failed to parse access token response: unexpected content type",
+ id="wrong content type",
+ ),
+ ],
+)
+@pytest.mark.parametrize("retries", [0, 1])
+def test_oauth_token_failures(
+ accept, openid_provider, retries, failure_mode, error_pattern
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ )
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ assert params["client_id"] == [client_id]
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": "https://example.com/device",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ retry_lock = threading.Lock()
+ final_sent = False
+
+ def token_endpoint(headers, params):
+ with retry_lock:
+ nonlocal retries, final_sent
+
+ # If the test wants to force the client to retry, return an
+ # authorization_pending response and decrement the retry count.
+ if retries > 0:
+ retries -= 1
+ return 400, {"error": "authorization_pending"}
+
+ # We should only return our failure_mode response once; any further
+ # requests indicate that the client isn't correctly bailing out.
+ assert not final_sent, "client continued after token error"
+
+ final_sent = True
+
+ return failure_mode
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "bad_value",
+ [
+ pytest.param({"device_code": 3}, id="object"),
+ pytest.param([1, 2, 3], id="array"),
+ pytest.param("some string", id="string"),
+ pytest.param(4, id="numeric"),
+ pytest.param(False, id="boolean"),
+ pytest.param(None, id="null"),
+ pytest.param(Missing, id="missing"),
+ ],
+)
+@pytest.mark.parametrize(
+ "field_name,ok_type,required",
+ [
+ ("access_token", str, True),
+ ("token_type", str, True),
+ ],
+)
+def test_oauth_token_bad_json_schema(
+ accept, openid_provider, field_name, ok_type, required, bad_value
+):
+ # To make the test matrix easy, just skip the tests that aren't actually
+ # interesting (field of the correct type, missing optional field).
+ if bad_value is Missing and not required:
+ pytest.skip("not interesting: optional field")
+ elif type(bad_value) == ok_type: # not isinstance(), because bool is an int
+ pytest.skip("not interesting: correct type")
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=secrets.token_hex(),
+ )
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ resp = {
+ "device_code": "my-device-code",
+ "user_code": "my-user-code",
+ "interval": 0,
+ "verification_uri": "https://example.com",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ # Begin with an acceptable base response...
+ resp = {
+ "access_token": secrets.token_urlsafe(),
+ "token_type": "bearer",
+ }
+
+ # ...then tweak it so the client fails.
+ if bad_value is Missing:
+ del resp[field_name]
+ else:
+ resp[field_name] = bad_value
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ error_pattern = "failed to parse access token response: "
+ if bad_value is Missing:
+ error_pattern += f'field "{field_name}" is missing'
+ elif ok_type == str:
+ error_pattern += f'field "{field_name}" must be a string'
+ elif ok_type == int:
+ error_pattern += f'field "{field_name}" must be a number'
+ else:
+ assert False, "update error_pattern for new failure mode"
+
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize("scope", [None, "openid email"])
+@pytest.mark.parametrize(
+ "base_response",
+ [
+ {"status": "invalid_token"},
+ {"extra_object": {"key": "value"}, "status": "invalid_token"},
+ {"extra_object": {"status": 1}, "status": "invalid_token"},
+ ],
+)
+def test_oauth_discovery(accept, openid_provider, base_response, scope):
+ sock, client = accept(oauth_client_id=secrets.token_hex())
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+ verification_url = "https://example.com/device"
+
+ access_token = secrets.token_urlsafe()
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ if scope:
+ assert params["scope"] == [scope]
+ else:
+ assert "scope" not in params
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": verification_url,
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert params["grant_type"] == ["urn:ietf:params:oauth:grant-type:device_code"]
+ assert params["device_code"] == [device_code]
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # For discovery, the client should send an empty auth header. See
+ # RFC 7628, Sec. 4.3.
+ auth = get_auth_value(initial)
+ assert auth == b""
+
+ # We will fail the first SASL exchange. First return a link to the
+ # discovery document, pointing to the test provider server.
+ resp = dict(base_response)
+
+ discovery_uri = f"{openid_provider.issuer}/.well-known/openid-configuration"
+ resp["openid-configuration"] = discovery_uri
+
+ if scope:
+ resp["scope"] = scope
+
+ resp = json.dumps(resp)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASLContinue,
+ body=resp.encode("ascii"),
+ )
+
+ # Per RFC, the client is required to send a dummy ^A response.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+ assert pkt.payload == b"\x01"
+
+ # Now fail the SASL exchange.
+ pq3.send(
+ conn,
+ pq3.types.ErrorResponse,
+ fields=[
+ b"SFATAL",
+ b"C28000",
+ b"Mdoesn't matter",
+ b"",
+ ],
+ )
+
+ # The client will connect to us a second time, using the parameters we sent
+ # it.
+ sock, _ = accept()
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+@pytest.mark.parametrize(
+ "response,expected_error",
+ [
+ pytest.param(
+ "abcde",
+ 'Token "abcde" is invalid',
+ id="bad JSON: invalid syntax",
+ ),
+ pytest.param(
+ '"abcde"',
+ "top-level element must be an object",
+ id="bad JSON: top-level element is a string",
+ ),
+ pytest.param(
+ "[]",
+ "top-level element must be an object",
+ id="bad JSON: top-level element is an array",
+ ),
+ pytest.param(
+ "{}",
+ "server sent error response without a status",
+ id="bad JSON: no status member",
+ ),
+ pytest.param(
+ '{ "status": null }',
+ 'field "status" must be a string',
+ id="bad JSON: null status member",
+ ),
+ pytest.param(
+ '{ "status": 0 }',
+ 'field "status" must be a string',
+ id="bad JSON: int status member",
+ ),
+ pytest.param(
+ '{ "status": [ "bad" ] }',
+ 'field "status" must be a string',
+ id="bad JSON: array status member",
+ ),
+ pytest.param(
+ '{ "status": { "bad": "bad" } }',
+ 'field "status" must be a string',
+ id="bad JSON: object status member",
+ ),
+ pytest.param(
+ '{ "nested": { "status": "bad" } }',
+ "server sent error response without a status",
+ id="bad JSON: nested status",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token" ',
+ "The input string ended unexpectedly",
+ id="bad JSON: unterminated object",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token" } { }',
+ 'Expected end of input, but found "{"',
+ id="bad JSON: trailing data",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "openid-configuration": 1 }',
+ 'field "openid-configuration" must be a string',
+ id="bad JSON: int openid-configuration member",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "openid-configuration": 1 }',
+ 'field "openid-configuration" must be a string',
+ id="bad JSON: int openid-configuration member",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "scope": 1 }',
+ 'field "scope" must be a string',
+ id="bad JSON: int scope member",
+ ),
+ ],
+)
+def test_oauth_discovery_server_error(accept, response, expected_error):
+ sock, client = accept(oauth_client_id=secrets.token_hex())
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # Fail the SASL exchange with an invalid JSON response.
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASLContinue,
+ body=response.encode("utf-8"),
+ )
+
+ # The client should disconnect, so the socket is closed here. (If
+ # the client doesn't disconnect, it will report a different error
+ # below and the test will fail.)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "bad_response,expected_error",
+ [
+ pytest.param(
+ (200, {"Content-Type": "text/plain"}, {}),
+ r'failed to parse OpenID discovery document: unexpected content type "text/plain"',
+ id="not JSON",
+ ),
+ pytest.param(
+ (200, {}, {}),
+ r"failed to parse OpenID discovery document: no content type was provided",
+ id="no Content-Type",
+ ),
+ pytest.param(
+ (204, {}, None),
+ r"failed to fetch OpenID discovery document: unexpected response code 204",
+ id="no content",
+ ),
+ pytest.param(
+ (301, {"Location": "https://localhost/"}, None),
+ r"failed to fetch OpenID discovery document: unexpected response code 301",
+ id="redirection",
+ ),
+ pytest.param(
+ (404, {}),
+ r"failed to fetch OpenID discovery document: unexpected response code 404",
+ id="not found",
+ ),
+ pytest.param(
+ (200, RawResponse("blah\x00blah")),
+ r"failed to parse OpenID discovery document: response contains embedded NULLs",
+ id="NULL bytes in document",
+ ),
+ pytest.param(
+ (200, 123),
+ r"failed to parse OpenID discovery document: top-level element must be an object",
+ id="scalar at top level",
+ ),
+ pytest.param(
+ (200, []),
+ r"failed to parse OpenID discovery document: top-level element must be an object",
+ id="array at top level",
+ ),
+ pytest.param(
+ (200, RawResponse("{")),
+ r"failed to parse OpenID discovery document.* input string ended unexpectedly",
+ id="unclosed object",
+ ),
+ pytest.param(
+ (200, RawResponse(r'{ "hello": ] }')),
+ r"failed to parse OpenID discovery document.* Expected JSON value",
+ id="bad array",
+ ),
+ pytest.param(
+ (200, {"issuer": 123}),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="non-string issuer",
+ ),
+ pytest.param(
+ (200, {"issuer": ["something"]}),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="issuer array",
+ ),
+ pytest.param(
+ (200, {"issuer": {}}),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="issuer object",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": 123}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="scalar grant types field",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": {}}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="object grant types field",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": [123]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="non-string grant types",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": ["something", 123]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="non-string grant types later in the list",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": ["something", {}]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="object grant types later in the list",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": ["something", ["something"]]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="embedded array grant types later in the list",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "grant_types_supported": ["something"],
+ "token_endpoint": "https://example.com/",
+ "issuer": 123,
+ },
+ ),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="non-string issuer after other valid fields",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "ignored": {"grant_types_supported": 123, "token_endpoint": 123},
+ "issuer": 123,
+ },
+ ),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="non-string issuer after other ignored fields",
+ ),
+ pytest.param(
+ (200, {"token_endpoint": "https://example.com/"}),
+ r'failed to parse OpenID discovery document: field "issuer" is missing',
+ id="missing issuer",
+ ),
+ pytest.param(
+ (200, {"issuer": "https://example.com/"}),
+ r'failed to parse OpenID discovery document: field "token_endpoint" is missing',
+ id="missing token endpoint",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "issuer": "https://example.com",
+ "token_endpoint": "https://example.com/token",
+ "device_authorization_endpoint": "https://example.com/dev",
+ },
+ ),
+ r'cannot run OAuth device authorization: issuer "https://example.com" does not support device code grants',
+ id="missing device code grants",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "issuer": "https://example.com",
+ "token_endpoint": "https://example.com/token",
+ "grant_types_supported": [
+ "urn:ietf:params:oauth:grant-type:device_code"
+ ],
+ },
+ ),
+ r'cannot run OAuth device authorization: issuer "https://example.com" does not provide a device authorization endpoint',
+ id="missing device_authorization_endpoint",
+ ),
+ #
+ # Exercise HTTP-level failures by breaking the protocol. Note that the
+ # error messages here are implementation-dependent.
+ #
+ pytest.param(
+ (1000, {}),
+ r"failed to fetch OpenID discovery document: Unsupported protocol \(.*\)",
+ id="invalid HTTP response code",
+ ),
+ pytest.param(
+ (200, {"Content-Length": -1}, {}),
+ r"failed to fetch OpenID discovery document: Weird server reply \(.*Content-Length.*\)",
+ id="bad HTTP Content-Length",
+ ),
+ ],
+)
+def test_oauth_discovery_provider_failure(
+ accept, openid_provider, bad_response, expected_error
+):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=secrets.token_hex(),
+ )
+
+ def failing_discovery_handler(headers, params):
+ return bad_response
+
+ openid_provider.register_endpoint(
+ None,
+ "GET",
+ "/.well-known/openid-configuration",
+ failing_discovery_handler,
+ )
+
+ expect_disconnected_handshake(sock)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "sasl_err,resp_type,resp_payload,expected_error",
+ [
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.ErrorResponse,
+ dict(
+ fields=[b"SFATAL", b"C28000", b"Mexpected error message", b""],
+ ),
+ "expected error message",
+ id="standard server error: invalid_request",
+ ),
+ pytest.param(
+ {"status": "invalid_token"},
+ pq3.types.ErrorResponse,
+ dict(
+ fields=[b"SFATAL", b"C28000", b"Mexpected error message", b""],
+ ),
+ "expected error message",
+ id="standard server error: invalid_token without discovery URI",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASLContinue, body=b""),
+ "server sent additional OAuth data",
+ id="broken server: additional challenge after error",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASLFinal),
+ "server sent additional OAuth data",
+ id="broken server: SASL success after error",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASL, body=[b"OAUTHBEARER", b""]),
+ "duplicate SASL authentication request",
+ id="broken server: SASL reinitialization after error",
+ ),
+ ],
+)
+def test_oauth_server_error(accept, sasl_err, resp_type, resp_payload, expected_error):
+ sock, client = accept()
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ start_oauth_handshake(conn)
+
+ # Ignore the client data. Return an error "challenge".
+ resp = json.dumps(sasl_err)
+ resp = resp.encode("utf-8")
+
+ pq3.send(
+ conn, pq3.types.AuthnRequest, type=pq3.authn.SASLContinue, body=resp
+ )
+
+ # Per RFC, the client is required to send a dummy ^A response.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+ assert pkt.payload == b"\x01"
+
+ # Now fail the SASL exchange (in either a valid way, or an invalid
+ # one, depending on the test).
+ pq3.send(conn, resp_type, **resp_payload)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
diff --git a/src/test/python/conftest.py b/src/test/python/conftest.py
new file mode 100644
index 0000000000..1a73865ee4
--- /dev/null
+++ b/src/test/python/conftest.py
@@ -0,0 +1,34 @@
+#
+# Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import os
+
+import pytest
+
+
+def pytest_addoption(parser):
+ """
+ Adds custom command line options to py.test. We add one to signal temporary
+ Postgres instance creation for the server tests.
+
+ Per pytest documentation, this must live in the top level test directory.
+ """
+ parser.addoption(
+ "--temp-instance",
+ metavar="DIR",
+ help="create a temporary Postgres instance in DIR",
+ )
+
+
+@pytest.fixture(scope="session", autouse=True)
+def _check_PG_TEST_EXTRA(request):
+ """
+ Automatically skips the whole suite if PG_TEST_EXTRA doesn't contain
+ 'python'. pytestmark doesn't seem to work in a top-level conftest.py, so
+ I've made this an autoused fixture instead.
+ """
+ extra_tests = os.getenv("PG_TEST_EXTRA", "").split()
+ if "python" not in extra_tests:
+ pytest.skip("Potentially unsafe test 'python' not enabled in PG_TEST_EXTRA")
diff --git a/src/test/python/meson.build b/src/test/python/meson.build
new file mode 100644
index 0000000000..045526a43a
--- /dev/null
+++ b/src/test/python/meson.build
@@ -0,0 +1,46 @@
+# Copyright (c) 2023, PostgreSQL Global Development Group
+
+subdir('server')
+
+pytest_env = {
+ 'with_oauth': oauth_library,
+
+ # Point to the default database; the tests will create their own databases as
+ # needed.
+ 'PGDATABASE': 'postgres',
+
+ # Avoid the need for a Rust compiler on platforms without prebuilt wheels for
+ # pyca/cryptography.
+ 'CRYPTOGRAPHY_DONT_BUILD_RUST': '1',
+}
+
+# Some modules (psycopg2) need OpenSSL at compile time; for platforms where we
+# might have multiple implementations installed (macOS+brew), try to use the
+# same one that libpq is using.
+if ssl.found()
+ pytest_incdir = ssl.get_variable(pkgconfig: 'includedir', default_value: '')
+ if pytest_incdir != ''
+ pytest_env += { 'CPPFLAGS': '-I@0@'.format(pytest_incdir) }
+ endif
+
+ pytest_libdir = ssl.get_variable(pkgconfig: 'libdir', default_value: '')
+ if pytest_libdir != ''
+ pytest_env += { 'LDFLAGS': '-L@0@'.format(pytest_libdir) }
+ endif
+endif
+
+tests += {
+ 'name': 'python',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'pytest': {
+ 'requirements': meson.current_source_dir() / 'requirements.txt',
+ 'tests': [
+ './client',
+ './server',
+ './test_internals.py',
+ './test_pq3.py',
+ ],
+ 'env': pytest_env,
+ },
+}
diff --git a/src/test/python/pq3.py b/src/test/python/pq3.py
new file mode 100644
index 0000000000..ef809e288a
--- /dev/null
+++ b/src/test/python/pq3.py
@@ -0,0 +1,740 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import getpass
+import io
+import os
+import platform
+import ssl
+import sys
+import textwrap
+
+from construct import *
+
+import tls
+
+
+def protocol(major, minor):
+ """
+ Returns the protocol version, in integer format, corresponding to the given
+ major and minor version numbers.
+ """
+ return (major << 16) | minor
+
+
+# Startup
+
+StringList = GreedyRange(NullTerminated(GreedyBytes))
+
+
+class KeyValueAdapter(Adapter):
+ """
+ Turns a key-value store into a null-terminated list of null-terminated
+ strings, as presented on the wire in the startup packet.
+ """
+
+ def _encode(self, obj, context, path):
+ if isinstance(obj, list):
+ return obj
+
+ l = []
+
+ for k, v in obj.items():
+ if isinstance(k, str):
+ k = k.encode("utf-8")
+ l.append(k)
+
+ if isinstance(v, str):
+ v = v.encode("utf-8")
+ l.append(v)
+
+ l.append(b"")
+ return l
+
+ def _decode(self, obj, context, path):
+ # TODO: turn a list back into a dict
+ return obj
+
+
+KeyValues = KeyValueAdapter(StringList)
+
+_startup_payload = Switch(
+ this.proto,
+ {
+ protocol(3, 0): KeyValues,
+ },
+ default=GreedyBytes,
+)
+
+
+def _default_protocol(this):
+ try:
+ if isinstance(this.payload, (list, dict)):
+ return protocol(3, 0)
+ except AttributeError:
+ pass # no payload passed during build
+
+ return 0
+
+
+def _startup_payload_len(this):
+ """
+ The payload field has a fixed size based on the length of the packet. But
+ if the caller hasn't supplied an explicit length at build time, we have to
+ build the payload to figure out how long it is, which requires us to know
+ the length first... This function exists solely to break the cycle.
+ """
+ assert this._building, "_startup_payload_len() cannot be called during parsing"
+
+ try:
+ payload = this.payload
+ except AttributeError:
+ return 0 # no payload
+
+ if isinstance(payload, bytes):
+ # already serialized; just use the given length
+ return len(payload)
+
+ try:
+ proto = this.proto
+ except AttributeError:
+ proto = _default_protocol(this)
+
+ data = _startup_payload.build(payload, proto=proto)
+ return len(data)
+
+
+Startup = Struct(
+ "len" / Default(Int32sb, lambda this: _startup_payload_len(this) + 8),
+ "proto" / Default(Hex(Int32sb), _default_protocol),
+ "payload" / FixedSized(this.len - 8, Default(_startup_payload, b"")),
+)
+
+# Pq3
+
+
+# Adapted from construct.core.EnumIntegerString
+class EnumNamedByte:
+ def __init__(self, val, name):
+ self._val = val
+ self._name = name
+
+ def __int__(self):
+ return ord(self._val)
+
+ def __str__(self):
+ return "(enum) %s %r" % (self._name, self._val)
+
+ def __repr__(self):
+ return "EnumNamedByte(%r)" % self._val
+
+ def __eq__(self, other):
+ if isinstance(other, EnumNamedByte):
+ other = other._val
+ if not isinstance(other, bytes):
+ return NotImplemented
+
+ return self._val == other
+
+ def __hash__(self):
+ return hash(self._val)
+
+
+# Adapted from construct.core.Enum
+class ByteEnum(Adapter):
+ def __init__(self, **mapping):
+ super(ByteEnum, self).__init__(Byte)
+ self.namemapping = {k: EnumNamedByte(v, k) for k, v in mapping.items()}
+ self.decmapping = {v: EnumNamedByte(v, k) for k, v in mapping.items()}
+
+ def __getattr__(self, name):
+ if name in self.namemapping:
+ return self.decmapping[self.namemapping[name]]
+ raise AttributeError
+
+ def _decode(self, obj, context, path):
+ b = bytes([obj])
+ try:
+ return self.decmapping[b]
+ except KeyError:
+ return EnumNamedByte(b, "(unknown)")
+
+ def _encode(self, obj, context, path):
+ if isinstance(obj, int):
+ return obj
+ elif isinstance(obj, bytes):
+ return ord(obj)
+ return int(obj)
+
+
+types = ByteEnum(
+ ErrorResponse=b"E",
+ ReadyForQuery=b"Z",
+ Query=b"Q",
+ EmptyQueryResponse=b"I",
+ AuthnRequest=b"R",
+ PasswordMessage=b"p",
+ BackendKeyData=b"K",
+ CommandComplete=b"C",
+ ParameterStatus=b"S",
+ DataRow=b"D",
+ Terminate=b"X",
+)
+
+
+authn = Enum(
+ Int32ub,
+ OK=0,
+ SASL=10,
+ SASLContinue=11,
+ SASLFinal=12,
+)
+
+
+_authn_body = Switch(
+ this.type,
+ {
+ authn.OK: Terminated,
+ authn.SASL: StringList,
+ },
+ default=GreedyBytes,
+)
+
+
+def _data_len(this):
+ assert this._building, "_data_len() cannot be called during parsing"
+
+ if not hasattr(this, "data") or this.data is None:
+ return -1
+
+ return len(this.data)
+
+
+# The protocol reuses the PasswordMessage for several authentication response
+# types, and there's no good way to figure out which is which without keeping
+# state for the entire stream. So this is a separate Construct that can be
+# explicitly parsed/built by code that knows it's needed.
+SASLInitialResponse = Struct(
+ "name" / NullTerminated(GreedyBytes),
+ "len" / Default(Int32sb, lambda this: _data_len(this)),
+ "data"
+ / IfThenElse(
+ # Allow tests to explicitly pass an incorrect length during testing, by
+ # not enforcing a FixedSized during build. (The len calculation above
+ # defaults to the correct size.)
+ this._building,
+ Optional(GreedyBytes),
+ If(this.len != -1, Default(FixedSized(this.len, GreedyBytes), b"")),
+ ),
+ Terminated, # make sure the entire response is consumed
+)
+
+
+_column = FocusedSeq(
+ "data",
+ "len" / Default(Int32sb, lambda this: _data_len(this)),
+ "data" / If(this.len != -1, FixedSized(this.len, GreedyBytes)),
+)
+
+
+_payload_map = {
+ types.ErrorResponse: Struct("fields" / StringList),
+ types.ReadyForQuery: Struct("status" / Bytes(1)),
+ types.Query: Struct("query" / NullTerminated(GreedyBytes)),
+ types.EmptyQueryResponse: Terminated,
+ types.AuthnRequest: Struct("type" / authn, "body" / Default(_authn_body, b"")),
+ types.BackendKeyData: Struct("pid" / Int32ub, "key" / Hex(Int32ub)),
+ types.CommandComplete: Struct("tag" / NullTerminated(GreedyBytes)),
+ types.ParameterStatus: Struct(
+ "name" / NullTerminated(GreedyBytes), "value" / NullTerminated(GreedyBytes)
+ ),
+ types.DataRow: Struct("columns" / Default(PrefixedArray(Int16sb, _column), b"")),
+ types.Terminate: Terminated,
+}
+
+
+_payload = FocusedSeq(
+ "_payload",
+ "_payload"
+ / Switch(
+ this._.type,
+ _payload_map,
+ default=GreedyBytes,
+ ),
+ Terminated, # make sure every payload consumes the entire packet
+)
+
+
+def _payload_len(this):
+ """
+ See _startup_payload_len() for an explanation.
+ """
+ assert this._building, "_payload_len() cannot be called during parsing"
+
+ try:
+ payload = this.payload
+ except AttributeError:
+ return 0 # no payload
+
+ if isinstance(payload, bytes):
+ # already serialized; just use the given length
+ return len(payload)
+
+ data = _payload.build(payload, type=this.type)
+ return len(data)
+
+
+Pq3 = Struct(
+ "type" / types,
+ "len" / Default(Int32ub, lambda this: _payload_len(this) + 4),
+ "payload"
+ / IfThenElse(
+ # Allow tests to explicitly pass an incorrect length during testing, by
+ # not enforcing a FixedSized during build. (The len calculation above
+ # defaults to the correct size.)
+ this._building,
+ Optional(_payload),
+ FixedSized(this.len - 4, Default(_payload, b"")),
+ ),
+)
+
+
+# Environment
+
+
+def pghost():
+ return os.environ.get("PGHOST", default="localhost")
+
+
+def pgport():
+ return int(os.environ.get("PGPORT", default=5432))
+
+
+def pguser():
+ try:
+ return os.environ["PGUSER"]
+ except KeyError:
+ if platform.system() == "Windows":
+ # libpq defaults to GetUserName() on Windows.
+ return os.getlogin()
+ return getpass.getuser()
+
+
+def pgdatabase():
+ return os.environ.get("PGDATABASE", default="postgres")
+
+
+# Connections
+
+
+def _hexdump_translation_map():
+ """
+ For hexdumps. Translates any unprintable or non-ASCII bytes into '.'.
+ """
+ input = bytearray()
+
+ for i in range(128):
+ c = chr(i)
+
+ if not c.isprintable():
+ input += bytes([i])
+
+ input += bytes(range(128, 256))
+
+ return bytes.maketrans(input, b"." * len(input))
+
+
+class _DebugStream(object):
+ """
+ Wraps a file-like object and adds hexdumps of the read and write data. Call
+ end_packet() on a _DebugStream to write the accumulated hexdumps to the
+ output stream, along with the packet that was sent.
+ """
+
+ _translation_map = _hexdump_translation_map()
+
+ def __init__(self, stream, out=sys.stdout):
+ """
+ Creates a new _DebugStream wrapping the given stream (which must have
+ been created by wrap()). All attributes not provided by the _DebugStream
+ are delegated to the wrapped stream. out is the text stream to which
+ hexdumps are written.
+ """
+ self.raw = stream
+ self._out = out
+ self._rbuf = io.BytesIO()
+ self._wbuf = io.BytesIO()
+
+ def __getattr__(self, name):
+ return getattr(self.raw, name)
+
+ def __setattr__(self, name, value):
+ if name in ("raw", "_out", "_rbuf", "_wbuf"):
+ return object.__setattr__(self, name, value)
+
+ setattr(self.raw, name, value)
+
+ def read(self, *args, **kwargs):
+ buf = self.raw.read(*args, **kwargs)
+
+ self._rbuf.write(buf)
+ return buf
+
+ def write(self, b):
+ self._wbuf.write(b)
+ return self.raw.write(b)
+
+ def recv(self, *args):
+ buf = self.raw.recv(*args)
+
+ self._rbuf.write(buf)
+ return buf
+
+ def _flush(self, buf, prefix):
+ width = 16
+ hexwidth = width * 3 - 1
+
+ count = 0
+ buf.seek(0)
+
+ while True:
+ line = buf.read(16)
+
+ if not line:
+ if count:
+ self._out.write("\n") # separate the output block with a newline
+ return
+
+ self._out.write("%s %04X:\t" % (prefix, count))
+ self._out.write("%*s\t" % (-hexwidth, line.hex(" ")))
+ self._out.write(line.translate(self._translation_map).decode("ascii"))
+ self._out.write("\n")
+
+ count += 16
+
+ def print_debug(self, obj, *, prefix=""):
+ contents = ""
+ if obj is not None:
+ contents = str(obj)
+
+ for line in contents.splitlines():
+ self._out.write("%s%s\n" % (prefix, line))
+
+ self._out.write("\n")
+
+ def flush_debug(self, *, prefix=""):
+ self._flush(self._rbuf, prefix + "<")
+ self._rbuf = io.BytesIO()
+
+ self._flush(self._wbuf, prefix + ">")
+ self._wbuf = io.BytesIO()
+
+ def end_packet(self, pkt, *, read=False, prefix="", indent=" "):
+ """
+ Marks the end of a logical "packet" of data. A string representation of
+ pkt will be printed, and the debug buffers will be flushed with an
+ indent. All lines can be optionally prefixed.
+
+ If read is True, the packet representation is written after the debug
+ buffers; otherwise the default of False (meaning write) causes the
+ packet representation to be dumped first. This is meant to capture the
+ logical flow of layer translation.
+ """
+ write = not read
+
+ if write:
+ self.print_debug(pkt, prefix=prefix + "> ")
+
+ self.flush_debug(prefix=prefix + indent)
+
+ if read:
+ self.print_debug(pkt, prefix=prefix + "< ")
+
+
+@contextlib.contextmanager
+def wrap(socket, *, debug_stream=None):
+ """
+ Transforms a raw socket into a connection that can be used for Construct
+ building and parsing. The return value is a context manager and can be used
+ in a with statement.
+ """
+ # It is critical that buffering be disabled here, so that we can still
+ # manipulate the raw socket without desyncing the stream.
+ with socket.makefile("rwb", buffering=0) as sfile:
+ # Expose the original socket's recv() on the SocketIO object we return.
+ def recv(self, *args):
+ return socket.recv(*args)
+
+ sfile.recv = recv.__get__(sfile)
+
+ conn = sfile
+ if debug_stream:
+ conn = _DebugStream(conn, debug_stream)
+
+ try:
+ yield conn
+ finally:
+ if debug_stream:
+ conn.flush_debug(prefix="? ")
+
+
+def _send(stream, cls, obj):
+ debugging = hasattr(stream, "flush_debug")
+ out = io.BytesIO()
+
+ # Ideally we would build directly to the passed stream, but because we need
+ # to reparse the generated output for the debugging case, build to an
+ # intermediate BytesIO and send it instead.
+ cls.build_stream(obj, out)
+ buf = out.getvalue()
+
+ stream.write(buf)
+ if debugging:
+ pkt = cls.parse(buf)
+ stream.end_packet(pkt)
+
+ stream.flush()
+
+
+def send(stream, packet_type, payload_data=None, **payloadkw):
+ """
+ Sends a packet on the given pq3 connection. type is the pq3.types member
+ that should be assigned to the packet. If payload_data is given, it will be
+ used as the packet payload; otherwise the key/value pairs in payloadkw will
+ be the payload contents.
+ """
+ data = payloadkw
+
+ if payload_data is not None:
+ if payloadkw:
+ raise ValueError(
+ "payload_data and payload keywords may not be used simultaneously"
+ )
+
+ data = payload_data
+
+ _send(stream, Pq3, dict(type=packet_type, payload=data))
+
+
+def send_startup(stream, proto=None, **kwargs):
+ """
+ Sends a startup packet on the given pq3 connection. In most cases you should
+ use the handshake functions instead, which will do this for you.
+
+ By default, a protocol version 3 packet will be sent. This can be overridden
+ with the proto parameter.
+ """
+ pkt = {}
+
+ if proto is not None:
+ pkt["proto"] = proto
+ if kwargs:
+ pkt["payload"] = kwargs
+
+ _send(stream, Startup, pkt)
+
+
+def recv1(stream, *, cls=Pq3):
+ """
+ Receives a single pq3 packet from the given stream and returns it.
+ """
+ resp = cls.parse_stream(stream)
+
+ debugging = hasattr(stream, "flush_debug")
+ if debugging:
+ stream.end_packet(resp, read=True)
+
+ return resp
+
+
+def handshake(stream, **kwargs):
+ """
+ Performs a libpq v3 startup handshake. kwargs should contain the key/value
+ parameters to send to the server in the startup packet.
+ """
+ # Send our startup parameters.
+ send_startup(stream, **kwargs)
+
+ # Receive and dump packets until the server indicates it's ready for our
+ # first query.
+ while True:
+ resp = recv1(stream)
+ if resp is None:
+ raise RuntimeError("server closed connection during handshake")
+
+ if resp.type == types.ReadyForQuery:
+ return
+ elif resp.type == types.ErrorResponse:
+ raise RuntimeError(
+ f"received error response from peer: {resp.payload.fields!r}"
+ )
+
+
+# TLS
+
+
+class _TLSStream(object):
+ """
+ A file-like object that performs TLS encryption/decryption on a wrapped
+ stream. Differs from ssl.SSLSocket in that we have full visibility and
+ control over the TLS layer.
+ """
+
+ def __init__(self, stream, context):
+ self._stream = stream
+ self._debugging = hasattr(stream, "flush_debug")
+
+ self._in = ssl.MemoryBIO()
+ self._out = ssl.MemoryBIO()
+ self._ssl = context.wrap_bio(self._in, self._out)
+
+ def handshake(self):
+ try:
+ self._pump(lambda: self._ssl.do_handshake())
+ finally:
+ self._flush_debug(prefix="? ")
+
+ def read(self, *args):
+ return self._pump(lambda: self._ssl.read(*args))
+
+ def write(self, *args):
+ return self._pump(lambda: self._ssl.write(*args))
+
+ def _decode(self, buf):
+ """
+ Attempts to decode a buffer of TLS data into a packet representation
+ that can be printed.
+
+ TODO: handle buffers (and record fragments) that don't align with packet
+ boundaries.
+ """
+ end = len(buf)
+ bio = io.BytesIO(buf)
+
+ ret = io.StringIO()
+
+ while bio.tell() < end:
+ record = tls.Plaintext.parse_stream(bio)
+
+ if ret.tell() > 0:
+ ret.write("\n")
+ ret.write("[Record] ")
+ ret.write(str(record))
+ ret.write("\n")
+
+ if record.type == tls.ContentType.handshake:
+ record_cls = tls.Handshake
+ else:
+ continue
+
+ innerlen = len(record.fragment)
+ inner = io.BytesIO(record.fragment)
+
+ while inner.tell() < innerlen:
+ msg = record_cls.parse_stream(inner)
+
+ indented = "[Message] " + str(msg)
+ indented = textwrap.indent(indented, " ")
+
+ ret.write("\n")
+ ret.write(indented)
+ ret.write("\n")
+
+ return ret.getvalue()
+
+ def flush(self):
+ if not self._out.pending:
+ self._stream.flush()
+ return
+
+ buf = self._out.read()
+ self._stream.write(buf)
+
+ if self._debugging:
+ pkt = self._decode(buf)
+ self._stream.end_packet(pkt, prefix=" ")
+
+ self._stream.flush()
+
+ def _pump(self, operation):
+ while True:
+ try:
+ return operation()
+ except (ssl.SSLWantReadError, ssl.SSLWantWriteError) as e:
+ want = e
+ self._read_write(want)
+
+ def _recv(self, maxsize):
+ buf = self._stream.recv(4096)
+ if not buf:
+ self._in.write_eof()
+ return
+
+ self._in.write(buf)
+
+ if not self._debugging:
+ return
+
+ pkt = self._decode(buf)
+ self._stream.end_packet(pkt, read=True, prefix=" ")
+
+ def _read_write(self, want):
+ # XXX This needs work. So many corner cases yet to handle. For one,
+ # doing blocking writes in flush may lead to distributed deadlock if the
+ # peer is already blocking on its writes.
+
+ if isinstance(want, ssl.SSLWantWriteError):
+ assert self._out.pending, "SSL backend wants write without data"
+
+ self.flush()
+
+ if isinstance(want, ssl.SSLWantReadError):
+ self._recv(4096)
+
+ def _flush_debug(self, prefix):
+ if not self._debugging:
+ return
+
+ self._stream.flush_debug(prefix=prefix)
+
+
+@contextlib.contextmanager
+def tls_handshake(stream, context):
+ """
+ Performs a TLS handshake over the given stream (which must have been created
+ via a call to wrap()), and returns a new stream which transparently tunnels
+ data over the TLS connection.
+
+ If the passed stream has debugging enabled, the returned stream will also
+ have debugging, using the same output IO.
+ """
+ debugging = hasattr(stream, "flush_debug")
+
+ # Send our startup parameters.
+ send_startup(stream, proto=protocol(1234, 5679))
+
+ # Look at the SSL response.
+ resp = stream.read(1)
+ if debugging:
+ stream.flush_debug(prefix=" ")
+
+ if resp == b"N":
+ raise RuntimeError("server does not support SSLRequest")
+ if resp != b"S":
+ raise RuntimeError(f"unexpected response of type {resp!r} during TLS startup")
+
+ tls = _TLSStream(stream, context)
+ tls.handshake()
+
+ if debugging:
+ tls = _DebugStream(tls, stream._out)
+
+ try:
+ yield tls
+ # TODO: teardown/unwrap the connection?
+ finally:
+ if debugging:
+ tls.flush_debug(prefix="? ")
diff --git a/src/test/python/pytest.ini b/src/test/python/pytest.ini
new file mode 100644
index 0000000000..ab7a6e7fb9
--- /dev/null
+++ b/src/test/python/pytest.ini
@@ -0,0 +1,4 @@
+[pytest]
+
+markers =
+ slow: mark test as slow
diff --git a/src/test/python/requirements.txt b/src/test/python/requirements.txt
new file mode 100644
index 0000000000..57ba1ced94
--- /dev/null
+++ b/src/test/python/requirements.txt
@@ -0,0 +1,9 @@
+black
+# cryptography 35.x and later add many platform/toolchain restrictions, beware
+cryptography~=3.4.8
+construct~=2.10.61
+isort~=5.6
+# TODO: update to psycopg[c] 3.1
+psycopg2~=2.9.7
+pytest~=7.3
+pytest-asyncio~=0.21.0
diff --git a/src/test/python/server/__init__.py b/src/test/python/server/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/src/test/python/server/conftest.py b/src/test/python/server/conftest.py
new file mode 100644
index 0000000000..c89f0756ae
--- /dev/null
+++ b/src/test/python/server/conftest.py
@@ -0,0 +1,141 @@
+#
+# Portions Copyright 2021 VMware, Inc.
+# Portions Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import collections
+import contextlib
+import os
+import shutil
+import socket
+import subprocess
+import sys
+
+import pytest
+
+import pq3
+
+BLOCKING_TIMEOUT = 2 # the number of seconds to wait for blocking calls
+
+
+def cleanup_prior_instance(datadir):
+ """
+ Clean up an existing data directory, but make sure it actually looks like a
+ data directory first. (Empty folders will remain untouched, since initdb can
+ populate them.)
+ """
+ required_entries = set(["base", "PG_VERSION", "postgresql.conf"])
+ empty = True
+
+ try:
+ with os.scandir(datadir) as entries:
+ for e in entries:
+ empty = False
+ required_entries.discard(e.name)
+
+ except FileNotFoundError:
+ return # nothing to clean up
+
+ if empty:
+ return # initdb can handle an empty datadir
+
+ if required_entries:
+ pytest.fail(
+ f"--temp-instance directory \"{datadir}\" is not empty and doesn't look like a data directory (missing {', '.join(required_entries)})"
+ )
+
+ # Okay, seems safe enough now.
+ shutil.rmtree(datadir)
+
+
+@pytest.fixture(scope="session")
+def postgres_instance(pytestconfig, unused_tcp_port_factory):
+ """
+ If --temp-instance has been passed to pytest, this fixture runs a temporary
+ Postgres instance on an available port. Otherwise, the fixture will attempt
+ to contact a running Postgres server on (PGHOST, PGPORT); dependent tests
+ will be skipped if the connection fails.
+
+ Yields a (host, port) tuple for connecting to the server.
+ """
+ PGInstance = collections.namedtuple("PGInstance", ["addr", "temporary"])
+
+ datadir = pytestconfig.getoption("temp_instance")
+ if datadir:
+ # We were told to create a temporary instance. Use pg_ctl to set it up
+ # on an unused port.
+ cleanup_prior_instance(datadir)
+ subprocess.run(["pg_ctl", "-D", datadir, "init"], check=True)
+
+ # The CI looks for *.log files to upload, so the file name here isn't
+ # completely arbitrary.
+ log = os.path.join(datadir, "postmaster.log")
+ port = unused_tcp_port_factory()
+
+ subprocess.run(
+ [
+ "pg_ctl",
+ "-D",
+ datadir,
+ "-l",
+ log,
+ "-o",
+ " ".join(
+ [
+ f"-c port={port}",
+ "-c listen_addresses=localhost",
+ "-c log_connections=on",
+ "-c shared_preload_libraries=oauthtest",
+ "-c oauth_validator_library=oauthtest",
+ ]
+ ),
+ "start",
+ ],
+ check=True,
+ )
+
+ yield ("localhost", port)
+
+ subprocess.run(["pg_ctl", "-D", datadir, "stop"], check=True)
+
+ else:
+ # Try to contact an already running server; skip the suite if we can't
+ # find one.
+ addr = (pq3.pghost(), pq3.pgport())
+
+ try:
+ with socket.create_connection(addr, timeout=BLOCKING_TIMEOUT):
+ pass
+ except ConnectionError as e:
+ pytest.skip(f"unable to connect to Postgres server at {addr}: {e}")
+
+ yield addr
+
+
+@pytest.fixture
+def connect(postgres_instance):
+ """
+ A factory fixture that, when called, returns a socket connected to a
+ Postgres server, wrapped in a pq3 connection. Dependent tests will be
+ skipped if no server is available.
+ """
+ addr = postgres_instance
+
+ # Set up an ExitStack to handle safe cleanup of all of the moving pieces.
+ with contextlib.ExitStack() as stack:
+
+ def conn_factory():
+ sock = socket.create_connection(addr, timeout=BLOCKING_TIMEOUT)
+
+ # Have ExitStack close our socket.
+ stack.enter_context(sock)
+
+ # Wrap the connection in a pq3 layer and have ExitStack clean it up
+ # too.
+ wrap_ctx = pq3.wrap(sock, debug_stream=sys.stdout)
+ conn = stack.enter_context(wrap_ctx)
+
+ return conn
+
+ yield conn_factory
diff --git a/src/test/python/server/meson.build b/src/test/python/server/meson.build
new file mode 100644
index 0000000000..85534b9cc9
--- /dev/null
+++ b/src/test/python/server/meson.build
@@ -0,0 +1,18 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+oauthtest_sources = files(
+ 'oauthtest.c',
+)
+
+if host_system == 'windows'
+ oauthtest_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'oauthtest',
+ '--FILEDESC', 'passthrough module to validate OAuth tests',
+ ])
+endif
+
+oauthtest = shared_module('oauthtest',
+ oauthtest_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += oauthtest
diff --git a/src/test/python/server/oauthtest.c b/src/test/python/server/oauthtest.c
new file mode 100644
index 0000000000..dbb8b8823c
--- /dev/null
+++ b/src/test/python/server/oauthtest.c
@@ -0,0 +1,118 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauthtest.c
+ * Test module for serverside OAuth token validation callbacks
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/test/python/server/oauthtest.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "libpq/oauth.h"
+#include "utils/guc.h"
+#include "utils/memutils.h"
+
+PG_MODULE_MAGIC;
+
+static void test_startup(ValidatorModuleState *state);
+static void test_shutdown(ValidatorModuleState *state);
+static ValidatorModuleResult * test_validate(ValidatorModuleState *state,
+ const char *token,
+ const char *role);
+
+static const OAuthValidatorCallbacks callbacks = {
+ .startup_cb = test_startup,
+ .shutdown_cb = test_shutdown,
+ .validate_cb = test_validate,
+};
+
+static char *expected_bearer = "";
+static bool set_authn_id = false;
+static char *authn_id = "";
+static bool reflect_role = false;
+
+void
+_PG_init(void)
+{
+ DefineCustomStringVariable("oauthtest.expected_bearer",
+ "Expected Bearer token for future connections",
+ NULL,
+ &expected_bearer,
+ "",
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("oauthtest.set_authn_id",
+ "Whether to set an authenticated identity",
+ NULL,
+ &set_authn_id,
+ false,
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+ DefineCustomStringVariable("oauthtest.authn_id",
+ "Authenticated identity to use for future connections",
+ NULL,
+ &authn_id,
+ "",
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("oauthtest.reflect_role",
+ "Ignore the bearer token; use the requested role as the authn_id",
+ NULL,
+ &reflect_role,
+ false,
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+
+ MarkGUCPrefixReserved("oauthtest");
+}
+
+const OAuthValidatorCallbacks *
+_PG_oauth_validator_module_init(void)
+{
+ return &callbacks;
+}
+
+static void
+test_startup(ValidatorModuleState *state)
+{
+}
+
+static void
+test_shutdown(ValidatorModuleState *state)
+{
+}
+
+static ValidatorModuleResult *
+test_validate(ValidatorModuleState *state, const char *token, const char *role)
+{
+ ValidatorModuleResult *res;
+
+ res = palloc0(sizeof(ValidatorModuleResult)); /* TODO: palloc context? */
+
+ if (reflect_role)
+ {
+ res->authorized = true;
+ res->authn_id = pstrdup(role); /* TODO: constify? */
+ }
+ else
+ {
+ if (*expected_bearer && !strcmp(token, expected_bearer))
+ res->authorized = true;
+ if (set_authn_id)
+ res->authn_id = authn_id;
+ }
+
+ return res;
+}
diff --git a/src/test/python/server/test_oauth.py b/src/test/python/server/test_oauth.py
new file mode 100644
index 0000000000..427ab063e6
--- /dev/null
+++ b/src/test/python/server/test_oauth.py
@@ -0,0 +1,1074 @@
+#
+# Copyright 2021 VMware, Inc.
+# Portions Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import contextlib
+import json
+import os
+import pathlib
+import platform
+import secrets
+import shlex
+import shutil
+import socket
+import struct
+from multiprocessing import shared_memory
+
+import psycopg2
+import pytest
+from construct import Container
+from psycopg2 import sql
+
+import pq3
+
+from .conftest import BLOCKING_TIMEOUT
+
+MAX_SASL_MESSAGE_LENGTH = 65535
+
+INVALID_AUTHORIZATION_ERRCODE = b"28000"
+PROTOCOL_VIOLATION_ERRCODE = b"08P01"
+FEATURE_NOT_SUPPORTED_ERRCODE = b"0A000"
+
+SHARED_MEM_NAME = "oauth-pytest"
+MAX_UINT16 = 2**16 - 1
+
+
+@contextlib.contextmanager
+def prepend_file(path, lines, *, suffix=".bak"):
+ """
+ A context manager that prepends a file on disk with the desired lines of
+ text. When the context manager is exited, the file will be restored to its
+ original contents.
+ """
+ # First make a backup of the original file.
+ bak = path + suffix
+ shutil.copy2(path, bak)
+
+ try:
+ # Write the new lines, followed by the original file content.
+ with open(path, "w") as new, open(bak, "r") as orig:
+ new.writelines(lines)
+ shutil.copyfileobj(orig, new)
+
+ # Return control to the calling code.
+ yield
+
+ finally:
+ # Put the backup back into place.
+ os.replace(bak, path)
+
+
+@pytest.fixture(scope="module")
+def oauth_ctx(postgres_instance):
+ """
+ Creates a database and user that use the oauth auth method. The context
+ object contains the dbname and user attributes as strings to be used during
+ connection, as well as the issuer and scope that have been set in the HBA
+ configuration.
+
+ This fixture assumes that the standard PG* environment variables point to a
+ server running on a local machine, and that the PGUSER has rights to create
+ databases and roles.
+ """
+ id = secrets.token_hex(4)
+
+ class Context:
+ dbname = "oauth_test_" + id
+
+ user = "oauth_user_" + id
+ punct_user = "oauth_\"'? ;&!_user_" + id # username w/ punctuation
+ map_user = "oauth_map_user_" + id
+ authz_user = "oauth_authz_user_" + id
+
+ issuer = "https://example.com/" + id
+ scope = "openid " + id
+
+ ctx = Context()
+ hba_lines = [
+ f'host {ctx.dbname} {ctx.map_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" map=oauth\n',
+ f'host {ctx.dbname} {ctx.authz_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" trust_validator_authz=1\n',
+ f'host {ctx.dbname} all samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}"\n',
+ ]
+ ident_lines = [r"oauth /^(.*)@example\.com$ \1"]
+
+ if platform.system() == "Windows":
+ # XXX why is 'samehost' not behaving as expected on Windows?
+ for l in list(hba_lines):
+ hba_lines.append(l.replace("samehost", "::1/128"))
+
+ host, port = postgres_instance
+ conn = psycopg2.connect(host=host, port=port)
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+
+ # Create our roles and database.
+ user = sql.Identifier(ctx.user)
+ punct_user = sql.Identifier(ctx.punct_user)
+ map_user = sql.Identifier(ctx.map_user)
+ authz_user = sql.Identifier(ctx.authz_user)
+ dbname = sql.Identifier(ctx.dbname)
+
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(punct_user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(map_user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(authz_user))
+ c.execute(sql.SQL("CREATE DATABASE {};").format(dbname))
+
+ # Replace pg_hba and pg_ident.
+ c.execute("SHOW hba_file;")
+ hba = c.fetchone()[0]
+
+ c.execute("SHOW ident_file;")
+ ident = c.fetchone()[0]
+
+ with prepend_file(hba, hba_lines), prepend_file(ident, ident_lines):
+ c.execute("SELECT pg_reload_conf();")
+
+ # Use the new database and user.
+ yield ctx
+
+ # Put things back the way they were.
+ c.execute("SELECT pg_reload_conf();")
+
+ c.execute(sql.SQL("DROP DATABASE {};").format(dbname))
+ c.execute(sql.SQL("DROP ROLE {};").format(authz_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(map_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(punct_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(user))
+
+
+@pytest.fixture()
+def conn(oauth_ctx, connect):
+ """
+ A convenience wrapper for connect(). The main purpose of this fixture is to
+ make sure oauth_ctx runs its setup code before the connection is made.
+ """
+ return connect()
+
+
+def bearer_token(*, size=16):
+ """
+ Generates a Bearer token using secrets.token_urlsafe(). The generated token
+ size in bytes may be specified; if unset, a small 16-byte token will be
+ generated.
+ """
+
+ if size % 4:
+ raise ValueError(f"requested token size {size} is not a multiple of 4")
+
+ token = secrets.token_urlsafe(size // 4 * 3)
+ assert len(token) == size
+
+ return token
+
+
+def begin_oauth_handshake(conn, oauth_ctx, *, user=None):
+ if user is None:
+ user = oauth_ctx.authz_user
+
+ pq3.send_startup(conn, user=user, database=oauth_ctx.dbname)
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.AuthnRequest
+
+ # The server should advertise exactly one mechanism.
+ assert resp.payload.type == pq3.authn.SASL
+ assert resp.payload.body == [b"OAUTHBEARER", b""]
+
+
+def send_initial_response(conn, *, auth=None, bearer=None):
+ """
+ Sends the OAUTHBEARER initial response on the connection, using the given
+ bearer token. Alternatively to a bearer token, the initial response's auth
+ field may be explicitly specified to test corner cases.
+ """
+ if bearer is not None and auth is not None:
+ raise ValueError("exactly one of the auth and bearer kwargs must be set")
+
+ if bearer is not None:
+ auth = b"Bearer " + bearer
+
+ if auth is None:
+ raise ValueError("exactly one of the auth and bearer kwargs must be set")
+
+ initial = pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"n,,\x01auth=" + auth + b"\x01\x01",
+ )
+ )
+ pq3.send(conn, pq3.types.PasswordMessage, initial)
+
+
+def expect_handshake_success(conn):
+ """
+ Validates that the server responds with an AuthnOK message, and then drains
+ the connection until a ReadyForQuery message is received.
+ """
+ resp = pq3.recv1(conn)
+
+ assert resp.type == pq3.types.AuthnRequest
+ assert resp.payload.type == pq3.authn.OK
+ assert not resp.payload.body
+
+ receive_until(conn, pq3.types.ReadyForQuery)
+
+
+def expect_handshake_failure(conn, oauth_ctx):
+ """
+ Performs the OAUTHBEARER SASL failure "handshake" and validates the server's
+ side of the conversation, including the final ErrorResponse.
+ """
+
+ # We expect a discovery "challenge" back from the server before the authn
+ # failure message.
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.AuthnRequest
+
+ req = resp.payload
+ assert req.type == pq3.authn.SASLContinue
+
+ body = json.loads(req.body)
+ assert body["status"] == "invalid_token"
+ assert body["scope"] == oauth_ctx.scope
+
+ expected_config = oauth_ctx.issuer + "/.well-known/openid-configuration"
+ assert body["openid-configuration"] == expected_config
+
+ # Send the dummy response to complete the failed handshake.
+ pq3.send(conn, pq3.types.PasswordMessage, b"\x01")
+ resp = pq3.recv1(conn)
+
+ err = ExpectedError(INVALID_AUTHORIZATION_ERRCODE, "bearer authentication failed")
+ err.match(resp)
+
+
+def receive_until(conn, type):
+ """
+ receive_until pulls packets off the pq3 connection until a packet with the
+ desired type is found, or an error response is received.
+ """
+ while True:
+ pkt = pq3.recv1(conn)
+
+ if pkt.type == type:
+ return pkt
+ elif pkt.type == pq3.types.ErrorResponse:
+ raise RuntimeError(
+ f"received error response from peer: {pkt.payload.fields!r}"
+ )
+
+
+@pytest.fixture()
+def setup_validator(postgres_instance):
+ """
+ A per-test fixture that sets up the test validator with expected behavior.
+ The setting will be reverted during teardown.
+ """
+ host, port = postgres_instance
+ conn = psycopg2.connect(host=host, port=port)
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+ prev = dict()
+
+ def setter(**gucs):
+ for guc, val in gucs.items():
+ # Save the previous value.
+ c.execute(sql.SQL("SHOW oauthtest.{};").format(sql.Identifier(guc)))
+ prev[guc] = c.fetchone()[0]
+
+ c.execute(
+ sql.SQL("ALTER SYSTEM SET oauthtest.{} TO %s;").format(
+ sql.Identifier(guc)
+ ),
+ (val,),
+ )
+ c.execute("SELECT pg_reload_conf();")
+
+ yield setter
+
+ # Restore the previous values.
+ for guc, val in prev.items():
+ c.execute(
+ sql.SQL("ALTER SYSTEM SET oauthtest.{} TO %s;").format(
+ sql.Identifier(guc)
+ ),
+ (val,),
+ )
+ c.execute("SELECT pg_reload_conf();")
+
+
+@pytest.mark.parametrize("token_len", [16, 1024, 4096])
+@pytest.mark.parametrize(
+ "auth_prefix",
+ [
+ b"Bearer ",
+ b"bearer ",
+ b"Bearer ",
+ ],
+)
+def test_oauth(setup_validator, connect, oauth_ctx, auth_prefix, token_len):
+ # Generate our bearer token with the desired length.
+ token = bearer_token(size=token_len)
+ setup_validator(expected_bearer=token)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ auth = auth_prefix + token.encode("ascii")
+ send_initial_response(conn, auth=auth)
+ expect_handshake_success(conn)
+
+ # Make sure that the server has not set an authenticated ID.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT system_user;")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ assert row.columns == [None]
+
+
+@pytest.mark.parametrize(
+ "token_value",
+ [
+ "abcdzA==",
+ "123456M=",
+ "x-._~+/x",
+ ],
+)
+def test_oauth_bearer_corner_cases(setup_validator, connect, oauth_ctx, token_value):
+ setup_validator(expected_bearer=token_value)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ send_initial_response(conn, bearer=token_value.encode("ascii"))
+
+ expect_handshake_success(conn)
+
+
+@pytest.mark.parametrize(
+ "user,authn_id,should_succeed",
+ [
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: ctx.user,
+ True,
+ id="validator authn: succeeds when authn_id == username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: None,
+ False,
+ id="validator authn: fails when authn_id is not set",
+ ),
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: ctx.authz_user,
+ False,
+ id="validator authn: fails when authn_id != username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: ctx.map_user + "@example.com",
+ True,
+ id="validator with map: succeeds when authn_id matches map",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: None,
+ False,
+ id="validator with map: fails when authn_id is not set",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: ctx.map_user + "@example.net",
+ False,
+ id="validator with map: fails when authn_id doesn't match map",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: None,
+ True,
+ id="validator authz: succeeds with no authn_id",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "",
+ True,
+ id="validator authz: succeeds with empty authn_id",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "postgres",
+ True,
+ id="validator authz: succeeds with basic username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "me@example.com",
+ True,
+ id="validator authz: succeeds with email address",
+ ),
+ ],
+)
+def test_oauth_authn_id(
+ setup_validator, connect, oauth_ctx, user, authn_id, should_succeed
+):
+ token = bearer_token()
+ authn_id = authn_id(oauth_ctx)
+
+ # Set up the validator appropriately.
+ gucs = dict(expected_bearer=token)
+ if authn_id is not None:
+ gucs["set_authn_id"] = True
+ gucs["authn_id"] = authn_id
+ setup_validator(**gucs)
+
+ conn = connect()
+ username = user(oauth_ctx)
+ begin_oauth_handshake(conn, oauth_ctx, user=username)
+ send_initial_response(conn, bearer=token.encode("ascii"))
+
+ if not should_succeed:
+ expect_handshake_failure(conn, oauth_ctx)
+ return
+
+ expect_handshake_success(conn)
+
+ # Check the reported authn_id.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT system_user;")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ expected = authn_id
+ if expected is not None:
+ expected = b"oauth:" + expected.encode("ascii")
+
+ row = resp.payload
+ assert row.columns == [expected]
+
+
+class ExpectedError(object):
+ def __init__(self, code, msg=None, detail=None):
+ self.code = code
+ self.msg = msg
+ self.detail = detail
+
+ # Protect against the footgun of an accidental empty string, which will
+ # "match" anything. If you don't want to match message or detail, just
+ # don't pass them.
+ if self.msg == "":
+ raise ValueError("msg must be non-empty or None")
+ if self.detail == "":
+ raise ValueError("detail must be non-empty or None")
+
+ def _getfield(self, resp, type):
+ """
+ Searches an ErrorResponse for a single field of the given type (e.g.
+ "M", "C", "D") and returns its value. Asserts if it doesn't find exactly
+ one field.
+ """
+ prefix = type.encode("ascii")
+ fields = [f for f in resp.payload.fields if f.startswith(prefix)]
+
+ assert len(fields) == 1
+ return fields[0][1:] # strip off the type byte
+
+ def match(self, resp):
+ """
+ Checks that the given response matches the expected code, message, and
+ detail (if given). The error code must match exactly. The expected
+ message and detail must be contained within the actual strings.
+ """
+ assert resp.type == pq3.types.ErrorResponse
+
+ code = self._getfield(resp, "C")
+ assert code == self.code
+
+ if self.msg:
+ msg = self._getfield(resp, "M")
+ expected = self.msg.encode("utf-8")
+ assert expected in msg
+
+ if self.detail:
+ detail = self._getfield(resp, "D")
+ expected = self.detail.encode("utf-8")
+ assert expected in detail
+
+
+def test_oauth_rejected_bearer(conn, oauth_ctx):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send a bearer token that doesn't match what the validator expects. It
+ # should fail the connection.
+ send_initial_response(conn, bearer=b"xxxxxx")
+
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.parametrize(
+ "bad_bearer",
+ [
+ b"Bearer ",
+ b"Bearer a===b",
+ b"Bearer hello!",
+ b"Bearer trailingspace ",
+ b"Bearer trailingtab\t",
+ b"Bearer me@example.com",
+ b"Beare abcd",
+ b" Bearer leadingspace",
+ b'OAuth realm="Example"',
+ b"",
+ ],
+)
+def test_oauth_invalid_bearer(setup_validator, connect, oauth_ctx, bad_bearer):
+ # Tell the validator to accept any token. This ensures that the invalid
+ # bearer tokens are rejected before the validation step.
+ setup_validator(reflect_role=True)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+ send_initial_response(conn, auth=bad_bearer)
+
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.slow
+@pytest.mark.parametrize(
+ "resp_type,resp,err",
+ [
+ pytest.param(
+ None,
+ None,
+ None,
+ marks=pytest.mark.slow,
+ id="no response (expect timeout)",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"hello",
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not send a kvsep response",
+ ),
+ id="bad dummy response",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"\x01\x01",
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not send a kvsep response",
+ ),
+ id="multiple kvseps",
+ ),
+ pytest.param(
+ pq3.types.Query,
+ dict(query=b""),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "expected SASL response"),
+ id="bad response message type",
+ ),
+ ],
+)
+def test_oauth_bad_response_to_error_challenge(conn, oauth_ctx, resp_type, resp, err):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send an empty auth initial response, which will force an authn failure.
+ send_initial_response(conn, auth=b"")
+
+ # We expect a discovery "challenge" back from the server before the authn
+ # failure message.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.AuthnRequest
+
+ req = pkt.payload
+ assert req.type == pq3.authn.SASLContinue
+
+ body = json.loads(req.body)
+ assert body["status"] == "invalid_token"
+
+ if resp_type is None:
+ # Do not send the dummy response. We should time out and not get a
+ # response from the server.
+ with pytest.raises(socket.timeout):
+ conn.read(1)
+
+ # Done with the test.
+ return
+
+ # Send the bad response.
+ pq3.send(conn, resp_type, resp)
+
+ # Make sure the server fails the connection correctly.
+ pkt = pq3.recv1(conn)
+ err.match(pkt)
+
+
+@pytest.mark.parametrize(
+ "type,payload,err",
+ [
+ pytest.param(
+ pq3.types.ErrorResponse,
+ dict(fields=[b""]),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "expected SASL response"),
+ id="error response in initial message",
+ ),
+ pytest.param(
+ None,
+ # Sending an actual 65k packet results in ECONNRESET on Windows, and
+ # it floods the tests' connection log uselessly, so just fake the
+ # length and send a smaller number of bytes.
+ dict(
+ type=pq3.types.PasswordMessage,
+ len=MAX_SASL_MESSAGE_LENGTH + 1,
+ payload=b"x" * 512,
+ ),
+ ExpectedError(
+ INVALID_AUTHORIZATION_ERRCODE, "bearer authentication failed"
+ ),
+ id="overlong initial response data",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"SCRAM-SHA-256")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE, "invalid SASL authentication mechanism"
+ ),
+ id="bad SASL mechanism selection",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", len=2, data=b"x")),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "insufficient data"),
+ id="SASL data underflow",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", len=0, data=b"x")),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "invalid message format"),
+ id="SASL data overflow",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "message is empty",
+ ),
+ id="empty",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"n,,\x01auth=\x01\x01\0")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "length does not match input length",
+ ),
+ id="contains null byte",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected channel-binding flag", # XXX this is a bit strange
+ ),
+ id="initial error response",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"p=tls-server-end-point,,\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "server does not support channel binding",
+ ),
+ id="uses channel binding",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"x,,\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected channel-binding flag",
+ ),
+ id="invalid channel binding specifier",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Comma expected",
+ ),
+ id="bad GS2 header: missing channel binding terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,a")),
+ ExpectedError(
+ FEATURE_NOT_SUPPORTED_ERRCODE,
+ "client uses authorization identity",
+ ),
+ id="bad GS2 header: authzid in use",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,b,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected attribute",
+ ),
+ id="bad GS2 header: extra attribute",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected attribute 0x00", # XXX this is a bit strange
+ ),
+ id="bad GS2 header: missing authzid terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Key-value separator expected",
+ ),
+ id="missing initial kvsep",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Key-value separator expected",
+ ),
+ id="missing initial kvsep",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "does not contain an auth value",
+ ),
+ id="missing auth value: empty key-value list",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01host=example.com\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "does not contain an auth value",
+ ),
+ id="missing auth value: other keys present",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01host=example.com")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "unterminated key/value pair",
+ ),
+ id="missing value terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not contain a final terminator",
+ ),
+ id="missing list terminator: empty list",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01auth=Bearer 0\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not contain a final terminator",
+ ),
+ id="missing list terminator: with auth value",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01auth=Bearer 0\x01\x01blah")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "additional data after the final terminator",
+ ),
+ id="additional key after terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01key\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "key without a value",
+ ),
+ id="key without value",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"y,,\x01auth=Bearer 0\x01auth=Bearer 1\x01\x01",
+ )
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "contains multiple auth values",
+ ),
+ id="multiple auth values",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"y,,\x01=\x01\x01",
+ )
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "empty key name",
+ ),
+ id="empty key",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"y,,\x01my key= \x01\x01",
+ )
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "invalid key name",
+ ),
+ id="whitespace in key name",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"y,,\x01key=a\x05b\x01\x01",
+ )
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "invalid value",
+ ),
+ id="junk in value",
+ ),
+ ],
+)
+def test_oauth_bad_initial_response(conn, oauth_ctx, type, payload, err):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # The server expects a SASL response; give it something else instead.
+ if type is not None:
+ # Build a new packet of the desired type.
+ if not isinstance(payload, dict):
+ payload = dict(payload_data=payload)
+ pq3.send(conn, type, **payload)
+ else:
+ # The test has a custom packet to send. (The only reason to do this is
+ # if the packet is corrupt or otherwise unbuildable/unparsable, so we
+ # don't use the standard pq3.send().)
+ conn.write(pq3.Pq3.build(payload))
+ conn.end_packet(Container(payload))
+
+ resp = pq3.recv1(conn)
+ err.match(resp)
+
+
+def test_oauth_empty_initial_response(setup_validator, connect, oauth_ctx):
+ token = bearer_token()
+ setup_validator(expected_bearer=token)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send an initial response without data.
+ initial = pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER"))
+ pq3.send(conn, pq3.types.PasswordMessage, initial)
+
+ # The server should respond with an empty challenge so we can send the data
+ # it wants.
+ pkt = pq3.recv1(conn)
+
+ assert pkt.type == pq3.types.AuthnRequest
+ assert pkt.payload.type == pq3.authn.SASLContinue
+ assert not pkt.payload.body
+
+ # Now send the initial data.
+ data = b"n,,\x01auth=Bearer " + token.encode("ascii") + b"\x01\x01"
+ pq3.send(conn, pq3.types.PasswordMessage, data)
+
+ # Server should now complete the handshake.
+ expect_handshake_success(conn)
+
+
+# TODO: see if there's a way to test this easily after the API switch
+def xtest_oauth_no_validator(setup_validator, oauth_ctx, connect):
+ # Clear out our validator command, then establish a new connection.
+ set_validator("")
+ conn = connect()
+
+ begin_oauth_handshake(conn, oauth_ctx)
+ send_initial_response(conn, bearer=bearer_token())
+
+ # The server should fail the connection.
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.parametrize(
+ "user",
+ [
+ pytest.param(
+ lambda ctx: ctx.user,
+ id="basic username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.punct_user,
+ id="'unsafe' characters are passed through correctly",
+ ),
+ ],
+)
+def test_oauth_validator_role(setup_validator, oauth_ctx, connect, user):
+ username = user(oauth_ctx)
+
+ # Tell the validator to reflect the PGUSER as the authenticated identity.
+ setup_validator(reflect_role=True)
+ conn = connect()
+
+ # Log in. Note that reflection ignores the bearer token.
+ begin_oauth_handshake(conn, oauth_ctx, user=username)
+ send_initial_response(conn, bearer=b"dontcare")
+ expect_handshake_success(conn)
+
+ # Check the user identity.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT system_user;")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ expected = b"oauth:" + username.encode("utf-8")
+ assert row.columns == [expected]
+
+
+@pytest.fixture
+def odd_oauth_ctx(postgres_instance, oauth_ctx):
+ """
+ Adds an HBA entry with messed up issuer/scope settings, to pin the server
+ behavior.
+
+ TODO: these should really be rejected in the HBA rather than passed through
+ by the server.
+ """
+ id = secrets.token_hex(4)
+
+ class Context:
+ user = oauth_ctx.user
+ dbname = oauth_ctx.dbname
+
+ # Both of these embedded double-quotes are invalid; they're prohibited
+ # in both URLs and OAuth scope identifiers.
+ issuer = oauth_ctx.issuer + '/"/'
+ scope = oauth_ctx.scope + ' quo"ted'
+
+ ctx = Context()
+ hba_issuer = ctx.issuer.replace('"', '""')
+ hba_scope = ctx.scope.replace('"', '""')
+ hba_lines = [
+ f'host {ctx.dbname} {ctx.user} samehost oauth issuer="{hba_issuer}" scope="{hba_scope}"\n',
+ ]
+
+ if platform.system() == "Windows":
+ # XXX why is 'samehost' not behaving as expected on Windows?
+ for l in list(hba_lines):
+ hba_lines.append(l.replace("samehost", "::1/128"))
+
+ host, port = postgres_instance
+ conn = psycopg2.connect(host=host, port=port)
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+
+ # Replace pg_hba. Note that it's already been replaced once by
+ # oauth_ctx, so use a different backup prefix in prepend_file().
+ c.execute("SHOW hba_file;")
+ hba = c.fetchone()[0]
+
+ with prepend_file(hba, hba_lines, suffix=".bak2"):
+ c.execute("SELECT pg_reload_conf();")
+
+ yield ctx
+
+ # Put things back the way they were.
+ c.execute("SELECT pg_reload_conf();")
+
+
+def test_odd_server_response(odd_oauth_ctx, connect):
+ """
+ Verifies that the server is correctly escaping the JSON in its failure
+ response.
+ """
+ conn = connect()
+ begin_oauth_handshake(conn, odd_oauth_ctx, user=odd_oauth_ctx.user)
+
+ # Send an empty auth initial response, which will force an authn failure.
+ send_initial_response(conn, auth=b"")
+
+ expect_handshake_failure(conn, odd_oauth_ctx)
diff --git a/src/test/python/server/test_server.py b/src/test/python/server/test_server.py
new file mode 100644
index 0000000000..02126dba79
--- /dev/null
+++ b/src/test/python/server/test_server.py
@@ -0,0 +1,21 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import pq3
+
+
+def test_handshake(connect):
+ """Basic sanity check."""
+ conn = connect()
+
+ pq3.handshake(conn, user=pq3.pguser(), database=pq3.pgdatabase())
+
+ pq3.send(conn, pq3.types.Query, query=b"")
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.EmptyQueryResponse
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.ReadyForQuery
diff --git a/src/test/python/test_internals.py b/src/test/python/test_internals.py
new file mode 100644
index 0000000000..dee4855fc0
--- /dev/null
+++ b/src/test/python/test_internals.py
@@ -0,0 +1,138 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import io
+
+from pq3 import _DebugStream
+
+
+def test_DebugStream_read():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+
+ stream = _DebugStream(under, out)
+
+ res = stream.read(5)
+ assert res == b"abcde"
+
+ res = stream.read(16)
+ assert res == b"fghijklmnopqrstu"
+
+ stream.flush_debug()
+
+ res = stream.read()
+ assert res == b"vwxyz"
+
+ stream.flush_debug()
+
+ expected = (
+ "< 0000:\t61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 70\tabcdefghijklmnop\n"
+ "< 0010:\t71 72 73 74 75 \tqrstu\n"
+ "\n"
+ "< 0000:\t76 77 78 79 7a \tvwxyz\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_write():
+ under = io.BytesIO()
+ out = io.StringIO()
+
+ stream = _DebugStream(under, out)
+
+ stream.write(b"\x00\x01\x02")
+ stream.flush()
+
+ assert under.getvalue() == b"\x00\x01\x02"
+
+ stream.write(b"\xc0\xc1\xc2")
+ stream.flush()
+
+ assert under.getvalue() == b"\x00\x01\x02\xc0\xc1\xc2"
+
+ stream.flush_debug()
+
+ expected = "> 0000:\t00 01 02 c0 c1 c2 \t......\n\n"
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_read_write():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+ stream = _DebugStream(under, out)
+
+ res = stream.read(5)
+ assert res == b"abcde"
+
+ stream.write(b"xxxxx")
+ stream.flush()
+
+ assert under.getvalue() == b"abcdexxxxxklmnopqrstuvwxyz"
+
+ res = stream.read(5)
+ assert res == b"klmno"
+
+ stream.write(b"xxxxx")
+ stream.flush()
+
+ assert under.getvalue() == b"abcdexxxxxklmnoxxxxxuvwxyz"
+
+ stream.flush_debug()
+
+ expected = (
+ "< 0000:\t61 62 63 64 65 6b 6c 6d 6e 6f \tabcdeklmno\n"
+ "\n"
+ "> 0000:\t78 78 78 78 78 78 78 78 78 78 \txxxxxxxxxx\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_end_packet():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+ stream = _DebugStream(under, out)
+
+ stream.read(5)
+ stream.end_packet("read description", read=True, indent=" ")
+
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("write description", indent=" ")
+
+ stream.read(5)
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("read/write combo for read", read=True, indent=" ")
+
+ stream.read(5)
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("read/write combo for write", indent=" ")
+
+ expected = (
+ " < 0000:\t61 62 63 64 65 \tabcde\n"
+ "\n"
+ "< read description\n"
+ "\n"
+ "> write description\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ " < 0000:\t6b 6c 6d 6e 6f \tklmno\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ "< read/write combo for read\n"
+ "\n"
+ "> read/write combo for write\n"
+ "\n"
+ " < 0000:\t75 76 77 78 79 \tuvwxy\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
diff --git a/src/test/python/test_pq3.py b/src/test/python/test_pq3.py
new file mode 100644
index 0000000000..7c6817de31
--- /dev/null
+++ b/src/test/python/test_pq3.py
@@ -0,0 +1,574 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import getpass
+import io
+import os
+import platform
+import struct
+import sys
+
+import pytest
+from construct import Container, PaddingError, StreamError, TerminatedError
+
+import pq3
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"\x00\x00\x00\x10\x00\x04\x00\x00abcdefgh",
+ Container(len=16, proto=0x40000, payload=b"abcdefgh"),
+ b"",
+ id="8-byte payload",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x08\x00\x04\x00\x00",
+ Container(len=8, proto=0x40000, payload=b""),
+ b"",
+ id="no payload",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x09\x00\x04\x00\x00abcde",
+ Container(len=9, proto=0x40000, payload=b"a"),
+ b"bcde",
+ id="1-byte payload and extra padding",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x0B\x00\x03\x00\x00hi\x00",
+ Container(len=11, proto=pq3.protocol(3, 0), payload=[b"hi"]),
+ b"",
+ id="implied parameter list when using proto version 3.0",
+ ),
+ ],
+)
+def test_Startup_parse(raw, expected, extra):
+ with io.BytesIO(raw) as stream:
+ actual = pq3.Startup.parse_stream(stream)
+
+ assert actual == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "packet,expected_bytes",
+ [
+ pytest.param(
+ dict(),
+ b"\x00\x00\x00\x08\x00\x00\x00\x00",
+ id="nothing set",
+ ),
+ pytest.param(
+ dict(len=10, proto=0x12345678),
+ b"\x00\x00\x00\x0A\x12\x34\x56\x78\x00\x00",
+ id="len and proto set explicitly",
+ ),
+ pytest.param(
+ dict(proto=0x12345678),
+ b"\x00\x00\x00\x08\x12\x34\x56\x78",
+ id="implied len with no payload",
+ ),
+ pytest.param(
+ dict(proto=0x12345678, payload=b"abcd"),
+ b"\x00\x00\x00\x0C\x12\x34\x56\x78abcd",
+ id="implied len with payload",
+ ),
+ pytest.param(
+ dict(payload=[b""]),
+ b"\x00\x00\x00\x09\x00\x03\x00\x00\x00",
+ id="implied proto version 3 when sending parameters",
+ ),
+ pytest.param(
+ dict(payload=[b"hi", b""]),
+ b"\x00\x00\x00\x0C\x00\x03\x00\x00hi\x00\x00",
+ id="implied proto version 3 and len when sending more than one parameter",
+ ),
+ pytest.param(
+ dict(payload=dict(user="jsmith", database="postgres")),
+ b"\x00\x00\x00\x27\x00\x03\x00\x00user\x00jsmith\x00database\x00postgres\x00\x00",
+ id="auto-serialization of dict parameters",
+ ),
+ ],
+)
+def test_Startup_build(packet, expected_bytes):
+ actual = pq3.Startup.build(packet)
+ assert actual == expected_bytes
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"*\x00\x00\x00\x08abcd",
+ dict(type=b"*", len=8, payload=b"abcd"),
+ b"",
+ id="4-byte payload",
+ ),
+ pytest.param(
+ b"*\x00\x00\x00\x04",
+ dict(type=b"*", len=4, payload=b""),
+ b"",
+ id="no payload",
+ ),
+ pytest.param(
+ b"*\x00\x00\x00\x05xabcd",
+ dict(type=b"*", len=5, payload=b"x"),
+ b"abcd",
+ id="1-byte payload with extra padding",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x08\x00\x00\x00\x00",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=8,
+ payload=dict(type=pq3.authn.OK, body=None),
+ ),
+ b"",
+ id="AuthenticationOk",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x12\x00\x00\x00\x0AEXTERNAL\x00\x00",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=18,
+ payload=dict(type=pq3.authn.SASL, body=[b"EXTERNAL", b""]),
+ ),
+ b"",
+ id="AuthenticationSASL",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0B12345",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=13,
+ payload=dict(type=pq3.authn.SASLContinue, body=b"12345"),
+ ),
+ b"",
+ id="AuthenticationSASLContinue",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0C12345",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=13,
+ payload=dict(type=pq3.authn.SASLFinal, body=b"12345"),
+ ),
+ b"",
+ id="AuthenticationSASLFinal",
+ ),
+ pytest.param(
+ b"p\x00\x00\x00\x0Bhunter2",
+ dict(
+ type=pq3.types.PasswordMessage,
+ len=11,
+ payload=b"hunter2",
+ ),
+ b"",
+ id="PasswordMessage",
+ ),
+ pytest.param(
+ b"K\x00\x00\x00\x0C\x00\x00\x00\x00\x12\x34\x56\x78",
+ dict(
+ type=pq3.types.BackendKeyData,
+ len=12,
+ payload=dict(pid=0, key=0x12345678),
+ ),
+ b"",
+ id="BackendKeyData",
+ ),
+ pytest.param(
+ b"C\x00\x00\x00\x08SET\x00",
+ dict(
+ type=pq3.types.CommandComplete,
+ len=8,
+ payload=dict(tag=b"SET"),
+ ),
+ b"",
+ id="CommandComplete",
+ ),
+ pytest.param(
+ b"E\x00\x00\x00\x11Mbad!\x00Mdog!\x00\x00",
+ dict(type=b"E", len=17, payload=dict(fields=[b"Mbad!", b"Mdog!", b""])),
+ b"",
+ id="ErrorResponse",
+ ),
+ pytest.param(
+ b"S\x00\x00\x00\x08a\x00b\x00",
+ dict(
+ type=pq3.types.ParameterStatus,
+ len=8,
+ payload=dict(name=b"a", value=b"b"),
+ ),
+ b"",
+ id="ParameterStatus",
+ ),
+ pytest.param(
+ b"Z\x00\x00\x00\x05x",
+ dict(type=b"Z", len=5, payload=dict(status=b"x")),
+ b"",
+ id="ReadyForQuery",
+ ),
+ pytest.param(
+ b"Q\x00\x00\x00\x06!\x00",
+ dict(type=pq3.types.Query, len=6, payload=dict(query=b"!")),
+ b"",
+ id="Query",
+ ),
+ pytest.param(
+ b"D\x00\x00\x00\x0B\x00\x01\x00\x00\x00\x01!",
+ dict(type=pq3.types.DataRow, len=11, payload=dict(columns=[b"!"])),
+ b"",
+ id="DataRow",
+ ),
+ pytest.param(
+ b"D\x00\x00\x00\x06\x00\x00extra",
+ dict(type=pq3.types.DataRow, len=6, payload=dict(columns=[])),
+ b"extra",
+ id="DataRow with extra data",
+ ),
+ pytest.param(
+ b"I\x00\x00\x00\x04",
+ dict(type=pq3.types.EmptyQueryResponse, len=4, payload=None),
+ b"",
+ id="EmptyQueryResponse",
+ ),
+ pytest.param(
+ b"I\x00\x00\x00\x04\xFF",
+ dict(type=b"I", len=4, payload=None),
+ b"\xFF",
+ id="EmptyQueryResponse with extra bytes",
+ ),
+ pytest.param(
+ b"X\x00\x00\x00\x04",
+ dict(type=pq3.types.Terminate, len=4, payload=None),
+ b"",
+ id="Terminate",
+ ),
+ ],
+)
+def test_Pq3_parse(raw, expected, extra):
+ with io.BytesIO(raw) as stream:
+ actual = pq3.Pq3.parse_stream(stream)
+
+ assert actual == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(type=b"*", len=5),
+ b"*\x00\x00\x00\x05",
+ id="type and len set explicitly",
+ ),
+ pytest.param(
+ dict(type=b"*"),
+ b"*\x00\x00\x00\x04",
+ id="implied len with no payload",
+ ),
+ pytest.param(
+ dict(type=b"*", payload=b"1234"),
+ b"*\x00\x00\x00\x081234",
+ id="implied len with payload",
+ ),
+ pytest.param(
+ dict(type=b"*", len=12, payload=b"1234"),
+ b"*\x00\x00\x00\x0C1234",
+ id="overridden len (payload underflow)",
+ ),
+ pytest.param(
+ dict(type=b"*", len=5, payload=b"1234"),
+ b"*\x00\x00\x00\x051234",
+ id="overridden len (payload overflow)",
+ ),
+ pytest.param(
+ dict(type=pq3.types.AuthnRequest, payload=dict(type=pq3.authn.OK)),
+ b"R\x00\x00\x00\x08\x00\x00\x00\x00",
+ id="implied len/type for AuthenticationOK",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(
+ type=pq3.authn.SASL,
+ body=[b"SCRAM-SHA-256-PLUS", b"SCRAM-SHA-256", b""],
+ ),
+ ),
+ b"R\x00\x00\x00\x2A\x00\x00\x00\x0ASCRAM-SHA-256-PLUS\x00SCRAM-SHA-256\x00\x00",
+ id="implied len/type for AuthenticationSASL",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(type=pq3.authn.SASLContinue, body=b"12345"),
+ ),
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0B12345",
+ id="implied len/type for AuthenticationSASLContinue",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(type=pq3.authn.SASLFinal, body=b"12345"),
+ ),
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0C12345",
+ id="implied len/type for AuthenticationSASLFinal",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.PasswordMessage,
+ payload=b"hunter2",
+ ),
+ b"p\x00\x00\x00\x0Bhunter2",
+ id="implied len/type for PasswordMessage",
+ ),
+ pytest.param(
+ dict(type=pq3.types.BackendKeyData, payload=dict(pid=1, key=7)),
+ b"K\x00\x00\x00\x0C\x00\x00\x00\x01\x00\x00\x00\x07",
+ id="implied len/type for BackendKeyData",
+ ),
+ pytest.param(
+ dict(type=pq3.types.CommandComplete, payload=dict(tag=b"SET")),
+ b"C\x00\x00\x00\x08SET\x00",
+ id="implied len/type for CommandComplete",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ErrorResponse, payload=dict(fields=[b"error", b""])),
+ b"E\x00\x00\x00\x0Berror\x00\x00",
+ id="implied len/type for ErrorResponse",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ParameterStatus, payload=dict(name=b"a", value=b"b")),
+ b"S\x00\x00\x00\x08a\x00b\x00",
+ id="implied len/type for ParameterStatus",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ReadyForQuery, payload=dict(status=b"I")),
+ b"Z\x00\x00\x00\x05I",
+ id="implied len/type for ReadyForQuery",
+ ),
+ pytest.param(
+ dict(type=pq3.types.Query, payload=dict(query=b"SELECT 1;")),
+ b"Q\x00\x00\x00\x0eSELECT 1;\x00",
+ id="implied len/type for Query",
+ ),
+ pytest.param(
+ dict(type=pq3.types.DataRow, payload=dict(columns=[b"abcd"])),
+ b"D\x00\x00\x00\x0E\x00\x01\x00\x00\x00\x04abcd",
+ id="implied len/type for DataRow",
+ ),
+ pytest.param(
+ dict(type=pq3.types.EmptyQueryResponse),
+ b"I\x00\x00\x00\x04",
+ id="implied len for EmptyQueryResponse",
+ ),
+ pytest.param(
+ dict(type=pq3.types.Terminate),
+ b"X\x00\x00\x00\x04",
+ id="implied len for Terminate",
+ ),
+ ],
+)
+def test_Pq3_build(fields, expected):
+ actual = pq3.Pq3.build(fields)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"\x00\x00",
+ dict(columns=[]),
+ b"",
+ id="no columns",
+ ),
+ pytest.param(
+ b"\x00\x01\x00\x00\x00\x04abcd",
+ dict(columns=[b"abcd"]),
+ b"",
+ id="one column",
+ ),
+ pytest.param(
+ b"\x00\x02\x00\x00\x00\x04abcd\x00\x00\x00\x01x",
+ dict(columns=[b"abcd", b"x"]),
+ b"",
+ id="multiple columns",
+ ),
+ pytest.param(
+ b"\x00\x02\x00\x00\x00\x00\x00\x00\x00\x01x",
+ dict(columns=[b"", b"x"]),
+ b"",
+ id="empty column value",
+ ),
+ pytest.param(
+ b"\x00\x02\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF",
+ dict(columns=[None, None]),
+ b"",
+ id="null columns",
+ ),
+ ],
+)
+def test_DataRow_parse(raw, expected, extra):
+ pkt = b"D" + struct.pack("!i", len(raw) + 4) + raw
+ with io.BytesIO(pkt) as stream:
+ actual = pq3.Pq3.parse_stream(stream)
+
+ assert actual.type == pq3.types.DataRow
+ assert actual.payload == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(),
+ b"\x00\x00",
+ id="no columns",
+ ),
+ pytest.param(
+ dict(columns=[None, None]),
+ b"\x00\x02\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF",
+ id="null columns",
+ ),
+ ],
+)
+def test_DataRow_build(fields, expected):
+ actual = pq3.Pq3.build(dict(type=pq3.types.DataRow, payload=fields))
+
+ expected = b"D" + struct.pack("!i", len(expected) + 4) + expected
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "raw,expected,exception",
+ [
+ pytest.param(
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ dict(name=b"EXTERNAL", len=-1, data=None),
+ None,
+ id="no initial response",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\x02me",
+ dict(name=b"EXTERNAL", len=2, data=b"me"),
+ None,
+ id="initial response",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\x02meextra",
+ None,
+ TerminatedError,
+ id="extra data",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\xFFme",
+ None,
+ StreamError,
+ id="underflow",
+ ),
+ ],
+)
+def test_SASLInitialResponse_parse(raw, expected, exception):
+ ctx = contextlib.nullcontext()
+ if exception:
+ ctx = pytest.raises(exception)
+
+ with ctx:
+ actual = pq3.SASLInitialResponse.parse(raw)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(name=b"EXTERNAL"),
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ id="no initial response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=None),
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ id="no initial response (explicit None)",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=b""),
+ b"EXTERNAL\x00\x00\x00\x00\x00",
+ id="empty response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=b"me@example.com"),
+ b"EXTERNAL\x00\x00\x00\x00\x0Eme@example.com",
+ id="initial response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", len=2, data=b"me@example.com"),
+ b"EXTERNAL\x00\x00\x00\x00\x02me@example.com",
+ id="data overflow",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", len=14, data=b"me"),
+ b"EXTERNAL\x00\x00\x00\x00\x0Eme",
+ id="data underflow",
+ ),
+ ],
+)
+def test_SASLInitialResponse_build(fields, expected):
+ actual = pq3.SASLInitialResponse.build(fields)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "version,expected_bytes",
+ [
+ pytest.param((3, 0), b"\x00\x03\x00\x00", id="version 3"),
+ pytest.param((1234, 5679), b"\x04\xd2\x16\x2f", id="SSLRequest"),
+ ],
+)
+def test_protocol(version, expected_bytes):
+ # Make sure the integer returned by protocol is correctly serialized on the
+ # wire.
+ assert struct.pack("!i", pq3.protocol(*version)) == expected_bytes
+
+
+@pytest.mark.parametrize(
+ "envvar,func,expected",
+ [
+ ("PGHOST", pq3.pghost, "localhost"),
+ ("PGPORT", pq3.pgport, 5432),
+ (
+ "PGUSER",
+ pq3.pguser,
+ os.getlogin() if platform.system() == "Windows" else getpass.getuser(),
+ ),
+ ("PGDATABASE", pq3.pgdatabase, "postgres"),
+ ],
+)
+def test_env_defaults(monkeypatch, envvar, func, expected):
+ monkeypatch.delenv(envvar, raising=False)
+
+ actual = func()
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "envvars,func,expected",
+ [
+ (dict(PGHOST="otherhost"), pq3.pghost, "otherhost"),
+ (dict(PGPORT="6789"), pq3.pgport, 6789),
+ (dict(PGUSER="postgres"), pq3.pguser, "postgres"),
+ (dict(PGDATABASE="template1"), pq3.pgdatabase, "template1"),
+ ],
+)
+def test_env(monkeypatch, envvars, func, expected):
+ for k, v in envvars.items():
+ monkeypatch.setenv(k, v)
+
+ actual = func()
+ assert actual == expected
diff --git a/src/test/python/tls.py b/src/test/python/tls.py
new file mode 100644
index 0000000000..075c02c1ca
--- /dev/null
+++ b/src/test/python/tls.py
@@ -0,0 +1,195 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+from construct import *
+
+#
+# TLS 1.3
+#
+# Most of the types below are transcribed from RFC 8446:
+#
+# https://tools.ietf.org/html/rfc8446
+#
+
+
+def _Vector(size_field, element):
+ return Prefixed(size_field, GreedyRange(element))
+
+
+# Alerts
+
+AlertLevel = Enum(
+ Byte,
+ warning=1,
+ fatal=2,
+)
+
+AlertDescription = Enum(
+ Byte,
+ close_notify=0,
+ unexpected_message=10,
+ bad_record_mac=20,
+ decryption_failed_RESERVED=21,
+ record_overflow=22,
+ decompression_failure=30,
+ handshake_failure=40,
+ no_certificate_RESERVED=41,
+ bad_certificate=42,
+ unsupported_certificate=43,
+ certificate_revoked=44,
+ certificate_expired=45,
+ certificate_unknown=46,
+ illegal_parameter=47,
+ unknown_ca=48,
+ access_denied=49,
+ decode_error=50,
+ decrypt_error=51,
+ export_restriction_RESERVED=60,
+ protocol_version=70,
+ insufficient_security=71,
+ internal_error=80,
+ user_canceled=90,
+ no_renegotiation=100,
+ unsupported_extension=110,
+)
+
+Alert = Struct(
+ "level" / AlertLevel,
+ "description" / AlertDescription,
+)
+
+
+# Extensions
+
+ExtensionType = Enum(
+ Int16ub,
+ server_name=0,
+ max_fragment_length=1,
+ status_request=5,
+ supported_groups=10,
+ signature_algorithms=13,
+ use_srtp=14,
+ heartbeat=15,
+ application_layer_protocol_negotiation=16,
+ signed_certificate_timestamp=18,
+ client_certificate_type=19,
+ server_certificate_type=20,
+ padding=21,
+ pre_shared_key=41,
+ early_data=42,
+ supported_versions=43,
+ cookie=44,
+ psk_key_exchange_modes=45,
+ certificate_authorities=47,
+ oid_filters=48,
+ post_handshake_auth=49,
+ signature_algorithms_cert=50,
+ key_share=51,
+)
+
+Extension = Struct(
+ "extension_type" / ExtensionType,
+ "extension_data" / Prefixed(Int16ub, GreedyBytes),
+)
+
+
+# ClientHello
+
+
+class _CipherSuiteAdapter(Adapter):
+ class _hextuple(tuple):
+ def __repr__(self):
+ return f"(0x{self[0]:02X}, 0x{self[1]:02X})"
+
+ def _encode(self, obj, context, path):
+ return bytes(obj)
+
+ def _decode(self, obj, context, path):
+ assert len(obj) == 2
+ return self._hextuple(obj)
+
+
+ProtocolVersion = Hex(Int16ub)
+
+Random = Hex(Bytes(32))
+
+CipherSuite = _CipherSuiteAdapter(Byte[2])
+
+ClientHello = Struct(
+ "legacy_version" / ProtocolVersion,
+ "random" / Random,
+ "legacy_session_id" / Prefixed(Byte, Hex(GreedyBytes)),
+ "cipher_suites" / _Vector(Int16ub, CipherSuite),
+ "legacy_compression_methods" / Prefixed(Byte, GreedyBytes),
+ "extensions" / _Vector(Int16ub, Extension),
+)
+
+# ServerHello
+
+ServerHello = Struct(
+ "legacy_version" / ProtocolVersion,
+ "random" / Random,
+ "legacy_session_id_echo" / Prefixed(Byte, Hex(GreedyBytes)),
+ "cipher_suite" / CipherSuite,
+ "legacy_compression_method" / Hex(Byte),
+ "extensions" / _Vector(Int16ub, Extension),
+)
+
+# Handshake
+
+HandshakeType = Enum(
+ Byte,
+ client_hello=1,
+ server_hello=2,
+ new_session_ticket=4,
+ end_of_early_data=5,
+ encrypted_extensions=8,
+ certificate=11,
+ certificate_request=13,
+ certificate_verify=15,
+ finished=20,
+ key_update=24,
+ message_hash=254,
+)
+
+Handshake = Struct(
+ "msg_type" / HandshakeType,
+ "length" / Int24ub,
+ "payload"
+ / Switch(
+ this.msg_type,
+ {
+ HandshakeType.client_hello: ClientHello,
+ HandshakeType.server_hello: ServerHello,
+ # HandshakeType.end_of_early_data: EndOfEarlyData,
+ # HandshakeType.encrypted_extensions: EncryptedExtensions,
+ # HandshakeType.certificate_request: CertificateRequest,
+ # HandshakeType.certificate: Certificate,
+ # HandshakeType.certificate_verify: CertificateVerify,
+ # HandshakeType.finished: Finished,
+ # HandshakeType.new_session_ticket: NewSessionTicket,
+ # HandshakeType.key_update: KeyUpdate,
+ },
+ default=FixedSized(this.length, GreedyBytes),
+ ),
+)
+
+# Records
+
+ContentType = Enum(
+ Byte,
+ invalid=0,
+ change_cipher_spec=20,
+ alert=21,
+ handshake=22,
+ application_data=23,
+)
+
+Plaintext = Struct(
+ "type" / ContentType,
+ "legacy_record_version" / ProtocolVersion,
+ "length" / Int16ub,
+ "fragment" / FixedSized(this.length, GreedyBytes),
+)
diff --git a/src/tools/make_venv b/src/tools/make_venv
new file mode 100755
index 0000000000..804307ee12
--- /dev/null
+++ b/src/tools/make_venv
@@ -0,0 +1,56 @@
+#!/usr/bin/env python3
+
+import argparse
+import subprocess
+import os
+import platform
+import sys
+
+parser = argparse.ArgumentParser()
+
+parser.add_argument('--requirements', help='path to pip requirements file', type=str)
+parser.add_argument('--privatedir', help='private directory for target', type=str)
+parser.add_argument('venv_path', help='desired venv location')
+
+args = parser.parse_args()
+
+# Decide whether or not to capture stdout into a log file. We only do this if
+# we've been given our own private directory.
+#
+# FIXME Unfortunately this interferes with debugging on Cirrus, because the
+# private directory isn't uploaded in the sanity check's artifacts. When we
+# don't capture the log file, it gets spammed to stdout during build... Is there
+# a way to push this into the meson-log somehow? For now, the capture
+# implementation is commented out.
+logfile = None
+
+if args.privatedir:
+ if not os.path.isdir(args.privatedir):
+ os.mkdir(args.privatedir)
+
+ # FIXME see above comment
+ # logpath = os.path.join(args.privatedir, 'stdout.txt')
+ # logfile = open(logpath, 'w')
+
+def run(*args):
+ kwargs = dict(check=True)
+ if logfile:
+ kwargs.update(stdout=logfile)
+
+ subprocess.run(args, **kwargs)
+
+# Create the virtualenv first.
+run(sys.executable, '-m', 'venv', args.venv_path)
+
+# Update pip next. This helps avoid old pip bugs; the version inside system
+# Pythons tends to be pretty out of date.
+bindir = 'Scripts' if platform.system() == 'Windows' else 'bin'
+python = os.path.join(args.venv_path, bindir, 'python3')
+run(python, '-m', 'pip', 'install', '-U', 'pip')
+
+# Finally, install the test's requirements. We need pytest and pytest-tap, no
+# matter what the test needs.
+pip = os.path.join(args.venv_path, bindir, 'pip')
+run(pip, 'install', 'pytest', 'pytest-tap')
+if args.requirements:
+ run(pip, 'install', '-r', args.requirements)
--
2.34.1
v20-0009-WIP-Python-OAuth-provider-implementation.patch (application/octet-stream, 10.8K) download
From c4d850a7c43c82de562b22b4ec1ad107adbe0cec Mon Sep 17 00:00:00 2001
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Mon, 26 Feb 2024 16:24:32 -0800
Subject: [PATCH v20 9/9] WIP: Python OAuth provider implementation
---
src/test/modules/oauth_validator/Makefile | 2 +
src/test/modules/oauth_validator/meson.build | 3 +
.../modules/oauth_validator/t/001_server.pl | 16 +-
.../modules/oauth_validator/t/oauth_server.py | 114 ++++++++++++
src/test/perl/PostgreSQL/Test/OAuthServer.pm | 166 +++---------------
5 files changed, 149 insertions(+), 152 deletions(-)
create mode 100755 src/test/modules/oauth_validator/t/oauth_server.py
diff --git a/src/test/modules/oauth_validator/Makefile b/src/test/modules/oauth_validator/Makefile
index 1f874cd7f2..e93e01455a 100644
--- a/src/test/modules/oauth_validator/Makefile
+++ b/src/test/modules/oauth_validator/Makefile
@@ -1,3 +1,5 @@
+export PYTHON
+
MODULES = validator
PGFILEDESC = "validator - test OAuth validator module"
diff --git a/src/test/modules/oauth_validator/meson.build b/src/test/modules/oauth_validator/meson.build
index d9c1d1d577..3feba6f826 100644
--- a/src/test/modules/oauth_validator/meson.build
+++ b/src/test/modules/oauth_validator/meson.build
@@ -29,5 +29,8 @@ tests += {
'tests': [
't/001_server.pl',
],
+ 'env': {
+ 'PYTHON': python.path(),
+ },
},
}
diff --git a/src/test/modules/oauth_validator/t/001_server.pl b/src/test/modules/oauth_validator/t/001_server.pl
index 14c7778298..ea610fcd28 100644
--- a/src/test/modules/oauth_validator/t/001_server.pl
+++ b/src/test/modules/oauth_validator/t/001_server.pl
@@ -19,7 +19,11 @@ $node->start;
$node->safe_psql('postgres', 'CREATE USER test;');
$node->safe_psql('postgres', 'CREATE USER testalt;');
-my $issuer = "127.0.0.1:18080";
+my $webserver = PostgreSQL::Test::OAuthServer->new();
+$webserver->run();
+
+my $port = $webserver->port();
+my $issuer = "127.0.0.1:$port";
unlink($node->data_dir . '/pg_hba.conf');
$node->append_conf('pg_hba.conf', qq{
@@ -28,15 +32,6 @@ local all testalt oauth issuer="$issuer/alternate" scope="openid postgres alt"
});
$node->reload;
-my $webserver = PostgreSQL::Test::OAuthServer->new(18080);
-
-my $port = $webserver->port();
-
-is($port, 18080, "Port is 18080");
-
-$webserver->setup();
-$webserver->run();
-
my ($log_start, $log_end);
$log_start = $node->wait_for_log(qr/reloading configuration files/);
@@ -73,6 +68,7 @@ $node->log_check("user $user: validator sets authenticated identity", $log_start
]);
$log_start = $log_end;
+$webserver->stop();
$node->stop;
done_testing();
diff --git a/src/test/modules/oauth_validator/t/oauth_server.py b/src/test/modules/oauth_validator/t/oauth_server.py
new file mode 100755
index 0000000000..77e3883a81
--- /dev/null
+++ b/src/test/modules/oauth_validator/t/oauth_server.py
@@ -0,0 +1,114 @@
+#! /usr/bin/env python3
+
+import http.server
+import json
+import os
+import sys
+
+
+class OAuthHandler(http.server.BaseHTTPRequestHandler):
+ JsonObject = dict[str, object] # TypeAlias is not available until 3.10
+
+ def _check_issuer(self):
+ """
+ Switches the behavior of the provider depending on the issuer URI.
+ """
+ self._alt_issuer = self.path.startswith("/alternate/")
+ if self._alt_issuer:
+ self.path = self.path.removeprefix("/alternate")
+
+ def do_GET(self):
+ self._check_issuer()
+
+ if self.path == "/.well-known/openid-configuration":
+ resp = self.config()
+ else:
+ self.send_error(404, "Not Found")
+ return
+
+ self._send_json(resp)
+
+ def do_POST(self):
+ self._check_issuer()
+
+ if self.path == "/authorize":
+ resp = self.authorization()
+ elif self.path == "/token":
+ resp = self.token()
+ else:
+ self.send_error(404, "Not Found")
+ return
+
+ self._send_json(resp)
+
+ def _send_json(self, js: JsonObject) -> None:
+ """
+ Sends the provided JSON dict as an application/json response.
+ """
+
+ resp = json.dumps(js).encode("ascii")
+
+ self.send_response(200, "OK")
+ self.send_header("Content-Type", "application/json")
+ self.send_header("Content-Length", str(len(resp)))
+ self.end_headers()
+
+ self.wfile.write(resp)
+
+ def config(self) -> JsonObject:
+ port = self.server.socket.getsockname()[1]
+ issuer = f"http://localhost:{port}"
+ if self._alt_issuer:
+ issuer += "/alternate"
+
+ return {
+ "issuer": issuer,
+ "token_endpoint": issuer + "/token",
+ "device_authorization_endpoint": issuer + "/authorize",
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ "grant_types_supported": ["urn:ietf:params:oauth:grant-type:device_code"],
+ }
+
+ def authorization(self) -> JsonObject:
+ uri = "https://example.com/"
+ if self._alt_issuer:
+ uri = "https://example.org/"
+
+ return {
+ "device_code": "postgres",
+ "user_code": "postgresuser",
+ "interval": 0,
+ "verification_uri": uri,
+ "expires-in": 5,
+ }
+
+ def token(self) -> JsonObject:
+ token = "9243959234"
+ if self._alt_issuer:
+ token += "-alt"
+
+ return {
+ "access_token": token,
+ "token_type": "bearer",
+ }
+
+
+def main():
+ s = http.server.HTTPServer(("127.0.0.1", 0), OAuthHandler)
+
+ # Give the parent the port number to contact (this is also the signal that
+ # we're ready to receive requests).
+ port = s.socket.getsockname()[1]
+ print(port)
+
+ stdout = sys.stdout.fileno()
+ sys.stdout.close()
+ os.close(stdout)
+
+ s.serve_forever() # we expect our parent to send a termination signal
+
+
+if __name__ == "__main__":
+ main()
diff --git a/src/test/perl/PostgreSQL/Test/OAuthServer.pm b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
index 5c195efb79..d96733f531 100644
--- a/src/test/perl/PostgreSQL/Test/OAuthServer.pm
+++ b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
@@ -5,6 +5,7 @@ package PostgreSQL::Test::OAuthServer;
use warnings;
use strict;
use threads;
+use Scalar::Util;
use Socket;
use IO::Select;
@@ -13,27 +14,13 @@ local *server_socket;
sub new
{
my $class = shift;
- my $port = shift;
my $self = {};
bless($self, $class);
- $self->{'port'} = $port;
-
return $self;
}
-sub setup
-{
- my $self = shift;
- my $tcp = getprotobyname('tcp');
-
- socket($self->{'socket'}, PF_INET, SOCK_STREAM, $tcp)
- or die "no socket";
- setsockopt($self->{'socket'}, SOL_SOCKET, SO_REUSEADDR, pack("l", 1));
- bind($self->{'socket'}, sockaddr_in($self->{'port'}, INADDR_ANY));
-}
-
sub port
{
my $self = shift;
@@ -44,140 +31,35 @@ sub port
sub run
{
my $self = shift;
+ my $port;
- my $server_thread = threads->create(\&_listen, $self);
- $server_thread->detach();
+ my $pid = open(my $read_fh, "-|", $ENV{PYTHON}, "t/oauth_server.py")
+ // die "failed to start OAuth server: $!";
+
+ read($read_fh, $port, 7) // die "failed to read port number: $!";
+ chomp $port;
+ die "server did not advertise a valid port"
+ unless Scalar::Util::looks_like_number($port);
+
+ $self->{'pid'} = $pid;
+ $self->{'port'} = $port;
+ $self->{'child'} = $read_fh;
+
+ print("# OAuth provider (PID $pid) is listening on port $port\n");
}
-sub _listen
+sub stop
{
my $self = shift;
- listen($self->{'socket'}, SOMAXCONN) or die "fail to listen: $!";
-
- while (1)
- {
- my $fh;
- my %request;
- my $remote = accept($fh, $self->{'socket'});
- binmode $fh;
-
- my ($method, $object, $prot) = split(/ /, <$fh>);
- $request{'method'} = $method;
- $request{'object'} = $object;
- chomp($request{'object'});
-
- local $/ = Socket::CRLF;
- my $c = 0;
- while(<$fh>)
- {
- chomp;
- # Headers
- if (/:/)
- {
- my ($field, $value) = split(/:/, $_, 2);
- $value =~ s/^\s+//;
- $request{'headers'}{lc $field} = $value;
- }
- # POST data
- elsif (/^$/)
- {
- read($fh, $request{'content'}, $request{'headers'}{'content-length'})
- if defined $request{'headers'}{'content-length'};
- last;
- }
- }
-
- # Debug printing
- # print ": read ".$request{'method'} . ";" . $request{'object'}.";\n";
- # foreach my $h (keys(%{$request{'headers'}}))
- #{
- # printf ": headers: " . $request{'headers'}{$h} . "\n";
- #}
- #printf ": POST: " . $request{'content'} . "\n" if defined $request{'content'};
-
- my $alternate = 0;
- if ($request{'object'} =~ qr|^/alternate(/.*)$|)
- {
- $alternate = 1;
- $request{'object'} = $1;
- }
-
- if ($request{'object'} eq '/.well-known/openid-configuration')
- {
- my $issuer = "http://localhost:$self->{'port'}";
- if ($alternate)
- {
- $issuer .= "/alternate";
- }
-
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: application/json\r\n";
- print $fh "\r\n";
- print $fh <<EOR;
- {
- "issuer": "$issuer",
- "token_endpoint": "$issuer/token",
- "device_authorization_endpoint": "$issuer/authorize",
- "response_types_supported": ["token"],
- "subject_types_supported": ["public"],
- "id_token_signing_alg_values_supported": ["RS256"],
- "grant_types_supported": ["urn:ietf:params:oauth:grant-type:device_code"]
- }
-EOR
- }
- elsif ($request{'object'} eq '/authorize')
- {
- my $uri = "https://example.com/";
- if ($alternate)
- {
- $uri = "https://example.org/";
- }
-
- print ": returning device_code\n";
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: application/json\r\n";
- print $fh "\r\n";
- print $fh <<EOR;
- {
- "device_code": "postgres",
- "user_code" : "postgresuser",
- "interval" : 0,
- "verification_uri" : "$uri",
- "expires-in": 5
- }
-EOR
- }
- elsif ($request{'object'} eq '/token')
- {
- my $token = "9243959234";
- if ($alternate)
- {
- $token .= "-alt";
- }
-
- print ": returning token\n";
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: application/json\r\n";
- print $fh "\r\n";
- print $fh <<EOR;
- {
- "access_token": "$token",
- "token_type": "bearer"
- }
-EOR
- }
- else
- {
- print ": returning default\n";
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: text/html\r\n";
- print $fh "\r\n";
- print $fh "Ok\n";
- }
-
- close($fh);
- }
+ print("# Sending SIGTERM to OAuth provider PID: $self->{'pid'}\n");
+
+ kill(15, $self->{'pid'});
+ $self->{'pid'} = undef;
+
+ # Closing the popen() handle waits for the process to exit.
+ close($self->{'child'});
+ $self->{'child'} = undef;
}
1;
--
2.34.1
v20-0008-XXX-temporary-patches-to-build-and-test.patch (application/octet-stream, 3.8K) download
From 185f9902fd85d1409d4ea749938991bb61ac4b21 Mon Sep 17 00:00:00 2001
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Tue, 20 Feb 2024 11:35:29 -0800
Subject: [PATCH v20 8/9] XXX temporary patches to build and test
- the new pg_combinebackup utility uses JSON in the frontend without
0001; has something changed?
- construct 2.10.70 has some incompatibilities with the current tests
- temporarily skip the exit check (from Daniel Gustafsson); this needs
to be turned into an exception for curl rather than a plain exit call
---
src/bin/pg_combinebackup/Makefile | 6 ++++--
src/bin/pg_combinebackup/meson.build | 3 ++-
src/bin/pg_verifybackup/Makefile | 2 +-
src/interfaces/libpq/Makefile | 2 +-
src/test/python/requirements.txt | 4 +++-
5 files changed, 11 insertions(+), 6 deletions(-)
diff --git a/src/bin/pg_combinebackup/Makefile b/src/bin/pg_combinebackup/Makefile
index 4f24b1aff6..2f7dc1ed87 100644
--- a/src/bin/pg_combinebackup/Makefile
+++ b/src/bin/pg_combinebackup/Makefile
@@ -18,6 +18,8 @@ include $(top_builddir)/src/Makefile.global
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils
+# TODO: fix this properly
+LDFLAGS_INTERNAL += -lpgcommon $(libpq_pgport)
OBJS = \
$(WIN32RES) \
@@ -30,8 +32,8 @@ OBJS = \
all: pg_combinebackup
-pg_combinebackup: $(OBJS) | submake-libpgport submake-libpgfeutils
- $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(libpq_pgport) $(LIBS) -o $@$(X)
+pg_combinebackup: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
install: all installdirs
$(INSTALL_PROGRAM) pg_combinebackup$(X) '$(DESTDIR)$(bindir)/pg_combinebackup$(X)'
diff --git a/src/bin/pg_combinebackup/meson.build b/src/bin/pg_combinebackup/meson.build
index 1d4b9c218f..6639987a58 100644
--- a/src/bin/pg_combinebackup/meson.build
+++ b/src/bin/pg_combinebackup/meson.build
@@ -17,7 +17,8 @@ endif
pg_combinebackup = executable('pg_combinebackup',
pg_combinebackup_sources,
- dependencies: [frontend_code],
+ # XXX linking against libpq isn't good, but how was JSON working?
+ dependencies: [frontend_code, libpq],
kwargs: default_bin_args,
)
bin_targets += pg_combinebackup
diff --git a/src/bin/pg_verifybackup/Makefile b/src/bin/pg_verifybackup/Makefile
index 7c045f142e..3372fada01 100644
--- a/src/bin/pg_verifybackup/Makefile
+++ b/src/bin/pg_verifybackup/Makefile
@@ -17,7 +17,7 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
# We need libpq only because fe_utils does.
-LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils -lpgcommon $(libpq_pgport)
OBJS = \
$(WIN32RES) \
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 2618c293af..e86d4803ff 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -124,7 +124,7 @@ libpq-refs-stamp: $(shlib)
ifneq ($(enable_coverage), yes)
ifeq (,$(filter solaris,$(PORTNAME)))
@if nm -A -u $< 2>/dev/null | grep -v -e __cxa_atexit -e __tsan_func_exit | grep exit; then \
- echo 'libpq must not be calling any function which invokes exit'; exit 1; \
+ echo 'libpq must not be calling any function which invokes exit'; \
fi
endif
endif
diff --git a/src/test/python/requirements.txt b/src/test/python/requirements.txt
index 57ba1ced94..0dfcffb83e 100644
--- a/src/test/python/requirements.txt
+++ b/src/test/python/requirements.txt
@@ -1,7 +1,9 @@
black
# cryptography 35.x and later add many platform/toolchain restrictions, beware
cryptography~=3.4.8
-construct~=2.10.61
+# TODO: figure out why 2.10.70 broke things
+# (probably https://github.com/construct/construct/pull/1015)
+construct==2.10.69
isort~=5.6
# TODO: update to psycopg[c] 3.1
psycopg2~=2.9.7
--
2.34.1
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-03-22 18:21 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2024-03-22 18:21 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: pgsql-hackers; Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>; Andrew Dunstan <andrew@dunslane.net>
v21 is a quick rebase over HEAD, which has adopted a few pieces of
v20. I've also fixed a race condition in the tests.
On Mon, Mar 11, 2024 at 3:51 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Andrew asked over at [2] if we could perhaps get 0001 in as well. I
> think the main thing to figure out there is, is requiring linkage
> against libpq (see 0008) going to be okay for the frontend binaries
> that need JSON support? Or do we need to do something like moving
> PQExpBuffer into src/common to simplify the dependency tree?
0001 has been pared down to the part that teaches jsonapi.c to use
PQExpBuffer and track out-of-memory conditions; the linkage questions
remain.
Thanks,
--Jacob
Attachments: download all (8 files, .tar.gz)
since-v20.diff.txt (text/plain, 22.1K) download
1: 231c6fb165 ! 1: 557370eabb common/jsonapi: support FRONTEND clients
@@ Metadata
Author: Jacob Champion <pchampion@vmware.com>
## Commit message ##
- common/jsonapi: support FRONTEND clients
+ common/jsonapi: support libpq as a client
Based on a patch by Michael Paquier.
For frontend code, use PQExpBuffer instead of StringInfo. This requires
us to track allocation failures so that we can return JSON_OUT_OF_MEMORY
- as needed. json_errdetail() now allocates its error message inside
- memory owned by the JsonLexContext, so clients don't need to worry about
- freeing it.
+ as needed rather than exit()ing.
- We can now partially revert b44669b2ca, now that json_errdetail() works
- correctly.
+ ## src/bin/pg_combinebackup/Makefile ##
+@@ src/bin/pg_combinebackup/Makefile: include $(top_builddir)/src/Makefile.global
- Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
+ override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
+ LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils
++# TODO: fix this properly
++LDFLAGS_INTERNAL += -lpgcommon $(libpq_pgport)
- ## src/bin/pg_verifybackup/t/005_bad_manifest.pl ##
-@@ src/bin/pg_verifybackup/t/005_bad_manifest.pl: use Test::More;
- my $tempdir = PostgreSQL::Test::Utils::tempdir;
+ OBJS = \
+ $(WIN32RES) \
+@@ src/bin/pg_combinebackup/Makefile: OBJS = \
- test_bad_manifest('input string ended unexpectedly',
-- qr/could not parse backup manifest: parsing failed/, <<EOM);
-+ qr/could not parse backup manifest: The input string ended unexpectedly/,
-+ <<EOM);
- {
- EOM
+ all: pg_combinebackup
+
+-pg_combinebackup: $(OBJS) | submake-libpgport submake-libpgfeutils
++pg_combinebackup: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
+
+ install: all installdirs
+
+ ## src/bin/pg_combinebackup/meson.build ##
+@@ src/bin/pg_combinebackup/meson.build: endif
+
+ pg_combinebackup = executable('pg_combinebackup',
+ pg_combinebackup_sources,
+- dependencies: [frontend_code],
++ dependencies: [frontend_code, libpq],
+ kwargs: default_bin_args,
+ )
+ bin_targets += pg_combinebackup
+ ## src/bin/pg_verifybackup/Makefile ##
+@@ src/bin/pg_verifybackup/Makefile: top_builddir = ../../..
+ include $(top_builddir)/src/Makefile.global
+
+ # We need libpq only because fe_utils does.
+-LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
++LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils -lpgcommon $(libpq_pgport)
+
+ OBJS = \
+ $(WIN32RES) \
## src/common/Makefile ##
@@ src/common/Makefile: override CPPFLAGS += -DVAL_LDFLAGS_EX="\"$(LDFLAGS_EX)\""
@@ src/common/jsonapi.c: makeJsonLexContextCstringLen(JsonLexContext *lex, char *js
+ static const JsonLexContext empty = {0};
+
if (lex->flags & JSONLEX_FREE_STRVAL)
-- {
-- pfree(lex->strval->data);
-- pfree(lex->strval);
-- }
+- destroyStringInfo(lex->strval);
+ destroyStrVal(lex->strval);
-+
-+ if (lex->errormsg)
+
+ if (lex->errormsg)
+- destroyStringInfo(lex->errormsg);
+ destroyStrVal(lex->errormsg);
-+
+
if (lex->flags & JSONLEX_FREE_STRUCT)
pfree(lex);
+ else
@@ src/common/jsonapi.c: json_lex_string(JsonLexContext *lex)
lex->prev_token_terminator = lex->token_terminator;
lex->token_terminator = s + 1;
@@ src/common/jsonapi.c: report_parse_error(JsonParseContext ctx, JsonLexContext *lex)
- return JSON_SUCCESS; /* silence stupider compilers */
- }
-
--
--#ifndef FRONTEND
--/*
-- * Extract the current token from a lexing context, for error reporting.
-- */
--static char *
--extract_token(JsonLexContext *lex)
--{
-- int toklen = lex->token_terminator - lex->token_start;
-- char *token = palloc(toklen + 1);
--
-- memcpy(token, lex->token_start, toklen);
-- token[toklen] = '\0';
-- return token;
--}
--
- /*
- * Construct an (already translated) detail message for a JSON error.
- *
-- * Note that the error message generated by this routine may not be
-- * palloc'd, making it unsafe for frontend code as there is no way to
-- * know if this can be safely pfree'd or not.
-+ * The returned allocation is either static or owned by the JsonLexContext and
-+ * should not be freed.
- */
char *
json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
{
-+ int toklen = lex->token_terminator - lex->token_start;
-+
+ if (error == JSON_OUT_OF_MEMORY)
+ {
+ /* Short circuit. Allocating anything for this case is unhelpful. */
+ return _("out of memory");
+ }
+
-+ if (lex->errormsg)
+ if (lex->errormsg)
+- resetStringInfo(lex->errormsg);
+ resetStrVal(lex->errormsg);
-+ else
+ else
+- lex->errormsg = makeStringInfo();
+ lex->errormsg = createStrVal();
-+
+
+ /*
+ * A helper for error messages that should print the current token. The
+ * format must contain exactly one %.*s specifier.
+ */
+ #define token_error(lex, format) \
+- appendStringInfo((lex)->errormsg, _(format), \
+- (int) ((lex)->token_terminator - (lex)->token_start), \
+- (lex)->token_start);
++ appendStrVal((lex)->errormsg, _(format), \
++ (int) ((lex)->token_terminator - (lex)->token_start), \
++ (lex)->token_start);
+
switch (error)
{
- case JSON_SUCCESS:
- /* fall through to the error code after switch */
+@@ src/common/jsonapi.c: json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
+ token_error(lex, "Escape sequence \"\\%.*s\" is invalid.");
break;
- case JSON_ESCAPING_INVALID:
-- return psprintf(_("Escape sequence \"\\%s\" is invalid."),
-- extract_token(lex));
-+ appendStrVal(lex->errormsg,
-+ _("Escape sequence \"\\%.*s\" is invalid."),
-+ toklen, lex->token_start);
-+ break;
case JSON_ESCAPING_REQUIRED:
-- return psprintf(_("Character with value 0x%02x must be escaped."),
+- appendStringInfo(lex->errormsg,
+- _("Character with value 0x%02x must be escaped."),
- (unsigned char) *(lex->token_terminator));
+ appendStrVal(lex->errormsg,
+ _("Character with value 0x%02x must be escaped."),
+ (unsigned char) *(lex->token_terminator));
-+ break;
+ break;
case JSON_EXPECTED_END:
-- return psprintf(_("Expected end of input, but found \"%s\"."),
-- extract_token(lex));
-+ appendStrVal(lex->errormsg,
-+ _("Expected end of input, but found \"%.*s\"."),
-+ toklen, lex->token_start);
-+ break;
- case JSON_EXPECTED_ARRAY_FIRST:
-- return psprintf(_("Expected array element or \"]\", but found \"%s\"."),
-- extract_token(lex));
-+ appendStrVal(lex->errormsg,
-+ _("Expected array element or \"]\", but found \"%.*s\"."),
-+ toklen, lex->token_start);
-+ break;
- case JSON_EXPECTED_ARRAY_NEXT:
-- return psprintf(_("Expected \",\" or \"]\", but found \"%s\"."),
-- extract_token(lex));
-+ appendStrVal(lex->errormsg,
-+ _("Expected \",\" or \"]\", but found \"%.*s\"."),
-+ toklen, lex->token_start);
-+ break;
- case JSON_EXPECTED_COLON:
-- return psprintf(_("Expected \":\", but found \"%s\"."),
-- extract_token(lex));
-+ appendStrVal(lex->errormsg,
-+ _("Expected \":\", but found \"%.*s\"."),
-+ toklen, lex->token_start);
-+ break;
- case JSON_EXPECTED_JSON:
-- return psprintf(_("Expected JSON value, but found \"%s\"."),
-- extract_token(lex));
-+ appendStrVal(lex->errormsg,
-+ _("Expected JSON value, but found \"%.*s\"."),
-+ toklen, lex->token_start);
-+ break;
- case JSON_EXPECTED_MORE:
- return _("The input string ended unexpectedly.");
- case JSON_EXPECTED_OBJECT_FIRST:
-- return psprintf(_("Expected string or \"}\", but found \"%s\"."),
-- extract_token(lex));
-+ appendStrVal(lex->errormsg,
-+ _("Expected string or \"}\", but found \"%.*s\"."),
-+ toklen, lex->token_start);
-+ break;
- case JSON_EXPECTED_OBJECT_NEXT:
-- return psprintf(_("Expected \",\" or \"}\", but found \"%s\"."),
-- extract_token(lex));
-+ appendStrVal(lex->errormsg,
-+ _("Expected \",\" or \"}\", but found \"%.*s\"."),
-+ toklen, lex->token_start);
-+ break;
- case JSON_EXPECTED_STRING:
-- return psprintf(_("Expected string, but found \"%s\"."),
-- extract_token(lex));
-+ appendStrVal(lex->errormsg,
-+ _("Expected string, but found \"%.*s\"."),
-+ toklen, lex->token_start);
-+ break;
+ token_error(lex, "Expected end of input, but found \"%.*s\".");
+@@ src/common/jsonapi.c: json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
case JSON_INVALID_TOKEN:
-- return psprintf(_("Token \"%s\" is invalid."),
-- extract_token(lex));
-+ appendStrVal(lex->errormsg,
-+ _("Token \"%.*s\" is invalid."),
-+ toklen, lex->token_start);
-+ break;
+ token_error(lex, "Token \"%.*s\" is invalid.");
+ break;
+ case JSON_OUT_OF_MEMORY:
+ /* should have been handled above; use the error path */
+ break;
@@ src/common/jsonapi.c: report_parse_error(JsonParseContext ctx, JsonLexContext *l
return _("\\u0000 cannot be converted to text.");
case JSON_UNICODE_ESCAPE_FORMAT:
@@ src/common/jsonapi.c: json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
- /* note: this case is only reachable in frontend not backend */
- return _("Unicode escape values cannot be used for code point values above 007F when the encoding is not UTF8.");
- case JSON_UNICODE_UNTRANSLATABLE:
-- /* note: this case is only reachable in backend not frontend */
-+
-+ /*
-+ * note: this case is only reachable in backend not frontend.
-+ * #ifdef it away so the frontend doesn't try to link against
-+ * backend functionality.
-+ */
-+#ifndef FRONTEND
- return psprintf(_("Unicode escape value could not be translated to the server's encoding %s."),
- GetDatabaseEncodingName());
-+#else
-+ Assert(false);
-+ break;
-+#endif
- case JSON_UNICODE_HIGH_SURROGATE:
- return _("Unicode high surrogate must not follow a high surrogate.");
- case JSON_UNICODE_LOW_SURROGATE:
-@@ src/common/jsonapi.c: json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
- break;
}
+ #undef token_error
- /*
- * We don't use a default: case, so that the compiler will warn about
- * unhandled enum values. But this needs to be here anyway to cover the
- * possibility of an incorrect input.
- */
-- elog(ERROR, "unexpected json parse error type: %d", (int) error);
-- return NULL;
--}
+- if (lex->errormsg->len == 0)
+- appendStringInfo(lex->errormsg,
+- _("unexpected json parse error type: %d"),
+- (int) error);
+ /* Note that lex->errormsg can be NULL in FRONTEND code. */
-+ if (lex->errormsg && !lex->errormsg->data[0])
++ if (lex->errormsg && lex->errormsg->len == 0)
+ {
+ /*
+ * We don't use a default: case, so that the compiler will warn about
@@ src/common/jsonapi.c: json_errdetail(JsonParseErrorType error, JsonLexContext *l
+ * the possibility of an incorrect input.
+ */
+ appendStrVal(lex->errormsg,
-+ "unexpected json parse error type: %d", (int) error);
++ _("unexpected json parse error type: %d"),
++ (int) error);
+ }
+
+#ifdef FRONTEND
+ if (PQExpBufferBroken(lex->errormsg))
+ return _("out of memory while constructing error description");
- #endif
-+
-+ return lex->errormsg->data;
-+}
++#endif
+
+ return lex->errormsg->data;
+ }
## src/common/meson.build ##
@@ src/common/meson.build: common_sources_frontend_static += files(
@@ src/common/meson.build: foreach name, opts : pgcommon_variants
'dependencies': opts['dependencies'] + [ssl],
}
- ## src/common/parse_manifest.c ##
-@@ src/common/parse_manifest.c: json_parse_manifest(JsonManifestParseContext *context, char *buffer,
- /* Run the actual JSON parser. */
- json_error = pg_parse_json(lex, &sem);
- if (json_error != JSON_SUCCESS)
-- json_manifest_parse_failure(context, "parsing failed");
-+ json_manifest_parse_failure(context, json_errdetail(json_error, lex));
- if (parse.state != JM_EXPECT_EOF)
- json_manifest_parse_failure(context, "manifest ended unexpectedly");
-
-
- ## src/common/stringinfo.c ##
-@@ src/common/stringinfo.c: enlargeStringInfo(StringInfo str, int needed)
-
- str->maxlen = newlen;
- }
-+
-+void
-+destroyStringInfo(StringInfo str)
-+{
-+ pfree(str->data);
-+ pfree(str);
-+}
-
## src/include/common/jsonapi.h ##
@@
#ifndef JSONAPI_H
@@ src/include/common/jsonapi.h: typedef struct JsonLexContext
int line_number; /* line number, starting from 1 */
char *line_start; /* where that line starts within input */
- StringInfo strval;
+- StringInfo errormsg;
+ bool parse_strval;
+ StrValType *strval; /* only used if parse_strval == true */
+ StrValType *errormsg;
} JsonLexContext;
typedef JsonParseErrorType (*json_struct_action) (void *state);
-
- ## src/include/lib/stringinfo.h ##
-@@ src/include/lib/stringinfo.h: extern void appendBinaryStringInfoNT(StringInfo str,
- */
- extern void enlargeStringInfo(StringInfo str, int needed);
-
-+
-+extern void destroyStringInfo(StringInfo str);
- #endif /* STRINGINFO_H */
2: f78c79ea68 < -: ---------- Refactor SASL exchange to return tri-state status
3: 10b6d2a6b9 < -: ---------- Explicitly require password for SCRAM exchange
4: 2a55d9c806 ! 2: b754a2f8fb libpq: add OAUTHBEARER SASL mechanism
@@ src/interfaces/libpq/Makefile: endif
endif
## src/interfaces/libpq/exports.txt ##
-@@ src/interfaces/libpq/exports.txt: PQsendClosePrepared 190
- PQsendClosePortal 191
- PQchangePassword 192
- PQsendPipelineSync 193
-+PQsetAuthDataHook 194
-+PQgetAuthDataHook 195
-+PQdefaultAuthDataHook 196
+@@ src/interfaces/libpq/exports.txt: PQcancelSocket 199
+ PQcancelErrorMessage 200
+ PQcancelReset 201
+ PQcancelFinish 202
++PQsetAuthDataHook 203
++PQgetAuthDataHook 204
++PQdefaultAuthDataHook 205
## src/interfaces/libpq/fe-auth-oauth-curl.c (new) ##
@@
@@ src/interfaces/libpq/fe-connect.c: static const internalPQconninfoOption PQconni
{NULL, NULL, NULL, NULL,
NULL, NULL, 0}
@@ src/interfaces/libpq/fe-connect.c: pqDropServerData(PGconn *conn)
+ conn->write_failed = false;
+ free(conn->write_err_msg);
conn->write_err_msg = NULL;
- conn->be_pid = 0;
- conn->be_key = 0;
+ /* conn->oauth_want_retry = false; TODO */
- }
-
+ /*
+ * Cancel connections need to retain their be_pid and be_key across
@@ src/interfaces/libpq/fe-connect.c: PQconnectPoll(PGconn *conn)
case CONNECTION_NEEDED:
case CONNECTION_GSS_STARTUP:
@@ src/interfaces/libpq/libpq-fe.h: extern "C"
/*
* Option flags for PQcopyResult
@@ src/interfaces/libpq/libpq-fe.h: typedef enum
- CONNECTION_CONSUME, /* Consuming any extra messages. */
- CONNECTION_GSS_STARTUP, /* Negotiating GSSAPI. */
- CONNECTION_CHECK_TARGET, /* Checking target server properties. */
-- CONNECTION_CHECK_STANDBY /* Checking if server is in standby mode. */
-+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+ CONNECTION_CHECK_TARGET, /* Internal state: checking target server
+ * properties. */
+ CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
+- CONNECTION_ALLOCATED /* Waiting for connection attempt to be
++ CONNECTION_ALLOCATED, /* Waiting for connection attempt to be
+ * started. */
+ CONNECTION_AUTHENTICATING, /* Authentication is in progress with some
+ * external system. */
} ConnStatusType;
@@ src/interfaces/libpq/libpq-int.h: typedef struct pg_conn_host
* PGconn stores all the state data associated with a single connection
* to a backend.
@@ src/interfaces/libpq/libpq-int.h: struct pg_conn
- char *require_auth; /* name of the expected auth method */
- char *load_balance_hosts; /* load balance over hosts */
+ * cancel request, instead of being a normal
+ * connection that's used for queries */
+ /* OAuth v2 */
+ char *oauth_issuer; /* token issuer URL */
5: 5488ac25f5 ! 3: 27251b63c1 backend: add OAUTHBEARER SASL mechanism
@@ src/backend/utils/misc/guc_tables.c
#include "nodes/queryjumble.h"
#include "optimizer/cost.h"
@@ src/backend/utils/misc/guc_tables.c: struct config_string ConfigureNamesString[] =
- check_debug_io_direct, assign_debug_io_direct, NULL
+ check_standby_slot_names, assign_standby_slot_names, NULL
},
+ {
6: fdbad1976a ! 4: 78617fa9ba Introduce OAuth validator libraries
@@ src/backend/utils/misc/guc_tables.c: struct config_string ConfigureNamesString[]
NULL, NULL, NULL
},
- ## src/bin/pg_combinebackup/Makefile ##
-@@ src/bin/pg_combinebackup/Makefile: OBJS = \
- all: pg_combinebackup
-
- pg_combinebackup: $(OBJS) | submake-libpgport submake-libpgfeutils
-- $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
-+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(libpq_pgport) $(LIBS) -o $@$(X)
-
- install: all installdirs
- $(INSTALL_PROGRAM) pg_combinebackup$(X) '$(DESTDIR)$(bindir)/pg_combinebackup$(X)'
-
## src/common/Makefile ##
@@ src/common/Makefile: override CPPFLAGS += -DVAL_LDFLAGS_SL="\"$(LDFLAGS_SL)\""
override CPPFLAGS += -DVAL_LIBS="\"$(LIBS)\""
7: e1da97fb50 ! 5: 1b871542f5 Add pytest suite for OAuth
@@ src/test/python/client/test_oauth.py (new)
+ discovery_uri = issuer + "/.well-known/openid-configuration"
+ access_token = secrets.token_urlsafe()
+
-+ sock, _ = accept(
++ sock, client = accept(
+ oauth_issuer=issuer,
+ oauth_client_id="some-id",
+ oauth_scope=scope,
@@ src/test/python/client/test_oauth.py (new)
+ assert call.openid_configuration.decode() == discovery_uri
+ assert call.scope == (None if scope is None else scope.encode())
+
-+ # Make sure we cleaned up after ourselves.
++ # Make sure we clean up after ourselves when the connection is finished.
++ client.check_completed()
+ assert cleanup_calls == 1
+
+
8: 185f9902fd ! 6: e6c5d94682 XXX temporary patches to build and test
@@ Metadata
## Commit message ##
XXX temporary patches to build and test
- - the new pg_combinebackup utility uses JSON in the frontend without
- 0001; has something changed?
- construct 2.10.70 has some incompatibilities with the current tests
- temporarily skip the exit check (from Daniel Gustafsson); this needs
to be turned into an exception for curl rather than a plain exit call
- ## src/bin/pg_combinebackup/Makefile ##
-@@ src/bin/pg_combinebackup/Makefile: include $(top_builddir)/src/Makefile.global
-
- override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
- LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils
-+# TODO: fix this properly
-+LDFLAGS_INTERNAL += -lpgcommon $(libpq_pgport)
-
- OBJS = \
- $(WIN32RES) \
-@@ src/bin/pg_combinebackup/Makefile: OBJS = \
-
- all: pg_combinebackup
-
--pg_combinebackup: $(OBJS) | submake-libpgport submake-libpgfeutils
-- $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(libpq_pgport) $(LIBS) -o $@$(X)
-+pg_combinebackup: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
-+ $(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
-
- install: all installdirs
- $(INSTALL_PROGRAM) pg_combinebackup$(X) '$(DESTDIR)$(bindir)/pg_combinebackup$(X)'
-
- ## src/bin/pg_combinebackup/meson.build ##
-@@ src/bin/pg_combinebackup/meson.build: endif
-
- pg_combinebackup = executable('pg_combinebackup',
- pg_combinebackup_sources,
-- dependencies: [frontend_code],
-+ # XXX linking against libpq isn't good, but how was JSON working?
-+ dependencies: [frontend_code, libpq],
- kwargs: default_bin_args,
- )
- bin_targets += pg_combinebackup
-
- ## src/bin/pg_verifybackup/Makefile ##
-@@ src/bin/pg_verifybackup/Makefile: top_builddir = ../../..
- include $(top_builddir)/src/Makefile.global
-
- # We need libpq only because fe_utils does.
--LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
-+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils -lpgcommon $(libpq_pgport)
-
- OBJS = \
- $(WIN32RES) \
-
## src/interfaces/libpq/Makefile ##
@@ src/interfaces/libpq/Makefile: libpq-refs-stamp: $(shlib)
ifneq ($(enable_coverage), yes)
9: c4d850a7c4 = 7: decc90579a WIP: Python OAuth provider implementation
v21-0003-backend-add-OAUTHBEARER-SASL-mechanism.patch (application/octet-stream, 41.7K) download
From 27251b63c1ee4288b6974b16d70085ee558a74b0 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 4 May 2021 16:21:11 -0700
Subject: [PATCH v21 3/7] backend: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) on the server side. This adds a new
auth method, oauth, to pg_hba.
Because OAuth implementations vary so wildly, and bearer token
validation is heavily dependent on the issuing party, authn/z is done by
communicating with an external program: the oauth_validator_command.
This command must do the following:
1. Receive the bearer token by reading its contents from a file
descriptor passed from the server. (The numeric value of this
descriptor may be inserted into the oauth_validator_command using the
%f specifier.)
This MUST be the first action the command performs. The server will
not begin reading stdout from the command until the token has been
read in full, so if the command tries to print anything and hits a
buffer limit, the backend will deadlock and time out.
2. Validate the bearer token. The correct way to do this depends on the
issuer, but it generally involves either cryptographic operations to
prove that the token was issued by a trusted party, or the
presentation of the bearer token to some other party so that _it_ can
perform validation.
The command MUST maintain confidentiality of the bearer token, since
in most cases it can be used just like a password. (There are ways to
cryptographically bind tokens to client certificates, but they are
way beyond the scope of this commit message.)
If the token cannot be validated, the command must exit with a
non-zero status. Further authentication/authorization is pointless if
the bearer token wasn't issued by someone you trust.
3. Authenticate the user, authorize the user, or both:
a. To authenticate the user, use the bearer token to retrieve some
trusted identifier string for the end user. The exact process for
this is, again, issuer-dependent. The command should print the
authenticated identity string to stdout, followed by a newline.
If the user cannot be authenticated, the validator should not
print anything to stdout. It should also exit with a non-zero
status, unless the token may be used to authorize the connection
through some other means (see below).
On a success, the command may then exit with a zero success code.
By default, the server will then check to make sure the identity
string matches the role that is being used (or matches a usermap
entry, if one is in use).
b. To optionally authorize the user, in combination with the HBA
option trust_validator_authz=1 (see below), the validator simply
returns a zero exit code if the client should be allowed to
connect with its presented role (which can be passed to the
command using the %r specifier), or a non-zero code otherwise.
The hard part is in determining whether the given token truly
authorizes the client to use the given role, which must
unfortunately be left as an exercise to the reader.
This obviously requires some care, as a poorly implemented token
validator may silently open the entire database to anyone with a
bearer token. But it may be a more portable approach, since OAuth
is designed as an authorization framework, not an authentication
framework. For example, the user's bearer token could carry an
"allow_superuser_access" claim, which would authorize pseudonymous
database access as any role. It's then up to the OAuth system
administrators to ensure that allow_superuser_access is doled out
only to the proper users.
c. It's possible that the user can be successfully authenticated but
isn't authorized to connect. In this case, the command may print
the authenticated ID and then fail with a non-zero exit code.
(This makes it easier to see what's going on in the Postgres
logs.)
4. Token validators may optionally log to stderr. This will be printed
verbatim into the Postgres server logs.
The oauth method supports the following HBA options (but note that two
of them are not optional, since we have no way of choosing sensible
defaults):
issuer: Required. The URL of the OAuth issuing party, which the client
must contact to receive a bearer token.
Some real-world examples as of time of writing:
- https://accounts.google.com
- https://login.microsoft.com/[tenant-id]/v2.0
scope: Required. The OAuth scope(s) required for the server to
authenticate and/or authorize the user. This is heavily
deployment-specific, but a simple example is "openid email".
map: Optional. Specify a standard PostgreSQL user map; this works
the same as with other auth methods such as peer. If a map is
not specified, the user ID returned by the token validator
must exactly match the role that's being requested (but see
trust_validator_authz, below).
trust_validator_authz:
Optional. When set to 1, this allows the token validator to
take full control of the authorization process. Standard user
mapping is skipped: if the validator command succeeds, the
client is allowed to connect under its desired role and no
further checks are done.
Unlike the client, servers support OAuth without needing to be built
against libiddawc (since the responsibility for "speaking" OAuth/OIDC
correctly is delegated entirely to the oauth_validator_command).
Several TODOs:
- port to platforms other than "modern Linux/BSD"
- overhaul the communication with oauth_validator_command, which is
currently a bad hack on OpenPipeStream()
- implement more helpful handling of HBA misconfigurations
- use logdetail during auth failures
- deal with role names that can't be safely passed to system() without
shell-escaping
- allow passing the configured issuer to the oauth_validator_command, to
deal with multi-issuer setups
- ...and more.
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
---
src/backend/libpq/Makefile | 1 +
src/backend/libpq/auth-oauth.c | 883 ++++++++++++++++++++++++++++
src/backend/libpq/auth-sasl.c | 10 +-
src/backend/libpq/auth-scram.c | 4 +-
src/backend/libpq/auth.c | 26 +-
src/backend/libpq/hba.c | 31 +-
src/backend/libpq/meson.build | 1 +
src/backend/utils/misc/guc_tables.c | 12 +
src/include/libpq/auth.h | 17 +
src/include/libpq/hba.h | 6 +-
src/include/libpq/oauth.h | 24 +
src/include/libpq/sasl.h | 11 +
src/tools/pgindent/typedefs.list | 1 +
13 files changed, 995 insertions(+), 32 deletions(-)
create mode 100644 src/backend/libpq/auth-oauth.c
create mode 100644 src/include/libpq/oauth.h
diff --git a/src/backend/libpq/Makefile b/src/backend/libpq/Makefile
index 6d385fd6a4..98eb2a8242 100644
--- a/src/backend/libpq/Makefile
+++ b/src/backend/libpq/Makefile
@@ -15,6 +15,7 @@ include $(top_builddir)/src/Makefile.global
# be-fsstubs is here for historical reasons, probably belongs elsewhere
OBJS = \
+ auth-oauth.o \
auth-sasl.o \
auth-scram.o \
auth.o \
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
new file mode 100644
index 0000000000..16596c089a
--- /dev/null
+++ b/src/backend/libpq/auth-oauth.c
@@ -0,0 +1,883 @@
+/*-------------------------------------------------------------------------
+ *
+ * auth-oauth.c
+ * Server-side implementation of the SASL OAUTHBEARER mechanism.
+ *
+ * See the following RFC for more details:
+ * - RFC 7628: https://tools.ietf.org/html/rfc7628
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/backend/libpq/auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <unistd.h>
+#include <fcntl.h>
+
+#include "common/oauth-common.h"
+#include "lib/stringinfo.h"
+#include "libpq/auth.h"
+#include "libpq/hba.h"
+#include "libpq/oauth.h"
+#include "libpq/sasl.h"
+#include "storage/fd.h"
+#include "utils/json.h"
+
+/* GUC */
+char *oauth_validator_command;
+
+static void oauth_get_mechanisms(Port *port, StringInfo buf);
+static void *oauth_init(Port *port, const char *selected_mech, const char *shadow_pass);
+static int oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, const char **logdetail);
+
+/* Mechanism declaration */
+const pg_be_sasl_mech pg_be_oauth_mech = {
+ oauth_get_mechanisms,
+ oauth_init,
+ oauth_exchange,
+
+ PG_MAX_AUTH_TOKEN_LENGTH,
+};
+
+
+typedef enum
+{
+ OAUTH_STATE_INIT = 0,
+ OAUTH_STATE_ERROR,
+ OAUTH_STATE_FINISHED,
+} oauth_state;
+
+struct oauth_ctx
+{
+ oauth_state state;
+ Port *port;
+ const char *issuer;
+ const char *scope;
+};
+
+static char *sanitize_char(char c);
+static char *parse_kvpairs_for_auth(char **input);
+static void generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen);
+static bool validate(Port *port, const char *auth, const char **logdetail);
+static bool run_validator_command(Port *port, const char *token);
+static bool check_exit(FILE **fh, const char *command);
+static bool set_cloexec(int fd);
+static bool username_ok_for_shell(const char *username);
+
+#define KVSEP 0x01
+#define AUTH_KEY "auth"
+#define BEARER_SCHEME "Bearer "
+
+static void
+oauth_get_mechanisms(Port *port, StringInfo buf)
+{
+ /* Only OAUTHBEARER is supported. */
+ appendStringInfoString(buf, OAUTHBEARER_NAME);
+ appendStringInfoChar(buf, '\0');
+}
+
+static void *
+oauth_init(Port *port, const char *selected_mech, const char *shadow_pass)
+{
+ struct oauth_ctx *ctx;
+
+ if (strcmp(selected_mech, OAUTHBEARER_NAME))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("client selected an invalid SASL authentication mechanism")));
+
+ ctx = palloc0(sizeof(*ctx));
+
+ ctx->state = OAUTH_STATE_INIT;
+ ctx->port = port;
+
+ Assert(port->hba);
+ ctx->issuer = port->hba->oauth_issuer;
+ ctx->scope = port->hba->oauth_scope;
+
+ return ctx;
+}
+
+static int
+oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, const char **logdetail)
+{
+ char *p;
+ char cbind_flag;
+ char *auth;
+
+ struct oauth_ctx *ctx = opaq;
+
+ *output = NULL;
+ *outputlen = -1;
+
+ /*
+ * If the client didn't include an "Initial Client Response" in the
+ * SASLInitialResponse message, send an empty challenge, to which the
+ * client will respond with the same data that usually comes in the
+ * Initial Client Response.
+ */
+ if (input == NULL)
+ {
+ Assert(ctx->state == OAUTH_STATE_INIT);
+
+ *output = pstrdup("");
+ *outputlen = 0;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ /*
+ * Check that the input length agrees with the string length of the input.
+ */
+ if (inputlen == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The message is empty.")));
+ if (inputlen != strlen(input))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message length does not match input length.")));
+
+ switch (ctx->state)
+ {
+ case OAUTH_STATE_INIT:
+ /* Handle this case below. */
+ break;
+
+ case OAUTH_STATE_ERROR:
+
+ /*
+ * Only one response is valid for the client during authentication
+ * failure: a single kvsep.
+ */
+ if (inputlen != 1 || *input != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Client did not send a kvsep response.")));
+
+ /* The (failed) handshake is now complete. */
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_FAILURE;
+
+ default:
+ elog(ERROR, "invalid OAUTHBEARER exchange state");
+ return PG_SASL_EXCHANGE_FAILURE;
+ }
+
+ /* Handle the client's initial message. */
+ p = pstrdup(input);
+
+ /*
+ * OAUTHBEARER does not currently define a channel binding (so there is no
+ * OAUTHBEARER-PLUS, and we do not accept a 'p' specifier). We accept a
+ * 'y' specifier purely for the remote chance that a future specification
+ * could define one; then future clients can still interoperate with this
+ * server implementation. 'n' is the expected case.
+ */
+ cbind_flag = *p;
+ switch (cbind_flag)
+ {
+ case 'p':
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The server does not support channel binding for OAuth, but the client message includes channel binding data.")));
+ break;
+
+ case 'y': /* fall through */
+ case 'n':
+ p++;
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Comma expected, but found character \"%s\".",
+ sanitize_char(*p))));
+ p++;
+ break;
+
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected channel-binding flag %s.",
+ sanitize_char(cbind_flag))));
+ }
+
+ /*
+ * Forbid optional authzid (authorization identity). We don't support it.
+ */
+ if (*p == 'a')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("client uses authorization identity, but it is not supported")));
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected attribute %s in client-first-message.",
+ sanitize_char(*p))));
+ p++;
+
+ /* All remaining fields are separated by the RFC's kvsep (\x01). */
+ if (*p != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Key-value separator expected, but found character %s.",
+ sanitize_char(*p))));
+ p++;
+
+ auth = parse_kvpairs_for_auth(&p);
+ if (!auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message does not contain an auth value.")));
+
+ /* We should be at the end of our message. */
+ if (*p)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains additional data after the final terminator.")));
+
+ if (!validate(ctx->port, auth, logdetail))
+ {
+ generate_error_response(ctx, output, outputlen);
+
+ ctx->state = OAUTH_STATE_ERROR;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_SUCCESS;
+}
+
+/*
+ * Convert an arbitrary byte to printable form. For error messages.
+ *
+ * If it's a printable ASCII character, print it as a single character.
+ * otherwise, print it in hex.
+ *
+ * The returned pointer points to a static buffer.
+ */
+static char *
+sanitize_char(char c)
+{
+ static char buf[5];
+
+ if (c >= 0x21 && c <= 0x7E)
+ snprintf(buf, sizeof(buf), "'%c'", c);
+ else
+ snprintf(buf, sizeof(buf), "0x%02x", (unsigned char) c);
+ return buf;
+}
+
+/*
+ * Performs syntactic validation of a key and value from the initial client
+ * response. (Semantic validation of interesting values must be performed
+ * later.)
+ */
+static void
+validate_kvpair(const char *key, const char *val)
+{
+ /*-----
+ * From Sec 3.1:
+ * key = 1*(ALPHA)
+ */
+ static const char *key_allowed_set =
+ "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
+
+ size_t span;
+
+ if (!key[0])
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an empty key name.")));
+
+ span = strspn(key, key_allowed_set);
+ if (key[span] != '\0')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an invalid key name.")));
+
+ /*-----
+ * From Sec 3.1:
+ * value = *(VCHAR / SP / HTAB / CR / LF )
+ *
+ * The VCHAR (visible character) class is large; a loop is more
+ * straightforward than strspn().
+ */
+ for (; *val; ++val)
+ {
+ if (0x21 <= *val && *val <= 0x7E)
+ continue; /* VCHAR */
+
+ switch (*val)
+ {
+ case ' ':
+ case '\t':
+ case '\r':
+ case '\n':
+ continue; /* SP, HTAB, CR, LF */
+
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an invalid value.")));
+ }
+ }
+}
+
+/*
+ * Consumes all kvpairs in an OAUTHBEARER exchange message. If the "auth" key is
+ * found, its value is returned.
+ */
+static char *
+parse_kvpairs_for_auth(char **input)
+{
+ char *pos = *input;
+ char *auth = NULL;
+
+ /*----
+ * The relevant ABNF, from Sec. 3.1:
+ *
+ * kvsep = %x01
+ * key = 1*(ALPHA)
+ * value = *(VCHAR / SP / HTAB / CR / LF )
+ * kvpair = key "=" value kvsep
+ * ;;gs2-header = See RFC 5801
+ * client-resp = (gs2-header kvsep *kvpair kvsep) / kvsep
+ *
+ * By the time we reach this code, the gs2-header and initial kvsep have
+ * already been validated. We start at the beginning of the first kvpair.
+ */
+
+ while (*pos)
+ {
+ char *end;
+ char *sep;
+ char *key;
+ char *value;
+
+ /*
+ * Find the end of this kvpair. Note that input is null-terminated by
+ * the SASL code, so the strchr() is bounded.
+ */
+ end = strchr(pos, KVSEP);
+ if (!end)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an unterminated key/value pair.")));
+ *end = '\0';
+
+ if (pos == end)
+ {
+ /* Empty kvpair, signifying the end of the list. */
+ *input = pos + 1;
+ return auth;
+ }
+
+ /*
+ * Find the end of the key name.
+ */
+ sep = strchr(pos, '=');
+ if (!sep)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains a key without a value.")));
+ *sep = '\0';
+
+ /* Both key and value are now safely terminated. */
+ key = pos;
+ value = sep + 1;
+ validate_kvpair(key, value);
+
+ if (!strcmp(key, AUTH_KEY))
+ {
+ if (auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains multiple auth values.")));
+
+ auth = value;
+ }
+ else
+ {
+ /*
+ * The RFC also defines the host and port keys, but they are not
+ * required for OAUTHBEARER and we do not use them. Also, per Sec.
+ * 3.1, any key/value pairs we don't recognize must be ignored.
+ */
+ }
+
+ /* Move to the next pair. */
+ pos = end + 1;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message did not contain a final terminator.")));
+
+ return NULL; /* unreachable */
+}
+
+static void
+generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
+{
+ StringInfoData buf;
+ StringInfoData issuer;
+
+ /*
+ * The admin needs to set an issuer and scope for OAuth to work. There's
+ * not really a way to hide this from the user, either, because we can't
+ * choose a "default" issuer, so be honest in the failure message.
+ *
+ * TODO: see if there's a better place to fail, earlier than this.
+ */
+ if (!ctx->issuer || !ctx->scope)
+ ereport(FATAL,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("OAuth is not properly configured for this user"),
+ errdetail_log("The issuer and scope parameters must be set in pg_hba.conf.")));
+
+ /*------
+ * Build the .well-known URI based on our issuer.
+ * TODO: RFC 8414 defines a competing well-known URI, so we'll probably
+ * have to make this configurable too.
+ */
+ initStringInfo(&issuer);
+ appendStringInfoString(&issuer, ctx->issuer);
+ appendStringInfoString(&issuer, "/.well-known/openid-configuration");
+
+ initStringInfo(&buf);
+
+ /*
+ * TODO: note that escaping here should be belt-and-suspenders, since
+ * escapable characters aren't valid in either the issuer URI or the scope
+ * list, but the HBA doesn't enforce that yet.
+ */
+ appendStringInfoString(&buf, "{ \"status\": \"invalid_token\", ");
+
+ appendStringInfoString(&buf, "\"openid-configuration\": ");
+ escape_json(&buf, issuer.data);
+ pfree(issuer.data);
+
+ appendStringInfoString(&buf, ", \"scope\": ");
+ escape_json(&buf, ctx->scope);
+
+ appendStringInfoString(&buf, " }");
+
+ *output = buf.data;
+ *outputlen = buf.len;
+}
+
+static bool
+validate(Port *port, const char *auth, const char **logdetail)
+{
+ static const char *const b64_set =
+ "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-._~+/";
+
+ const char *token;
+ size_t span;
+ int ret;
+
+ /* TODO: handle logdetail when the test framework can check it */
+
+ /*-----
+ * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
+ * 2.1:
+ *
+ * b64token = 1*( ALPHA / DIGIT /
+ * "-" / "." / "_" / "~" / "+" / "/" ) *"="
+ * credentials = "Bearer" 1*SP b64token
+ *
+ * The "credentials" construction is what we receive in our auth value.
+ *
+ * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
+ * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
+ * compared case-insensitively. (This is not mentioned in RFC 6750, but
+ * it's pointed out in RFC 7628 Sec. 4.)
+ *
+ * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
+ */
+ if (pg_strncasecmp(auth, BEARER_SCHEME, strlen(BEARER_SCHEME)))
+ return false;
+
+ /* Pull the bearer token out of the auth value. */
+ token = auth + strlen(BEARER_SCHEME);
+
+ /* Swallow any additional spaces. */
+ while (*token == ' ')
+ token++;
+
+ /*
+ * Before invoking the validator command, sanity-check the token format to
+ * avoid any injection attacks later in the chain. Invalid formats are
+ * technically a protocol violation, but don't reflect any information
+ * about the sensitive Bearer token back to the client; log at COMMERROR
+ * instead.
+ */
+
+ /* Tokens must not be empty. */
+ if (!*token)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Bearer token is empty.")));
+ return false;
+ }
+
+ /*
+ * Make sure the token contains only allowed characters. Tokens may end
+ * with any number of '=' characters.
+ */
+ span = strspn(token, b64_set);
+ while (token[span] == '=')
+ span++;
+
+ if (token[span] != '\0')
+ {
+ /*
+ * This error message could be more helpful by printing the
+ * problematic character(s), but that'd be a bit like printing a piece
+ * of someone's password into the logs.
+ */
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Bearer token is not in the correct format.")));
+ return false;
+ }
+
+ /* Have the validator check the token. */
+ if (!run_validator_command(port, token))
+ return false;
+
+ if (port->hba->oauth_skip_usermap)
+ {
+ /*
+ * If the validator is our authorization authority, we're done.
+ * Authentication may or may not have been performed depending on the
+ * validator implementation; all that matters is that the validator
+ * says the user can log in with the target role.
+ */
+ return true;
+ }
+
+ /* Make sure the validator authenticated the user. */
+ if (!MyClientConnectionInfo.authn_id)
+ {
+ /* TODO: use logdetail; reduce message duplication */
+ ereport(LOG,
+ (errmsg("OAuth bearer authentication failed for user \"%s\": validator provided no identity",
+ port->user_name)));
+ return false;
+ }
+
+ /* Finally, check the user map. */
+ ret = check_usermap(port->hba->usermap, port->user_name,
+ MyClientConnectionInfo.authn_id, false);
+ return (ret == STATUS_OK);
+}
+
+static bool
+run_validator_command(Port *port, const char *token)
+{
+ bool success = false;
+ int rc;
+ int pipefd[2];
+ int rfd = -1;
+ int wfd = -1;
+
+ StringInfoData command = {0};
+ char *p;
+ FILE *fh = NULL;
+
+ ssize_t written;
+ char *line = NULL;
+ size_t size = 0;
+ ssize_t len;
+
+ Assert(oauth_validator_command);
+
+ if (!oauth_validator_command[0])
+ {
+ ereport(COMMERROR,
+ (errmsg("oauth_validator_command is not set"),
+ errhint("To allow OAuth authenticated connections, set "
+ "oauth_validator_command in postgresql.conf.")));
+ return false;
+ }
+
+ /*------
+ * Since popen() is unidirectional, open up a pipe for the other
+ * direction. Use CLOEXEC to ensure that our write end doesn't
+ * accidentally get copied into child processes, which would prevent us
+ * from closing it cleanly.
+ *
+ * XXX this is ugly. We should just read from the child process's stdout,
+ * but that's a lot more code.
+ * XXX by bypassing the popen API, we open the potential of process
+ * deadlock. Clearly document child process requirements (i.e. the child
+ * MUST read all data off of the pipe before writing anything).
+ * TODO: port to Windows using _pipe().
+ */
+ rc = pipe(pipefd);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create child pipe: %m")));
+ return false;
+ }
+
+ rfd = pipefd[0];
+ wfd = pipefd[1];
+
+ if (!set_cloexec(wfd))
+ {
+ /* error message was already logged */
+ goto cleanup;
+ }
+
+ /*----------
+ * Construct the command, substituting any recognized %-specifiers:
+ *
+ * %f: the file descriptor of the input pipe
+ * %r: the role that the client wants to assume (port->user_name)
+ * %%: a literal '%'
+ */
+ initStringInfo(&command);
+
+ for (p = oauth_validator_command; *p; p++)
+ {
+ if (p[0] == '%')
+ {
+ switch (p[1])
+ {
+ case 'f':
+ appendStringInfo(&command, "%d", rfd);
+ p++;
+ break;
+ case 'r':
+
+ /*
+ * TODO: decide how this string should be escaped. The
+ * role is controlled by the client, so if we don't escape
+ * it, command injections are inevitable.
+ *
+ * This is probably an indication that the role name needs
+ * to be communicated to the validator process in some
+ * other way. For this proof of concept, just be
+ * incredibly strict about the characters that are allowed
+ * in user names.
+ */
+ if (!username_ok_for_shell(port->user_name))
+ goto cleanup;
+
+ appendStringInfoString(&command, port->user_name);
+ p++;
+ break;
+ case '%':
+ appendStringInfoChar(&command, '%');
+ p++;
+ break;
+ default:
+ appendStringInfoChar(&command, p[0]);
+ }
+ }
+ else
+ appendStringInfoChar(&command, p[0]);
+ }
+
+ /* Execute the command. */
+ fh = OpenPipeStream(command.data, "r");
+ if (!fh)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("opening pipe to OAuth validator: %m")));
+ goto cleanup;
+ }
+
+ /* We don't need the read end of the pipe anymore. */
+ close(rfd);
+ rfd = -1;
+
+ /* Give the command the token to validate. */
+ written = write(wfd, token, strlen(token));
+ if (written != strlen(token))
+ {
+ /* TODO must loop for short writes, EINTR et al */
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write token to child pipe: %m")));
+ goto cleanup;
+ }
+
+ close(wfd);
+ wfd = -1;
+
+ /*-----
+ * Read the command's response.
+ *
+ * TODO: getline() is probably too new to use, unfortunately.
+ * TODO: loop over all lines
+ */
+ if ((len = getline(&line, &size, fh)) >= 0)
+ {
+ /* TODO: fail if the authn_id doesn't end with a newline */
+ if (len > 0)
+ line[len - 1] = '\0';
+
+ set_authn_id(port, line);
+ }
+ else if (ferror(fh))
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from command \"%s\": %m",
+ command.data)));
+ goto cleanup;
+ }
+
+ /* Make sure the command exits cleanly. */
+ if (!check_exit(&fh, command.data))
+ {
+ /* error message already logged */
+ goto cleanup;
+ }
+
+ /* Done. */
+ success = true;
+
+cleanup:
+ if (line)
+ free(line);
+
+ /*
+ * In the successful case, the pipe fds are already closed. For the error
+ * case, always close out the pipe before waiting for the command, to
+ * prevent deadlock.
+ */
+ if (rfd >= 0)
+ close(rfd);
+ if (wfd >= 0)
+ close(wfd);
+
+ if (fh)
+ {
+ Assert(!success);
+ check_exit(&fh, command.data);
+ }
+
+ if (command.data)
+ pfree(command.data);
+
+ return success;
+}
+
+static bool
+check_exit(FILE **fh, const char *command)
+{
+ int rc;
+
+ rc = ClosePipeStream(*fh);
+ *fh = NULL;
+
+ if (rc == -1)
+ {
+ /* pclose() itself failed. */
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close pipe to command \"%s\": %m",
+ command)));
+ }
+ else if (rc != 0)
+ {
+ char *reason = wait_result_to_str(rc);
+
+ ereport(COMMERROR,
+ (errmsg("failed to execute command \"%s\": %s",
+ command, reason)));
+
+ pfree(reason);
+ }
+
+ return (rc == 0);
+}
+
+static bool
+set_cloexec(int fd)
+{
+ int flags;
+ int rc;
+
+ flags = fcntl(fd, F_GETFD);
+ if (flags == -1)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not get fd flags for child pipe: %m")));
+ return false;
+ }
+
+ rc = fcntl(fd, F_SETFD, flags | FD_CLOEXEC);
+ if (rc < 0)
+ {
+ ereport(COMMERROR,
+ (errcode_for_file_access(),
+ errmsg("could not set FD_CLOEXEC for child pipe: %m")));
+ return false;
+ }
+
+ return true;
+}
+
+/*
+ * XXX This should go away eventually and be replaced with either a proper
+ * escape or a different strategy for communication with the validator command.
+ */
+static bool
+username_ok_for_shell(const char *username)
+{
+ /* This set is borrowed from fe_utils' appendShellStringNoError(). */
+ static const char *const allowed =
+ "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-_./:";
+ size_t span;
+
+ Assert(username && username[0]); /* should have already been checked */
+
+ span = strspn(username, allowed);
+ if (username[span] != '\0')
+ {
+ ereport(COMMERROR,
+ (errmsg("PostgreSQL user name contains unsafe characters and cannot be passed to the OAuth validator")));
+ return false;
+ }
+
+ return true;
+}
diff --git a/src/backend/libpq/auth-sasl.c b/src/backend/libpq/auth-sasl.c
index 08b24d90b4..4039e7fa3e 100644
--- a/src/backend/libpq/auth-sasl.c
+++ b/src/backend/libpq/auth-sasl.c
@@ -20,14 +20,6 @@
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
-/*
- * Maximum accepted size of SASL messages.
- *
- * The messages that the server or libpq generate are much smaller than this,
- * but have some headroom.
- */
-#define PG_MAX_SASL_MESSAGE_LENGTH 1024
-
/*
* Perform a SASL exchange with a libpq client, using a specific mechanism
* implementation.
@@ -103,7 +95,7 @@ CheckSASLAuth(const pg_be_sasl_mech *mech, Port *port, char *shadow_pass,
/* Get the actual SASL message */
initStringInfo(&buf);
- if (pq_getmessage(&buf, PG_MAX_SASL_MESSAGE_LENGTH))
+ if (pq_getmessage(&buf, mech->max_message_length))
{
/* EOF - pq_getmessage already logged error */
pfree(buf.data);
diff --git a/src/backend/libpq/auth-scram.c b/src/backend/libpq/auth-scram.c
index 4161959914..486a34e719 100644
--- a/src/backend/libpq/auth-scram.c
+++ b/src/backend/libpq/auth-scram.c
@@ -113,7 +113,9 @@ static int scram_exchange(void *opaq, const char *input, int inputlen,
const pg_be_sasl_mech pg_be_scram_mech = {
scram_get_mechanisms,
scram_init,
- scram_exchange
+ scram_exchange,
+
+ PG_MAX_SASL_MESSAGE_LENGTH
};
/*
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 2b607c5270..0a5a8640fc 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -29,6 +29,7 @@
#include "libpq/auth.h"
#include "libpq/crypt.h"
#include "libpq/libpq.h"
+#include "libpq/oauth.h"
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
#include "libpq/scram.h"
@@ -45,7 +46,6 @@
*/
static void auth_failed(Port *port, int status, const char *logdetail);
static char *recv_password_packet(Port *port);
-static void set_authn_id(Port *port, const char *id);
/*----------------------------------------------------------------
@@ -201,22 +201,6 @@ static int CheckRADIUSAuth(Port *port);
static int PerformRadiusTransaction(const char *server, const char *secret, const char *portstr, const char *identifier, const char *user_name, const char *passwd);
-/*
- * Maximum accepted size of GSS and SSPI authentication tokens.
- * We also use this as a limit on ordinary password packet lengths.
- *
- * Kerberos tickets are usually quite small, but the TGTs issued by Windows
- * domain controllers include an authorization field known as the Privilege
- * Attribute Certificate (PAC), which contains the user's Windows permissions
- * (group memberships etc.). The PAC is copied into all tickets obtained on
- * the basis of this TGT (even those issued by Unix realms which the Windows
- * realm trusts), and can be several kB in size. The maximum token size
- * accepted by Windows systems is determined by the MaxAuthToken Windows
- * registry setting. Microsoft recommends that it is not set higher than
- * 65535 bytes, so that seems like a reasonable limit for us as well.
- */
-#define PG_MAX_AUTH_TOKEN_LENGTH 65535
-
/*----------------------------------------------------------------
* Global authentication functions
*----------------------------------------------------------------
@@ -305,6 +289,9 @@ auth_failed(Port *port, int status, const char *logdetail)
case uaRADIUS:
errstr = gettext_noop("RADIUS authentication failed for user \"%s\"");
break;
+ case uaOAuth:
+ errstr = gettext_noop("OAuth bearer authentication failed for user \"%s\"");
+ break;
default:
errstr = gettext_noop("authentication failed for user \"%s\": invalid authentication method");
break;
@@ -340,7 +327,7 @@ auth_failed(Port *port, int status, const char *logdetail)
* lifetime of MyClientConnectionInfo, so it is safe to pass a string that is
* managed by an external library.
*/
-static void
+void
set_authn_id(Port *port, const char *id)
{
Assert(id);
@@ -627,6 +614,9 @@ ClientAuthentication(Port *port)
case uaTrust:
status = STATUS_OK;
break;
+ case uaOAuth:
+ status = CheckSASLAuth(&pg_be_oauth_mech, port, NULL, NULL);
+ break;
}
if ((status == STATUS_OK && port->hba->clientcert == clientCertFull)
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index d506c3c0b7..e592bedf9f 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -114,7 +114,8 @@ static const char *const UserAuthName[] =
"ldap",
"cert",
"radius",
- "peer"
+ "peer",
+ "oauth",
};
/*
@@ -1743,6 +1744,8 @@ parse_hba_line(TokenizedAuthLine *tok_line, int elevel)
#endif
else if (strcmp(token->string, "radius") == 0)
parsedline->auth_method = uaRADIUS;
+ else if (strcmp(token->string, "oauth") == 0)
+ parsedline->auth_method = uaOAuth;
else
{
ereport(elevel,
@@ -2062,8 +2065,9 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->auth_method != uaPeer &&
hbaline->auth_method != uaGSS &&
hbaline->auth_method != uaSSPI &&
- hbaline->auth_method != uaCert)
- INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, and cert"));
+ hbaline->auth_method != uaCert &&
+ hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, cert, and oauth"));
hbaline->usermap = pstrdup(val);
}
else if (strcmp(name, "clientcert") == 0)
@@ -2446,6 +2450,27 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->radiusidentifiers = parsed_identifiers;
hbaline->radiusidentifiers_s = pstrdup(val);
}
+ else if (strcmp(name, "issuer") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("issuer", gettext_noop("oauth"));
+ hbaline->oauth_issuer = pstrdup(val);
+ }
+ else if (strcmp(name, "scope") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("scope", gettext_noop("oauth"));
+ hbaline->oauth_scope = pstrdup(val);
+ }
+ else if (strcmp(name, "trust_validator_authz") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("trust_validator_authz", gettext_noop("oauth"));
+ if (strcmp(val, "1") == 0)
+ hbaline->oauth_skip_usermap = true;
+ else
+ hbaline->oauth_skip_usermap = false;
+ }
else
{
ereport(elevel,
diff --git a/src/backend/libpq/meson.build b/src/backend/libpq/meson.build
index 7c65314512..c85527fb01 100644
--- a/src/backend/libpq/meson.build
+++ b/src/backend/libpq/meson.build
@@ -1,6 +1,7 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
backend_sources += files(
+ 'auth-oauth.c',
'auth-sasl.c',
'auth-scram.c',
'auth.c',
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 1e71e7db4a..773a75c96f 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -48,6 +48,7 @@
#include "jit/jit.h"
#include "libpq/auth.h"
#include "libpq/libpq.h"
+#include "libpq/oauth.h"
#include "libpq/scram.h"
#include "nodes/queryjumble.h"
#include "optimizer/cost.h"
@@ -4684,6 +4685,17 @@ struct config_string ConfigureNamesString[] =
check_standby_slot_names, assign_standby_slot_names, NULL
},
+ {
+ {"oauth_validator_command", PGC_SIGHUP, CONN_AUTH_AUTH,
+ gettext_noop("Command to validate OAuth v2 bearer tokens."),
+ NULL,
+ GUC_SUPERUSER_ONLY | GUC_NOT_IN_SAMPLE
+ },
+ &oauth_validator_command,
+ "",
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, NULL, NULL, NULL, NULL
diff --git a/src/include/libpq/auth.h b/src/include/libpq/auth.h
index 227b41daf6..22f6ab9f1d 100644
--- a/src/include/libpq/auth.h
+++ b/src/include/libpq/auth.h
@@ -16,6 +16,22 @@
#include "libpq/libpq-be.h"
+/*
+ * Maximum accepted size of GSS and SSPI authentication tokens.
+ * We also use this as a limit on ordinary password packet lengths.
+ *
+ * Kerberos tickets are usually quite small, but the TGTs issued by Windows
+ * domain controllers include an authorization field known as the Privilege
+ * Attribute Certificate (PAC), which contains the user's Windows permissions
+ * (group memberships etc.). The PAC is copied into all tickets obtained on
+ * the basis of this TGT (even those issued by Unix realms which the Windows
+ * realm trusts), and can be several kB in size. The maximum token size
+ * accepted by Windows systems is determined by the MaxAuthToken Windows
+ * registry setting. Microsoft recommends that it is not set higher than
+ * 65535 bytes, so that seems like a reasonable limit for us as well.
+ */
+#define PG_MAX_AUTH_TOKEN_LENGTH 65535
+
extern PGDLLIMPORT char *pg_krb_server_keyfile;
extern PGDLLIMPORT bool pg_krb_caseins_users;
extern PGDLLIMPORT bool pg_gss_accept_delegation;
@@ -23,6 +39,7 @@ extern PGDLLIMPORT bool pg_gss_accept_delegation;
extern void ClientAuthentication(Port *port);
extern void sendAuthRequest(Port *port, AuthRequest areq, const char *extradata,
int extralen);
+extern void set_authn_id(Port *port, const char *id);
/* Hook for plugins to get control in ClientAuthentication() */
typedef void (*ClientAuthentication_hook_type) (Port *, int);
diff --git a/src/include/libpq/hba.h b/src/include/libpq/hba.h
index 8ea837ae82..53c999a69f 100644
--- a/src/include/libpq/hba.h
+++ b/src/include/libpq/hba.h
@@ -39,7 +39,8 @@ typedef enum UserAuth
uaCert,
uaRADIUS,
uaPeer,
-#define USER_AUTH_LAST uaPeer /* Must be last value of this enum */
+ uaOAuth,
+#define USER_AUTH_LAST uaOAuth /* Must be last value of this enum */
} UserAuth;
/*
@@ -135,6 +136,9 @@ typedef struct HbaLine
char *radiusidentifiers_s;
List *radiusports;
char *radiusports_s;
+ char *oauth_issuer;
+ char *oauth_scope;
+ bool oauth_skip_usermap;
} HbaLine;
typedef struct IdentLine
diff --git a/src/include/libpq/oauth.h b/src/include/libpq/oauth.h
new file mode 100644
index 0000000000..5edab3b25a
--- /dev/null
+++ b/src/include/libpq/oauth.h
@@ -0,0 +1,24 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth.h
+ * Interface to libpq/auth-oauth.c
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/libpq/oauth.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_OAUTH_H
+#define PG_OAUTH_H
+
+#include "libpq/libpq-be.h"
+#include "libpq/sasl.h"
+
+extern char *oauth_validator_command;
+
+/* Implementation */
+extern const pg_be_sasl_mech pg_be_oauth_mech;
+
+#endif /* PG_OAUTH_H */
diff --git a/src/include/libpq/sasl.h b/src/include/libpq/sasl.h
index 7a1f970cca..3f2c02b8f2 100644
--- a/src/include/libpq/sasl.h
+++ b/src/include/libpq/sasl.h
@@ -26,6 +26,14 @@
#define PG_SASL_EXCHANGE_SUCCESS 1
#define PG_SASL_EXCHANGE_FAILURE 2
+/*
+ * Maximum accepted size of SASL messages.
+ *
+ * The messages that the server or libpq generate are much smaller than this,
+ * but have some headroom.
+ */
+#define PG_MAX_SASL_MESSAGE_LENGTH 1024
+
/*
* Backend SASL mechanism callbacks.
*
@@ -127,6 +135,9 @@ typedef struct pg_be_sasl_mech
const char *input, int inputlen,
char **output, int *outputlen,
const char **logdetail);
+
+ /* The maximum size allowed for client SASLResponses. */
+ int max_message_length;
} pg_be_sasl_mech;
/* Common implementation for auth.c */
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index f535fbbd5c..86b1cc6e5c 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3564,6 +3564,7 @@ normal_rand_fctx
nsphash_hash
ntile_context
numeric
+oauth_state
object_access_hook_type
object_access_hook_type_str
off_t
--
2.34.1
v21-0002-libpq-add-OAUTHBEARER-SASL-mechanism.patch (application/octet-stream, 106.4K) download
From b754a2f8fb918921e32cbf2b59ac823ae01fc301 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 13 Apr 2021 10:27:27 -0700
Subject: [PATCH v21 2/7] libpq: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) and OAuth 2.0 Device Authorization
Grants (RFC 8628) on the client side. When speaking to a OAuth-enabled
server, it looks a bit like this:
$ psql 'host=example.org oauth_client_id=f02c6361-0635-...'
Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG
The OAuth issuer must support device authorization. No other OAuth flows
are currently implemented (but clients may provide their own flows; see
below).
The client implementation requires libcurl and its development headers.
Pass `curl` to --with-oauth/-Doauth during configuration.
Thomas Munro wrote the kqueue() implementation for oauth-curl; thanks!
= PQauthDataHook =
Clients may override two pieces of OAuth handling using the new
PQsetAuthDataHook():
- PQAUTHDATA_PROMPT_OAUTH_DEVICE: replaces the default user prompt to
standard error when using the builtin device authorization flow
- PQAUTHDATA_OAUTH_BEARER_TOKEN: replaces the entire OAuth flow with a
custom asynchronous implementation
In general, a hook implementation should examine the incoming `type` to
decide whether or not to handle a specific piece of authdata; if not, it
should delegate to the previous hook in the chain (retrievable via
PQgetAuthDataHook()). Otherwise, it should return an integer > 0 and
follow the authdata-specific instructions. Returning an integer < 0
signals an error condition and abandons the connection attempt.
== PQAUTHDATA_PROMPT_OAUTH_DEVICE ==
The hook should display the device prompt (URL + code) using whatever
method it prefers.
== PQAUTHDATA_OAUTH_BEARER_TOKEN ==
The hook should either directly return a Bearer token for the current
user/issuer/scope combination, if one is available without blocking, or
else set up an asynchronous callback to retrieve one. See the
documentation for PQoauthBearerRequest.
Several TODOs:
- don't retry forever if the server won't accept our token
- perform several sanity checks on the OAuth issuer's responses
- handle cases where the client has been set up with an issuer and
scope, but the Postgres server wants to use something different
- improve error debuggability during the OAuth handshake
- fix libcurl initialization thread-safety
- harden the libcurl flow implementation
- figure out pgsocket/int difference on Windows
- fix intermittent failure in the cleanup callback tests (race
condition?)
- support require_auth
- ...and more.
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
---
configure | 110 ++
configure.ac | 28 +
meson.build | 29 +
meson_options.txt | 4 +
src/Makefile.global.in | 1 +
src/include/common/oauth-common.h | 19 +
src/include/pg_config.h.in | 9 +
src/interfaces/libpq/Makefile | 10 +-
src/interfaces/libpq/exports.txt | 3 +
src/interfaces/libpq/fe-auth-oauth-curl.c | 1982 +++++++++++++++++++++
src/interfaces/libpq/fe-auth-oauth.c | 659 +++++++
src/interfaces/libpq/fe-auth-oauth.h | 42 +
src/interfaces/libpq/fe-auth-sasl.h | 10 +-
src/interfaces/libpq/fe-auth-scram.c | 6 +-
src/interfaces/libpq/fe-auth.c | 105 +-
src/interfaces/libpq/fe-auth.h | 9 +-
src/interfaces/libpq/fe-connect.c | 85 +-
src/interfaces/libpq/fe-misc.c | 7 +-
src/interfaces/libpq/libpq-fe.h | 77 +-
src/interfaces/libpq/libpq-int.h | 14 +
src/interfaces/libpq/meson.build | 7 +
src/makefiles/meson.build | 1 +
src/tools/pgindent/typedefs.list | 10 +
23 files changed, 3200 insertions(+), 27 deletions(-)
create mode 100644 src/include/common/oauth-common.h
create mode 100644 src/interfaces/libpq/fe-auth-oauth-curl.c
create mode 100644 src/interfaces/libpq/fe-auth-oauth.c
create mode 100644 src/interfaces/libpq/fe-auth-oauth.h
diff --git a/configure b/configure
index 36feeafbb2..d785ca846b 100755
--- a/configure
+++ b/configure
@@ -712,6 +712,7 @@ with_uuid
with_readline
with_systemd
with_selinux
+with_oauth
with_ldap
with_krb_srvnam
krb_srvtab
@@ -858,6 +859,7 @@ with_krb_srvnam
with_pam
with_bsd_auth
with_ldap
+with_oauth
with_bonjour
with_selinux
with_systemd
@@ -1568,6 +1570,7 @@ Optional Packages:
--with-pam build with PAM support
--with-bsd-auth build with BSD Authentication support
--with-ldap build with LDAP support
+ --with-oauth=LIB use LIB for OAuth 2.0 support (curl)
--with-bonjour build with Bonjour support
--with-selinux build with SELinux support
--with-systemd build with systemd support
@@ -8485,6 +8488,52 @@ $as_echo "$with_ldap" >&6; }
+#
+# OAuth 2.0
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with OAuth support" >&5
+$as_echo_n "checking whether to build with OAuth support... " >&6; }
+
+
+
+# Check whether --with-oauth was given.
+if test "${with_oauth+set}" = set; then :
+ withval=$with_oauth;
+ case $withval in
+ yes)
+ as_fn_error $? "argument required for --with-oauth option" "$LINENO" 5
+ ;;
+ no)
+ as_fn_error $? "argument required for --with-oauth option" "$LINENO" 5
+ ;;
+ *)
+
+ ;;
+ esac
+
+fi
+
+
+if test x"$with_oauth" = x"" ; then
+ with_oauth=no
+fi
+
+if test x"$with_oauth" = x"curl"; then
+
+$as_echo "#define USE_OAUTH 1" >>confdefs.h
+
+
+$as_echo "#define USE_OAUTH_CURL 1" >>confdefs.h
+
+elif test x"$with_oauth" != x"no"; then
+ as_fn_error $? "--with-oauth must specify curl" "$LINENO" 5
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_oauth" >&5
+$as_echo "$with_oauth" >&6; }
+
+
+
#
# Bonjour
#
@@ -13037,6 +13086,56 @@ fi
+if test "$with_oauth" = curl ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for curl_multi_init in -lcurl" >&5
+$as_echo_n "checking for curl_multi_init in -lcurl... " >&6; }
+if ${ac_cv_lib_curl_curl_multi_init+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lcurl $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char curl_multi_init ();
+int
+main ()
+{
+return curl_multi_init ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_curl_curl_multi_init=yes
+else
+ ac_cv_lib_curl_curl_multi_init=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_curl_curl_multi_init" >&5
+$as_echo "$ac_cv_lib_curl_curl_multi_init" >&6; }
+if test "x$ac_cv_lib_curl_curl_multi_init" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBCURL 1
+_ACEOF
+
+ LIBS="-lcurl $LIBS"
+
+else
+ as_fn_error $? "library 'curl' is required for --with-oauth=curl" "$LINENO" 5
+fi
+
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for security_compute_create_name in -lselinux" >&5
@@ -14062,6 +14161,17 @@ fi
done
+fi
+
+if test "$with_oauth" = curl; then
+ ac_fn_c_check_header_mongrel "$LINENO" "curl/curl.h" "ac_cv_header_curl_curl_h" "$ac_includes_default"
+if test "x$ac_cv_header_curl_curl_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <curl/curl.h> is required for OAuth" "$LINENO" 5
+fi
+
+
fi
if test "$PORTNAME" = "win32" ; then
diff --git a/configure.ac b/configure.ac
index 57f734879e..1ec0712ed5 100644
--- a/configure.ac
+++ b/configure.ac
@@ -927,6 +927,26 @@ AC_MSG_RESULT([$with_ldap])
AC_SUBST(with_ldap)
+#
+# OAuth 2.0
+#
+AC_MSG_CHECKING([whether to build with OAuth support])
+PGAC_ARG_REQ(with, oauth, [LIB], [use LIB for OAuth 2.0 support (curl)])
+if test x"$with_oauth" = x"" ; then
+ with_oauth=no
+fi
+
+if test x"$with_oauth" = x"curl"; then
+ AC_DEFINE([USE_OAUTH], 1, [Define to 1 to build with OAuth 2.0 support. (--with-oauth)])
+ AC_DEFINE([USE_OAUTH_CURL], 1, [Define to 1 to use libcurl for OAuth support.])
+elif test x"$with_oauth" != x"no"; then
+ AC_MSG_ERROR([--with-oauth must specify curl])
+fi
+
+AC_MSG_RESULT([$with_oauth])
+AC_SUBST(with_oauth)
+
+
#
# Bonjour
#
@@ -1423,6 +1443,10 @@ fi
AC_SUBST(LDAP_LIBS_FE)
AC_SUBST(LDAP_LIBS_BE)
+if test "$with_oauth" = curl ; then
+ AC_CHECK_LIB(curl, curl_multi_init, [], [AC_MSG_ERROR([library 'curl' is required for --with-oauth=curl])])
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
AC_CHECK_LIB(selinux, security_compute_create_name, [],
@@ -1614,6 +1638,10 @@ elif test "$with_uuid" = ossp ; then
[AC_MSG_ERROR([header file <ossp/uuid.h> or <uuid.h> is required for OSSP UUID])])])
fi
+if test "$with_oauth" = curl; then
+ AC_CHECK_HEADER(curl/curl.h, [], [AC_MSG_ERROR([header file <curl/curl.h> is required for OAuth])])
+fi
+
if test "$PORTNAME" = "win32" ; then
AC_CHECK_HEADERS(crtdefs.h)
fi
diff --git a/meson.build b/meson.build
index c8fdfeb0ec..34a0226ccf 100644
--- a/meson.build
+++ b/meson.build
@@ -840,6 +840,33 @@ endif
+###############################################################
+# Library: oauth
+###############################################################
+
+oauth = not_found_dep
+oauth_library = 'none'
+oauthopt = get_option('oauth')
+
+if oauthopt == 'auto' and auto_features.disabled()
+ oauthopt = 'none'
+endif
+
+if oauthopt in ['auto', 'curl']
+ oauth = dependency('libcurl', required: (oauthopt == 'curl'))
+
+ if oauth.found()
+ oauth_library = 'curl'
+ cdata.set('USE_OAUTH', 1)
+ cdata.set('USE_OAUTH_CURL', 1)
+ endif
+endif
+
+if oauthopt == 'auto' and auto_features.enabled() and not oauth.found()
+ error('no OAuth implementation library found')
+endif
+
+
###############################################################
# Library: Tcl (for pltcl)
#
@@ -2845,6 +2872,7 @@ libpq_deps += [
gssapi,
ldap_r,
libintl,
+ oauth,
ssl,
]
@@ -3446,6 +3474,7 @@ if meson.version().version_compare('>=0.57')
'llvm': llvm,
'lz4': lz4,
'nls': libintl,
+ 'oauth': oauth,
'openssl': ssl,
'pam': pam,
'plperl': perl_dep,
diff --git a/meson_options.txt b/meson_options.txt
index 249ecc5ffd..3248b9cc1c 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -121,6 +121,10 @@ option('lz4', type: 'feature', value: 'auto',
option('nls', type: 'feature', value: 'auto',
description: 'Native language support')
+option('oauth', type : 'combo', choices : ['auto', 'none', 'curl'],
+ value: 'auto',
+ description: 'use LIB for OAuth 2.0 support (curl)')
+
option('pam', type: 'feature', value: 'auto',
description: 'PAM support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 8b3f8c24e0..79b3647834 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -193,6 +193,7 @@ with_ldap = @with_ldap@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
+with_oauth = @with_oauth@
with_system_tzdata = @with_system_tzdata@
with_uuid = @with_uuid@
with_zlib = @with_zlib@
diff --git a/src/include/common/oauth-common.h b/src/include/common/oauth-common.h
new file mode 100644
index 0000000000..5ff3488bfb
--- /dev/null
+++ b/src/include/common/oauth-common.h
@@ -0,0 +1,19 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth-common.h
+ * Declarations for helper functions used for OAuth/OIDC authentication
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/common/oauth-common.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef OAUTH_COMMON_H
+#define OAUTH_COMMON_H
+
+/* Name of SASL mechanism per IANA */
+#define OAUTHBEARER_NAME "OAUTHBEARER"
+
+#endif /* OAUTH_COMMON_H */
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 591e1ca3df..399578cebb 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -246,6 +246,9 @@
/* Define to 1 if you have the `crypto' library (-lcrypto). */
#undef HAVE_LIBCRYPTO
+/* Define to 1 if you have the `curl' library (-lcurl). */
+#undef HAVE_LIBCURL
+
/* Define to 1 if you have the `ldap' library (-lldap). */
#undef HAVE_LIBLDAP
@@ -714,6 +717,12 @@
/* Define to select named POSIX semaphores. */
#undef USE_NAMED_POSIX_SEMAPHORES
+/* Define to 1 to build with OAuth 2.0 support. (--with-oauth) */
+#undef USE_OAUTH
+
+/* Define to 1 to use libcurl for OAuth support. */
+#undef USE_OAUTH_CURL
+
/* Define to 1 to build with OpenSSL support. (--with-ssl=openssl) */
#undef USE_OPENSSL
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index fe2af575c5..2618c293af 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -61,6 +61,14 @@ OBJS += \
fe-secure-gssapi.o
endif
+ifneq ($(with_oauth),no)
+OBJS += fe-auth-oauth.o
+
+ifeq ($(with_oauth),curl)
+OBJS += fe-auth-oauth-curl.o
+endif
+endif
+
ifeq ($(PORTNAME), cygwin)
override shlib = cyg$(NAME)$(DLSUFFIX)
endif
@@ -79,7 +87,7 @@ endif
# that are built correctly for use in a shlib.
SHLIB_LINK_INTERNAL = -lpgcommon_shlib -lpgport_shlib
ifneq ($(PORTNAME), win32)
-SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
+SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lcurl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
else
SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi32 -lssl -lsocket -lnsl -lresolv -lintl -lm $(PTHREAD_LIBS), $(LIBS)) $(LDAP_LIBS_FE)
endif
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 9fbd3d3407..9946764c4a 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -202,3 +202,6 @@ PQcancelSocket 199
PQcancelErrorMessage 200
PQcancelReset 201
PQcancelFinish 202
+PQsetAuthDataHook 203
+PQgetAuthDataHook 204
+PQdefaultAuthDataHook 205
diff --git a/src/interfaces/libpq/fe-auth-oauth-curl.c b/src/interfaces/libpq/fe-auth-oauth-curl.c
new file mode 100644
index 0000000000..0504f96e4e
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth-curl.c
@@ -0,0 +1,1982 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth-curl.c
+ * The libcurl implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth-curl.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <curl/curl.h>
+#include <math.h>
+#ifdef HAVE_SYS_EPOLL_H
+#include <sys/epoll.h>
+#include <sys/timerfd.h>
+#endif
+#ifdef HAVE_SYS_EVENT_H
+#include <sys/event.h>
+#endif
+#include <unistd.h>
+
+#include "common/jsonapi.h"
+#include "fe-auth.h"
+#include "fe-auth-oauth.h"
+#include "libpq-int.h"
+#include "mb/pg_wchar.h"
+
+/*
+ * Parsed JSON Representations
+ *
+ * As a general rule, we parse and cache only the fields we're currently using.
+ * When adding new fields, ensure the corresponding free_*() function is updated
+ * too.
+ */
+
+/*
+ * The OpenID Provider configuration (alternatively named "authorization server
+ * metadata") jointly described by OpenID Connect Discovery 1.0 and RFC 8414:
+ *
+ * https://openid.net/specs/openid-connect-discovery-1_0.html
+ * https://www.rfc-editor.org/rfc/rfc8414#section-3.2
+ */
+struct provider
+{
+ char *issuer;
+ char *token_endpoint;
+ char *device_authorization_endpoint;
+ struct curl_slist *grant_types_supported;
+};
+
+static void
+free_provider(struct provider *provider)
+{
+ free(provider->issuer);
+ free(provider->token_endpoint);
+ free(provider->device_authorization_endpoint);
+ curl_slist_free_all(provider->grant_types_supported);
+}
+
+/*
+ * The Device Authorization response, described by RFC 8628:
+ *
+ * https://www.rfc-editor.org/rfc/rfc8628#section-3.2
+ */
+struct device_authz
+{
+ char *device_code;
+ char *user_code;
+ char *verification_uri;
+ char *interval_str;
+
+ /* Fields below are parsed from the corresponding string above. */
+ int interval;
+};
+
+static void
+free_device_authz(struct device_authz *authz)
+{
+ free(authz->device_code);
+ free(authz->user_code);
+ free(authz->verification_uri);
+ free(authz->interval_str);
+}
+
+/*
+ * The Token Endpoint error response, as described by RFC 6749:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-5.2
+ *
+ * Note that this response type can also be returned from the Device
+ * Authorization Endpoint.
+ */
+struct token_error
+{
+ char *error;
+ char *error_description;
+};
+
+static void
+free_token_error(struct token_error *err)
+{
+ free(err->error);
+ free(err->error_description);
+}
+
+/*
+ * The Access Token response, as described by RFC 6749:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-4.1.4
+ *
+ * During the Device Authorization flow, several temporary errors are expected
+ * as part of normal operation. To make it easy to handle these in the happy
+ * path, this contains an embedded token_error that is filled in if needed.
+ */
+struct token
+{
+ /* for successful responses */
+ char *access_token;
+ char *token_type;
+
+ /* for error responses */
+ struct token_error err;
+};
+
+static void
+free_token(struct token *tok)
+{
+ free(tok->access_token);
+ free(tok->token_type);
+ free_token_error(&tok->err);
+}
+
+/*
+ * Asynchronous State
+ */
+
+/* States for the overall async machine. */
+typedef enum
+{
+ OAUTH_STEP_INIT,
+ OAUTH_STEP_DISCOVERY,
+ OAUTH_STEP_DEVICE_AUTHORIZATION,
+ OAUTH_STEP_TOKEN_REQUEST,
+ OAUTH_STEP_WAIT_INTERVAL,
+} OAuthStep;
+
+/*
+ * The async_ctx holds onto state that needs to persist across multiple calls to
+ * pg_fe_run_oauth_flow(). Almost everything interacts with this in some way.
+ */
+struct async_ctx
+{
+ OAuthStep step; /* where are we in the flow? */
+
+#ifdef HAVE_SYS_EPOLL_H
+ int timerfd; /* a timerfd for signaling async timeouts */
+#endif
+ pgsocket mux; /* the multiplexer socket containing all
+ * descriptors tracked by cURL, plus the
+ * timerfd */
+ CURLM *curlm; /* top-level multi handle for cURL operations */
+ CURL *curl; /* the (single) easy handle for serial
+ * requests */
+
+ struct curl_slist *headers; /* common headers for all requests */
+ PQExpBufferData work_data; /* scratch buffer for general use (remember to
+ * clear out prior contents first!) */
+
+ /*------
+ * Since a single logical operation may stretch across multiple calls to
+ * our entry point, errors have three parts:
+ *
+ * - errctx: an optional static string, describing the global operation
+ * currently in progress. It'll be translated for you.
+ *
+ * - errbuf: contains the actual error message. Generally speaking, use
+ * actx_error[_str] to manipulate this. This must be filled
+ * with something useful on an error.
+ *
+ * - curl_err: an optional static error buffer used by cURL to put
+ * detailed information about failures. Unfortunately
+ * untranslatable.
+ *
+ * These pieces will be combined into a single error message looking
+ * something like the following, with errctx and/or curl_err omitted when
+ * absent:
+ *
+ * connection to server ... failed: errctx: errbuf (curl_err)
+ */
+ const char *errctx; /* not freed; must point to static allocation */
+ PQExpBufferData errbuf;
+ char curl_err[CURL_ERROR_SIZE];
+
+ /*
+ * These documents need to survive over multiple calls, and are therefore
+ * cached directly in the async_ctx.
+ */
+ struct provider provider;
+ struct device_authz authz;
+
+ bool user_prompted; /* have we already sent the authz prompt? */
+};
+
+/*
+ * Frees the async_ctx, which is stored directly on the PGconn. This is called
+ * during pqDropConnection() so that we don't leak resources even if
+ * PQconnectPoll() never calls us back.
+ *
+ * TODO: we should probably call this at the end of a successful authentication,
+ * too, to proactively free up resources.
+ */
+static void
+free_curl_async_ctx(PGconn *conn, void *ctx)
+{
+ struct async_ctx *actx = ctx;
+
+ Assert(actx); /* oauth_free() shouldn't call us otherwise */
+
+ /*
+ * TODO: in general, none of the error cases below should ever happen if
+ * we have no bugs above. But if we do hit them, surfacing those errors
+ * somehow might be the only way to have a chance to debug them. What's
+ * the best way to do that? Assertions? Spraying messages on stderr?
+ * Bubbling an error code to the top? Appending to the connection's error
+ * message only helps if the bug caused a connection failure; otherwise
+ * it'll be buried...
+ */
+
+ if (actx->curlm && actx->curl)
+ {
+ CURLMcode err = curl_multi_remove_handle(actx->curlm, actx->curl);
+
+ if (err)
+ libpq_append_conn_error(conn,
+ "cURL easy handle removal failed: %s",
+ curl_multi_strerror(err));
+ }
+
+ if (actx->curl)
+ {
+ /*
+ * curl_multi_cleanup() doesn't free any associated easy handles; we
+ * need to do that separately. We only ever have one easy handle per
+ * multi handle.
+ */
+ curl_easy_cleanup(actx->curl);
+ }
+
+ if (actx->curlm)
+ {
+ CURLMcode err = curl_multi_cleanup(actx->curlm);
+
+ if (err)
+ libpq_append_conn_error(conn,
+ "cURL multi handle cleanup failed: %s",
+ curl_multi_strerror(err));
+ }
+
+ free_provider(&actx->provider);
+ free_device_authz(&actx->authz);
+
+ curl_slist_free_all(actx->headers);
+ termPQExpBuffer(&actx->work_data);
+ termPQExpBuffer(&actx->errbuf);
+
+ if (actx->mux != PGINVALID_SOCKET)
+ close(actx->mux);
+#ifdef HAVE_SYS_EPOLL_H
+ if (actx->timerfd >= 0)
+ close(actx->timerfd);
+#endif
+
+ free(actx);
+}
+
+/*
+ * Macros for manipulating actx->errbuf. actx_error() translates and formats a
+ * string for you; actx_error_str() appends a string directly without
+ * translation.
+ */
+
+#define actx_error(ACTX, FMT, ...) \
+ appendPQExpBuffer(&(ACTX)->errbuf, libpq_gettext(FMT), ##__VA_ARGS__)
+
+#define actx_error_str(ACTX, S) \
+ appendPQExpBufferStr(&(ACTX)->errbuf, S)
+
+/*
+ * Macros for getting and setting state for the connection's two cURL handles,
+ * so you don't have to write out the error handling every time.
+ */
+
+#define CHECK_MSETOPT(ACTX, OPT, VAL, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLMcode _setopterr = curl_multi_setopt(_actx->curlm, OPT, VAL); \
+ if (_setopterr) { \
+ actx_error(_actx, "failed to set %s on OAuth connection: %s",\
+ #OPT, curl_multi_strerror(_setopterr)); \
+ FAILACTION; \
+ } \
+ } while (0)
+
+#define CHECK_SETOPT(ACTX, OPT, VAL, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLcode _setopterr = curl_easy_setopt(_actx->curl, OPT, VAL); \
+ if (_setopterr) { \
+ actx_error(_actx, "failed to set %s on OAuth connection: %s",\
+ #OPT, curl_easy_strerror(_setopterr)); \
+ FAILACTION; \
+ } \
+ } while (0)
+
+#define CHECK_GETINFO(ACTX, INFO, OUT, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLcode _getinfoerr = curl_easy_getinfo(_actx->curl, INFO, OUT); \
+ if (_getinfoerr) { \
+ actx_error(_actx, "failed to get %s from OAuth response: %s",\
+ #INFO, curl_easy_strerror(_getinfoerr)); \
+ FAILACTION; \
+ } \
+ } while (0)
+
+/*
+ * General JSON Parsing for OAuth Responses
+ */
+
+/*
+ * Represents a single name/value pair in a JSON object. This is the primary
+ * interface to parse_oauth_json().
+ *
+ * All fields are stored internally as strings or lists of strings, so clients
+ * have to explicitly parse other scalar types (though they will have gone
+ * through basic lexical validation). Storing nested objects is not currently
+ * supported, nor is parsing arrays of anything other than strings.
+ */
+struct json_field
+{
+ const char *name; /* name (key) of the member */
+
+ JsonTokenType type; /* currently supports JSON_TOKEN_STRING,
+ * JSON_TOKEN_NUMBER, and
+ * JSON_TOKEN_ARRAY_START */
+ union
+ {
+ char **scalar; /* for all scalar types */
+ struct curl_slist **array; /* for type == JSON_TOKEN_ARRAY_START */
+ };
+
+ bool required; /* REQUIRED field, or just OPTIONAL? */
+};
+
+/* Documentation macros for json_field.required. */
+#define REQUIRED true
+#define OPTIONAL false
+
+/* Parse state for parse_oauth_json(). */
+struct oauth_parse
+{
+ PQExpBuffer errbuf; /* detail message for JSON_SEM_ACTION_FAILED */
+ int nested; /* nesting level (zero is the top) */
+
+ const struct json_field *fields; /* field definition array */
+ const struct json_field *active; /* points inside the fields array */
+};
+
+#define oauth_parse_set_error(ctx, fmt, ...) \
+ appendPQExpBuffer((ctx)->errbuf, libpq_gettext(fmt), ##__VA_ARGS__)
+
+static void
+report_type_mismatch(struct oauth_parse *ctx)
+{
+ char *msgfmt;
+
+ Assert(ctx->active);
+
+ /*
+ * At the moment, the only fields we're interested in are strings,
+ * numbers, and arrays of strings.
+ */
+ switch (ctx->active->type)
+ {
+ case JSON_TOKEN_STRING:
+ msgfmt = "field \"%s\" must be a string";
+ break;
+
+ case JSON_TOKEN_NUMBER:
+ msgfmt = "field \"%s\" must be a number";
+ break;
+
+ case JSON_TOKEN_ARRAY_START:
+ msgfmt = "field \"%s\" must be an array of strings";
+ break;
+
+ default:
+ Assert(false);
+ msgfmt = "field \"%s\" has unexpected type";
+ }
+
+ oauth_parse_set_error(ctx, msgfmt, ctx->active->name);
+}
+
+static JsonParseErrorType
+oauth_json_object_start(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (ctx->active)
+ {
+ /*
+ * Currently, none of the fields we're interested in can be or contain
+ * objects, so we can reject this case outright.
+ */
+ report_type_mismatch(ctx);
+ return JSON_SEM_ACTION_FAILED;
+ }
+
+ ++ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_field_start(void *state, char *name, bool isnull)
+{
+ struct oauth_parse *ctx = state;
+
+ /* We care only about the top-level fields. */
+ if (ctx->nested == 1)
+ {
+ const struct json_field *field = ctx->fields;
+
+ /*
+ * We should never start parsing a new field while a previous one is
+ * still active.
+ *
+ * TODO: this code relies on assertions too much. We need to exit
+ * sanely on internal logic errors, to avoid turning bugs into
+ * vulnerabilities.
+ */
+ Assert(!ctx->active);
+
+ while (field->name)
+ {
+ if (strcmp(name, field->name) == 0)
+ {
+ ctx->active = field;
+ break;
+ }
+
+ ++field;
+ }
+ }
+
+ free(name);
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_end(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_start(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (!ctx->nested)
+ {
+ oauth_parse_set_error(ctx, "top-level element must be an object");
+ return JSON_SEM_ACTION_FAILED;
+ }
+
+ if (ctx->active)
+ {
+ if (ctx->active->type != JSON_TOKEN_ARRAY_START
+ /* The arrays we care about must not have arrays as values. */
+ || ctx->nested > 1)
+ {
+ report_type_mismatch(ctx);
+ return JSON_SEM_ACTION_FAILED;
+ }
+ }
+
+ ++ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_end(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (ctx->active)
+ {
+ /*
+ * This assumes that no target arrays can contain other arrays, which
+ * we check in the array_start callback.
+ */
+ Assert(ctx->nested == 2);
+ Assert(ctx->active->type == JSON_TOKEN_ARRAY_START);
+
+ ctx->active = NULL;
+ }
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_scalar(void *state, char *token, JsonTokenType type)
+{
+ struct oauth_parse *ctx = state;
+ JsonParseErrorType result = JSON_SUCCESS;
+
+ if (!ctx->nested)
+ {
+ oauth_parse_set_error(ctx, "top-level element must be an object");
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ if (ctx->active)
+ {
+ JsonTokenType expected;
+
+ /*
+ * Make sure this matches what the active field expects. Arrays must
+ * contain only strings with the current implementation.
+ */
+ if (ctx->active->type == JSON_TOKEN_ARRAY_START)
+ expected = JSON_TOKEN_STRING;
+ else
+ expected = ctx->active->type;
+
+ if (type != expected)
+ {
+ report_type_mismatch(ctx);
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ /*
+ * FIXME if the JSON field is duplicated, we'll leak the prior value.
+ * Error out in that case instead.
+ */
+ if (ctx->active->type != JSON_TOKEN_ARRAY_START)
+ {
+ Assert(ctx->nested == 1);
+
+ *ctx->active->scalar = token;
+ ctx->active = NULL;
+
+ return JSON_SUCCESS; /* don't free the token */
+ }
+ else /* ctx->target_array */
+ {
+ struct curl_slist *temp;
+
+ Assert(ctx->nested == 2);
+
+ temp = curl_slist_append(*ctx->active->array, token);
+ if (!temp)
+ {
+ oauth_parse_set_error(ctx, "out of memory");
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ *ctx->active->array = temp;
+
+ /*
+ * Note that curl_slist_append() makes a copy of the token, so we
+ * can free it below.
+ */
+ }
+ }
+ else
+ {
+ /* otherwise we just ignore it */
+ }
+
+cleanup:
+ free(token);
+ return result;
+}
+
+/*
+ * A helper function for general JSON parsing. fields is the array of field
+ * definitions with their backing pointers. The response will be parsed from
+ * actx->curl and actx->work_data (as set up by start_request()), and any
+ * parsing errors will be placed into actx->errbuf.
+ */
+static bool
+parse_oauth_json(struct async_ctx *actx, const struct json_field *fields)
+{
+ PQExpBuffer resp = &actx->work_data;
+ char *content_type;
+ JsonLexContext lex = {0};
+ JsonSemAction sem = {0};
+ JsonParseErrorType err;
+ struct oauth_parse ctx = {0};
+ bool success = false;
+
+ /* Make sure the server thinks it's given us JSON. */
+ CHECK_GETINFO(actx, CURLINFO_CONTENT_TYPE, &content_type, return false);
+
+ if (!content_type)
+ {
+ actx_error(actx, "no content type was provided");
+ goto cleanup;
+ }
+ else if (strcasecmp(content_type, "application/json") != 0)
+ {
+ actx_error(actx, "unexpected content type \"%s\"", content_type);
+ goto cleanup;
+ }
+
+ if (strlen(resp->data) != resp->len)
+ {
+ actx_error(actx, "response contains embedded NULLs");
+ goto cleanup;
+ }
+
+ makeJsonLexContextCstringLen(&lex, resp->data, resp->len, PG_UTF8, true);
+
+ ctx.errbuf = &actx->errbuf;
+ ctx.fields = fields;
+ sem.semstate = &ctx;
+
+ sem.object_start = oauth_json_object_start;
+ sem.object_field_start = oauth_json_object_field_start;
+ sem.object_end = oauth_json_object_end;
+ sem.array_start = oauth_json_array_start;
+ sem.array_end = oauth_json_array_end;
+ sem.scalar = oauth_json_scalar;
+
+ err = pg_parse_json(&lex, &sem);
+
+ if (err != JSON_SUCCESS)
+ {
+ /*
+ * For JSON_SEM_ACTION_FAILED, we've already written the error
+ * message. Other errors come directly from pg_parse_json(), already
+ * translated.
+ */
+ if (err != JSON_SEM_ACTION_FAILED)
+ actx_error_str(actx, json_errdetail(err, &lex));
+
+ goto cleanup;
+ }
+
+ /* Check all required fields. */
+ while (fields->name)
+ {
+ if (fields->required && !*fields->scalar && !*fields->array)
+ {
+ actx_error(actx, "field \"%s\" is missing", fields->name);
+ goto cleanup;
+ }
+
+ fields++;
+ }
+
+ success = true;
+
+cleanup:
+ freeJsonLexContext(&lex);
+ return success;
+}
+
+/*
+ * JSON Parser Definitions
+ */
+
+static bool
+parse_provider(struct async_ctx *actx, struct provider *provider)
+{
+ struct json_field fields[] = {
+ {"issuer", JSON_TOKEN_STRING, {&provider->issuer}, REQUIRED},
+ {"token_endpoint", JSON_TOKEN_STRING, {&provider->token_endpoint}, REQUIRED},
+
+ /*----
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - jwks_uri
+ * - response_types_supported
+ * - subject_types_supported
+ * - id_token_signing_alg_values_supported
+ */
+
+ {"device_authorization_endpoint", JSON_TOKEN_STRING, {&provider->device_authorization_endpoint}, OPTIONAL},
+ {"grant_types_supported", JSON_TOKEN_ARRAY_START, {.array = &provider->grant_types_supported}, OPTIONAL},
+
+ {0},
+ };
+
+ return parse_oauth_json(actx, fields);
+}
+
+/*
+ * Parses the "interval" JSON number, corresponding to the number of seconds to
+ * wait between token endpoint requests.
+ *
+ * RFC 8628 is pretty silent on sanity checks for the interval. As a matter of
+ * practicality, round any fractional intervals up to the next second, and clamp
+ * the result at a minimum of one. (Zero-second intervals would result in an
+ * expensive network polling loop.)
+ *
+ * TODO: maybe clamp the upper bound too, based on the libpq timeout and/or the
+ * code expiration time?
+ */
+static int
+parse_interval(const char *interval_str)
+{
+ double parsed;
+ int cnt;
+
+ /*
+ * The JSON lexer has already validated the number, which is stricter than
+ * the %f format, so we should be good to use sscanf().
+ */
+ cnt = sscanf(interval_str, "%lf", &parsed);
+
+ if (cnt != 1)
+ {
+ /*
+ * Either the lexer screwed up or our assumption above isn't true, and
+ * either way a developer needs to take a look.
+ */
+ Assert(cnt == 1);
+ return 1; /* don't fall through in release builds */
+ }
+
+ parsed = ceil(parsed);
+
+ if (parsed < 1)
+ return 1; /* TODO this slows down the tests
+ * considerably... */
+ else if (INT_MAX <= parsed)
+ return INT_MAX;
+
+ return parsed;
+}
+
+static bool
+parse_device_authz(struct async_ctx *actx, struct device_authz *authz)
+{
+ struct json_field fields[] = {
+ {"device_code", JSON_TOKEN_STRING, {&authz->device_code}, REQUIRED},
+ {"user_code", JSON_TOKEN_STRING, {&authz->user_code}, REQUIRED},
+ {"verification_uri", JSON_TOKEN_STRING, {&authz->verification_uri}, REQUIRED},
+
+ /*
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - expires_in
+ */
+
+ {"interval", JSON_TOKEN_NUMBER, {&authz->interval_str}, OPTIONAL},
+
+ {0},
+ };
+
+ if (!parse_oauth_json(actx, fields))
+ return false;
+
+ /*
+ * Parse our numeric fields. Lexing has already completed by this time, so
+ * we at least know they're valid JSON numbers.
+ */
+ if (authz->interval_str)
+ authz->interval = parse_interval(authz->interval_str);
+ else
+ {
+ /* TODO: handle default interval of 5 seconds */
+ }
+
+ return true;
+}
+
+static bool
+parse_token_error(struct async_ctx *actx, struct token_error *err)
+{
+ bool result;
+ struct json_field fields[] = {
+ {"error", JSON_TOKEN_STRING, {&err->error}, REQUIRED},
+
+ {"error_description", JSON_TOKEN_STRING, {&err->error_description}, OPTIONAL},
+
+ {0},
+ };
+
+ result = parse_oauth_json(actx, fields);
+
+ /*
+ * Since token errors are parsed during other active error paths, only
+ * override the errctx if parsing explicitly fails.
+ */
+ if (!result)
+ actx->errctx = "failed to parse token error response";
+
+ return result;
+}
+
+static bool
+parse_access_token(struct async_ctx *actx, struct token *tok)
+{
+ struct json_field fields[] = {
+ {"access_token", JSON_TOKEN_STRING, {&tok->access_token}, REQUIRED},
+ {"token_type", JSON_TOKEN_STRING, {&tok->token_type}, REQUIRED},
+
+ /*
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - scope (only required if different than requested -- TODO check)
+ */
+
+ {0},
+ };
+
+ return parse_oauth_json(actx, fields);
+}
+
+/*
+ * cURL Multi Setup/Callbacks
+ */
+
+/*
+ * Sets up the actx->mux, which is the altsock that PQconnectPoll clients will
+ * select() on instead of the Postgres socket during OAuth negotiation.
+ *
+ * This is just an epoll set or kqueue abstracting multiple other descriptors.
+ * A timerfd is always part of the set when using epoll; it's just disabled
+ * when we're not using it.
+ */
+static bool
+setup_multiplexer(struct async_ctx *actx)
+{
+#ifdef HAVE_SYS_EPOLL_H
+ struct epoll_event ev = {.events = EPOLLIN};
+
+ actx->mux = epoll_create1(EPOLL_CLOEXEC);
+ if (actx->mux < 0)
+ {
+ actx_error(actx, "failed to create epoll set: %m");
+ return false;
+ }
+
+ actx->timerfd = timerfd_create(CLOCK_MONOTONIC, TFD_CLOEXEC);
+ if (actx->timerfd < 0)
+ {
+ actx_error(actx, "failed to create timerfd: %m");
+ return false;
+ }
+
+ if (epoll_ctl(actx->mux, EPOLL_CTL_ADD, actx->timerfd, &ev) < 0)
+ {
+ actx_error(actx, "failed to add timerfd to epoll set: %m");
+ return false;
+ }
+
+ return true;
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ actx->mux = kqueue();
+ if (actx->mux < 0)
+ {
+ actx_error(actx, "failed to create kqueue: %m");
+ return false;
+ }
+
+ return true;
+#endif
+
+ actx_error(actx, "here's a nickel kid, get yourself a better computer");
+ return false;
+}
+
+/*
+ * Adds and removes sockets from the multiplexer set, as directed by the
+ * cURL multi handle.
+ */
+static int
+register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
+ void *socketp)
+{
+#ifdef HAVE_SYS_EPOLL_H
+ struct async_ctx *actx = ctx;
+ struct epoll_event ev = {0};
+ int res;
+ int op = EPOLL_CTL_ADD;
+
+ switch (what)
+ {
+ case CURL_POLL_IN:
+ ev.events = EPOLLIN;
+ break;
+
+ case CURL_POLL_OUT:
+ ev.events = EPOLLOUT;
+ break;
+
+ case CURL_POLL_INOUT:
+ ev.events = EPOLLIN | EPOLLOUT;
+ break;
+
+ case CURL_POLL_REMOVE:
+ op = EPOLL_CTL_DEL;
+ break;
+
+ default:
+ actx_error(actx, "unknown cURL socket operation (%d)", what);
+ return -1;
+ }
+
+ res = epoll_ctl(actx->mux, op, socket, &ev);
+ if (res < 0 && errno == EEXIST)
+ {
+ /* We already had this socket in the pollset. */
+ op = EPOLL_CTL_MOD;
+ res = epoll_ctl(actx->mux, op, socket, &ev);
+ }
+
+ if (res < 0)
+ {
+ switch (op)
+ {
+ case EPOLL_CTL_ADD:
+ actx_error(actx, "could not add to epoll set: %m");
+ break;
+
+ case EPOLL_CTL_DEL:
+ actx_error(actx, "could not delete from epoll set: %m");
+ break;
+
+ default:
+ actx_error(actx, "could not update epoll set: %m");
+ }
+
+ return -1;
+ }
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct async_ctx *actx = ctx;
+ struct kevent ev[2] = {{0}};
+ struct kevent ev_out[2];
+ struct timespec timeout = {0};
+ int nev = 0;
+ int res;
+
+ switch (what)
+ {
+ case CURL_POLL_IN:
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_OUT:
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_INOUT:
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_REMOVE:
+
+ /*
+ * We don't know which of these is currently registered, perhaps
+ * both, so we try to remove both. This means we need to tolerate
+ * ENOENT below.
+ */
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_DELETE | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_DELETE | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ default:
+ actx_error(actx, "unknown cURL socket operation (%d)", what);
+ return -1;
+ }
+
+ res = kevent(actx->mux, ev, nev, ev_out, lengthof(ev_out), &timeout);
+ if (res < 0)
+ {
+ actx_error(actx, "could not modify kqueue: %m");
+ return -1;
+ }
+
+ /*
+ * We can't use the simple errno version of kevent, because we need to
+ * skip over ENOENT while still allowing a second change to be processed.
+ * So we need a longer-form error checking loop.
+ */
+ for (int i = 0; i < res; ++i)
+ {
+ /*
+ * EV_RECEIPT should guarantee one EV_ERROR result for every change,
+ * whether successful or not. Failed entries contain a non-zero errno
+ * in the `data` field.
+ */
+ Assert(ev_out[i].flags & EV_ERROR);
+
+ errno = ev_out[i].data;
+ if (errno && errno != ENOENT)
+ {
+ switch (what)
+ {
+ case CURL_POLL_REMOVE:
+ actx_error(actx, "could not delete from kqueue: %m");
+ break;
+ default:
+ actx_error(actx, "could not add to kqueue: %m");
+ }
+ return -1;
+ }
+ }
+#endif
+
+ return 0;
+}
+
+/*
+ * Adds or removes timeouts from the multiplexer set, as directed by the
+ * cURL multi handle. Rather than continually adding and removing the timer,
+ * we keep it in the set at all times and just disarm it when it's not
+ * needed.
+ */
+static int
+register_timer(CURLM *curlm, long timeout, void *ctx)
+{
+#if HAVE_SYS_EPOLL_H
+ struct async_ctx *actx = ctx;
+ struct itimerspec spec = {0};
+
+ if (timeout < 0)
+ {
+ /* the zero itimerspec will disarm the timer below */
+ }
+ else if (timeout == 0)
+ {
+ /*
+ * A zero timeout means cURL wants us to call back immediately. That's
+ * not technically an option for timerfd, but we can make the timeout
+ * ridiculously short.
+ *
+ * TODO: maybe just signal drive_request() to immediately call back in
+ * this case?
+ */
+ spec.it_value.tv_nsec = 1;
+ }
+ else
+ {
+ spec.it_value.tv_sec = timeout / 1000;
+ spec.it_value.tv_nsec = (timeout % 1000) * 1000000;
+ }
+
+ if (timerfd_settime(actx->timerfd, 0 /* no flags */ , &spec, NULL) < 0)
+ {
+ actx_error(actx, "setting timerfd to %ld: %m", timeout);
+ return -1;
+ }
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct async_ctx *actx = ctx;
+ struct kevent ev;
+
+ EV_SET(&ev, 1, EVFILT_TIMER, timeout < 0 ? EV_DELETE : EV_ADD,
+ 0, timeout, 0);
+ if (kevent(actx->mux, &ev, 1, NULL, 0, NULL) < 0 && errno != ENOENT)
+ {
+ actx_error(actx, "setting kqueue timer to %ld: %m", timeout);
+ return -1;
+ }
+#endif
+
+ return 0;
+}
+
+/*
+ * Initializes the two cURL handles in the async_ctx. The multi handle,
+ * actx->curlm, is what drives the asynchronous engine and tells us what to do
+ * next. The easy handle, actx->curl, encapsulates the state for a single
+ * request/response. It's added to the multi handle as needed, during
+ * start_request().
+ */
+static bool
+setup_curl_handles(struct async_ctx *actx)
+{
+ curl_version_info_data *curl_info;
+
+ /*
+ * Create our multi handle. This encapsulates the entire conversation with
+ * cURL for this connection.
+ */
+ actx->curlm = curl_multi_init();
+ if (!actx->curlm)
+ {
+ /* We don't get a lot of feedback on the failure reason. */
+ actx_error(actx, "failed to create cURL multi handle");
+ return false;
+ }
+
+ /*
+ * Extract information about the libcurl we are linked against.
+ */
+ curl_info = curl_version_info(CURLVERSION_NOW);
+
+ /*
+ * The multi handle tells us what to wait on using two callbacks. These
+ * will manipulate actx->mux as needed.
+ */
+ CHECK_MSETOPT(actx, CURLMOPT_SOCKETFUNCTION, register_socket, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_SOCKETDATA, actx, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_TIMERFUNCTION, register_timer, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_TIMERDATA, actx, return false);
+
+ /*
+ * Set up an easy handle. All of our requests are made serially, so we
+ * only ever need to keep track of one.
+ */
+ actx->curl = curl_easy_init();
+ if (!actx->curl)
+ {
+ actx_error(actx, "failed to create cURL handle");
+ return false;
+ }
+
+ /*
+ * Multi-threaded applications must set CURLOPT_NOSIGNAL. This requires us
+ * to handle the possibility of SIGPIPE ourselves.
+ *
+ * TODO: handle SIGPIPE via pq_block_sigpipe(), or via a
+ * CURLOPT_SOCKOPTFUNCTION maybe...
+ */
+ CHECK_SETOPT(actx, CURLOPT_NOSIGNAL, 1L, return false);
+ if (!curl_info->ares_num)
+ {
+ /* No alternative resolver, TODO: warn about timeouts */
+ }
+
+ /* TODO investigate using conn->Pfdebug and CURLOPT_DEBUGFUNCTION here */
+ CHECK_SETOPT(actx, CURLOPT_VERBOSE, 1L, return false);
+ CHECK_SETOPT(actx, CURLOPT_ERRORBUFFER, actx->curl_err, return false);
+
+ /*
+ * Only HTTP[S] is allowed. TODO: disallow HTTP without user opt-in
+ */
+ CHECK_SETOPT(actx, CURLOPT_PROTOCOLS, CURLPROTO_HTTP | CURLPROTO_HTTPS, return false);
+
+ /*
+ * Suppress the Accept header to make our request as minimal as possible.
+ * (Ideally we would set it to "application/json" instead, but OpenID is
+ * pretty strict when it comes to provider behavior, so we have to check
+ * what comes back anyway.)
+ */
+ actx->headers = curl_slist_append(actx->headers, "Accept:"); /* TODO: check result */
+ CHECK_SETOPT(actx, CURLOPT_HTTPHEADER, actx->headers, return false);
+
+ return true;
+}
+
+/*
+ * Generic HTTP Request Handlers
+ */
+
+/*
+ * Response callback from cURL; appends the response body into actx->work_data.
+ * See start_request().
+ */
+static size_t
+append_data(char *buf, size_t size, size_t nmemb, void *userdata)
+{
+ PQExpBuffer resp = userdata;
+ size_t len = size * nmemb;
+
+ /* TODO: cap the maximum size */
+ appendBinaryPQExpBuffer(resp, buf, len);
+ /* TODO: check for broken buffer */
+
+ return len;
+}
+
+/*
+ * Begins an HTTP request on the multi handle. The caller should have set up all
+ * request-specific options on actx->curl first. The server's response body will
+ * be accumulated in actx->work_data (which will be reset, so don't store
+ * anything important there across this call).
+ *
+ * Once a request is queued, it can be driven to completion via drive_request().
+ */
+static bool
+start_request(struct async_ctx *actx)
+{
+ CURLMcode err;
+ int running;
+
+ resetPQExpBuffer(&actx->work_data);
+ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data, return false);
+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, &actx->work_data, return false);
+
+ err = curl_multi_add_handle(actx->curlm, actx->curl);
+ if (err)
+ {
+ actx_error(actx, "failed to queue HTTP request: %s",
+ curl_multi_strerror(err));
+ return false;
+ }
+
+ err = curl_multi_socket_action(actx->curlm, CURL_SOCKET_TIMEOUT, 0, &running);
+ if (err)
+ {
+ actx_error(actx, "asynchronous HTTP request failed: %s",
+ curl_multi_strerror(err));
+ return false;
+ }
+
+ /*
+ * Sanity check.
+ *
+ * TODO: even though this is nominally an asynchronous process, there are
+ * apparently operations that can synchronously fail by this point, such
+ * as connections to closed local ports. Maybe we need to let this case
+ * fall through to drive_request instead, or else perform a
+ * curl_multi_info_read immediately.
+ */
+ if (running != 1)
+ {
+ actx_error(actx, "failed to queue HTTP request");
+ return false;
+ }
+
+ return true;
+}
+
+/*
+ * Drives the multi handle towards completion. The caller should have already
+ * set up an asynchronous request via start_request().
+ */
+static PostgresPollingStatusType
+drive_request(struct async_ctx *actx)
+{
+ CURLMcode err;
+ int running;
+ CURLMsg *msg;
+ int msgs_left;
+ bool done;
+
+ err = curl_multi_socket_all(actx->curlm, &running);
+ if (err)
+ {
+ actx_error(actx, "asynchronous HTTP request failed: %s",
+ curl_multi_strerror(err));
+ return PGRES_POLLING_FAILED;
+ }
+
+ if (running)
+ {
+ /* We'll come back again. */
+ return PGRES_POLLING_READING;
+ }
+
+ done = false;
+ while ((msg = curl_multi_info_read(actx->curlm, &msgs_left)) != NULL)
+ {
+ if (msg->msg != CURLMSG_DONE)
+ {
+ /*
+ * Future cURL versions may define new message types; we don't
+ * know how to handle them, so we'll ignore them.
+ */
+ continue;
+ }
+
+ /* First check the status of the request itself. */
+ if (msg->data.result != CURLE_OK)
+ {
+ actx_error_str(actx, curl_easy_strerror(msg->data.result));
+ return PGRES_POLLING_FAILED;
+ }
+
+ /* Now remove the finished handle; we'll add it back later if needed. */
+ err = curl_multi_remove_handle(actx->curlm, msg->easy_handle);
+ if (err)
+ {
+ actx_error(actx, "cURL easy handle removal failed: %s",
+ curl_multi_strerror(err));
+ return PGRES_POLLING_FAILED;
+ }
+
+ done = true;
+ }
+
+ /* Sanity check. */
+ if (!done)
+ {
+ actx_error(actx, "no result was retrieved for the finished handle");
+ return PGRES_POLLING_FAILED;
+ }
+
+ return PGRES_POLLING_OK;
+}
+
+/*
+ * Specific HTTP Request Handlers
+ *
+ * This is finally the beginning of the actual application logic. Generally
+ * speaking, a single request consists of a start_* and a finish_* step, with
+ * drive_request() pumping the machine in between.
+ */
+
+/*
+ * Queue an OpenID Provider Configuration Request:
+ *
+ * https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderConfigurationRequest
+ * https://www.rfc-editor.org/rfc/rfc8414#section-3.1
+ *
+ * This is done first to get the endpoint URIs we need to contact and to make
+ * sure the provider provides a device authorization flow. finish_discovery()
+ * will fill in actx->provider.
+ */
+static bool
+start_discovery(struct async_ctx *actx, const char *discovery_uri)
+{
+ CHECK_SETOPT(actx, CURLOPT_HTTPGET, 1L, return false);
+ CHECK_SETOPT(actx, CURLOPT_URL, discovery_uri, return false);
+
+ return start_request(actx);
+}
+
+static bool
+finish_discovery(struct async_ctx *actx)
+{
+ long response_code;
+
+ /*----
+ * Now check the response. OIDC Discovery 1.0 is pretty strict:
+ *
+ * A successful response MUST use the 200 OK HTTP status code and
+ * return a JSON object using the application/json content type that
+ * contains a set of Claims as its members that are a subset of the
+ * Metadata values defined in Section 3.
+ *
+ * Compared to standard HTTP semantics, this makes life easy -- we don't
+ * need to worry about redirections (which would call the Issuer host
+ * validation into question), or non-authoritative responses, or any other
+ * complications.
+ */
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ if (response_code != 200)
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ actx->errctx = "failed to parse OpenID discovery document";
+ if (!parse_provider(actx, &actx->provider))
+ return false; /* error message already set */
+
+ /*
+ * Fill in any defaults for OPTIONAL/RECOMMENDED fields we care about.
+ */
+ if (!actx->provider.grant_types_supported)
+ {
+ /*
+ * Per Section 3, the default is ["authorization_code", "implicit"].
+ */
+ struct curl_slist *temp = actx->provider.grant_types_supported;
+
+ temp = curl_slist_append(temp, "authorization_code");
+ if (temp)
+ {
+ temp = curl_slist_append(temp, "implicit");
+ }
+
+ if (!temp)
+ {
+ actx_error(actx, "out of memory");
+ return false;
+ }
+
+ actx->provider.grant_types_supported = temp;
+ }
+
+ return true;
+}
+
+#define OAUTH_GRANT_TYPE_DEVICE_CODE "urn:ietf:params:oauth:grant-type:device_code"
+
+/*
+ * Ensure that the provider supports the Device Authorization flow (i.e. it
+ * accepts the device_code grant type and provides an authorization endpoint).
+ */
+static bool
+check_for_device_flow(struct async_ctx *actx)
+{
+ const struct provider *provider = &actx->provider;
+ const struct curl_slist *grant;
+ bool device_grant_found = false;
+
+ Assert(provider->issuer); /* ensured by get_discovery_document() */
+
+ /*------
+ * First, sanity checks for discovery contents that are OPTIONAL in the
+ * spec but required for our flow:
+ * - the issuer must support the device_code grant
+ * - the issuer must have actually given us a
+ * device_authorization_endpoint
+ */
+
+ grant = provider->grant_types_supported;
+ while (grant)
+ {
+ if (strcmp(grant->data, OAUTH_GRANT_TYPE_DEVICE_CODE) == 0)
+ {
+ device_grant_found = true;
+ break;
+ }
+
+ grant = grant->next;
+ }
+
+ if (!device_grant_found)
+ {
+ actx_error(actx, "issuer \"%s\" does not support device code grants",
+ provider->issuer);
+ return false;
+ }
+
+ if (!provider->device_authorization_endpoint)
+ {
+ actx_error(actx,
+ "issuer \"%s\" does not provide a device authorization endpoint",
+ provider->issuer);
+ return false;
+ }
+
+ /* TODO: check that the endpoint uses HTTPS */
+
+ return true;
+}
+
+/*
+ * Queue a Device Authorization Request:
+ *
+ * https://www.rfc-editor.org/rfc/rfc8628#section-3.1
+ *
+ * This is the second step. We ask the provider to verify the end user out of
+ * band and authorize us to act on their behalf; it will give us the required
+ * nonces for us to later poll the request status, which we'll grab in
+ * finish_device_authz().
+ */
+static bool
+start_device_authz(struct async_ctx *actx, PGconn *conn)
+{
+ const char *device_authz_uri = actx->provider.device_authorization_endpoint;
+ PQExpBuffer work_buffer = &actx->work_data;
+
+ Assert(conn->oauth_client_id); /* ensured by get_auth_token() */
+ Assert(device_authz_uri); /* ensured by check_for_device_flow() */
+
+ /* Construct our request body. TODO: url-encode */
+ resetPQExpBuffer(work_buffer);
+ appendPQExpBuffer(work_buffer, "client_id=%s", conn->oauth_client_id);
+ if (conn->oauth_scope)
+ appendPQExpBuffer(work_buffer, "&scope=%s", conn->oauth_scope);
+ /* TODO check for broken buffer */
+
+ /* Make our request. */
+ CHECK_SETOPT(actx, CURLOPT_URL, device_authz_uri, return false);
+ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data, return false);
+
+ if (conn->oauth_client_secret)
+ {
+ /*----
+ * Use HTTP Basic auth to send the password. Per RFC 6749, Sec. 2.3.1,
+ *
+ * Including the client credentials in the request-body using the
+ * two parameters is NOT RECOMMENDED and SHOULD be limited to
+ * clients unable to directly utilize the HTTP Basic authentication
+ * scheme (or other password-based HTTP authentication schemes).
+ *
+ * TODO: should we omit client_id from the body in this case?
+ */
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC, return false);
+ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id, return false);
+ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret, return false);
+ }
+ else
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE, return false);
+
+ return start_request(actx);
+}
+
+static bool
+finish_device_authz(struct async_ctx *actx)
+{
+ long response_code;
+
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ /*
+ * The device authorization endpoint uses the same error response as the
+ * token endpoint, so we have to handle 400/401 here too.
+ */
+ if (response_code != 200
+ && response_code != 400
+ /* && response_code != 401 TODO */ )
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ if (response_code != 200)
+ {
+ struct token_error err = {0};
+
+ if (!parse_token_error(actx, &err))
+ {
+ free_token_error(&err);
+ return false;
+ }
+
+ if (err.error_description)
+ appendPQExpBuffer(&actx->errbuf, "%s ", err.error_description);
+
+ appendPQExpBuffer(&actx->errbuf, "(%s)", err.error);
+
+ free_token_error(&err);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ actx->errctx = "failed to parse device authorization";
+ if (!parse_device_authz(actx, &actx->authz))
+ return false; /* error message already set */
+
+ return true;
+}
+
+/*
+ * Queue an Access Token Request:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-4.1.3
+ *
+ * This is the final step. We continually poll the token endpoint to see if the
+ * user has authorized us yet. finish_token_request() will pull either the token
+ * or a (ideally temporary) error status from the provider.
+ */
+static bool
+start_token_request(struct async_ctx *actx, PGconn *conn)
+{
+ const char *token_uri = actx->provider.token_endpoint;
+ const char *device_code = actx->authz.device_code;
+ PQExpBuffer work_buffer = &actx->work_data;
+
+ Assert(conn->oauth_client_id); /* ensured by get_auth_token() */
+ Assert(token_uri); /* ensured by get_discovery_document() */
+ Assert(device_code); /* ensured by run_device_authz() */
+
+ /* Construct our request body. TODO: url-encode */
+ resetPQExpBuffer(work_buffer);
+ appendPQExpBuffer(work_buffer, "client_id=%s", conn->oauth_client_id);
+ appendPQExpBuffer(work_buffer, "&device_code=%s", device_code);
+ appendPQExpBuffer(work_buffer, "&grant_type=%s",
+ OAUTH_GRANT_TYPE_DEVICE_CODE);
+ /* TODO check for broken buffer */
+
+ /* Make our request. */
+ CHECK_SETOPT(actx, CURLOPT_URL, token_uri, return false);
+ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data, return false);
+
+ if (conn->oauth_client_secret)
+ {
+ /*----
+ * Use HTTP Basic auth to send the password. Per RFC 6749, Sec. 2.3.1,
+ *
+ * Including the client credentials in the request-body using the
+ * two parameters is NOT RECOMMENDED and SHOULD be limited to
+ * clients unable to directly utilize the HTTP Basic authentication
+ * scheme (or other password-based HTTP authentication schemes).
+ *
+ * TODO: should we omit client_id from the body in this case?
+ */
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC, return false);
+ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id, return false);
+ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret, return false);
+ }
+ else
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE, return false);
+
+ resetPQExpBuffer(work_buffer);
+ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data, return false);
+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, work_buffer, return false);
+
+ return start_request(actx);
+}
+
+static bool
+finish_token_request(struct async_ctx *actx, struct token *tok)
+{
+ long response_code;
+
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ /*
+ * Per RFC 6749, Section 5, a successful response uses 200 OK. An error
+ * response uses either 400 Bad Request or 401 Unauthorized.
+ *
+ * TODO: there are references online to 403 appearing in the wild...
+ */
+ if (response_code != 200
+ && response_code != 400
+ /* && response_code != 401 TODO */ )
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ if (response_code == 200)
+ {
+ actx->errctx = "failed to parse access token response";
+ if (!parse_access_token(actx, tok))
+ return false; /* error message already set */
+ }
+ else if (!parse_token_error(actx, &tok->err))
+ return false;
+
+ return true;
+}
+
+/*
+ * The top-level, nonblocking entry point for the cURL implementation. This will
+ * be called several times to pump the async engine.
+ *
+ * The architecture is based on PQconnectPoll(). The first half drives the
+ * connection state forward as necessary, returning if we're not ready to
+ * proceed to the next step yet. The second half performs the actual transition
+ * between states.
+ *
+ * You can trace the overall OAuth flow through the second half. It's linear
+ * until we get to the end, where we flip back and forth between
+ * OAUTH_STEP_TOKEN_REQUEST and OAUTH_STEP_WAIT_INTERVAL to regularly ping the
+ * provider.
+ */
+PostgresPollingStatusType
+pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
+{
+ fe_oauth_state *state = conn->sasl_state;
+ struct async_ctx *actx;
+
+ struct token tok = {0};
+
+ /*
+ * XXX This is not safe. cURL has stringent requirements for the thread
+ * context in which you call curl_global_init(), because it's going to try
+ * initializing a bunch of other libraries (OpenSSL, Winsock...). And we
+ * probably need to consider both the TLS backend libcurl is compiled
+ * against and what the user has asked us to do via PQinit[Open]SSL.
+ *
+ * Recent versions of libcurl have improved the thread-safety situation,
+ * but you apparently can't check at compile time whether the
+ * implementation is thread-safe, and there's a chicken-and-egg problem
+ * where you can't check the thread safety until you've initialized cURL,
+ * which you can't do before you've made sure it's thread-safe...
+ *
+ * We know we've already initialized Winsock by this point, so we should
+ * be able to safely skip that bit. But we have to tell cURL to initialize
+ * everything else, because other pieces of our client executable may
+ * already be using cURL for their own purposes. If we initialize libcurl
+ * first, with only a subset of its features, we could break those other
+ * clients nondeterministically, and that would probably be a nightmare to
+ * debug.
+ */
+ curl_global_init(CURL_GLOBAL_ALL
+ & ~CURL_GLOBAL_WIN32); /* we already initialized Winsock */
+
+ if (!state->async_ctx)
+ {
+ /*
+ * Create our asynchronous state, and hook it into the upper-level
+ * OAuth state immediately, so any failures below won't leak the
+ * context allocation.
+ */
+ actx = calloc(1, sizeof(*actx));
+ if (!actx)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return PGRES_POLLING_FAILED;
+ }
+
+ actx->mux = PGINVALID_SOCKET;
+#ifdef HAVE_SYS_EPOLL_H
+ actx->timerfd = -1;
+#endif
+
+ state->async_ctx = actx;
+ state->free_async_ctx = free_curl_async_ctx;
+
+ initPQExpBuffer(&actx->work_data);
+ initPQExpBuffer(&actx->errbuf);
+
+ if (!setup_multiplexer(actx))
+ goto error_return;
+
+ if (!setup_curl_handles(actx))
+ goto error_return;
+ }
+
+ actx = state->async_ctx;
+
+ /* By default, the multiplexer is the altsock. Reassign as desired. */
+ *altsock = actx->mux;
+
+ switch (actx->step)
+ {
+ case OAUTH_STEP_INIT:
+ break;
+
+ case OAUTH_STEP_DISCOVERY:
+ case OAUTH_STEP_DEVICE_AUTHORIZATION:
+ case OAUTH_STEP_TOKEN_REQUEST:
+ {
+ PostgresPollingStatusType status;
+
+ status = drive_request(actx);
+
+ if (status == PGRES_POLLING_FAILED)
+ goto error_return;
+ else if (status != PGRES_POLLING_OK)
+ {
+ /* not done yet */
+ free_token(&tok);
+ return status;
+ }
+ }
+
+ case OAUTH_STEP_WAIT_INTERVAL:
+ /* TODO check that the timer has expired */
+ break;
+ }
+
+ switch (actx->step)
+ {
+ case OAUTH_STEP_INIT:
+ actx->errctx = "failed to fetch OpenID discovery document";
+ if (!start_discovery(actx, conn->oauth_discovery_uri))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_DISCOVERY;
+ break;
+
+ case OAUTH_STEP_DISCOVERY:
+ if (!finish_discovery(actx))
+ goto error_return;
+
+ /* TODO: check issuer */
+
+ actx->errctx = "cannot run OAuth device authorization";
+ if (!check_for_device_flow(actx))
+ goto error_return;
+
+ actx->errctx = "failed to obtain device authorization";
+ if (!start_device_authz(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_DEVICE_AUTHORIZATION;
+ break;
+
+ case OAUTH_STEP_DEVICE_AUTHORIZATION:
+ if (!finish_device_authz(actx))
+ goto error_return;
+
+ actx->errctx = "failed to obtain access token";
+ if (!start_token_request(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_TOKEN_REQUEST;
+ break;
+
+ case OAUTH_STEP_TOKEN_REQUEST:
+ {
+ const struct token_error *err;
+#ifdef HAVE_SYS_EPOLL_H
+ struct itimerspec spec = {0};
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct kevent ev = {0};
+#endif
+
+ if (!finish_token_request(actx, &tok))
+ goto error_return;
+
+ if (!actx->user_prompted)
+ {
+ int res;
+ PQpromptOAuthDevice prompt = {
+ .verification_uri = actx->authz.verification_uri,
+ .user_code = actx->authz.user_code,
+ /* TODO: optional fields */
+ };
+
+ /*
+ * Now that we know the token endpoint isn't broken, give
+ * the user the login instructions.
+ */
+ res = PQauthDataHook(PQAUTHDATA_PROMPT_OAUTH_DEVICE, conn,
+ &prompt);
+
+ if (!res)
+ {
+ fprintf(stderr, "Visit %s and enter the code: %s",
+ prompt.verification_uri, prompt.user_code);
+ }
+ else if (res < 0)
+ {
+ actx_error(actx, "device prompt failed");
+ goto error_return;
+ }
+
+ actx->user_prompted = true;
+ }
+
+ if (tok.access_token)
+ {
+ /* Construct our Bearer token. */
+ resetPQExpBuffer(&actx->work_data);
+ appendPQExpBuffer(&actx->work_data, "Bearer %s",
+ tok.access_token);
+
+ if (PQExpBufferDataBroken(actx->work_data))
+ {
+ actx_error(actx, "out of memory");
+ goto error_return;
+ }
+
+ state->token = strdup(actx->work_data.data);
+ break;
+ }
+
+ /*
+ * authorization_pending and slow_down are the only acceptable
+ * errors; anything else and we bail.
+ */
+ err = &tok.err;
+ if (!err->error || (strcmp(err->error, "authorization_pending")
+ && strcmp(err->error, "slow_down")))
+ {
+ /* TODO handle !err->error */
+ if (err->error_description)
+ appendPQExpBuffer(&actx->errbuf, "%s ",
+ err->error_description);
+
+ appendPQExpBuffer(&actx->errbuf, "(%s)", err->error);
+
+ goto error_return;
+ }
+
+ /*
+ * A slow_down error requires us to permanently increase our
+ * retry interval by five seconds. RFC 8628, Sec. 3.5.
+ */
+ if (strcmp(err->error, "slow_down") == 0)
+ {
+ actx->authz.interval += 5; /* TODO check for overflow? */
+ }
+
+ /*
+ * Wait for the required interval before issuing the next
+ * request.
+ */
+ Assert(actx->authz.interval > 0);
+#ifdef HAVE_SYS_EPOLL_H
+ spec.it_value.tv_sec = actx->authz.interval;
+
+ if (timerfd_settime(actx->timerfd, 0 /* no flags */ , &spec, NULL) < 0)
+ {
+ actx_error(actx, "failed to set timerfd: %m");
+ goto error_return;
+ }
+
+ *altsock = actx->timerfd;
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ /* XXX: I guess this wants to be hidden in a routine */
+ EV_SET(&ev, 1, EVFILT_TIMER, EV_ADD, 0,
+ actx->authz.interval * 1000, 0);
+ if (kevent(actx->mux, &ev, 1, NULL, 0, NULL) < 0)
+ {
+ actx_error(actx, "failed to set kqueue timer: %m");
+ goto error_return;
+ }
+ /* XXX: why did we change the altsock in the epoll version? */
+#endif
+ actx->step = OAUTH_STEP_WAIT_INTERVAL;
+ break;
+ }
+
+ case OAUTH_STEP_WAIT_INTERVAL:
+ actx->errctx = "failed to obtain access token";
+ if (!start_token_request(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_TOKEN_REQUEST;
+ break;
+ }
+
+ free_token(&tok);
+
+ /* If we've stored a token, we're done. Otherwise come back later. */
+ return state->token ? PGRES_POLLING_OK : PGRES_POLLING_READING;
+
+error_return:
+
+ /*
+ * Assemble the three parts of our error: context, body, and detail. See
+ * also the documentation for struct async_ctx.
+ */
+ if (actx->errctx)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext(actx->errctx));
+ appendPQExpBufferStr(&conn->errorMessage, ": ");
+ }
+
+ if (PQExpBufferDataBroken(actx->errbuf))
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("out of memory"));
+ else
+ appendPQExpBufferStr(&conn->errorMessage, actx->errbuf.data);
+
+ if (actx->curl_err[0])
+ {
+ size_t len;
+
+ appendPQExpBuffer(&conn->errorMessage, " (%s)", actx->curl_err);
+
+ /* Sometimes libcurl adds a newline to the error buffer. :( */
+ len = conn->errorMessage.len;
+ if (len >= 2 && conn->errorMessage.data[len - 2] == '\n')
+ {
+ conn->errorMessage.data[len - 2] = ')';
+ conn->errorMessage.data[len - 1] = '\0';
+ conn->errorMessage.len--;
+ }
+ }
+
+ appendPQExpBufferStr(&conn->errorMessage, "\n");
+
+ free_token(&tok);
+ return PGRES_POLLING_FAILED;
+}
diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
new file mode 100644
index 0000000000..66ee8ff076
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth.c
@@ -0,0 +1,659 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth.c
+ * The front-end (client) implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "common/base64.h"
+#include "common/hmac.h"
+#include "common/jsonapi.h"
+#include "common/oauth-common.h"
+#include "fe-auth.h"
+#include "fe-auth-oauth.h"
+#include "mb/pg_wchar.h"
+
+/* The exported OAuth callback mechanism. */
+static void *oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism);
+static SASLStatus oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen);
+static bool oauth_channel_bound(void *opaq);
+static void oauth_free(void *opaq);
+
+const pg_fe_sasl_mech pg_oauth_mech = {
+ oauth_init,
+ oauth_exchange,
+ oauth_channel_bound,
+ oauth_free,
+};
+
+static void *
+oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism)
+{
+ fe_oauth_state *state;
+
+ /*
+ * We only support one SASL mechanism here; anything else is programmer
+ * error.
+ */
+ Assert(sasl_mechanism != NULL);
+ Assert(strcmp(sasl_mechanism, OAUTHBEARER_NAME) == 0);
+
+ state = calloc(1, sizeof(*state));
+ if (!state)
+ return NULL;
+
+ state->state = FE_OAUTH_INIT;
+ state->conn = conn;
+
+ return state;
+}
+
+#define kvsep "\x01"
+
+static char *
+client_initial_response(PGconn *conn, const char *token)
+{
+ static const char *const resp_format = "n,," kvsep "auth=%s" kvsep kvsep;
+
+ PQExpBufferData buf;
+ char *response = NULL;
+
+ if (!token)
+ {
+ /*
+ * Either programmer error, or something went badly wrong during the
+ * asynchronous fetch.
+ *
+ * TODO: users shouldn't see this; what action should they take if
+ * they do?
+ */
+ libpq_append_conn_error(conn, "no OAuth token was set for the connection");
+ return NULL;
+ }
+
+ initPQExpBuffer(&buf);
+ appendPQExpBuffer(&buf, resp_format, token);
+
+ if (!PQExpBufferDataBroken(buf))
+ response = strdup(buf.data);
+
+ termPQExpBuffer(&buf);
+ return response;
+}
+
+#define ERROR_STATUS_FIELD "status"
+#define ERROR_SCOPE_FIELD "scope"
+#define ERROR_OPENID_CONFIGURATION_FIELD "openid-configuration"
+
+struct json_ctx
+{
+ char *errmsg; /* any non-NULL value stops all processing */
+ PQExpBufferData errbuf; /* backing memory for errmsg */
+ int nested; /* nesting level (zero is the top) */
+
+ const char *target_field_name; /* points to a static allocation */
+ char **target_field; /* see below */
+
+ /* target_field, if set, points to one of the following: */
+ char *status;
+ char *scope;
+ char *discovery_uri;
+};
+
+#define oauth_json_has_error(ctx) \
+ (PQExpBufferDataBroken((ctx)->errbuf) || (ctx)->errmsg)
+
+#define oauth_json_set_error(ctx, ...) \
+ do { \
+ appendPQExpBuffer(&(ctx)->errbuf, __VA_ARGS__); \
+ (ctx)->errmsg = (ctx)->errbuf.data; \
+ } while (0)
+
+static JsonParseErrorType
+oauth_json_object_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ ++ctx->nested;
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_end(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_field_start(void *state, char *name, bool isnull)
+{
+ struct json_ctx *ctx = state;
+
+ if (ctx->nested == 1)
+ {
+ if (strcmp(name, ERROR_STATUS_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_STATUS_FIELD;
+ ctx->target_field = &ctx->status;
+ }
+ else if (strcmp(name, ERROR_SCOPE_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_SCOPE_FIELD;
+ ctx->target_field = &ctx->scope;
+ }
+ else if (strcmp(name, ERROR_OPENID_CONFIGURATION_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_OPENID_CONFIGURATION_FIELD;
+ ctx->target_field = &ctx->discovery_uri;
+ }
+ }
+
+ free(name);
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_scalar(void *state, char *token, JsonTokenType type)
+{
+ struct json_ctx *ctx = state;
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ if (type == JSON_TOKEN_STRING)
+ {
+ *ctx->target_field = token;
+
+ ctx->target_field = NULL;
+ ctx->target_field_name = NULL;
+
+ return JSON_SUCCESS; /* don't free the token we're using */
+ }
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ free(token);
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static bool
+handle_oauth_sasl_error(PGconn *conn, char *msg, int msglen)
+{
+ JsonLexContext lex = {0};
+ JsonSemAction sem = {0};
+ JsonParseErrorType err;
+ struct json_ctx ctx = {0};
+ char *errmsg = NULL;
+
+ /* Sanity check. */
+ if (strlen(msg) != msglen)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server's error message contained an embedded NULL, and was discarded"));
+ return false;
+ }
+
+ makeJsonLexContextCstringLen(&lex, msg, msglen, PG_UTF8, true);
+
+ initPQExpBuffer(&ctx.errbuf);
+ sem.semstate = &ctx;
+
+ sem.object_start = oauth_json_object_start;
+ sem.object_end = oauth_json_object_end;
+ sem.object_field_start = oauth_json_object_field_start;
+ sem.array_start = oauth_json_array_start;
+ sem.scalar = oauth_json_scalar;
+
+ err = pg_parse_json(&lex, &sem);
+
+ if (err == JSON_SEM_ACTION_FAILED)
+ {
+ if (PQExpBufferDataBroken(ctx.errbuf))
+ errmsg = libpq_gettext("out of memory");
+ else if (ctx.errmsg)
+ errmsg = ctx.errmsg;
+ else
+ {
+ /*
+ * Developer error: one of the action callbacks didn't call
+ * oauth_json_set_error() before erroring out.
+ */
+ Assert(oauth_json_has_error(&ctx));
+ errmsg = "<unexpected empty error>";
+ }
+ }
+ else if (err != JSON_SUCCESS)
+ errmsg = json_errdetail(err, &lex);
+
+ if (errmsg)
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("failed to parse server's error response: %s"),
+ errmsg);
+
+ /* Don't need the error buffer or the JSON lexer anymore. */
+ termPQExpBuffer(&ctx.errbuf);
+ freeJsonLexContext(&lex);
+
+ if (errmsg)
+ return false;
+
+ /* TODO: what if these override what the user already specified? */
+ if (ctx.discovery_uri)
+ {
+ if (conn->oauth_discovery_uri)
+ free(conn->oauth_discovery_uri);
+
+ conn->oauth_discovery_uri = ctx.discovery_uri;
+ }
+
+ if (ctx.scope)
+ {
+ if (conn->oauth_scope)
+ free(conn->oauth_scope);
+
+ conn->oauth_scope = ctx.scope;
+ }
+ /* TODO: missing error scope should clear any existing connection scope */
+
+ if (!ctx.status)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("server sent error response without a status"));
+ return false;
+ }
+
+ if (strcmp(ctx.status, "invalid_token") == 0)
+ {
+ /*
+ * invalid_token is the only error code we'll automatically retry for,
+ * but only if we have enough information to do so.
+ */
+ if (conn->oauth_discovery_uri)
+ conn->oauth_want_retry = true;
+ }
+ /* TODO: include status in hard failure message */
+
+ return true;
+}
+
+static void
+free_request(PGconn *conn, void *vreq)
+{
+ PQoauthBearerRequest *request = vreq;
+
+ if (request->cleanup)
+ request->cleanup(conn, request);
+
+ free(request);
+}
+
+static PostgresPollingStatusType
+run_user_oauth_flow(PGconn *conn, pgsocket *altsock)
+{
+ fe_oauth_state *state = conn->sasl_state;
+ PQoauthBearerRequest *request = state->async_ctx;
+ PostgresPollingStatusType status;
+
+ if (!request->async)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow provided neither a token nor an async callback");
+ return PGRES_POLLING_FAILED;
+ }
+
+ status = request->async(conn, request, altsock);
+ if (status == PGRES_POLLING_FAILED)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow failed");
+ return status;
+ }
+ else if (status == PGRES_POLLING_OK)
+ {
+ /*
+ * We already have a token, so copy it into the state. (We can't hold
+ * onto the original string, since it may not be safe for us to free()
+ * it.)
+ */
+ PQExpBufferData token;
+
+ if (!request->token)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow did not provide a token");
+ return PGRES_POLLING_FAILED;
+ }
+
+ initPQExpBuffer(&token);
+ appendPQExpBuffer(&token, "Bearer %s", request->token);
+
+ if (PQExpBufferDataBroken(token))
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return PGRES_POLLING_FAILED;
+ }
+
+ state->token = token.data;
+ return PGRES_POLLING_OK;
+ }
+
+ /* TODO: what if no altsock was set? */
+ return status;
+}
+
+static bool
+setup_token_request(PGconn *conn, fe_oauth_state *state)
+{
+ int res;
+ PQoauthBearerRequest request = {
+ .openid_configuration = conn->oauth_discovery_uri,
+ .scope = conn->oauth_scope,
+ };
+
+ Assert(request.openid_configuration);
+
+ /* The client may have overridden the OAuth flow. */
+ res = PQauthDataHook(PQAUTHDATA_OAUTH_BEARER_TOKEN, conn, &request);
+ if (res > 0)
+ {
+ PQoauthBearerRequest *request_copy;
+
+ if (request.token)
+ {
+ /*
+ * We already have a token, so copy it into the state. (We can't
+ * hold onto the original string, since it may not be safe for us
+ * to free() it.)
+ */
+ PQExpBufferData token;
+
+ initPQExpBuffer(&token);
+ appendPQExpBuffer(&token, "Bearer %s", request.token);
+
+ if (PQExpBufferDataBroken(token))
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ goto fail;
+ }
+
+ state->token = token.data;
+
+ /* short-circuit */
+ if (request.cleanup)
+ request.cleanup(conn, &request);
+ return true;
+ }
+
+ request_copy = malloc(sizeof(*request_copy));
+ if (!request_copy)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ goto fail;
+ }
+
+ memcpy(request_copy, &request, sizeof(request));
+
+ conn->async_auth = run_user_oauth_flow;
+ state->async_ctx = request_copy;
+ state->free_async_ctx = free_request;
+ }
+ else if (res < 0)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow failed");
+ goto fail;
+ }
+ else
+ {
+ /* Use our built-in OAuth flow. */
+ conn->async_auth = pg_fe_run_oauth_flow;
+ }
+
+ return true;
+
+fail:
+ if (request.cleanup)
+ request.cleanup(conn, &request);
+ return false;
+}
+
+static bool
+derive_discovery_uri(PGconn *conn)
+{
+ PQExpBufferData discovery_buf;
+
+ if (conn->oauth_discovery_uri || !conn->oauth_issuer)
+ {
+ /*
+ * Either we already have one, or we aren't able to derive one
+ * ourselves. The latter case is not an error condition; we'll just
+ * ask the server to provide one for us.
+ */
+ return true;
+ }
+
+ initPQExpBuffer(&discovery_buf);
+
+ Assert(!conn->oauth_discovery_uri);
+ Assert(conn->oauth_issuer);
+
+ /*
+ * If we don't yet have a discovery URI, but the user gave us an explicit
+ * issuer, use the .well-known discovery URI for that issuer.
+ */
+ appendPQExpBufferStr(&discovery_buf, conn->oauth_issuer);
+ appendPQExpBufferStr(&discovery_buf, "/.well-known/openid-configuration");
+
+ if (PQExpBufferDataBroken(discovery_buf))
+ goto cleanup;
+
+ conn->oauth_discovery_uri = strdup(discovery_buf.data);
+
+cleanup:
+ termPQExpBuffer(&discovery_buf);
+ return (conn->oauth_discovery_uri != NULL);
+}
+
+static SASLStatus
+oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen)
+{
+ fe_oauth_state *state = opaq;
+ PGconn *conn = state->conn;
+
+ *output = NULL;
+ *outputlen = 0;
+
+ switch (state->state)
+ {
+ case FE_OAUTH_INIT:
+ Assert(inputlen == -1);
+
+ if (!derive_discovery_uri(conn))
+ return SASL_FAILED;
+
+ if (conn->oauth_discovery_uri)
+ {
+ if (!conn->oauth_client_id)
+ {
+ /* We can't talk to a server without a client identifier. */
+ libpq_append_conn_error(conn, "no oauth_client_id is set for the connection");
+ return SASL_FAILED;
+ }
+
+ /*
+ * Decide whether we're using a user-provided OAuth flow, or
+ * the one we have built in.
+ */
+ if (!setup_token_request(conn, state))
+ return SASL_FAILED;
+
+ if (state->token)
+ {
+ /*
+ * A really smart user implementation may have already
+ * given us the token (e.g. if there was an unexpired copy
+ * already cached). In that case, we can just fall
+ * through.
+ */
+ }
+ else
+ {
+ /*
+ * Otherwise, we have to hand the connection over to our
+ * OAuth implementation. This involves a number of HTTP
+ * connections and timed waits, so we escape the
+ * synchronous auth processing and tell PQconnectPoll to
+ * transfer control to our async implementation.
+ */
+ Assert(conn->async_auth); /* should have been set
+ * already */
+ state->state = FE_OAUTH_REQUESTING_TOKEN;
+ return SASL_ASYNC;
+ }
+ }
+ else
+ {
+ /*
+ * If we don't have a discovery URI to be able to request a
+ * token, we ask the server for one explicitly with an empty
+ * token. This doesn't require any asynchronous work.
+ */
+ state->token = strdup("");
+ if (!state->token)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return SASL_FAILED;
+ }
+ }
+
+ /* fall through */
+
+ case FE_OAUTH_REQUESTING_TOKEN:
+ /* We should still be in the initial response phase. */
+ Assert(inputlen == -1);
+
+ *output = client_initial_response(conn, state->token);
+ if (!*output)
+ return SASL_FAILED;
+
+ *outputlen = strlen(*output);
+ state->state = FE_OAUTH_BEARER_SENT;
+
+ return SASL_CONTINUE;
+
+ case FE_OAUTH_BEARER_SENT:
+ if (final)
+ {
+ /* TODO: ensure there is no message content here. */
+ return SASL_COMPLETE;
+ }
+
+ /*
+ * Error message sent by the server.
+ */
+ if (!handle_oauth_sasl_error(conn, input, inputlen))
+ return SASL_FAILED;
+
+ /*
+ * Respond with the required dummy message (RFC 7628, sec. 3.2.3).
+ */
+ *output = strdup(kvsep);
+ *outputlen = strlen(*output); /* == 1 */
+
+ state->state = FE_OAUTH_SERVER_ERROR;
+ return SASL_CONTINUE;
+
+ case FE_OAUTH_SERVER_ERROR:
+
+ /*
+ * After an error, the server should send an error response to
+ * fail the SASL handshake, which is handled in higher layers.
+ *
+ * If we get here, the server either sent *another* challenge
+ * which isn't defined in the RFC, or completed the handshake
+ * successfully after telling us it was going to fail. Neither is
+ * acceptable.
+ */
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server sent additional OAuth data after error\n"));
+ return SASL_FAILED;
+
+ default:
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("invalid OAuth exchange state\n"));
+ break;
+ }
+
+ return SASL_FAILED;
+}
+
+static bool
+oauth_channel_bound(void *opaq)
+{
+ /* This mechanism does not support channel binding. */
+ return false;
+}
+
+static void
+oauth_free(void *opaq)
+{
+ fe_oauth_state *state = opaq;
+
+ free(state->token);
+ if (state->async_ctx)
+ state->free_async_ctx(state->conn, state->async_ctx);
+
+ free(state);
+}
diff --git a/src/interfaces/libpq/fe-auth-oauth.h b/src/interfaces/libpq/fe-auth-oauth.h
new file mode 100644
index 0000000000..8d4ea45aa8
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth.h
@@ -0,0 +1,42 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth.h
+ *
+ * Definitions for OAuth authentication implementations
+ *
+ * Portions Copyright (c) 2023, PostgreSQL Global Development Group
+ *
+ * src/interfaces/libpq/fe-auth-oauth.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FE_AUTH_OAUTH_H
+#define FE_AUTH_OAUTH_H
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+
+
+typedef enum
+{
+ FE_OAUTH_INIT,
+ FE_OAUTH_REQUESTING_TOKEN,
+ FE_OAUTH_BEARER_SENT,
+ FE_OAUTH_SERVER_ERROR,
+} fe_oauth_state_enum;
+
+typedef struct
+{
+ fe_oauth_state_enum state;
+
+ PGconn *conn;
+ char *token;
+
+ void *async_ctx;
+ void (*free_async_ctx) (PGconn *conn, void *ctx);
+} fe_oauth_state;
+
+extern PostgresPollingStatusType pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock);
+
+#endif /* FE_AUTH_OAUTH_H */
diff --git a/src/interfaces/libpq/fe-auth-sasl.h b/src/interfaces/libpq/fe-auth-sasl.h
index 4eecf53a15..83bdab1f40 100644
--- a/src/interfaces/libpq/fe-auth-sasl.h
+++ b/src/interfaces/libpq/fe-auth-sasl.h
@@ -30,6 +30,7 @@ typedef enum
SASL_COMPLETE = 0,
SASL_FAILED,
SASL_CONTINUE,
+ SASL_ASYNC,
} SASLStatus;
/*
@@ -77,6 +78,8 @@ typedef struct pg_fe_sasl_mech
*
* state: The opaque mechanism state returned by init()
*
+ * final: true if the server has sent a final exchange outcome
+ *
* input: The challenge data sent by the server, or NULL when
* generating a client-first initial response (that is, when
* the server expects the client to send a message to start
@@ -101,12 +104,17 @@ typedef struct pg_fe_sasl_mech
*
* SASL_CONTINUE: The output buffer is filled with a client response.
* Additional server challenge is expected
+ * SASL_ASYNC: Some asynchronous processing external to the
+ * connection needs to be done before a response can be
+ * generated. The mechanism is responsible for setting up
+ * conn->async_auth appropriately before returning.
* SASL_COMPLETE: The SASL exchange has completed successfully.
* SASL_FAILED: The exchange has failed and the connection should be
* dropped.
*--------
*/
- SASLStatus (*exchange) (void *state, char *input, int inputlen,
+ SASLStatus (*exchange) (void *state, bool final,
+ char *input, int inputlen,
char **output, int *outputlen);
/*--------
diff --git a/src/interfaces/libpq/fe-auth-scram.c b/src/interfaces/libpq/fe-auth-scram.c
index 0bb820e0d9..da168eb2f5 100644
--- a/src/interfaces/libpq/fe-auth-scram.c
+++ b/src/interfaces/libpq/fe-auth-scram.c
@@ -24,7 +24,8 @@
/* The exported SCRAM callback mechanism. */
static void *scram_init(PGconn *conn, const char *password,
const char *sasl_mechanism);
-static SASLStatus scram_exchange(void *opaq, char *input, int inputlen,
+static SASLStatus scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen);
static bool scram_channel_bound(void *opaq);
static void scram_free(void *opaq);
@@ -202,7 +203,8 @@ scram_free(void *opaq)
* Exchange a SCRAM message with backend.
*/
static SASLStatus
-scram_exchange(void *opaq, char *input, int inputlen,
+scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen)
{
fe_scram_state *state = (fe_scram_state *) opaq;
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index 81ec08485d..9cd5c8cfb1 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -39,6 +39,7 @@
#endif
#include "common/md5.h"
+#include "common/oauth-common.h"
#include "common/scram-common.h"
#include "fe-auth.h"
#include "fe-auth-sasl.h"
@@ -419,7 +420,7 @@ pg_SSPI_startup(PGconn *conn, int use_negotiate, int payloadlen)
* Initialize SASL authentication exchange.
*/
static int
-pg_SASL_init(PGconn *conn, int payloadlen)
+pg_SASL_init(PGconn *conn, int payloadlen, bool *async)
{
char *initialresponse = NULL;
int initialresponselen;
@@ -437,7 +438,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
goto error;
}
- if (conn->sasl_state)
+ if (conn->sasl_state && !conn->async_auth)
{
libpq_append_conn_error(conn, "duplicate SASL authentication request");
goto error;
@@ -524,6 +525,15 @@ pg_SASL_init(PGconn *conn, int payloadlen)
conn->sasl = &pg_scram_mech;
conn->password_needed = true;
}
+#ifdef USE_OAUTH
+ else if (strcmp(mechanism_buf.data, OAUTHBEARER_NAME) == 0 &&
+ !selected_mechanism)
+ {
+ selected_mechanism = OAUTHBEARER_NAME;
+ conn->sasl = &pg_oauth_mech;
+ conn->password_needed = false;
+ }
+#endif
}
if (!selected_mechanism)
@@ -563,26 +573,48 @@ pg_SASL_init(PGconn *conn, int payloadlen)
Assert(conn->sasl);
- /*
- * Initialize the SASL state information with all the information gathered
- * during the initial exchange.
- *
- * Note: Only tls-unique is supported for the moment.
- */
- conn->sasl_state = conn->sasl->init(conn,
- password,
- selected_mechanism);
if (!conn->sasl_state)
- goto oom_error;
+ {
+ /*
+ * Initialize the SASL state information with all the information
+ * gathered during the initial exchange.
+ *
+ * Note: Only tls-unique is supported for the moment.
+ */
+ conn->sasl_state = conn->sasl->init(conn,
+ password,
+ selected_mechanism);
+ if (!conn->sasl_state)
+ goto oom_error;
+ }
+ else
+ {
+ /*
+ * This is only possible if we're returning from an async loop.
+ * Disconnect it now.
+ */
+ Assert(conn->async_auth);
+ conn->async_auth = NULL;
+ }
/* Get the mechanism-specific Initial Client Response, if any */
- status = conn->sasl->exchange(conn->sasl_state,
+ status = conn->sasl->exchange(conn->sasl_state, false,
NULL, -1,
&initialresponse, &initialresponselen);
if (status == SASL_FAILED)
goto error;
+ if (status == SASL_ASYNC)
+ {
+ /*
+ * The mechanism should have set up the necessary callbacks; all we
+ * need to do is signal the caller.
+ */
+ *async = true;
+ return STATUS_OK;
+ }
+
/*
* Build a SASLInitialResponse message, and send it.
*/
@@ -625,7 +657,7 @@ oom_error:
* the protocol.
*/
static int
-pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
+pg_SASL_continue(PGconn *conn, int payloadlen, bool final, bool *async)
{
char *output;
int outputlen;
@@ -650,11 +682,21 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
/* For safety and convenience, ensure the buffer is NULL-terminated. */
challenge[payloadlen] = '\0';
- status = conn->sasl->exchange(conn->sasl_state,
+ status = conn->sasl->exchange(conn->sasl_state, final,
challenge, payloadlen,
&output, &outputlen);
free(challenge); /* don't need the input anymore */
+ if (status == SASL_ASYNC)
+ {
+ /*
+ * The mechanism should have set up the necessary callbacks; all we
+ * need to do is signal the caller.
+ */
+ *async = true;
+ return STATUS_OK;
+ }
+
if (final && status == SASL_CONTINUE)
{
if (outputlen != 0)
@@ -955,12 +997,18 @@ check_expected_areq(AuthRequest areq, PGconn *conn)
* it. We are responsible for reading any remaining extra data, specific
* to the authentication method. 'payloadlen' is the remaining length in
* the message.
+ *
+ * If *async is set to true on return, the client doesn't yet have enough
+ * information to respond, and the caller must temporarily switch to
+ * conn->async_auth() to continue driving the exchange.
*/
int
-pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
+pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn, bool *async)
{
int oldmsglen;
+ *async = false;
+
if (!check_expected_areq(areq, conn))
return STATUS_ERROR;
@@ -1118,7 +1166,7 @@ pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
* The request contains the name (as assigned by IANA) of the
* authentication mechanism.
*/
- if (pg_SASL_init(conn, payloadlen) != STATUS_OK)
+ if (pg_SASL_init(conn, payloadlen, async) != STATUS_OK)
{
/* pg_SASL_init already set the error message */
return STATUS_ERROR;
@@ -1135,7 +1183,8 @@ pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
}
oldmsglen = conn->errorMessage.len;
if (pg_SASL_continue(conn, payloadlen,
- (areq == AUTH_REQ_SASL_FIN)) != STATUS_OK)
+ (areq == AUTH_REQ_SASL_FIN),
+ async) != STATUS_OK)
{
/* Use this message if pg_SASL_continue didn't supply one */
if (conn->errorMessage.len == oldmsglen)
@@ -1451,3 +1500,23 @@ PQchangePassword(PGconn *conn, const char *user, const char *passwd)
}
}
}
+
+PQauthDataHook_type PQauthDataHook = PQdefaultAuthDataHook;
+
+PQauthDataHook_type
+PQgetAuthDataHook(void)
+{
+ return PQauthDataHook;
+}
+
+void
+PQsetAuthDataHook(PQauthDataHook_type hook)
+{
+ PQauthDataHook = hook ? hook : PQdefaultAuthDataHook;
+}
+
+int
+PQdefaultAuthDataHook(PGAuthData type, PGconn *conn, void *data)
+{
+ return 0; /* handle nothing */
+}
diff --git a/src/interfaces/libpq/fe-auth.h b/src/interfaces/libpq/fe-auth.h
index a18c508688..3e2bc1333f 100644
--- a/src/interfaces/libpq/fe-auth.h
+++ b/src/interfaces/libpq/fe-auth.h
@@ -18,8 +18,12 @@
#include "libpq-int.h"
+extern PQauthDataHook_type PQauthDataHook;
+
+
/* Prototypes for functions in fe-auth.c */
-extern int pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn);
+extern int pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn,
+ bool *async);
extern char *pg_fe_getusername(uid_t user_id, PQExpBuffer errorMessage);
extern char *pg_fe_getauthname(PQExpBuffer errorMessage);
@@ -29,4 +33,7 @@ extern char *pg_fe_scram_build_secret(const char *password,
int iterations,
const char **errstr);
+/* Mechanisms in fe-auth-oauth.c */
+extern const pg_fe_sasl_mech pg_oauth_mech;
+
#endif /* FE_AUTH_H */
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 01e49c6975..714594324e 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -359,6 +359,23 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"Load-Balance-Hosts", "", 8, /* sizeof("disable") = 8 */
offsetof(struct pg_conn, load_balance_hosts)},
+ /* OAuth v2 */
+ {"oauth_issuer", NULL, NULL, NULL,
+ "OAuth-Issuer", "", 40,
+ offsetof(struct pg_conn, oauth_issuer)},
+
+ {"oauth_client_id", NULL, NULL, NULL,
+ "OAuth-Client-ID", "", 40,
+ offsetof(struct pg_conn, oauth_client_id)},
+
+ {"oauth_client_secret", NULL, NULL, NULL,
+ "OAuth-Client-Secret", "", 40,
+ offsetof(struct pg_conn, oauth_client_secret)},
+
+ {"oauth_scope", NULL, NULL, NULL,
+ "OAuth-Scope", "", 15,
+ offsetof(struct pg_conn, oauth_scope)},
+
/* Terminating entry --- MUST BE LAST */
{NULL, NULL, NULL, NULL,
NULL, NULL, 0}
@@ -616,6 +633,7 @@ pqDropServerData(PGconn *conn)
conn->write_failed = false;
free(conn->write_err_msg);
conn->write_err_msg = NULL;
+ /* conn->oauth_want_retry = false; TODO */
/*
* Cancel connections need to retain their be_pid and be_key across
@@ -2595,6 +2613,7 @@ PQconnectPoll(PGconn *conn)
case CONNECTION_NEEDED:
case CONNECTION_GSS_STARTUP:
case CONNECTION_CHECK_TARGET:
+ case CONNECTION_AUTHENTICATING:
break;
default:
@@ -3603,6 +3622,7 @@ keep_going: /* We will come back to here until there is
int avail;
AuthRequest areq;
int res;
+ bool async;
/*
* Scan the message from current point (note that if we find
@@ -3758,6 +3778,16 @@ keep_going: /* We will come back to here until there is
/* Check to see if we should mention pgpassfile */
pgpassfileWarning(conn);
+#ifdef USE_OAUTH
+ if (conn->sasl == &pg_oauth_mech
+ && conn->oauth_want_retry)
+ {
+ /* TODO: only allow retry once */
+ need_new_connection = true;
+ goto keep_going;
+ }
+#endif
+
#ifdef ENABLE_GSS
/*
@@ -3839,7 +3869,17 @@ keep_going: /* We will come back to here until there is
* Note that conn->pghost must be non-NULL if we are going to
* avoid the Kerberos code doing a hostname look-up.
*/
- res = pg_fe_sendauth(areq, msgLength, conn);
+ res = pg_fe_sendauth(areq, msgLength, conn, &async);
+
+ if (async && (res == STATUS_OK))
+ {
+ /*
+ * We'll come back later once we're ready to respond.
+ * Don't consume the request yet.
+ */
+ conn->status = CONNECTION_AUTHENTICATING;
+ goto keep_going;
+ }
/* OK, we have processed the message; mark data consumed */
conn->inStart = conn->inCursor;
@@ -3872,6 +3912,41 @@ keep_going: /* We will come back to here until there is
goto keep_going;
}
+ case CONNECTION_AUTHENTICATING:
+ {
+ PostgresPollingStatusType status;
+ pgsocket altsock = PGINVALID_SOCKET;
+
+ if (!conn->async_auth)
+ {
+ /* programmer error; should not happen */
+ libpq_append_conn_error(conn, "async authentication has no handler");
+ goto error_return;
+ }
+
+ status = conn->async_auth(conn, &altsock);
+
+ if (status == PGRES_POLLING_FAILED)
+ goto error_return;
+
+ if (status == PGRES_POLLING_OK)
+ {
+ /*
+ * Reenter the authentication exchange with the server. We
+ * didn't consume the message that started external
+ * authentication, so it'll be reprocessed as if we just
+ * received it.
+ */
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ conn->altsock = PGINVALID_SOCKET; /* TODO: what frees
+ * this? */
+ goto keep_going;
+ }
+
+ conn->altsock = altsock;
+ return status;
+ }
+
case CONNECTION_AUTH_OK:
{
/*
@@ -4377,6 +4452,7 @@ pqMakeEmptyPGconn(void)
conn->verbosity = PQERRORS_DEFAULT;
conn->show_context = PQSHOW_CONTEXT_ERRORS;
conn->sock = PGINVALID_SOCKET;
+ conn->altsock = PGINVALID_SOCKET;
conn->Pfdebug = NULL;
/*
@@ -4493,6 +4569,11 @@ freePGconn(PGconn *conn)
free(conn->rowBuf);
free(conn->target_session_attrs);
free(conn->load_balance_hosts);
+ free(conn->oauth_issuer);
+ free(conn->oauth_discovery_uri);
+ free(conn->oauth_client_id);
+ free(conn->oauth_client_secret);
+ free(conn->oauth_scope);
termPQExpBuffer(&conn->errorMessage);
termPQExpBuffer(&conn->workBuffer);
@@ -6975,6 +7056,8 @@ PQsocket(const PGconn *conn)
{
if (!conn)
return -1;
+ if (conn->altsock != PGINVALID_SOCKET)
+ return conn->altsock;
return (conn->sock != PGINVALID_SOCKET) ? conn->sock : -1;
}
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index f2fc78a481..663b1c1acf 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -1039,10 +1039,13 @@ static int
pqSocketCheck(PGconn *conn, int forRead, int forWrite, time_t end_time)
{
int result;
+ pgsocket sock;
if (!conn)
return -1;
- if (conn->sock == PGINVALID_SOCKET)
+
+ sock = (conn->altsock != PGINVALID_SOCKET) ? conn->altsock : conn->sock;
+ if (sock == PGINVALID_SOCKET)
{
libpq_append_conn_error(conn, "invalid socket");
return -1;
@@ -1059,7 +1062,7 @@ pqSocketCheck(PGconn *conn, int forRead, int forWrite, time_t end_time)
/* We will retry as long as we get EINTR */
do
- result = pqSocketPoll(conn->sock, forRead, forWrite, end_time);
+ result = pqSocketPoll(sock, forRead, forWrite, end_time);
while (result < 0 && SOCK_ERRNO == EINTR);
if (result < 0)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 09b485bd2b..454ec8236b 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -38,6 +38,8 @@ extern "C"
#define LIBPQ_HAS_TRACE_FLAGS 1
/* Indicates that PQsslAttribute(NULL, "library") is useful */
#define LIBPQ_HAS_SSL_LIBRARY_DETECTION 1
+/* Indicates presence of the PQAUTHDATA_PROMPT_OAUTH_DEVICE authdata hook */
+#define LIBPQ_HAS_PROMPT_OAUTH_DEVICE 1
/*
* Option flags for PQcopyResult
@@ -80,8 +82,10 @@ typedef enum
CONNECTION_CHECK_TARGET, /* Internal state: checking target server
* properties. */
CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
- CONNECTION_ALLOCATED /* Waiting for connection attempt to be
+ CONNECTION_ALLOCATED, /* Waiting for connection attempt to be
* started. */
+ CONNECTION_AUTHENTICATING, /* Authentication is in progress with some
+ * external system. */
} ConnStatusType;
typedef enum
@@ -163,6 +167,13 @@ typedef enum
PQ_PIPELINE_ABORTED
} PGpipelineStatus;
+typedef enum
+{
+ PQAUTHDATA_PROMPT_OAUTH_DEVICE, /* user must visit a device-authorization
+ * URL */
+ PQAUTHDATA_OAUTH_BEARER_TOKEN, /* server requests an OAuth Bearer token */
+} PGAuthData;
+
/* PGconn encapsulates a connection to the backend.
* The contents of this struct are not supposed to be known to applications.
*/
@@ -684,10 +695,74 @@ extern int PQenv2encoding(void);
/* === in fe-auth.c === */
+typedef struct _PQpromptOAuthDevice
+{
+ const char *verification_uri; /* verification URI to visit */
+ const char *user_code; /* user code to enter */
+} PQpromptOAuthDevice;
+
+typedef struct _PQoauthBearerRequest
+{
+ /* Hook inputs (constant across all calls) */
+ const char *const openid_configuration; /* OIDC discovery URI */
+ const char *const scope; /* required scope(s), or NULL */
+
+ /* Hook outputs */
+
+ /*---------
+ * Callback implementing a custom asynchronous OAuth flow.
+ *
+ * The callback may return
+ * - PGRES_POLLING_READING/WRITING, to indicate that a file descriptor
+ * has been stored in *altsock and libpq should wait until it is
+ * readable or writable before calling back;
+ * - PGRES_POLLING_OK, to indicate that the flow is complete and
+ * request->token has been set; or
+ * - PGRES_POLLING_FAILED, to indicate that token retrieval has failed.
+ *
+ * This callback is optional. If the token can be obtained without
+ * blocking during the original call to the PQAUTHDATA_OAUTH_BEARER_TOKEN
+ * hook, it may be returned directly, but one of request->async or
+ * request->token must be set by the hook.
+ */
+ PostgresPollingStatusType (*async) (PGconn *conn,
+ struct _PQoauthBearerRequest *request,
+ int *altsock);
+
+ /*
+ * Callback to clean up custom allocations. A hook implementation may use
+ * this to free request->token and any resources in request->user.
+ *
+ * This is technically optional, but highly recommended, because there is
+ * no other indication as to when it is safe to free the token.
+ */
+ void (*cleanup) (PGconn *conn, struct _PQoauthBearerRequest *request);
+
+ /*
+ * The hook should set this to the Bearer token contents for the
+ * connection, once the flow is completed. The token contents must remain
+ * available to libpq until the hook's cleanup callback is called.
+ */
+ char *token;
+
+ /*
+ * Hook-defined data. libpq will not modify this pointer across calls to
+ * the async callback, so it can be used to keep track of
+ * application-specific state. Resources allocated here should be freed by
+ * the cleanup callback.
+ */
+ void *user;
+} PQoauthBearerRequest;
+
extern char *PQencryptPassword(const char *passwd, const char *user);
extern char *PQencryptPasswordConn(PGconn *conn, const char *passwd, const char *user, const char *algorithm);
extern PGresult *PQchangePassword(PGconn *conn, const char *user, const char *passwd);
+typedef int (*PQauthDataHook_type) (PGAuthData type, PGconn *conn, void *data);
+extern void PQsetAuthDataHook(PQauthDataHook_type hook);
+extern PQauthDataHook_type PQgetAuthDataHook(void);
+extern int PQdefaultAuthDataHook(PGAuthData type, PGconn *conn, void *data);
+
/* === in encnames.c === */
extern int pg_char_to_encoding(const char *name);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 9c05f11a6e..5f3a0b00db 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -351,6 +351,8 @@ typedef struct pg_conn_host
* found in password file. */
} pg_conn_host;
+typedef PostgresPollingStatusType (*AsyncAuthFunc) (PGconn *conn, pgsocket *altsock);
+
/*
* PGconn stores all the state data associated with a single connection
* to a backend.
@@ -413,6 +415,15 @@ struct pg_conn
* cancel request, instead of being a normal
* connection that's used for queries */
+ /* OAuth v2 */
+ char *oauth_issuer; /* token issuer URL */
+ char *oauth_discovery_uri; /* URI of the issuer's discovery
+ * document */
+ char *oauth_client_id; /* client identifier */
+ char *oauth_client_secret; /* client secret */
+ char *oauth_scope; /* access token scope */
+ bool oauth_want_retry; /* should we retry on failure? */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -481,6 +492,9 @@ struct pg_conn
bool client_finished_auth; /* have we finished our half of the
* authentication exchange? */
+ AsyncAuthFunc async_auth; /* callback for external async authentication */
+ pgsocket altsock; /* alternative socket for client to poll */
+
/* Transient state needed while establishing connection */
PGTargetServerType target_server_type; /* desired session properties */
diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build
index be6fadaea2..0d4b7ac17d 100644
--- a/src/interfaces/libpq/meson.build
+++ b/src/interfaces/libpq/meson.build
@@ -40,6 +40,13 @@ if gssapi.found()
)
endif
+if oauth.found()
+ libpq_sources += files('fe-auth-oauth.c')
+ if oauth_library == 'curl'
+ libpq_sources += files('fe-auth-oauth-curl.c')
+ endif
+endif
+
export_file = custom_target('libpq.exports',
kwargs: gen_export_kwargs,
)
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index b0f4178b3d..f803c1200b 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -231,6 +231,7 @@ pgxs_deps = {
'llvm': llvm,
'lz4': lz4,
'nls': libintl,
+ 'oauth': oauth,
'pam': pam,
'perl': perl_dep,
'python': python3_dep,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index e2a0525dd4..f535fbbd5c 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -153,6 +153,7 @@ ArrayMetaState
ArraySubWorkspace
ArrayToken
ArrayType
+AsyncAuthFunc
AsyncQueueControl
AsyncQueueEntry
AsyncRequest
@@ -354,6 +355,8 @@ CState
CTECycleClause
CTEMaterialize
CTESearchClause
+CURL
+CURLM
CV
CachedExpression
CachedPlan
@@ -1669,6 +1672,7 @@ NumericDigit
NumericSortSupport
NumericSumAccum
NumericVar
+OAuthStep
OM_uint32
OP
OSAPerGroupState
@@ -1734,6 +1738,7 @@ PFN
PGAlignedBlock
PGAlignedXLogBlock
PGAsyncStatusType
+PGAuthData
PGCALL2
PGChecksummablePage
PGContextVisibility
@@ -1894,11 +1899,14 @@ PQArgBlock
PQEnvironmentOption
PQExpBuffer
PQExpBufferData
+PQauthDataHook_type
PQcommMethods
PQconninfoOption
PQnoticeProcessor
PQnoticeReceiver
+PQoauthBearerRequest
PQprintOpt
+PQpromptOAuthDevice
PQsslKeyPassHook_OpenSSL_type
PREDICATELOCK
PREDICATELOCKTAG
@@ -3364,6 +3372,8 @@ explain_get_index_name_hook_type
f_smgr
fasthash_state
fd_set
+fe_oauth_state
+fe_oauth_state_enum
fe_scram_state
fe_scram_state_enum
fetch_range_request
--
2.34.1
v21-0004-Introduce-OAuth-validator-libraries.patch (application/octet-stream, 33.4K) download
From 78617fa9ba7af77b64ec9c821d6a7a3cab0e38ef Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Wed, 21 Feb 2024 17:04:26 +0100
Subject: [PATCH v21 4/7] Introduce OAuth validator libraries
This replaces the serverside validation code with an module API
for loading in extensions for validating bearer tokens. A lot of
code is left to be written.
Co-authored-by: Jacob Champion <jacob.champion@enterprisedb.com>
---
src/backend/libpq/auth-oauth.c | 431 +++++-------------
src/backend/utils/misc/guc_tables.c | 6 +-
src/common/Makefile | 2 +-
src/include/libpq/oauth.h | 29 +-
src/test/modules/meson.build | 1 +
src/test/modules/oauth_validator/.gitignore | 4 +
src/test/modules/oauth_validator/Makefile | 19 +
.../oauth_validator/expected/validator.out | 6 +
src/test/modules/oauth_validator/meson.build | 33 ++
.../modules/oauth_validator/sql/validator.sql | 1 +
.../modules/oauth_validator/t/001_server.pl | 78 ++++
src/test/modules/oauth_validator/validator.c | 82 ++++
src/test/perl/PostgreSQL/Test/Cluster.pm | 14 +-
src/test/perl/PostgreSQL/Test/OAuthServer.pm | 183 ++++++++
src/tools/pgindent/typedefs.list | 2 +
15 files changed, 560 insertions(+), 331 deletions(-)
create mode 100644 src/test/modules/oauth_validator/.gitignore
create mode 100644 src/test/modules/oauth_validator/Makefile
create mode 100644 src/test/modules/oauth_validator/expected/validator.out
create mode 100644 src/test/modules/oauth_validator/meson.build
create mode 100644 src/test/modules/oauth_validator/sql/validator.sql
create mode 100644 src/test/modules/oauth_validator/t/001_server.pl
create mode 100644 src/test/modules/oauth_validator/validator.c
create mode 100644 src/test/perl/PostgreSQL/Test/OAuthServer.pm
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
index 16596c089a..024f304e4d 100644
--- a/src/backend/libpq/auth-oauth.c
+++ b/src/backend/libpq/auth-oauth.c
@@ -6,7 +6,7 @@
* See the following RFC for more details:
* - RFC 7628: https://tools.ietf.org/html/rfc7628
*
- * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* src/backend/libpq/auth-oauth.c
@@ -19,22 +19,30 @@
#include <fcntl.h>
#include "common/oauth-common.h"
+#include "fmgr.h"
#include "lib/stringinfo.h"
#include "libpq/auth.h"
#include "libpq/hba.h"
#include "libpq/oauth.h"
#include "libpq/sasl.h"
#include "storage/fd.h"
+#include "storage/ipc.h"
#include "utils/json.h"
/* GUC */
-char *oauth_validator_command;
+char *OAuthValidatorLibrary = "";
static void oauth_get_mechanisms(Port *port, StringInfo buf);
static void *oauth_init(Port *port, const char *selected_mech, const char *shadow_pass);
static int oauth_exchange(void *opaq, const char *input, int inputlen,
char **output, int *outputlen, const char **logdetail);
+static void load_validator_library(void);
+static void shutdown_validator_library(int code, Datum arg);
+
+static ValidatorModuleState *validator_module_state;
+static const OAuthValidatorCallbacks *ValidatorCallbacks;
+
/* Mechanism declaration */
const pg_be_sasl_mech pg_be_oauth_mech = {
oauth_get_mechanisms,
@@ -63,11 +71,7 @@ struct oauth_ctx
static char *sanitize_char(char c);
static char *parse_kvpairs_for_auth(char **input);
static void generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen);
-static bool validate(Port *port, const char *auth, const char **logdetail);
-static bool run_validator_command(Port *port, const char *token);
-static bool check_exit(FILE **fh, const char *command);
-static bool set_cloexec(int fd);
-static bool username_ok_for_shell(const char *username);
+static bool validate(Port *port, const char *auth);
#define KVSEP 0x01
#define AUTH_KEY "auth"
@@ -100,6 +104,8 @@ oauth_init(Port *port, const char *selected_mech, const char *shadow_pass)
ctx->issuer = port->hba->oauth_issuer;
ctx->scope = port->hba->oauth_scope;
+ load_validator_library();
+
return ctx;
}
@@ -250,7 +256,7 @@ oauth_exchange(void *opaq, const char *input, int inputlen,
errmsg("malformed OAUTHBEARER message"),
errdetail("Message contains additional data after the final terminator.")));
- if (!validate(ctx->port, auth, logdetail))
+ if (!validate(ctx->port, auth))
{
generate_error_response(ctx, output, outputlen);
@@ -489,70 +495,73 @@ generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
*outputlen = buf.len;
}
-static bool
-validate(Port *port, const char *auth, const char **logdetail)
+/*-----
+ * Validates the provided Authorization header and returns the token from
+ * within it. NULL is returned on validation failure.
+ *
+ * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
+ * 2.1:
+ *
+ * b64token = 1*( ALPHA / DIGIT /
+ * "-" / "." / "_" / "~" / "+" / "/" ) *"="
+ * credentials = "Bearer" 1*SP b64token
+ *
+ * The "credentials" construction is what we receive in our auth value.
+ *
+ * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
+ * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
+ * compared case-insensitively. (This is not mentioned in RFC 6750, but
+ * it's pointed out in RFC 7628 Sec. 4.)
+ *
+ * Invalid formats are technically a protocol violation, but we shouldn't
+ * reflect any information about the sensitive Bearer token back to the
+ * client; log at COMMERROR instead.
+ *
+ * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
+ */
+static const char *
+validate_token_format(const char *header)
{
- static const char *const b64_set =
+ size_t span;
+ const char *token;
+ static const char *const b64token_allowed_set =
"abcdefghijklmnopqrstuvwxyz"
"ABCDEFGHIJKLMNOPQRSTUVWXYZ"
"0123456789-._~+/";
- const char *token;
- size_t span;
- int ret;
+ /* If the token is empty or simply too short to be correct */
+ if (!header || strlen(header) <= 7)
+ {
+ ereport(COMMERROR,
+ (errmsg("malformed OAuth bearer token 1")));
+ return NULL;
+ }
- /* TODO: handle logdetail when the test framework can check it */
-
- /*-----
- * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
- * 2.1:
- *
- * b64token = 1*( ALPHA / DIGIT /
- * "-" / "." / "_" / "~" / "+" / "/" ) *"="
- * credentials = "Bearer" 1*SP b64token
- *
- * The "credentials" construction is what we receive in our auth value.
- *
- * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
- * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
- * compared case-insensitively. (This is not mentioned in RFC 6750, but
- * it's pointed out in RFC 7628 Sec. 4.)
- *
- * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
- */
- if (pg_strncasecmp(auth, BEARER_SCHEME, strlen(BEARER_SCHEME)))
- return false;
+ if (pg_strncasecmp(header, BEARER_SCHEME, strlen(BEARER_SCHEME)))
+ return NULL;
/* Pull the bearer token out of the auth value. */
- token = auth + strlen(BEARER_SCHEME);
+ token = header + strlen(BEARER_SCHEME);
/* Swallow any additional spaces. */
while (*token == ' ')
token++;
- /*
- * Before invoking the validator command, sanity-check the token format to
- * avoid any injection attacks later in the chain. Invalid formats are
- * technically a protocol violation, but don't reflect any information
- * about the sensitive Bearer token back to the client; log at COMMERROR
- * instead.
- */
-
/* Tokens must not be empty. */
if (!*token)
{
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("malformed OAUTHBEARER message"),
+ errmsg("malformed OAuth bearer token 2"),
errdetail("Bearer token is empty.")));
- return false;
+ return NULL;
}
/*
* Make sure the token contains only allowed characters. Tokens may end
* with any number of '=' characters.
*/
- span = strspn(token, b64_set);
+ span = strspn(token, b64token_allowed_set);
while (token[span] == '=')
span++;
@@ -565,15 +574,35 @@ validate(Port *port, const char *auth, const char **logdetail)
*/
ereport(COMMERROR,
(errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("malformed OAUTHBEARER message"),
+ errmsg("malformed OAuth bearer token 3"),
errdetail("Bearer token is not in the correct format.")));
- return false;
+ return NULL;
}
- /* Have the validator check the token. */
- if (!run_validator_command(port, token))
+ return token;
+}
+
+static bool
+validate(Port *port, const char *auth)
+{
+ int map_status;
+ ValidatorModuleResult *ret;
+ const char *token;
+
+ /* Ensure that we have a correct token to validate */
+ if (!(token = validate_token_format(auth)))
+ return false;
+
+ /* Call the validation function from the validator module */
+ ret = ValidatorCallbacks->validate_cb(validator_module_state,
+ token, port->user_name);
+
+ if (!ret->authorized)
return false;
+ if (ret->authn_id)
+ set_authn_id(port, ret->authn_id);
+
if (port->hba->oauth_skip_usermap)
{
/*
@@ -586,7 +615,7 @@ validate(Port *port, const char *auth, const char **logdetail)
}
/* Make sure the validator authenticated the user. */
- if (!MyClientConnectionInfo.authn_id)
+ if (ret->authn_id == NULL || ret->authn_id[0] == '\0')
{
/* TODO: use logdetail; reduce message duplication */
ereport(LOG,
@@ -596,288 +625,42 @@ validate(Port *port, const char *auth, const char **logdetail)
}
/* Finally, check the user map. */
- ret = check_usermap(port->hba->usermap, port->user_name,
- MyClientConnectionInfo.authn_id, false);
- return (ret == STATUS_OK);
-}
-
-static bool
-run_validator_command(Port *port, const char *token)
-{
- bool success = false;
- int rc;
- int pipefd[2];
- int rfd = -1;
- int wfd = -1;
-
- StringInfoData command = {0};
- char *p;
- FILE *fh = NULL;
-
- ssize_t written;
- char *line = NULL;
- size_t size = 0;
- ssize_t len;
-
- Assert(oauth_validator_command);
-
- if (!oauth_validator_command[0])
- {
- ereport(COMMERROR,
- (errmsg("oauth_validator_command is not set"),
- errhint("To allow OAuth authenticated connections, set "
- "oauth_validator_command in postgresql.conf.")));
- return false;
- }
-
- /*------
- * Since popen() is unidirectional, open up a pipe for the other
- * direction. Use CLOEXEC to ensure that our write end doesn't
- * accidentally get copied into child processes, which would prevent us
- * from closing it cleanly.
- *
- * XXX this is ugly. We should just read from the child process's stdout,
- * but that's a lot more code.
- * XXX by bypassing the popen API, we open the potential of process
- * deadlock. Clearly document child process requirements (i.e. the child
- * MUST read all data off of the pipe before writing anything).
- * TODO: port to Windows using _pipe().
- */
- rc = pipe(pipefd);
- if (rc < 0)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not create child pipe: %m")));
- return false;
- }
-
- rfd = pipefd[0];
- wfd = pipefd[1];
-
- if (!set_cloexec(wfd))
- {
- /* error message was already logged */
- goto cleanup;
- }
-
- /*----------
- * Construct the command, substituting any recognized %-specifiers:
- *
- * %f: the file descriptor of the input pipe
- * %r: the role that the client wants to assume (port->user_name)
- * %%: a literal '%'
- */
- initStringInfo(&command);
-
- for (p = oauth_validator_command; *p; p++)
- {
- if (p[0] == '%')
- {
- switch (p[1])
- {
- case 'f':
- appendStringInfo(&command, "%d", rfd);
- p++;
- break;
- case 'r':
-
- /*
- * TODO: decide how this string should be escaped. The
- * role is controlled by the client, so if we don't escape
- * it, command injections are inevitable.
- *
- * This is probably an indication that the role name needs
- * to be communicated to the validator process in some
- * other way. For this proof of concept, just be
- * incredibly strict about the characters that are allowed
- * in user names.
- */
- if (!username_ok_for_shell(port->user_name))
- goto cleanup;
-
- appendStringInfoString(&command, port->user_name);
- p++;
- break;
- case '%':
- appendStringInfoChar(&command, '%');
- p++;
- break;
- default:
- appendStringInfoChar(&command, p[0]);
- }
- }
- else
- appendStringInfoChar(&command, p[0]);
- }
-
- /* Execute the command. */
- fh = OpenPipeStream(command.data, "r");
- if (!fh)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("opening pipe to OAuth validator: %m")));
- goto cleanup;
- }
-
- /* We don't need the read end of the pipe anymore. */
- close(rfd);
- rfd = -1;
-
- /* Give the command the token to validate. */
- written = write(wfd, token, strlen(token));
- if (written != strlen(token))
- {
- /* TODO must loop for short writes, EINTR et al */
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not write token to child pipe: %m")));
- goto cleanup;
- }
-
- close(wfd);
- wfd = -1;
-
- /*-----
- * Read the command's response.
- *
- * TODO: getline() is probably too new to use, unfortunately.
- * TODO: loop over all lines
- */
- if ((len = getline(&line, &size, fh)) >= 0)
- {
- /* TODO: fail if the authn_id doesn't end with a newline */
- if (len > 0)
- line[len - 1] = '\0';
-
- set_authn_id(port, line);
- }
- else if (ferror(fh))
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not read from command \"%s\": %m",
- command.data)));
- goto cleanup;
- }
-
- /* Make sure the command exits cleanly. */
- if (!check_exit(&fh, command.data))
- {
- /* error message already logged */
- goto cleanup;
- }
-
- /* Done. */
- success = true;
-
-cleanup:
- if (line)
- free(line);
-
- /*
- * In the successful case, the pipe fds are already closed. For the error
- * case, always close out the pipe before waiting for the command, to
- * prevent deadlock.
- */
- if (rfd >= 0)
- close(rfd);
- if (wfd >= 0)
- close(wfd);
-
- if (fh)
- {
- Assert(!success);
- check_exit(&fh, command.data);
- }
-
- if (command.data)
- pfree(command.data);
-
- return success;
+ map_status = check_usermap(port->hba->usermap, port->user_name,
+ MyClientConnectionInfo.authn_id, false);
+ return (map_status == STATUS_OK);
}
-static bool
-check_exit(FILE **fh, const char *command)
+static void
+load_validator_library(void)
{
- int rc;
-
- rc = ClosePipeStream(*fh);
- *fh = NULL;
-
- if (rc == -1)
- {
- /* pclose() itself failed. */
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not close pipe to command \"%s\": %m",
- command)));
- }
- else if (rc != 0)
- {
- char *reason = wait_result_to_str(rc);
+ OAuthValidatorModuleInit validator_init;
- ereport(COMMERROR,
- (errmsg("failed to execute command \"%s\": %s",
- command, reason)));
+ if (OAuthValidatorLibrary[0] == '\0')
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("oauth_validator_library is not set")));
- pfree(reason);
- }
+ validator_init = (OAuthValidatorModuleInit)
+ load_external_function(OAuthValidatorLibrary,
+ "_PG_oauth_validator_module_init", false, NULL);
- return (rc == 0);
-}
+ if (validator_init == NULL)
+ ereport(ERROR,
+ (errmsg("%s module \"%s\" have to define the symbol %s",
+ "OAuth validator", OAuthValidatorLibrary, "_PG_oauth_validator_module_init")));
-static bool
-set_cloexec(int fd)
-{
- int flags;
- int rc;
+ ValidatorCallbacks = (*validator_init) ();
- flags = fcntl(fd, F_GETFD);
- if (flags == -1)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not get fd flags for child pipe: %m")));
- return false;
- }
+ validator_module_state = (ValidatorModuleState *) palloc0(sizeof(ValidatorModuleState));
+ if (ValidatorCallbacks->startup_cb != NULL)
+ ValidatorCallbacks->startup_cb(validator_module_state);
- rc = fcntl(fd, F_SETFD, flags | FD_CLOEXEC);
- if (rc < 0)
- {
- ereport(COMMERROR,
- (errcode_for_file_access(),
- errmsg("could not set FD_CLOEXEC for child pipe: %m")));
- return false;
- }
-
- return true;
+ before_shmem_exit(shutdown_validator_library, 0);
}
-/*
- * XXX This should go away eventually and be replaced with either a proper
- * escape or a different strategy for communication with the validator command.
- */
-static bool
-username_ok_for_shell(const char *username)
+static void
+shutdown_validator_library(int code, Datum arg)
{
- /* This set is borrowed from fe_utils' appendShellStringNoError(). */
- static const char *const allowed =
- "abcdefghijklmnopqrstuvwxyz"
- "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
- "0123456789-_./:";
- size_t span;
-
- Assert(username && username[0]); /* should have already been checked */
-
- span = strspn(username, allowed);
- if (username[span] != '\0')
- {
- ereport(COMMERROR,
- (errmsg("PostgreSQL user name contains unsafe characters and cannot be passed to the OAuth validator")));
- return false;
- }
-
- return true;
+ if (ValidatorCallbacks->shutdown_cb != NULL)
+ ValidatorCallbacks->shutdown_cb(validator_module_state);
}
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 773a75c96f..62d4f15d2b 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -4686,12 +4686,12 @@ struct config_string ConfigureNamesString[] =
},
{
- {"oauth_validator_command", PGC_SIGHUP, CONN_AUTH_AUTH,
- gettext_noop("Command to validate OAuth v2 bearer tokens."),
+ {"oauth_validator_library", PGC_SIGHUP, CONN_AUTH_AUTH,
+ gettext_noop("Sets the library that will be called to validate OAuth v2 bearer tokens."),
NULL,
GUC_SUPERUSER_ONLY | GUC_NOT_IN_SAMPLE
},
- &oauth_validator_command,
+ &OAuthValidatorLibrary,
"",
NULL, NULL, NULL
},
diff --git a/src/common/Makefile b/src/common/Makefile
index c4f4448e2f..98b8dac0b4 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -41,7 +41,7 @@ override CPPFLAGS += -DVAL_LDFLAGS_SL="\"$(LDFLAGS_SL)\""
override CPPFLAGS += -DVAL_LIBS="\"$(LIBS)\""
override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common -I$(libpq_srcdir) $(CPPFLAGS)
-LIBS += $(PTHREAD_LIBS)
+LIBS += $(PTHREAD_LIBS) $(libpq_pgport)
OBJS_COMMON = \
archive.o \
diff --git a/src/include/libpq/oauth.h b/src/include/libpq/oauth.h
index 5edab3b25a..6f98e84cc9 100644
--- a/src/include/libpq/oauth.h
+++ b/src/include/libpq/oauth.h
@@ -3,7 +3,7 @@
* oauth.h
* Interface to libpq/auth-oauth.c
*
- * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* src/include/libpq/oauth.h
@@ -16,7 +16,32 @@
#include "libpq/libpq-be.h"
#include "libpq/sasl.h"
-extern char *oauth_validator_command;
+extern PGDLLIMPORT char *OAuthValidatorLibrary;
+
+typedef struct ValidatorModuleState
+{
+ void *private_data;
+} ValidatorModuleState;
+
+typedef struct ValidatorModuleResult
+{
+ bool authorized;
+ char *authn_id;
+} ValidatorModuleResult;
+
+typedef void (*ValidatorStartupCB) (ValidatorModuleState *state);
+typedef void (*ValidatorShutdownCB) (ValidatorModuleState *state);
+typedef ValidatorModuleResult *(*ValidatorValidateCB) (ValidatorModuleState *state, const char *token, const char *role);
+
+typedef struct OAuthValidatorCallbacks
+{
+ ValidatorStartupCB startup_cb;
+ ValidatorShutdownCB shutdown_cb;
+ ValidatorValidateCB validate_cb;
+} OAuthValidatorCallbacks;
+
+typedef const OAuthValidatorCallbacks *(*OAuthValidatorModuleInit) (void);
+extern PGDLLEXPORT const OAuthValidatorCallbacks *_PG_oauth_validator_module_init(void);
/* Implementation */
extern const pg_be_sasl_mech pg_be_oauth_mech;
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index 7c11fb97f2..a08105e0c2 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -9,6 +9,7 @@ subdir('gin')
subdir('injection_points')
subdir('ldap_password_func')
subdir('libpq_pipeline')
+subdir('oauth_validator')
subdir('plsample')
subdir('spgist_name_ops')
subdir('ssl_passphrase_callback')
diff --git a/src/test/modules/oauth_validator/.gitignore b/src/test/modules/oauth_validator/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/src/test/modules/oauth_validator/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/oauth_validator/Makefile b/src/test/modules/oauth_validator/Makefile
new file mode 100644
index 0000000000..1f874cd7f2
--- /dev/null
+++ b/src/test/modules/oauth_validator/Makefile
@@ -0,0 +1,19 @@
+MODULES = validator
+PGFILEDESC = "validator - test OAuth validator module"
+
+NO_INSTALLCHECK = 1
+
+TAP_TESTS = 1
+
+REGRESS = validator
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/oauth_validator
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/oauth_validator/expected/validator.out b/src/test/modules/oauth_validator/expected/validator.out
new file mode 100644
index 0000000000..360caa2cb3
--- /dev/null
+++ b/src/test/modules/oauth_validator/expected/validator.out
@@ -0,0 +1,6 @@
+SELECT 1;
+ ?column?
+----------
+ 1
+(1 row)
+
diff --git a/src/test/modules/oauth_validator/meson.build b/src/test/modules/oauth_validator/meson.build
new file mode 100644
index 0000000000..d9c1d1d577
--- /dev/null
+++ b/src/test/modules/oauth_validator/meson.build
@@ -0,0 +1,33 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+validator_sources = files(
+ 'validator.c',
+)
+
+if host_system == 'windows'
+ validator_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'validator',
+ '--FILEDESC', 'validator - test OAuth validator module',])
+endif
+
+validator = shared_module('validator',
+ validator_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += validator
+
+tests += {
+ 'name': 'oauth_validator',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'regress': {
+ 'sql': [
+ 'validator',
+ ],
+ },
+ 'tap': {
+ 'tests': [
+ 't/001_server.pl',
+ ],
+ },
+}
diff --git a/src/test/modules/oauth_validator/sql/validator.sql b/src/test/modules/oauth_validator/sql/validator.sql
new file mode 100644
index 0000000000..e0ac49d1ec
--- /dev/null
+++ b/src/test/modules/oauth_validator/sql/validator.sql
@@ -0,0 +1 @@
+SELECT 1;
diff --git a/src/test/modules/oauth_validator/t/001_server.pl b/src/test/modules/oauth_validator/t/001_server.pl
new file mode 100644
index 0000000000..14c7778298
--- /dev/null
+++ b/src/test/modules/oauth_validator/t/001_server.pl
@@ -0,0 +1,78 @@
+
+# Copyright (c) 2021-2024, PostgreSQL Global Development Group
+
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::OAuthServer;
+use Test::More;
+
+my $node = PostgreSQL::Test::Cluster->new('primary');
+$node->init;
+$node->append_conf('postgresql.conf', "log_connections = on\n");
+$node->append_conf('postgresql.conf', "shared_preload_libraries = 'validator'\n");
+$node->append_conf('postgresql.conf', "oauth_validator_library = 'validator'\n");
+$node->start;
+
+$node->safe_psql('postgres', 'CREATE USER test;');
+$node->safe_psql('postgres', 'CREATE USER testalt;');
+
+my $issuer = "127.0.0.1:18080";
+
+unlink($node->data_dir . '/pg_hba.conf');
+$node->append_conf('pg_hba.conf', qq{
+local all test oauth issuer="$issuer" scope="openid postgres"
+local all testalt oauth issuer="$issuer/alternate" scope="openid postgres alt"
+});
+$node->reload;
+
+my $webserver = PostgreSQL::Test::OAuthServer->new(18080);
+
+my $port = $webserver->port();
+
+is($port, 18080, "Port is 18080");
+
+$webserver->setup();
+$webserver->run();
+
+my ($log_start, $log_end);
+$log_start = $node->wait_for_log(qr/reloading configuration files/);
+
+my $user = "test";
+$node->connect_ok("user=$user dbname=postgres oauth_client_id=f02c6361-0635", "connect",
+ expected_stderr => qr@Visit https://example\.com/ and enter the code: postgresuser@);
+
+$log_end = $node->wait_for_log(qr/connection authorized/, $log_start);
+$node->log_check("user $user: validator receives correct parameters", $log_start,
+ log_like => [
+ qr/oauth_validator: token="9243959234", role="$user"/,
+ qr/oauth_validator: issuer="\Q$issuer\E", scope="openid postgres"/,
+ ]);
+$node->log_check("user $user: validator sets authenticated identity", $log_start,
+ log_like => [
+ qr/connection authenticated: identity="test" method=oauth/,
+ ]);
+$log_start = $log_end;
+
+# The /alternate issuer uses slightly different parameters.
+$user = "testalt";
+$node->connect_ok("user=$user dbname=postgres oauth_client_id=f02c6361-0636", "connect",
+ expected_stderr => qr@Visit https://example\.org/ and enter the code: postgresuser@);
+
+$log_end = $node->wait_for_log(qr/connection authorized/, $log_start);
+$node->log_check("user $user: validator receives correct parameters", $log_start,
+ log_like => [
+ qr/oauth_validator: token="9243959234-alt", role="$user"/,
+ qr|oauth_validator: issuer="\Q$issuer/alternate\E", scope="openid postgres alt"|,
+ ]);
+$node->log_check("user $user: validator sets authenticated identity", $log_start,
+ log_like => [
+ qr/connection authenticated: identity="testalt" method=oauth/,
+ ]);
+$log_start = $log_end;
+
+$node->stop;
+
+done_testing();
diff --git a/src/test/modules/oauth_validator/validator.c b/src/test/modules/oauth_validator/validator.c
new file mode 100644
index 0000000000..09a4bf61d2
--- /dev/null
+++ b/src/test/modules/oauth_validator/validator.c
@@ -0,0 +1,82 @@
+/*-------------------------------------------------------------------------
+ *
+ * validator.c
+ * Test module for serverside OAuth token validation callbacks
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/test/modules/oauth_validator/validator.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "libpq/oauth.h"
+#include "miscadmin.h"
+#include "utils/memutils.h"
+
+PG_MODULE_MAGIC;
+
+static void validator_startup(ValidatorModuleState *state);
+static void validator_shutdown(ValidatorModuleState *state);
+static ValidatorModuleResult * validate_token(ValidatorModuleState *state,
+ const char *token,
+ const char *role);
+
+static const OAuthValidatorCallbacks validator_callbacks = {
+ .startup_cb = validator_startup,
+ .shutdown_cb = validator_shutdown,
+ .validate_cb = validate_token
+};
+
+void
+_PG_init(void)
+{
+ /* no-op */
+}
+
+const OAuthValidatorCallbacks *
+_PG_oauth_validator_module_init(void)
+{
+ return &validator_callbacks;
+}
+
+#define PRIVATE_COOKIE ((void *) 13579)
+
+static void
+validator_startup(ValidatorModuleState *state)
+{
+ state->private_data = PRIVATE_COOKIE;
+}
+
+static void
+validator_shutdown(ValidatorModuleState *state)
+{
+ /* do nothing */
+}
+
+static ValidatorModuleResult *
+validate_token(ValidatorModuleState *state, const char *token, const char *role)
+{
+ ValidatorModuleResult *res;
+
+ /* Check to make sure our private state still exists. */
+ if (state->private_data != PRIVATE_COOKIE)
+ elog(ERROR, "oauth_validator: private state cookie changed to %p",
+ state->private_data);
+
+ res = palloc(sizeof(ValidatorModuleResult));
+
+ elog(LOG, "oauth_validator: token=\"%s\", role=\"%s\"", token, role);
+ elog(LOG, "oauth_validator: issuer=\"%s\", scope=\"%s\"",
+ MyProcPort->hba->oauth_issuer,
+ MyProcPort->hba->oauth_scope);
+
+ res->authorized = true;
+ res->authn_id = pstrdup(role);
+
+ return res;
+}
diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm
index b08296605c..4b7c93f19d 100644
--- a/src/test/perl/PostgreSQL/Test/Cluster.pm
+++ b/src/test/perl/PostgreSQL/Test/Cluster.pm
@@ -2299,6 +2299,11 @@ instead of the default.
If this regular expression is set, matches it with the output generated.
+=item expected_stderr => B<value>
+
+If this regular expression is set, matches it against the standard error
+stream; otherwise the stderr must be empty.
+
=item log_like => [ qr/required message/ ]
=item log_unlike => [ qr/prohibited message/ ]
@@ -2342,7 +2347,14 @@ sub connect_ok
like($stdout, $params{expected_stdout}, "$test_name: stdout matches");
}
- is($stderr, "", "$test_name: no stderr");
+ if (defined($params{expected_stderr}))
+ {
+ like($stderr, $params{expected_stderr}, "$test_name: stderr matches");
+ }
+ else
+ {
+ is($stderr, "", "$test_name: no stderr");
+ }
$self->log_check($test_name, $log_location, %params);
}
diff --git a/src/test/perl/PostgreSQL/Test/OAuthServer.pm b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
new file mode 100644
index 0000000000..5c195efb79
--- /dev/null
+++ b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
@@ -0,0 +1,183 @@
+#!/usr/bin/perl
+
+package PostgreSQL::Test::OAuthServer;
+
+use warnings;
+use strict;
+use threads;
+use Socket;
+use IO::Select;
+
+local *server_socket;
+
+sub new
+{
+ my $class = shift;
+ my $port = shift;
+
+ my $self = {};
+ bless($self, $class);
+
+ $self->{'port'} = $port;
+
+ return $self;
+}
+
+sub setup
+{
+ my $self = shift;
+ my $tcp = getprotobyname('tcp');
+
+ socket($self->{'socket'}, PF_INET, SOCK_STREAM, $tcp)
+ or die "no socket";
+ setsockopt($self->{'socket'}, SOL_SOCKET, SO_REUSEADDR, pack("l", 1));
+ bind($self->{'socket'}, sockaddr_in($self->{'port'}, INADDR_ANY));
+}
+
+sub port
+{
+ my $self = shift;
+
+ return $self->{'port'};
+}
+
+sub run
+{
+ my $self = shift;
+
+ my $server_thread = threads->create(\&_listen, $self);
+ $server_thread->detach();
+}
+
+sub _listen
+{
+ my $self = shift;
+
+ listen($self->{'socket'}, SOMAXCONN) or die "fail to listen: $!";
+
+ while (1)
+ {
+ my $fh;
+ my %request;
+ my $remote = accept($fh, $self->{'socket'});
+ binmode $fh;
+
+ my ($method, $object, $prot) = split(/ /, <$fh>);
+ $request{'method'} = $method;
+ $request{'object'} = $object;
+ chomp($request{'object'});
+
+ local $/ = Socket::CRLF;
+ my $c = 0;
+ while(<$fh>)
+ {
+ chomp;
+ # Headers
+ if (/:/)
+ {
+ my ($field, $value) = split(/:/, $_, 2);
+ $value =~ s/^\s+//;
+ $request{'headers'}{lc $field} = $value;
+ }
+ # POST data
+ elsif (/^$/)
+ {
+ read($fh, $request{'content'}, $request{'headers'}{'content-length'})
+ if defined $request{'headers'}{'content-length'};
+ last;
+ }
+ }
+
+ # Debug printing
+ # print ": read ".$request{'method'} . ";" . $request{'object'}.";\n";
+ # foreach my $h (keys(%{$request{'headers'}}))
+ #{
+ # printf ": headers: " . $request{'headers'}{$h} . "\n";
+ #}
+ #printf ": POST: " . $request{'content'} . "\n" if defined $request{'content'};
+
+ my $alternate = 0;
+ if ($request{'object'} =~ qr|^/alternate(/.*)$|)
+ {
+ $alternate = 1;
+ $request{'object'} = $1;
+ }
+
+ if ($request{'object'} eq '/.well-known/openid-configuration')
+ {
+ my $issuer = "http://localhost:$self->{'port'}";
+ if ($alternate)
+ {
+ $issuer .= "/alternate";
+ }
+
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: application/json\r\n";
+ print $fh "\r\n";
+ print $fh <<EOR;
+ {
+ "issuer": "$issuer",
+ "token_endpoint": "$issuer/token",
+ "device_authorization_endpoint": "$issuer/authorize",
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ "grant_types_supported": ["urn:ietf:params:oauth:grant-type:device_code"]
+ }
+EOR
+ }
+ elsif ($request{'object'} eq '/authorize')
+ {
+ my $uri = "https://example.com/";
+ if ($alternate)
+ {
+ $uri = "https://example.org/";
+ }
+
+ print ": returning device_code\n";
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: application/json\r\n";
+ print $fh "\r\n";
+ print $fh <<EOR;
+ {
+ "device_code": "postgres",
+ "user_code" : "postgresuser",
+ "interval" : 0,
+ "verification_uri" : "$uri",
+ "expires-in": 5
+ }
+EOR
+ }
+ elsif ($request{'object'} eq '/token')
+ {
+ my $token = "9243959234";
+ if ($alternate)
+ {
+ $token .= "-alt";
+ }
+
+ print ": returning token\n";
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: application/json\r\n";
+ print $fh "\r\n";
+ print $fh <<EOR;
+ {
+ "access_token": "$token",
+ "token_type": "bearer"
+ }
+EOR
+ }
+ else
+ {
+ print ": returning default\n";
+ print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
+ print $fh "Content-Type: text/html\r\n";
+ print $fh "\r\n";
+ print $fh "Ok\n";
+ }
+
+ close($fh);
+ }
+}
+
+1;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 86b1cc6e5c..0eaa079261 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1673,6 +1673,7 @@ NumericSortSupport
NumericSumAccum
NumericVar
OAuthStep
+OAuthValidatorCallbacks
OM_uint32
OP
OSAPerGroupState
@@ -3000,6 +3001,7 @@ VacuumRelation
VacuumStmt
ValidIOData
ValidateIndexState
+ValidatorModuleState
ValuesScan
ValuesScanState
Var
--
2.34.1
v21-0001-common-jsonapi-support-libpq-as-a-client.patch (application/octet-stream, 16.6K) download
From 557370eabb9e7c8004267dd617ac8936f499b2cf Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Mon, 3 May 2021 15:38:26 -0700
Subject: [PATCH v21 1/7] common/jsonapi: support libpq as a client
Based on a patch by Michael Paquier.
For frontend code, use PQExpBuffer instead of StringInfo. This requires
us to track allocation failures so that we can return JSON_OUT_OF_MEMORY
as needed rather than exit()ing.
---
src/bin/pg_combinebackup/Makefile | 4 +-
src/bin/pg_combinebackup/meson.build | 2 +-
src/bin/pg_verifybackup/Makefile | 2 +-
src/common/Makefile | 2 +-
src/common/jsonapi.c | 173 ++++++++++++++++++++-------
src/common/meson.build | 8 +-
src/include/common/jsonapi.h | 19 ++-
7 files changed, 158 insertions(+), 52 deletions(-)
diff --git a/src/bin/pg_combinebackup/Makefile b/src/bin/pg_combinebackup/Makefile
index c3729755ba..2f7dc1ed87 100644
--- a/src/bin/pg_combinebackup/Makefile
+++ b/src/bin/pg_combinebackup/Makefile
@@ -18,6 +18,8 @@ include $(top_builddir)/src/Makefile.global
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils
+# TODO: fix this properly
+LDFLAGS_INTERNAL += -lpgcommon $(libpq_pgport)
OBJS = \
$(WIN32RES) \
@@ -30,7 +32,7 @@ OBJS = \
all: pg_combinebackup
-pg_combinebackup: $(OBJS) | submake-libpgport submake-libpgfeutils
+pg_combinebackup: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
$(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
install: all installdirs
diff --git a/src/bin/pg_combinebackup/meson.build b/src/bin/pg_combinebackup/meson.build
index 1d4b9c218f..cab677b574 100644
--- a/src/bin/pg_combinebackup/meson.build
+++ b/src/bin/pg_combinebackup/meson.build
@@ -17,7 +17,7 @@ endif
pg_combinebackup = executable('pg_combinebackup',
pg_combinebackup_sources,
- dependencies: [frontend_code],
+ dependencies: [frontend_code, libpq],
kwargs: default_bin_args,
)
bin_targets += pg_combinebackup
diff --git a/src/bin/pg_verifybackup/Makefile b/src/bin/pg_verifybackup/Makefile
index 7c045f142e..3372fada01 100644
--- a/src/bin/pg_verifybackup/Makefile
+++ b/src/bin/pg_verifybackup/Makefile
@@ -17,7 +17,7 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
# We need libpq only because fe_utils does.
-LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils -lpgcommon $(libpq_pgport)
OBJS = \
$(WIN32RES) \
diff --git a/src/common/Makefile b/src/common/Makefile
index 3d83299432..c4f4448e2f 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -40,7 +40,7 @@ override CPPFLAGS += -DVAL_LDFLAGS_EX="\"$(LDFLAGS_EX)\""
override CPPFLAGS += -DVAL_LDFLAGS_SL="\"$(LDFLAGS_SL)\""
override CPPFLAGS += -DVAL_LIBS="\"$(LIBS)\""
-override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common $(CPPFLAGS)
+override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common -I$(libpq_srcdir) $(CPPFLAGS)
LIBS += $(PTHREAD_LIBS)
OBJS_COMMON = \
diff --git a/src/common/jsonapi.c b/src/common/jsonapi.c
index 98d6e66a21..4aeedc0bc5 100644
--- a/src/common/jsonapi.c
+++ b/src/common/jsonapi.c
@@ -21,10 +21,43 @@
#include "mb/pg_wchar.h"
#include "port/pg_lfind.h"
-#ifndef FRONTEND
+#ifdef FRONTEND
+#include "pqexpbuffer.h"
+#else
+#include "lib/stringinfo.h"
#include "miscadmin.h"
#endif
+/*
+ * In backend, we will use palloc/pfree along with StringInfo. In frontend, use
+ * malloc and PQExpBuffer, and return JSON_OUT_OF_MEMORY on out-of-memory.
+ */
+#ifdef FRONTEND
+
+#define STRDUP(s) strdup(s)
+#define ALLOC(size) malloc(size)
+
+#define appendStrVal appendPQExpBuffer
+#define appendBinaryStrVal appendBinaryPQExpBuffer
+#define appendStrValChar appendPQExpBufferChar
+#define createStrVal createPQExpBuffer
+#define resetStrVal resetPQExpBuffer
+#define destroyStrVal destroyPQExpBuffer
+
+#else /* !FRONTEND */
+
+#define STRDUP(s) pstrdup(s)
+#define ALLOC(size) palloc(size)
+
+#define appendStrVal appendStringInfo
+#define appendBinaryStrVal appendBinaryStringInfo
+#define appendStrValChar appendStringInfoChar
+#define createStrVal makeStringInfo
+#define resetStrVal resetStringInfo
+#define destroyStrVal destroyStringInfo
+
+#endif
+
/*
* The context of the parser is maintained by the recursive descent
* mechanism, but is passed explicitly to the error reporting routine
@@ -168,9 +201,16 @@ makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
lex->input_encoding = encoding;
if (need_escapes)
{
- lex->strval = makeStringInfo();
+ /*
+ * This call can fail in FRONTEND code. We defer error handling to
+ * time of use (json_lex_string()) since we might not need to parse
+ * any strings anyway.
+ */
+ lex->strval = createStrVal();
lex->flags |= JSONLEX_FREE_STRVAL;
+ lex->parse_strval = true;
}
+ lex->errormsg = NULL;
return lex;
}
@@ -185,14 +225,18 @@ makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
void
freeJsonLexContext(JsonLexContext *lex)
{
+ static const JsonLexContext empty = {0};
+
if (lex->flags & JSONLEX_FREE_STRVAL)
- destroyStringInfo(lex->strval);
+ destroyStrVal(lex->strval);
if (lex->errormsg)
- destroyStringInfo(lex->errormsg);
+ destroyStrVal(lex->errormsg);
if (lex->flags & JSONLEX_FREE_STRUCT)
pfree(lex);
+ else
+ *lex = empty;
}
/*
@@ -258,7 +302,7 @@ json_count_array_elements(JsonLexContext *lex, int *elements)
* etc, so doing this with a copy makes that safe.
*/
memcpy(©lex, lex, sizeof(JsonLexContext));
- copylex.strval = NULL; /* not interested in values here */
+ copylex.parse_strval = false; /* not interested in values here */
copylex.lex_level++;
count = 0;
@@ -320,14 +364,21 @@ parse_scalar(JsonLexContext *lex, JsonSemAction *sem)
/* extract the de-escaped string value, or the raw lexeme */
if (lex_peek(lex) == JSON_TOKEN_STRING)
{
- if (lex->strval != NULL)
- val = pstrdup(lex->strval->data);
+ if (lex->parse_strval)
+ {
+ val = STRDUP(lex->strval->data);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
}
else
{
int len = (lex->token_terminator - lex->token_start);
- val = palloc(len + 1);
+ val = ALLOC(len + 1);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+
memcpy(val, lex->token_start, len);
val[len] = '\0';
}
@@ -361,8 +412,12 @@ parse_object_field(JsonLexContext *lex, JsonSemAction *sem)
if (lex_peek(lex) != JSON_TOKEN_STRING)
return report_parse_error(JSON_PARSE_STRING, lex);
- if ((ostart != NULL || oend != NULL) && lex->strval != NULL)
- fname = pstrdup(lex->strval->data);
+ if ((ostart != NULL || oend != NULL) && lex->parse_strval)
+ {
+ fname = STRDUP(lex->strval->data);
+ if (fname == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
result = json_lex(lex);
if (result != JSON_SUCCESS)
return result;
@@ -418,6 +473,11 @@ parse_object(JsonLexContext *lex, JsonSemAction *sem)
JsonParseErrorType result;
#ifndef FRONTEND
+
+ /*
+ * TODO: clients need some way to put a bound on stack growth. Parse level
+ * limits maybe?
+ */
check_stack_depth();
#endif
@@ -766,8 +826,15 @@ json_lex_string(JsonLexContext *lex)
return code; \
} while (0)
- if (lex->strval != NULL)
- resetStringInfo(lex->strval);
+ if (lex->parse_strval)
+ {
+#ifdef FRONTEND
+ /* make sure initialization succeeded */
+ if (lex->strval == NULL)
+ return JSON_OUT_OF_MEMORY;
+#endif
+ resetStrVal(lex->strval);
+ }
Assert(lex->input_length > 0);
s = lex->token_start;
@@ -804,7 +871,7 @@ json_lex_string(JsonLexContext *lex)
else
FAIL_AT_CHAR_END(JSON_UNICODE_ESCAPE_FORMAT);
}
- if (lex->strval != NULL)
+ if (lex->parse_strval)
{
/*
* Combine surrogate pairs.
@@ -861,19 +928,19 @@ json_lex_string(JsonLexContext *lex)
unicode_to_utf8(ch, (unsigned char *) utf8str);
utf8len = pg_utf_mblen((unsigned char *) utf8str);
- appendBinaryStringInfo(lex->strval, utf8str, utf8len);
+ appendBinaryPQExpBuffer(lex->strval, utf8str, utf8len);
}
else if (ch <= 0x007f)
{
/* The ASCII range is the same in all encodings */
- appendStringInfoChar(lex->strval, (char) ch);
+ appendPQExpBufferChar(lex->strval, (char) ch);
}
else
FAIL_AT_CHAR_END(JSON_UNICODE_HIGH_ESCAPE);
#endif /* FRONTEND */
}
}
- else if (lex->strval != NULL)
+ else if (lex->parse_strval)
{
if (hi_surrogate != -1)
FAIL_AT_CHAR_END(JSON_UNICODE_LOW_SURROGATE);
@@ -883,22 +950,22 @@ json_lex_string(JsonLexContext *lex)
case '"':
case '\\':
case '/':
- appendStringInfoChar(lex->strval, *s);
+ appendStrValChar(lex->strval, *s);
break;
case 'b':
- appendStringInfoChar(lex->strval, '\b');
+ appendStrValChar(lex->strval, '\b');
break;
case 'f':
- appendStringInfoChar(lex->strval, '\f');
+ appendStrValChar(lex->strval, '\f');
break;
case 'n':
- appendStringInfoChar(lex->strval, '\n');
+ appendStrValChar(lex->strval, '\n');
break;
case 'r':
- appendStringInfoChar(lex->strval, '\r');
+ appendStrValChar(lex->strval, '\r');
break;
case 't':
- appendStringInfoChar(lex->strval, '\t');
+ appendStrValChar(lex->strval, '\t');
break;
default:
@@ -933,7 +1000,7 @@ json_lex_string(JsonLexContext *lex)
/*
* Skip to the first byte that requires special handling, so we
- * can batch calls to appendBinaryStringInfo.
+ * can batch calls to appendBinaryStrVal.
*/
while (p < end - sizeof(Vector8) &&
!pg_lfind8('\\', (uint8 *) p, sizeof(Vector8)) &&
@@ -957,8 +1024,8 @@ json_lex_string(JsonLexContext *lex)
}
}
- if (lex->strval != NULL)
- appendBinaryStringInfo(lex->strval, s, p - s);
+ if (lex->parse_strval)
+ appendBinaryStrVal(lex->strval, s, p - s);
/*
* s will be incremented at the top of the loop, so set it to just
@@ -974,6 +1041,11 @@ json_lex_string(JsonLexContext *lex)
return JSON_UNICODE_LOW_SURROGATE;
}
+#ifdef FRONTEND
+ if (lex->parse_strval && PQExpBufferBroken(lex->strval))
+ return JSON_OUT_OF_MEMORY;
+#endif
+
/* Hooray, we found the end of the string! */
lex->prev_token_terminator = lex->token_terminator;
lex->token_terminator = s + 1;
@@ -1158,19 +1230,25 @@ report_parse_error(JsonParseContext ctx, JsonLexContext *lex)
char *
json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
{
+ if (error == JSON_OUT_OF_MEMORY)
+ {
+ /* Short circuit. Allocating anything for this case is unhelpful. */
+ return _("out of memory");
+ }
+
if (lex->errormsg)
- resetStringInfo(lex->errormsg);
+ resetStrVal(lex->errormsg);
else
- lex->errormsg = makeStringInfo();
+ lex->errormsg = createStrVal();
/*
* A helper for error messages that should print the current token. The
* format must contain exactly one %.*s specifier.
*/
#define token_error(lex, format) \
- appendStringInfo((lex)->errormsg, _(format), \
- (int) ((lex)->token_terminator - (lex)->token_start), \
- (lex)->token_start);
+ appendStrVal((lex)->errormsg, _(format), \
+ (int) ((lex)->token_terminator - (lex)->token_start), \
+ (lex)->token_start);
switch (error)
{
@@ -1181,9 +1259,9 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
token_error(lex, "Escape sequence \"\\%.*s\" is invalid.");
break;
case JSON_ESCAPING_REQUIRED:
- appendStringInfo(lex->errormsg,
- _("Character with value 0x%02x must be escaped."),
- (unsigned char) *(lex->token_terminator));
+ appendStrVal(lex->errormsg,
+ _("Character with value 0x%02x must be escaped."),
+ (unsigned char) *(lex->token_terminator));
break;
case JSON_EXPECTED_END:
token_error(lex, "Expected end of input, but found \"%.*s\".");
@@ -1214,6 +1292,9 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
case JSON_INVALID_TOKEN:
token_error(lex, "Token \"%.*s\" is invalid.");
break;
+ case JSON_OUT_OF_MEMORY:
+ /* should have been handled above; use the error path */
+ break;
case JSON_UNICODE_CODE_POINT_ZERO:
return _("\\u0000 cannot be converted to text.");
case JSON_UNICODE_ESCAPE_FORMAT:
@@ -1245,15 +1326,23 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
}
#undef token_error
- /*
- * We don't use a default: case, so that the compiler will warn about
- * unhandled enum values. But this needs to be here anyway to cover the
- * possibility of an incorrect input.
- */
- if (lex->errormsg->len == 0)
- appendStringInfo(lex->errormsg,
- _("unexpected json parse error type: %d"),
- (int) error);
+ /* Note that lex->errormsg can be NULL in FRONTEND code. */
+ if (lex->errormsg && lex->errormsg->len == 0)
+ {
+ /*
+ * We don't use a default: case, so that the compiler will warn about
+ * unhandled enum values. But this needs to be here anyway to cover
+ * the possibility of an incorrect input.
+ */
+ appendStrVal(lex->errormsg,
+ _("unexpected json parse error type: %d"),
+ (int) error);
+ }
+
+#ifdef FRONTEND
+ if (PQExpBufferBroken(lex->errormsg))
+ return _("out of memory while constructing error description");
+#endif
return lex->errormsg->data;
}
diff --git a/src/common/meson.build b/src/common/meson.build
index de68e408fa..379b228d86 100644
--- a/src/common/meson.build
+++ b/src/common/meson.build
@@ -125,13 +125,18 @@ common_sources_frontend_static += files(
# least cryptohash_openssl.c, hmac_openssl.c depend on it.
# controldata_utils.c depends on wait_event_types_h. That's arguably a
# layering violation, but ...
+#
+# XXX Frontend builds need libpq's pqexpbuffer.h, so adjust the include paths
+# appropriately. This seems completely broken.
pgcommon = {}
pgcommon_variants = {
'_srv': internal_lib_args + {
+ 'include_directories': include_directories('.'),
'sources': common_sources + [lwlocknames_h] + [wait_event_types_h],
'dependencies': [backend_common_code],
},
'': default_lib_args + {
+ 'include_directories': include_directories('../interfaces/libpq', '.'),
'sources': common_sources_frontend_static,
'dependencies': [frontend_common_code],
# Files in libpgcommon.a should use/export the "xxx_private" versions
@@ -140,6 +145,7 @@ pgcommon_variants = {
},
'_shlib': default_lib_args + {
'pic': true,
+ 'include_directories': include_directories('../interfaces/libpq', '.'),
'sources': common_sources_frontend_shlib,
'dependencies': [frontend_common_code],
},
@@ -157,7 +163,6 @@ foreach name, opts : pgcommon_variants
c_args = opts.get('c_args', []) + common_cflags[cflagname]
cflag_libs += static_library('libpgcommon@0@_@1@'.format(name, cflagname),
c_pch: pch_c_h,
- include_directories: include_directories('.'),
kwargs: opts + {
'sources': sources,
'c_args': c_args,
@@ -170,7 +175,6 @@ foreach name, opts : pgcommon_variants
lib = static_library('libpgcommon@0@'.format(name),
link_with: cflag_libs,
c_pch: pch_c_h,
- include_directories: include_directories('.'),
kwargs: opts + {
'dependencies': opts['dependencies'] + [ssl],
}
diff --git a/src/include/common/jsonapi.h b/src/include/common/jsonapi.h
index 86a0fc2d00..75d444c17a 100644
--- a/src/include/common/jsonapi.h
+++ b/src/include/common/jsonapi.h
@@ -14,8 +14,6 @@
#ifndef JSONAPI_H
#define JSONAPI_H
-#include "lib/stringinfo.h"
-
typedef enum JsonTokenType
{
JSON_TOKEN_INVALID,
@@ -48,6 +46,7 @@ typedef enum JsonParseErrorType
JSON_EXPECTED_OBJECT_NEXT,
JSON_EXPECTED_STRING,
JSON_INVALID_TOKEN,
+ JSON_OUT_OF_MEMORY,
JSON_UNICODE_CODE_POINT_ZERO,
JSON_UNICODE_ESCAPE_FORMAT,
JSON_UNICODE_HIGH_ESCAPE,
@@ -57,6 +56,17 @@ typedef enum JsonParseErrorType
JSON_SEM_ACTION_FAILED, /* error should already be reported */
} JsonParseErrorType;
+/*
+ * Don't depend on the internal type header for strval; if callers need access
+ * then they can include the appropriate header themselves.
+ */
+#ifdef FRONTEND
+#define StrValType PQExpBufferData
+#else
+#define StrValType StringInfoData
+#endif
+
+typedef struct StrValType StrValType;
/*
* All the fields in this structure should be treated as read-only.
@@ -88,8 +98,9 @@ typedef struct JsonLexContext
bits32 flags;
int line_number; /* line number, starting from 1 */
char *line_start; /* where that line starts within input */
- StringInfo strval;
- StringInfo errormsg;
+ bool parse_strval;
+ StrValType *strval; /* only used if parse_strval == true */
+ StrValType *errormsg;
} JsonLexContext;
typedef JsonParseErrorType (*json_struct_action) (void *state);
--
2.34.1
v21-0005-Add-pytest-suite-for-OAuth.patch (application/octet-stream, 176.9K) download
From 1b871542f59a447cd04d2e60e6a4628315a4c94f Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Fri, 4 Jun 2021 09:06:38 -0700
Subject: [PATCH v21 5/7] Add pytest suite for OAuth
Requires Python 3. On the first run of `make installcheck` or `meson
test` the dependencies will be installed into a local virtualenv for
you. See the README for more details.
Cirrus has been updated to build OAuth support on Debian and FreeBSD.
The suite contains a --temp-instance option, analogous to pg_regress's
option of the same name, which allows an ephemeral server to be spun up
during a test run.
TODOs:
- The --tap-stream option to pytest-tap is slightly broken during test
failures (it suppresses error information), which impedes debugging.
- pyca/cryptography is pinned at an old version. Since we use it for
testing and not security, this isn't a critical problem yet, but it's
not ideal. Newer versions require a Rust compiler to build, and while
many platforms have precompiled wheels, some (FreeBSD) do not. Even
with the Rust pieces bypassed, compilation on FreeBSD takes a while.
- The with_oauth test skip logic should probably be integrated into the
Makefile side as well...
- See if 32-bit tests can be enabled with a 32-bit Python.
---
.cirrus.tasks.yml | 22 +-
meson.build | 107 ++
src/test/meson.build | 1 +
src/test/python/.gitignore | 2 +
src/test/python/Makefile | 38 +
src/test/python/README | 66 +
src/test/python/client/__init__.py | 0
src/test/python/client/conftest.py | 137 ++
src/test/python/client/test_client.py | 186 +++
src/test/python/client/test_oauth.py | 1721 +++++++++++++++++++++++++
src/test/python/conftest.py | 34 +
src/test/python/meson.build | 46 +
src/test/python/pq3.py | 740 +++++++++++
src/test/python/pytest.ini | 4 +
src/test/python/requirements.txt | 9 +
src/test/python/server/__init__.py | 0
src/test/python/server/conftest.py | 141 ++
src/test/python/server/meson.build | 18 +
src/test/python/server/oauthtest.c | 118 ++
src/test/python/server/test_oauth.py | 1074 +++++++++++++++
src/test/python/server/test_server.py | 21 +
src/test/python/test_internals.py | 138 ++
src/test/python/test_pq3.py | 574 +++++++++
src/test/python/tls.py | 195 +++
src/tools/make_venv | 56 +
25 files changed, 5441 insertions(+), 7 deletions(-)
create mode 100644 src/test/python/.gitignore
create mode 100644 src/test/python/Makefile
create mode 100644 src/test/python/README
create mode 100644 src/test/python/client/__init__.py
create mode 100644 src/test/python/client/conftest.py
create mode 100644 src/test/python/client/test_client.py
create mode 100644 src/test/python/client/test_oauth.py
create mode 100644 src/test/python/conftest.py
create mode 100644 src/test/python/meson.build
create mode 100644 src/test/python/pq3.py
create mode 100644 src/test/python/pytest.ini
create mode 100644 src/test/python/requirements.txt
create mode 100644 src/test/python/server/__init__.py
create mode 100644 src/test/python/server/conftest.py
create mode 100644 src/test/python/server/meson.build
create mode 100644 src/test/python/server/oauthtest.c
create mode 100644 src/test/python/server/test_oauth.py
create mode 100644 src/test/python/server/test_server.py
create mode 100644 src/test/python/test_internals.py
create mode 100644 src/test/python/test_pq3.py
create mode 100644 src/test/python/tls.py
create mode 100755 src/tools/make_venv
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 1adfdfdd45..b7dc66bd14 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -20,7 +20,7 @@ env:
MTEST_ARGS: --print-errorlogs --no-rebuild -C build
PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests
TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf
- PG_TEST_EXTRA: kerberos ldap ssl load_balance
+ PG_TEST_EXTRA: kerberos ldap ssl load_balance python
# What files to preserve in case tests fail
@@ -163,7 +163,7 @@ task:
chown root:postgres /tmp/cores
sysctl kern.corefile='/tmp/cores/%N.%P.core'
setup_additional_packages_script: |
- #pkg install -y ...
+ pkg install -y curl
# NB: Intentionally build without -Dllvm. The freebsd image size is already
# large enough to make VM startup slow, and even without llvm freebsd
@@ -175,6 +175,7 @@ task:
-Dcassert=true -Dinjection_points=true \
-Duuid=bsd -Dtcl_version=tcl86 -Ddtrace=auto \
-DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
+ -Doauth=curl \
-Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \
build
EOF
@@ -223,6 +224,7 @@ LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
--with-libxslt
--with-llvm
--with-lz4
+ --with-oauth=curl
--with-pam
--with-perl
--with-python
@@ -235,6 +237,7 @@ LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
LINUX_MESON_FEATURES: &LINUX_MESON_FEATURES >-
-Dllvm=enabled
+ -Doauth=curl
-Duuid=e2fs
@@ -310,8 +313,11 @@ task:
EOF
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install \
+ libcurl4-openssl-dev \
+ libcurl4-openssl-dev:i386 \
+ python3-venv \
matrix:
- name: Linux - Debian Bullseye - Autoconf
@@ -366,6 +372,8 @@ task:
# Also build & test in a 32bit build - it's gotten rare to test that
# locally.
+ # XXX 32-bit Python tests are currently disabled, as the system's 64-bit
+ # Python modules can't link against libpq.
configure_32_script: |
su postgres <<-EOF
export CC='ccache gcc -m32'
@@ -376,7 +384,7 @@ task:
-Dllvm=disabled \
--pkg-config-path /usr/lib/i386-linux-gnu/pkgconfig/ \
-DPERL=perl5.32-i386-linux-gnu \
- -DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
+ -DPG_TEST_EXTRA="${PG_TEST_EXTRA//"python"}" \
build-32
EOF
@@ -674,8 +682,8 @@ task:
folder: $CCACHE_DIR
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install libcurl4-openssl-dev
###
# Test that code can be built with gcc/clang without warnings
diff --git a/meson.build b/meson.build
index 34a0226ccf..feb159abbf 100644
--- a/meson.build
+++ b/meson.build
@@ -3185,6 +3185,9 @@ else
endif
testwrap = files('src/tools/testwrap')
+make_venv = files('src/tools/make_venv')
+
+checked_working_venv = false
foreach test_dir : tests
testwrap_base = [
@@ -3346,6 +3349,110 @@ foreach test_dir : tests
)
endforeach
install_suites += test_group
+ elif kind == 'pytest'
+ venv_name = test_dir['name'] + '_venv'
+ venv_path = meson.build_root() / venv_name
+
+ # The Python tests require a working venv module. This is part of the
+ # standard library, but some platforms disable it until a separate package
+ # is installed. Those same platforms don't provide an easy way to check
+ # whether the venv command will work until the first time you try it, so
+ # we decide whether or not to enable these tests on the fly.
+ if not checked_working_venv
+ cmd = run_command(python, '-m', 'venv', venv_path, check: false)
+
+ have_working_venv = (cmd.returncode() == 0)
+ if not have_working_venv
+ warning('A working Python venv module is required to run Python tests.')
+ endif
+
+ checked_working_venv = true
+ endif
+
+ if not have_working_venv
+ continue
+ endif
+
+ # Make sure the temporary installation is in PATH (necessary both for
+ # --temp-instance and for any pip modules compiling against libpq, like
+ # psycopg2).
+ env = test_env
+ env.prepend('PATH', temp_install_bindir, test_dir['bd'])
+
+ foreach name, value : t.get('env', {})
+ env.set(name, value)
+ endforeach
+
+ if get_option('PG_TEST_EXTRA').contains('python')
+ reqs = files(t['requirements'])
+ test('install_' + venv_name,
+ python,
+ args: [ make_venv, '--requirements', reqs, venv_path ],
+ env: env,
+ priority: setup_tests_priority - 1, # must run after tmp_install
+ is_parallel: false,
+ suite: ['setup'],
+ timeout: 60, # 30s is too short for the cryptography package compile
+ )
+ endif
+
+ test_group = test_dir['name']
+ test_output = test_result_dir / test_group / kind
+ test_kwargs = {
+ #'protocol': 'tap',
+ 'suite': test_group,
+ 'timeout': 1000,
+ 'depends': test_deps,
+ 'env': env,
+ }
+
+ if fs.is_dir(venv_path / 'Scripts')
+ # Windows virtualenv layout
+ pytest = venv_path / 'Scripts' / 'py.test'
+ else
+ pytest = venv_path / 'bin' / 'py.test'
+ endif
+
+ test_command = [
+ pytest,
+ # Avoid running these tests against an existing database.
+ '--temp-instance', test_output / 'data',
+
+ # FIXME pytest-tap's stream feature accidentally suppresses errors that
+ # are critical for debugging:
+ # https://github.com/python-tap/pytest-tap/issues/30
+ # Don't use the meson TAP protocol for now...
+ #'--tap-stream',
+ ]
+
+ foreach pyt : t['tests']
+ # Similarly to TAP, strip ./ and .py to make the names prettier
+ pyt_p = pyt
+ if pyt_p.startswith('./')
+ pyt_p = pyt_p.split('./')[1]
+ endif
+ if pyt_p.endswith('.py')
+ pyt_p = fs.stem(pyt_p)
+ endif
+
+ testwrap_pytest = testwrap_base + [
+ '--testgroup', test_group,
+ '--testname', pyt_p,
+ ]
+ if not get_option('PG_TEST_EXTRA').contains('python')
+ testwrap_pytest += ['--skip', '"python" tests not enabled in PG_TEST_EXTRA']
+ endif
+
+ test(test_group / pyt_p,
+ python,
+ kwargs: test_kwargs,
+ args: testwrap_pytest + [
+ '--', test_command,
+ test_dir['sd'] / pyt,
+ ],
+ )
+ endforeach
+ install_suites += test_group
else
error('unknown kind @0@ of test in @1@'.format(kind, test_dir['sd']))
endif
diff --git a/src/test/meson.build b/src/test/meson.build
index c3d0dfedf1..f401ec179e 100644
--- a/src/test/meson.build
+++ b/src/test/meson.build
@@ -7,6 +7,7 @@ subdir('authentication')
subdir('recovery')
subdir('subscription')
subdir('modules')
+subdir('python')
if ssl.found()
subdir('ssl')
diff --git a/src/test/python/.gitignore b/src/test/python/.gitignore
new file mode 100644
index 0000000000..0e8f027b2e
--- /dev/null
+++ b/src/test/python/.gitignore
@@ -0,0 +1,2 @@
+__pycache__/
+/venv/
diff --git a/src/test/python/Makefile b/src/test/python/Makefile
new file mode 100644
index 0000000000..b0695b6287
--- /dev/null
+++ b/src/test/python/Makefile
@@ -0,0 +1,38 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+# Only Python 3 is supported, but if it's named something different on your
+# system you can override it with the PYTHON3 variable.
+PYTHON3 := python3
+
+# All dependencies are placed into this directory. The default is .gitignored
+# for you, but you can override it if you'd like.
+VENV := ./venv
+
+override VBIN := $(VENV)/bin
+override PIP := $(VBIN)/pip
+override PYTEST := $(VBIN)/py.test
+override ISORT := $(VBIN)/isort
+override BLACK := $(VBIN)/black
+
+.PHONY: installcheck indent
+
+installcheck: $(PYTEST)
+ $(PYTEST) -v -rs
+
+indent: $(ISORT) $(BLACK)
+ $(ISORT) --profile black *.py client/*.py server/*.py
+ $(BLACK) *.py client/*.py server/*.py
+
+$(PYTEST) $(ISORT) $(BLACK) &: requirements.txt | $(PIP)
+ $(PIP) install --force-reinstall -r $<
+
+$(PIP):
+ $(PYTHON3) -m venv $(VENV)
+
+# A convenience recipe to rebuild psycopg2 against the local libpq.
+.PHONY: rebuild-psycopg2
+rebuild-psycopg2: | $(PIP)
+ $(PIP) install --force-reinstall --no-binary :all: $(shell grep psycopg2 requirements.txt)
diff --git a/src/test/python/README b/src/test/python/README
new file mode 100644
index 0000000000..acf339a589
--- /dev/null
+++ b/src/test/python/README
@@ -0,0 +1,66 @@
+A test suite for exercising both the libpq client and the server backend at the
+protocol level, based on pytest and Construct.
+
+WARNING! This suite takes superuser-level control of the cluster under test,
+writing to the server config, creating and destroying databases, etc. It also
+spins up various ephemeral TCP services. This is not safe for production servers
+and therefore must be explicitly opted into by setting PG_TEST_EXTRA=python in
+the environment.
+
+The test suite currently assumes that the standard PG* environment variables
+point to the database under test and are sufficient to log in a superuser on
+that system. In other words, a bare `psql` needs to Just Work before the test
+suite can do its thing. For a newly built dev cluster, typically all that I need
+to do is a
+
+ export PGDATABASE=postgres
+
+but you can adjust as needed for your setup. See also 'Advanced Usage' below.
+
+## Requirements
+
+A supported version (3.6+) of Python.
+
+The first run of
+
+ make installcheck PG_TEST_EXTRA=python
+
+will install a local virtual environment and all needed dependencies. During
+development, if libpq changes incompatibly, you can issue
+
+ $ make rebuild-psycopg2
+
+to force a rebuild of the client library.
+
+## Hacking
+
+The code style is enforced by a _very_ opinionated autoformatter. Running the
+
+ make indent
+
+recipe will invoke it for you automatically. Don't fight the tool; part of the
+zen is in knowing that if the formatter makes your code ugly, there's probably a
+cleaner way to write your code.
+
+## Advanced Usage
+
+The Makefile is there for convenience, but you don't have to use it. Activate
+the virtualenv to be able to use pytest directly:
+
+ $ export PG_TEST_EXTRA=python
+ $ source venv/bin/activate
+ $ py.test -k oauth
+ ...
+ $ py.test ./server/test_server.py
+ ...
+ $ deactivate # puts the PATH et al back the way it was before
+
+To make quick smoke tests possible, slow tests have been marked explicitly. You
+can skip them by saying e.g.
+
+ $ py.test -m 'not slow'
+
+If you'd rather not test against an existing server, you can have the suite spin
+up a temporary one using whatever pg_ctl it finds in PATH:
+
+ $ py.test --temp-instance=./tmp_check
diff --git a/src/test/python/client/__init__.py b/src/test/python/client/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/src/test/python/client/conftest.py b/src/test/python/client/conftest.py
new file mode 100644
index 0000000000..94f3620af3
--- /dev/null
+++ b/src/test/python/client/conftest.py
@@ -0,0 +1,137 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import socket
+import sys
+import threading
+
+import psycopg2
+import psycopg2.extras
+import pytest
+
+import pq3
+
+BLOCKING_TIMEOUT = 2 # the number of seconds to wait for blocking calls
+
+
+@pytest.fixture
+def server_socket(unused_tcp_port_factory):
+ """
+ Returns a listening socket bound to an ephemeral port.
+ """
+ with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+ s.bind(("127.0.0.1", unused_tcp_port_factory()))
+ s.listen(1)
+ s.settimeout(BLOCKING_TIMEOUT)
+ yield s
+
+
+class ClientHandshake(threading.Thread):
+ """
+ A thread that connects to a local Postgres server using psycopg2. Once the
+ opening handshake completes, the connection will be immediately closed.
+ """
+
+ def __init__(self, *, port, **kwargs):
+ super().__init__()
+
+ kwargs["port"] = port
+ self._kwargs = kwargs
+
+ self.exception = None
+
+ def run(self):
+ try:
+ conn = psycopg2.connect(host="127.0.0.1", **self._kwargs)
+ self._pump_async(conn)
+ conn.close()
+ except Exception as e:
+ self.exception = e
+
+ def check_completed(self, timeout=BLOCKING_TIMEOUT):
+ """
+ Joins the client thread. Raises an exception if the thread could not be
+ joined, or if it threw an exception itself. (The exception will be
+ cleared, so future calls to check_completed will succeed.)
+ """
+ self.join(timeout)
+
+ if self.is_alive():
+ raise TimeoutError("client thread did not handshake within the timeout")
+ elif self.exception:
+ e = self.exception
+ self.exception = None
+ raise e
+
+ def _pump_async(self, conn):
+ """
+ Polls a psycopg2 connection until it's completed. (Synchronous
+ connections will work here too; they'll just immediately return OK.)
+ """
+ psycopg2.extras.wait_select(conn)
+
+
+@pytest.fixture
+def accept(server_socket):
+ """
+ Returns a factory function that, when called, returns a pair (sock, client)
+ where sock is a server socket that has accepted a connection from client,
+ and client is an instance of ClientHandshake. Clients will complete their
+ handshakes and cleanly disconnect.
+
+ The default connstring options may be extended or overridden by passing
+ arbitrary keyword arguments. Keep in mind that you generally should not
+ override the host or port, since they point to the local test server.
+
+ For situations where a client needs to connect more than once to complete a
+ handshake, the accept function may be called more than once. (The client
+ returned for subsequent calls will always be the same client that was
+ returned for the first call.)
+
+ Tests must either complete the handshake so that the client thread can be
+ automatically joined during teardown, or else call client.check_completed()
+ and manually handle any expected errors.
+ """
+ _, port = server_socket.getsockname()
+
+ client = None
+ default_opts = dict(
+ port=port,
+ user=pq3.pguser(),
+ sslmode="disable",
+ )
+
+ def factory(**kwargs):
+ nonlocal client
+
+ if client is None:
+ opts = dict(default_opts)
+ opts.update(kwargs)
+
+ # The server_socket is already listening, so the client thread can
+ # be safely started; it'll block on the connection until we accept.
+ client = ClientHandshake(**opts)
+ client.start()
+
+ sock, _ = server_socket.accept()
+ return sock, client
+
+ yield factory
+
+ if client is not None:
+ client.check_completed()
+
+
+@pytest.fixture
+def conn(accept):
+ """
+ Returns an accepted, wrapped pq3 connection to a psycopg2 client. The socket
+ will be closed when the test finishes, and the client will be checked for a
+ cleanly completed handshake.
+ """
+ sock, client = accept()
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ yield conn
diff --git a/src/test/python/client/test_client.py b/src/test/python/client/test_client.py
new file mode 100644
index 0000000000..8372376ede
--- /dev/null
+++ b/src/test/python/client/test_client.py
@@ -0,0 +1,186 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import sys
+
+import psycopg2
+import pytest
+from cryptography.hazmat.primitives import hashes, hmac
+
+import pq3
+
+from .test_oauth import alt_patterns
+
+
+def finish_handshake(conn):
+ """
+ Sends the AuthenticationOK message and the standard opening salvo of server
+ messages, then asserts that the client immediately sends a Terminate message
+ to close the connection cleanly.
+ """
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.OK)
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"client_encoding", value=b"UTF-8")
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"DateStyle", value=b"ISO, MDY")
+ pq3.send(conn, pq3.types.ReadyForQuery, status=b"I")
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.Terminate
+
+
+def test_handshake(conn):
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ finish_handshake(conn)
+
+
+def test_aborted_connection(accept):
+ """
+ Make sure the client correctly reports an early close during handshakes.
+ """
+ sock, client = accept()
+ sock.close()
+
+ expected = alt_patterns(
+ "server closed the connection unexpectedly",
+ # On some platforms, ECONNABORTED gets set instead.
+ "Software caused connection abort",
+ )
+ with pytest.raises(psycopg2.OperationalError, match=expected):
+ client.check_completed()
+
+
+#
+# SCRAM-SHA-256 (see RFC 5802: https://tools.ietf.org/html/rfc5802)
+#
+
+
+@pytest.fixture
+def password():
+ """
+ Returns a password for use by both client and server.
+ """
+ # TODO: parameterize this with passwords that require SASLprep.
+ return "secret"
+
+
+@pytest.fixture
+def pwconn(accept, password):
+ """
+ Like the conn fixture, but uses a password in the connection.
+ """
+ sock, client = accept(password=password)
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ yield conn
+
+
+def sha256(data):
+ """The H(str) function from Section 2.2."""
+ digest = hashes.Hash(hashes.SHA256())
+ digest.update(data)
+ return digest.finalize()
+
+
+def hmac_256(key, data):
+ """The HMAC(key, str) function from Section 2.2."""
+ h = hmac.HMAC(key, hashes.SHA256())
+ h.update(data)
+ return h.finalize()
+
+
+def xor(a, b):
+ """The XOR operation from Section 2.2."""
+ res = bytearray(a)
+ for i, byte in enumerate(b):
+ res[i] ^= byte
+ return bytes(res)
+
+
+def h_i(data, salt, i):
+ """The Hi(str, salt, i) function from Section 2.2."""
+ assert i > 0
+
+ acc = hmac_256(data, salt + b"\x00\x00\x00\x01")
+ last = acc
+ i -= 1
+
+ while i:
+ u = hmac_256(data, last)
+ acc = xor(acc, u)
+
+ last = u
+ i -= 1
+
+ return acc
+
+
+def test_scram(pwconn, password):
+ startup = pq3.recv1(pwconn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ pwconn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"SCRAM-SHA-256", b""],
+ )
+
+ # Get the client-first-message.
+ pkt = pq3.recv1(pwconn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ initial = pq3.SASLInitialResponse.parse(pkt.payload)
+ assert initial.name == b"SCRAM-SHA-256"
+
+ c_bind, authzid, c_name, c_nonce = initial.data.split(b",")
+ assert c_bind == b"n" # no channel bindings on a plaintext connection
+ assert authzid == b"" # we don't support authzid currently
+ assert c_name == b"n=" # libpq doesn't honor the GS2 username
+ assert c_nonce.startswith(b"r=")
+
+ # Send the server-first-message.
+ salt = b"12345"
+ iterations = 2
+
+ s_nonce = c_nonce + b"somenonce"
+ s_salt = b"s=" + base64.b64encode(salt)
+ s_iterations = b"i=%d" % iterations
+
+ msg = b",".join([s_nonce, s_salt, s_iterations])
+ pq3.send(pwconn, pq3.types.AuthnRequest, type=pq3.authn.SASLContinue, body=msg)
+
+ # Get the client-final-message.
+ pkt = pq3.recv1(pwconn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ c_bind_final, c_nonce_final, c_proof = pkt.payload.split(b",")
+ assert c_bind_final == b"c=" + base64.b64encode(c_bind + b"," + authzid + b",")
+ assert c_nonce_final == s_nonce
+
+ # Calculate what the client proof should be.
+ salted_password = h_i(password.encode("ascii"), salt, iterations)
+ client_key = hmac_256(salted_password, b"Client Key")
+ stored_key = sha256(client_key)
+
+ auth_message = b",".join(
+ [c_name, c_nonce, s_nonce, s_salt, s_iterations, c_bind_final, c_nonce_final]
+ )
+ client_signature = hmac_256(stored_key, auth_message)
+ client_proof = xor(client_key, client_signature)
+
+ expected = b"p=" + base64.b64encode(client_proof)
+ assert c_proof == expected
+
+ # Send the correct server signature.
+ server_key = hmac_256(salted_password, b"Server Key")
+ server_signature = hmac_256(server_key, auth_message)
+
+ s_verify = b"v=" + base64.b64encode(server_signature)
+ pq3.send(pwconn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal, body=s_verify)
+
+ # Done!
+ finish_handshake(pwconn)
diff --git a/src/test/python/client/test_oauth.py b/src/test/python/client/test_oauth.py
new file mode 100644
index 0000000000..14fe139e73
--- /dev/null
+++ b/src/test/python/client/test_oauth.py
@@ -0,0 +1,1721 @@
+#
+# Copyright 2021 VMware, Inc.
+# Portions Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import collections
+import ctypes
+import http.server
+import json
+import logging
+import os
+import platform
+import secrets
+import sys
+import threading
+import time
+import traceback
+import types
+import urllib.parse
+from numbers import Number
+
+import psycopg2
+import pytest
+
+import pq3
+
+from .conftest import BLOCKING_TIMEOUT
+
+# The client tests need libpq to have been compiled with OAuth support; skip
+# them otherwise.
+pytestmark = pytest.mark.skipif(
+ os.getenv("with_oauth") == "none",
+ reason="OAuth client tests require --with-oauth support",
+)
+
+if platform.system() == "Darwin":
+ libpq = ctypes.cdll.LoadLibrary("libpq.5.dylib")
+elif platform.system() == "Windows":
+ pass # TODO
+else:
+ libpq = ctypes.cdll.LoadLibrary("libpq.so.5")
+
+
+def finish_handshake(conn):
+ """
+ Sends the AuthenticationOK message and the standard opening salvo of server
+ messages, then asserts that the client immediately sends a Terminate message
+ to close the connection cleanly.
+ """
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.OK)
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"client_encoding", value=b"UTF-8")
+ pq3.send(conn, pq3.types.ParameterStatus, name=b"DateStyle", value=b"ISO, MDY")
+ pq3.send(conn, pq3.types.ReadyForQuery, status=b"I")
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.Terminate
+
+
+#
+# OAUTHBEARER (see RFC 7628: https://tools.ietf.org/html/rfc7628)
+#
+
+
+def start_oauth_handshake(conn):
+ """
+ Negotiates an OAUTHBEARER SASL challenge. Returns the client's initial
+ response data.
+ """
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn, pq3.types.AuthnRequest, type=pq3.authn.SASL, body=[b"OAUTHBEARER", b""]
+ )
+
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+
+ initial = pq3.SASLInitialResponse.parse(pkt.payload)
+ assert initial.name == b"OAUTHBEARER"
+
+ return initial.data
+
+
+def get_auth_value(initial):
+ """
+ Finds the auth value (e.g. "Bearer somedata..." in the client's initial SASL
+ response.
+ """
+ kvpairs = initial.split(b"\x01")
+ assert kvpairs[0] == b"n,," # no channel binding or authzid
+ assert kvpairs[2] == b"" # ends with an empty kvpair
+ assert kvpairs[3] == b"" # ...and there's nothing after it
+ assert len(kvpairs) == 4
+
+ key, value = kvpairs[1].split(b"=", 2)
+ assert key == b"auth"
+
+ return value
+
+
+def xtest_oauth_success(conn): # TODO
+ initial = start_oauth_handshake(conn)
+
+ auth = get_auth_value(initial)
+ assert auth.startswith(b"Bearer ")
+
+ # Accept the token. TODO actually validate
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+class RawResponse(str):
+ """
+ Returned by registered endpoint callbacks to take full control of the
+ response. Usually, return values are converted to JSON; a RawResponse body
+ will be passed to the client as-is, allowing endpoint implementations to
+ issue invalid JSON.
+ """
+
+ pass
+
+
+class OpenIDProvider(threading.Thread):
+ """
+ A thread that runs a mock OpenID provider server.
+ """
+
+ def __init__(self, *, port):
+ super().__init__()
+
+ self.exception = None
+
+ addr = ("", port)
+ self.server = self._Server(addr, self._Handler)
+
+ # TODO: allow HTTPS only, somehow
+ oauth = self._OAuthState()
+ oauth.host = f"localhost:{port}"
+ oauth.issuer = f"http://localhost:{port}"
+
+ # The following endpoints are required to be advertised by providers,
+ # even though our chosen client implementation does not actually make
+ # use of them.
+ oauth.register_endpoint(
+ "authorization_endpoint", "POST", "/authorize", self._authorization_handler
+ )
+ oauth.register_endpoint("jwks_uri", "GET", "/keys", self._jwks_handler)
+
+ self.server.oauth = oauth
+
+ def run(self):
+ try:
+ # XXX socketserver.serve_forever() has a serious architectural
+ # issue: its select loop wakes up every `poll_interval` seconds to
+ # see if the server is shutting down. The default, 500 ms, only lets
+ # us run two tests every second. But the faster we go, the more CPU
+ # we burn unnecessarily...
+ self.server.serve_forever(poll_interval=0.01)
+ except Exception as e:
+ self.exception = e
+
+ def stop(self, timeout=BLOCKING_TIMEOUT):
+ """
+ Shuts down the server and joins its thread. Raises an exception if the
+ thread could not be joined, or if it threw an exception itself. Must
+ only be called once, after start().
+ """
+ self.server.shutdown()
+ self.join(timeout)
+
+ if self.is_alive():
+ raise TimeoutError("client thread did not handshake within the timeout")
+ elif self.exception:
+ e = self.exception
+ raise e
+
+ class _OAuthState(object):
+ def __init__(self):
+ self.endpoint_paths = {}
+ self._endpoints = {}
+
+ # Provide a standard discovery document by default; tests can
+ # override it.
+ self.register_endpoint(
+ None,
+ "GET",
+ "/.well-known/openid-configuration",
+ self._default_discovery_handler,
+ )
+
+ def register_endpoint(self, name, method, path, func):
+ if method not in self._endpoints:
+ self._endpoints[method] = {}
+
+ self._endpoints[method][path] = func
+
+ if name is not None:
+ self.endpoint_paths[name] = path
+
+ def endpoint(self, method, path):
+ if method not in self._endpoints:
+ return None
+
+ return self._endpoints[method].get(path)
+
+ def _default_discovery_handler(self, headers, params):
+ doc = {
+ "issuer": self.issuer,
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ "grant_types_supported": [
+ "urn:ietf:params:oauth:grant-type:device_code"
+ ],
+ }
+
+ for name, path in self.endpoint_paths.items():
+ doc[name] = self.issuer + path
+
+ return 200, doc
+
+ class _Server(http.server.HTTPServer):
+ def handle_error(self, request, addr):
+ self.shutdown_request(request)
+ raise
+
+ @staticmethod
+ def _jwks_handler(headers, params):
+ return 200, {"keys": []}
+
+ @staticmethod
+ def _authorization_handler(headers, params):
+ # We don't actually want this to be called during these tests -- we
+ # should be using the device authorization endpoint instead.
+ assert (
+ False
+ ), "authorization handler called instead of device authorization handler"
+
+ class _Handler(http.server.BaseHTTPRequestHandler):
+ timeout = BLOCKING_TIMEOUT
+
+ def _handle(self, *, params=None, handler=None):
+ oauth = self.server.oauth
+ assert self.headers["Host"] == oauth.host
+
+ if handler is None:
+ handler = oauth.endpoint(self.command, self.path)
+ assert (
+ handler is not None
+ ), f"no registered endpoint for {self.command} {self.path}"
+
+ result = handler(self.headers, params)
+
+ if len(result) == 2:
+ headers = {"Content-Type": "application/json"}
+ code, resp = result
+ else:
+ code, headers, resp = result
+
+ self.send_response(code)
+ for h, v in headers.items():
+ self.send_header(h, v)
+ self.end_headers()
+
+ if resp is not None:
+ if not isinstance(resp, RawResponse):
+ resp = json.dumps(resp)
+ resp = resp.encode("utf-8")
+ self.wfile.write(resp)
+
+ self.close_connection = True
+
+ def do_GET(self):
+ self._handle()
+
+ def _request_body(self):
+ length = self.headers["Content-Length"]
+
+ # Handle only an explicit content-length.
+ assert length is not None
+ length = int(length)
+
+ return self.rfile.read(length).decode("utf-8")
+
+ def do_POST(self):
+ assert self.headers["Content-Type"] == "application/x-www-form-urlencoded"
+
+ body = self._request_body()
+ params = urllib.parse.parse_qs(body)
+
+ self._handle(params=params)
+
+
+@pytest.fixture
+def openid_provider(unused_tcp_port_factory):
+ """
+ A fixture that returns the OAuth state of a running OpenID provider server. The
+ server will be stopped when the fixture is torn down.
+ """
+ thread = OpenIDProvider(port=unused_tcp_port_factory())
+ thread.start()
+
+ try:
+ yield thread.server.oauth
+ finally:
+ thread.stop()
+
+
+#
+# PQAuthDataHook implementation, matching libpq.h
+#
+
+
+PQAUTHDATA_PROMPT_OAUTH_DEVICE = 0
+PQAUTHDATA_OAUTH_BEARER_TOKEN = 1
+
+PGRES_POLLING_FAILED = 0
+PGRES_POLLING_READING = 1
+PGRES_POLLING_WRITING = 2
+PGRES_POLLING_OK = 3
+
+
+class PQPromptOAuthDevice(ctypes.Structure):
+ _fields_ = [
+ ("verification_uri", ctypes.c_char_p),
+ ("user_code", ctypes.c_char_p),
+ ]
+
+
+class PQOAuthBearerRequest(ctypes.Structure):
+ pass
+
+
+PQOAuthBearerRequest._fields_ = [
+ ("openid_configuration", ctypes.c_char_p),
+ ("scope", ctypes.c_char_p),
+ (
+ "async_",
+ ctypes.CFUNCTYPE(
+ ctypes.c_int,
+ ctypes.c_void_p,
+ ctypes.POINTER(PQOAuthBearerRequest),
+ ctypes.POINTER(ctypes.c_int),
+ ),
+ ),
+ (
+ "cleanup",
+ ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(PQOAuthBearerRequest)),
+ ),
+ ("token", ctypes.c_char_p),
+ ("user", ctypes.c_void_p),
+]
+
+
+@pytest.fixture
+def auth_data_cb():
+ """
+ Tracks calls to the libpq authdata hook. The yielded object contains a calls
+ member that records the data sent to the hook. If a test needs to perform
+ custom actions during a call, it can set the yielded object's impl callback;
+ beware that the callback takes place on a different thread.
+
+ This is done differently from the other callback implementations on purpose.
+ For the others, we can declare test-specific callbacks and have them perform
+ direct assertions on the data they receive. But that won't work for a C
+ callback, because there's no way for us to bubble up the assertion through
+ libpq. Instead, this mock-style approach is taken, where we just record the
+ calls and let the test examine them later.
+ """
+
+ class _Call:
+ pass
+
+ class _cb(object):
+ def __init__(self):
+ self.calls = []
+
+ cb = _cb()
+ cb.impl = None
+
+ # The callback will occur on a different thread, so protect the cb object.
+ cb_lock = threading.Lock()
+
+ @ctypes.CFUNCTYPE(ctypes.c_int, ctypes.c_byte, ctypes.c_void_p, ctypes.c_void_p)
+ def auth_data_cb(typ, pgconn, data):
+ handle_by_default = 0 # does an implementation have to be provided?
+
+ if typ == PQAUTHDATA_PROMPT_OAUTH_DEVICE:
+ cls = PQPromptOAuthDevice
+ handle_by_default = 1
+ elif typ == PQAUTHDATA_OAUTH_BEARER_TOKEN:
+ cls = PQOAuthBearerRequest
+ else:
+ return 0
+
+ call = _Call()
+ call.type = typ
+
+ # The lifetime of the underlying data being pointed to doesn't
+ # necessarily match the lifetime of the Python object, so we can't
+ # reference a Structure's fields after returning. Explicitly copy the
+ # contents over, field by field.
+ data = ctypes.cast(data, ctypes.POINTER(cls))
+ for name, _ in cls._fields_:
+ setattr(call, name, getattr(data.contents, name))
+
+ with cb_lock:
+ cb.calls.append(call)
+
+ if cb.impl:
+ # Pass control back to the test.
+ try:
+ return cb.impl(typ, pgconn, data.contents)
+ except Exception:
+ # This can't escape into the C stack, but we can fail the flow
+ # and hope the traceback gives us enough detail.
+ logging.error(
+ "Exception during authdata hook callback:\n"
+ + traceback.format_exc()
+ )
+ return -1
+
+ return handle_by_default
+
+ libpq.PQsetAuthDataHook(auth_data_cb)
+ try:
+ yield cb
+ finally:
+ # The callback is about to go out of scope, so make sure libpq is
+ # disconnected from it. (We wouldn't want to accidentally influence
+ # later tests anyway.)
+ libpq.PQsetAuthDataHook(None)
+
+
+@pytest.mark.parametrize("secret", [None, "", "hunter2"])
+@pytest.mark.parametrize("scope", [None, "", "openid email"])
+@pytest.mark.parametrize("retries", [0, 1])
+@pytest.mark.parametrize(
+ "asynchronous",
+ [
+ pytest.param(False, id="synchronous"),
+ pytest.param(True, id="asynchronous"),
+ ],
+)
+def test_oauth_with_explicit_issuer(
+ accept, openid_provider, asynchronous, retries, scope, secret, auth_data_cb
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ oauth_client_secret=secret,
+ oauth_scope=scope,
+ async_=asynchronous,
+ )
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+ verification_url = "https://example.com/device"
+
+ access_token = secrets.token_urlsafe()
+
+ def check_client_authn(headers, params):
+ if not secret:
+ assert params["client_id"] == [client_id]
+ return
+
+ # Require the client to use Basic authn; request-body credentials are
+ # NOT RECOMMENDED (RFC 6749, Sec. 2.3.1).
+ assert "Authorization" in headers
+
+ method, creds = headers["Authorization"].split()
+ assert method == "Basic"
+
+ expected = f"{client_id}:{secret}"
+ assert base64.b64decode(creds) == expected.encode("ascii")
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ check_client_authn(headers, params)
+
+ if scope:
+ assert params["scope"] == [scope]
+ else:
+ assert "scope" not in params
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": verification_url,
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ attempts = 0
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ check_client_authn(headers, params)
+
+ assert params["grant_type"] == ["urn:ietf:params:oauth:grant-type:device_code"]
+ assert params["device_code"] == [device_code]
+
+ now = time.monotonic()
+
+ with retry_lock:
+ nonlocal attempts
+
+ # If the test wants to force the client to retry, return an
+ # authorization_pending response and decrement the retry count.
+ if attempts < retries:
+ attempts += 1
+ return 400, {"error": "authorization_pending"}
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+ if retries:
+ # Finally, make sure that the client prompted the user once with the
+ # expected authorization URL and user code.
+ assert len(auth_data_cb.calls) == 2
+
+ # First call should have been for a custom flow, which we ignored.
+ assert auth_data_cb.calls[0].type == PQAUTHDATA_OAUTH_BEARER_TOKEN
+
+ # Second call is for our user prompt.
+ call = auth_data_cb.calls[1]
+ assert call.type == PQAUTHDATA_PROMPT_OAUTH_DEVICE
+ assert call.verification_uri.decode() == verification_url
+ assert call.user_code.decode() == user_code
+
+
+def expect_disconnected_handshake(sock):
+ """
+ Helper for any tests that expect the client to disconnect immediately after
+ being sent the OAUTHBEARER SASL method. Generally speaking, this requires
+ the client to have an oauth_issuer set so that it doesn't try to go through
+ discovery.
+ """
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake.
+ startup = pq3.recv1(conn, cls=pq3.Startup)
+ assert startup.proto == pq3.protocol(3, 0)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASL,
+ body=[b"OAUTHBEARER", b""],
+ )
+
+ # The client should disconnect at this point.
+ assert not conn.read()
+
+
+def test_oauth_requires_client_id(accept, openid_provider):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ # Do not set a client ID; this should cause a client error after the
+ # server asks for OAUTHBEARER and the client tries to contact the
+ # issuer.
+ )
+
+ expect_disconnected_handshake(sock)
+
+ expected_error = "no oauth_client_id is set"
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.slow
+@pytest.mark.parametrize("error_code", ["authorization_pending", "slow_down"])
+@pytest.mark.parametrize("retries", [1, 2])
+def test_oauth_retry_interval(accept, openid_provider, retries, error_code):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id="some-id",
+ )
+
+ expected_retry_interval = 1
+ access_token = secrets.token_urlsafe()
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ resp = {
+ "device_code": "my-device-code",
+ "user_code": "my-user-code",
+ "interval": expected_retry_interval,
+ "verification_uri": "https://example.com",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ attempts = 0
+ last_retry = None
+ retry_lock = threading.Lock()
+
+ def token_endpoint(headers, params):
+ now = time.monotonic()
+
+ with retry_lock:
+ nonlocal attempts, last_retry, expected_retry_interval
+
+ # Make sure the retry interval is being respected by the client.
+ if last_retry is not None:
+ interval = now - last_retry
+ assert interval >= expected_retry_interval
+
+ last_retry = now
+
+ # If the test wants to force the client to retry, return the desired
+ # error response and decrement the retry count.
+ if attempts < retries:
+ attempts += 1
+
+ # A slow_down code requires the client to additionally increase
+ # its interval by five seconds.
+ if error_code == "slow_down":
+ expected_retry_interval += 5
+
+ return 400, {"error": error_code}
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in the above endpoints
+ # being called.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+@pytest.fixture
+def self_pipe():
+ """
+ Yields a pipe fd pair.
+ """
+
+ class _Pipe:
+ pass
+
+ p = _Pipe()
+ p.readfd, p.writefd = os.pipe()
+
+ try:
+ yield p
+ finally:
+ os.close(p.readfd)
+ os.close(p.writefd)
+
+
+@pytest.mark.parametrize("scope", [None, "", "openid email"])
+@pytest.mark.parametrize(
+ "retries",
+ [
+ -1, # no async callback
+ 0, # async callback immediately returns token
+ 1, # async callback waits on altsock once
+ 2, # async callback waits on altsock twice
+ ],
+)
+@pytest.mark.parametrize(
+ "asynchronous",
+ [
+ pytest.param(False, id="synchronous"),
+ pytest.param(True, id="asynchronous"),
+ ],
+)
+def test_user_defined_flow(
+ accept, auth_data_cb, self_pipe, scope, retries, asynchronous
+):
+ issuer = "http://localhost"
+ discovery_uri = issuer + "/.well-known/openid-configuration"
+ access_token = secrets.token_urlsafe()
+
+ sock, client = accept(
+ oauth_issuer=issuer,
+ oauth_client_id="some-id",
+ oauth_scope=scope,
+ async_=asynchronous,
+ )
+
+ # Track callbacks.
+ attempts = 0
+ wakeup_called = False
+ cleanup_calls = 0
+ lock = threading.Lock()
+
+ def wakeup():
+ """Writes a byte to the wakeup pipe."""
+ nonlocal wakeup_called
+ with lock:
+ wakeup_called = True
+ os.write(self_pipe.writefd, b"\0")
+
+ def get_token(pgconn, request, p_altsock):
+ """
+ Async token callback. While attempts < retries, libpq will be instructed
+ to wait on the self_pipe. When attempts == retries, the token will be
+ set.
+
+ Note that assertions and exceptions raised here are allowed but not very
+ helpful, since they can't bubble through the libpq stack to be collected
+ by the test suite. Try not to rely too heavily on them.
+ """
+ # Make sure libpq passed our user data through.
+ assert request.user == 42
+
+ with lock:
+ nonlocal attempts, wakeup_called
+
+ if attempts:
+ # If we've already started the timer, we shouldn't get a
+ # call back before it trips.
+ assert wakeup_called, "authdata hook was called before the timer"
+
+ # Drain the wakeup byte.
+ os.read(self_pipe.readfd, 1)
+
+ if attempts < retries:
+ attempts += 1
+
+ # Wake up the client in a little bit of time.
+ wakeup_called = False
+ threading.Timer(0.1, wakeup).start()
+
+ # Tell libpq to wait on the other end of the wakeup pipe.
+ p_altsock[0] = self_pipe.readfd
+ return PGRES_POLLING_READING
+
+ # Done!
+ request.token = access_token.encode()
+ return PGRES_POLLING_OK
+
+ @ctypes.CFUNCTYPE(
+ ctypes.c_int,
+ ctypes.c_void_p,
+ ctypes.POINTER(PQOAuthBearerRequest),
+ ctypes.POINTER(ctypes.c_int),
+ )
+ def get_token_wrapper(pgconn, p_request, p_altsock):
+ """
+ Translation layer between C and Python for the async callback.
+ Assertions and exceptions will be swallowed at the boundary, so make
+ sure they don't escape here.
+ """
+ try:
+ return get_token(pgconn, p_request.contents, p_altsock)
+ except Exception:
+ logging.error("Exception during async callback:\n" + traceback.format_exc())
+ return PGRES_POLLING_FAILED
+
+ @ctypes.CFUNCTYPE(None, ctypes.c_void_p, ctypes.POINTER(PQOAuthBearerRequest))
+ def cleanup(pgconn, p_request):
+ """
+ Should be called exactly once per connection.
+ """
+ nonlocal cleanup_calls
+ with lock:
+ cleanup_calls += 1
+
+ def bearer_hook(typ, pgconn, request):
+ """
+ Implementation of the PQAuthDataHook, which either sets up an async
+ callback or returns the token directly, depending on the value of
+ retries.
+
+ As above, try not to rely too much on assertions/exceptions here.
+ """
+ assert typ == PQAUTHDATA_OAUTH_BEARER_TOKEN
+ request.cleanup = cleanup
+
+ if retries < 0:
+ # Special case: return a token immediately without a callback.
+ request.token = access_token.encode()
+ return 1
+
+ # Tell libpq to call us back.
+ request.async_ = get_token_wrapper
+ request.user = ctypes.c_void_p(42) # will be checked in the callback
+ return 1
+
+ auth_data_cb.impl = bearer_hook
+
+ # Now drive the server side.
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ # Initiate a handshake, which should result in our custom callback
+ # being invoked to fetch the token.
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+ # Check the data provided to the hook.
+ assert len(auth_data_cb.calls) == 1
+
+ call = auth_data_cb.calls[0]
+ assert call.type == PQAUTHDATA_OAUTH_BEARER_TOKEN
+ assert call.openid_configuration.decode() == discovery_uri
+ assert call.scope == (None if scope is None else scope.encode())
+
+ # Make sure we clean up after ourselves when the connection is finished.
+ client.check_completed()
+ assert cleanup_calls == 1
+
+
+def alt_patterns(*patterns):
+ """
+ Just combines multiple alternative regexes into one. It's not very efficient
+ but IMO it's easier to read and maintain.
+ """
+ pat = ""
+
+ for p in patterns:
+ if pat:
+ pat += "|"
+ pat += f"({p})"
+
+ return pat
+
+
+@pytest.mark.parametrize(
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
+ (
+ 400,
+ {
+ "error": "invalid_client",
+ "error_description": "client authentication failed",
+ },
+ ),
+ r"failed to obtain device authorization: client authentication failed \(invalid_client\)",
+ id="authentication failure with description",
+ ),
+ pytest.param(
+ (400, {"error": "invalid_request"}),
+ r"failed to obtain device authorization: \(invalid_request\)",
+ id="invalid request without description",
+ ),
+ pytest.param(
+ (400, {}),
+ r'failed to parse token error response: field "error" is missing',
+ id="broken error response",
+ ),
+ pytest.param(
+ (200, RawResponse(r'{ "interval": 3.5.8 }')),
+ r"failed to parse device authorization: Token .* is invalid",
+ id="non-numeric interval",
+ ),
+ pytest.param(
+ (200, RawResponse(r'{ "interval": 08 }')),
+ r"failed to parse device authorization: Token .* is invalid",
+ id="invalid numeric interval",
+ ),
+ ],
+)
+def test_oauth_device_authorization_failures(
+ accept, openid_provider, failure_mode, error_pattern
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ )
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ return failure_mode
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert False, "token endpoint was invoked unexpectedly"
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+Missing = object() # sentinel for test_oauth_device_authorization_bad_json()
+
+
+@pytest.mark.parametrize(
+ "bad_value",
+ [
+ pytest.param({"device_code": 3}, id="object"),
+ pytest.param([1, 2, 3], id="array"),
+ pytest.param("some string", id="string"),
+ pytest.param(4, id="numeric"),
+ pytest.param(False, id="boolean"),
+ pytest.param(None, id="null"),
+ pytest.param(Missing, id="missing"),
+ ],
+)
+@pytest.mark.parametrize(
+ "field_name,ok_type,required",
+ [
+ ("device_code", str, True),
+ ("user_code", str, True),
+ ("verification_uri", str, True),
+ ("interval", int, False),
+ ],
+)
+def test_oauth_device_authorization_bad_json_schema(
+ accept, openid_provider, field_name, ok_type, required, bad_value
+):
+ # To make the test matrix easy, just skip the tests that aren't actually
+ # interesting (field of the correct type, missing optional field).
+ if bad_value is Missing and not required:
+ pytest.skip("not interesting: optional field")
+ elif type(bad_value) == ok_type: # not isinstance(), because bool is an int
+ pytest.skip("not interesting: correct type")
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=secrets.token_hex(),
+ )
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ # Begin with an acceptable base response...
+ resp = {
+ "device_code": "my-device-code",
+ "user_code": "my-user-code",
+ "interval": 0,
+ "verification_uri": "https://example.com",
+ "expires_in": 5,
+ }
+
+ # ...then tweak it so the client fails.
+ if bad_value is Missing:
+ del resp[field_name]
+ else:
+ resp[field_name] = bad_value
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert False, "token endpoint was invoked unexpectedly"
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ if bad_value is Missing:
+ error_pattern = f'field "{field_name}" is missing'
+ elif ok_type == str:
+ error_pattern = f'field "{field_name}" must be a string'
+ elif ok_type == int:
+ error_pattern = f'field "{field_name}" must be a number'
+ else:
+ assert False, "update error_pattern for new failure mode"
+
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "failure_mode, error_pattern",
+ [
+ pytest.param(
+ (
+ 400,
+ {
+ "error": "expired_token",
+ "error_description": "the device code has expired",
+ },
+ ),
+ r"failed to obtain access token: the device code has expired \(expired_token\)",
+ id="expired token with description",
+ ),
+ pytest.param(
+ (400, {"error": "access_denied"}),
+ r"failed to obtain access token: \(access_denied\)",
+ id="access denied without description",
+ ),
+ pytest.param(
+ (400, {}),
+ r'failed to parse token error response: field "error" is missing',
+ id="empty error response",
+ ),
+ pytest.param(
+ (200, {}, {}),
+ r"failed to parse access token response: no content type was provided",
+ id="missing content type",
+ ),
+ pytest.param(
+ (200, {"Content-Type": "text/plain"}, {}),
+ r"failed to parse access token response: unexpected content type",
+ id="wrong content type",
+ ),
+ ],
+)
+@pytest.mark.parametrize("retries", [0, 1])
+def test_oauth_token_failures(
+ accept, openid_provider, retries, failure_mode, error_pattern
+):
+ client_id = secrets.token_hex()
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=client_id,
+ )
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ assert params["client_id"] == [client_id]
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": "https://example.com/device",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ retry_lock = threading.Lock()
+ final_sent = False
+
+ def token_endpoint(headers, params):
+ with retry_lock:
+ nonlocal retries, final_sent
+
+ # If the test wants to force the client to retry, return an
+ # authorization_pending response and decrement the retry count.
+ if retries > 0:
+ retries -= 1
+ return 400, {"error": "authorization_pending"}
+
+ # We should only return our failure_mode response once; any further
+ # requests indicate that the client isn't correctly bailing out.
+ assert not final_sent, "client continued after token error"
+
+ final_sent = True
+
+ return failure_mode
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "bad_value",
+ [
+ pytest.param({"device_code": 3}, id="object"),
+ pytest.param([1, 2, 3], id="array"),
+ pytest.param("some string", id="string"),
+ pytest.param(4, id="numeric"),
+ pytest.param(False, id="boolean"),
+ pytest.param(None, id="null"),
+ pytest.param(Missing, id="missing"),
+ ],
+)
+@pytest.mark.parametrize(
+ "field_name,ok_type,required",
+ [
+ ("access_token", str, True),
+ ("token_type", str, True),
+ ],
+)
+def test_oauth_token_bad_json_schema(
+ accept, openid_provider, field_name, ok_type, required, bad_value
+):
+ # To make the test matrix easy, just skip the tests that aren't actually
+ # interesting (field of the correct type, missing optional field).
+ if bad_value is Missing and not required:
+ pytest.skip("not interesting: optional field")
+ elif type(bad_value) == ok_type: # not isinstance(), because bool is an int
+ pytest.skip("not interesting: correct type")
+
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=secrets.token_hex(),
+ )
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ resp = {
+ "device_code": "my-device-code",
+ "user_code": "my-user-code",
+ "interval": 0,
+ "verification_uri": "https://example.com",
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ # Begin with an acceptable base response...
+ resp = {
+ "access_token": secrets.token_urlsafe(),
+ "token_type": "bearer",
+ }
+
+ # ...then tweak it so the client fails.
+ if bad_value is Missing:
+ del resp[field_name]
+ else:
+ resp[field_name] = bad_value
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ expect_disconnected_handshake(sock)
+
+ # Now make sure the client correctly failed.
+ error_pattern = "failed to parse access token response: "
+ if bad_value is Missing:
+ error_pattern += f'field "{field_name}" is missing'
+ elif ok_type == str:
+ error_pattern += f'field "{field_name}" must be a string'
+ elif ok_type == int:
+ error_pattern += f'field "{field_name}" must be a number'
+ else:
+ assert False, "update error_pattern for new failure mode"
+
+ with pytest.raises(psycopg2.OperationalError, match=error_pattern):
+ client.check_completed()
+
+
+@pytest.mark.parametrize("scope", [None, "openid email"])
+@pytest.mark.parametrize(
+ "base_response",
+ [
+ {"status": "invalid_token"},
+ {"extra_object": {"key": "value"}, "status": "invalid_token"},
+ {"extra_object": {"status": 1}, "status": "invalid_token"},
+ ],
+)
+def test_oauth_discovery(accept, openid_provider, base_response, scope):
+ sock, client = accept(oauth_client_id=secrets.token_hex())
+
+ device_code = secrets.token_hex()
+ user_code = f"{secrets.token_hex(2)}-{secrets.token_hex(2)}"
+ verification_url = "https://example.com/device"
+
+ access_token = secrets.token_urlsafe()
+
+ # Set up our provider callbacks.
+ # NOTE that these callbacks will be called on a background thread. Don't do
+ # any unprotected state mutation here.
+
+ def authorization_endpoint(headers, params):
+ if scope:
+ assert params["scope"] == [scope]
+ else:
+ assert "scope" not in params
+
+ resp = {
+ "device_code": device_code,
+ "user_code": user_code,
+ "interval": 0,
+ "verification_uri": verification_url,
+ "expires_in": 5,
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "device_authorization_endpoint", "POST", "/device", authorization_endpoint
+ )
+
+ def token_endpoint(headers, params):
+ assert params["grant_type"] == ["urn:ietf:params:oauth:grant-type:device_code"]
+ assert params["device_code"] == [device_code]
+
+ # Successfully finish the request by sending the access bearer token.
+ resp = {
+ "access_token": access_token,
+ "token_type": "bearer",
+ }
+
+ return 200, resp
+
+ openid_provider.register_endpoint(
+ "token_endpoint", "POST", "/token", token_endpoint
+ )
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # For discovery, the client should send an empty auth header. See
+ # RFC 7628, Sec. 4.3.
+ auth = get_auth_value(initial)
+ assert auth == b""
+
+ # We will fail the first SASL exchange. First return a link to the
+ # discovery document, pointing to the test provider server.
+ resp = dict(base_response)
+
+ discovery_uri = f"{openid_provider.issuer}/.well-known/openid-configuration"
+ resp["openid-configuration"] = discovery_uri
+
+ if scope:
+ resp["scope"] = scope
+
+ resp = json.dumps(resp)
+
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASLContinue,
+ body=resp.encode("ascii"),
+ )
+
+ # Per RFC, the client is required to send a dummy ^A response.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+ assert pkt.payload == b"\x01"
+
+ # Now fail the SASL exchange.
+ pq3.send(
+ conn,
+ pq3.types.ErrorResponse,
+ fields=[
+ b"SFATAL",
+ b"C28000",
+ b"Mdoesn't matter",
+ b"",
+ ],
+ )
+
+ # The client will connect to us a second time, using the parameters we sent
+ # it.
+ sock, _ = accept()
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # Validate and accept the token.
+ auth = get_auth_value(initial)
+ assert auth == f"Bearer {access_token}".encode("ascii")
+
+ pq3.send(conn, pq3.types.AuthnRequest, type=pq3.authn.SASLFinal)
+ finish_handshake(conn)
+
+
+@pytest.mark.parametrize(
+ "response,expected_error",
+ [
+ pytest.param(
+ "abcde",
+ 'Token "abcde" is invalid',
+ id="bad JSON: invalid syntax",
+ ),
+ pytest.param(
+ '"abcde"',
+ "top-level element must be an object",
+ id="bad JSON: top-level element is a string",
+ ),
+ pytest.param(
+ "[]",
+ "top-level element must be an object",
+ id="bad JSON: top-level element is an array",
+ ),
+ pytest.param(
+ "{}",
+ "server sent error response without a status",
+ id="bad JSON: no status member",
+ ),
+ pytest.param(
+ '{ "status": null }',
+ 'field "status" must be a string',
+ id="bad JSON: null status member",
+ ),
+ pytest.param(
+ '{ "status": 0 }',
+ 'field "status" must be a string',
+ id="bad JSON: int status member",
+ ),
+ pytest.param(
+ '{ "status": [ "bad" ] }',
+ 'field "status" must be a string',
+ id="bad JSON: array status member",
+ ),
+ pytest.param(
+ '{ "status": { "bad": "bad" } }',
+ 'field "status" must be a string',
+ id="bad JSON: object status member",
+ ),
+ pytest.param(
+ '{ "nested": { "status": "bad" } }',
+ "server sent error response without a status",
+ id="bad JSON: nested status",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token" ',
+ "The input string ended unexpectedly",
+ id="bad JSON: unterminated object",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token" } { }',
+ 'Expected end of input, but found "{"',
+ id="bad JSON: trailing data",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "openid-configuration": 1 }',
+ 'field "openid-configuration" must be a string',
+ id="bad JSON: int openid-configuration member",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "openid-configuration": 1 }',
+ 'field "openid-configuration" must be a string',
+ id="bad JSON: int openid-configuration member",
+ ),
+ pytest.param(
+ '{ "status": "invalid_token", "scope": 1 }',
+ 'field "scope" must be a string',
+ id="bad JSON: int scope member",
+ ),
+ ],
+)
+def test_oauth_discovery_server_error(accept, response, expected_error):
+ sock, client = accept(oauth_client_id=secrets.token_hex())
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ initial = start_oauth_handshake(conn)
+
+ # Fail the SASL exchange with an invalid JSON response.
+ pq3.send(
+ conn,
+ pq3.types.AuthnRequest,
+ type=pq3.authn.SASLContinue,
+ body=response.encode("utf-8"),
+ )
+
+ # The client should disconnect, so the socket is closed here. (If
+ # the client doesn't disconnect, it will report a different error
+ # below and the test will fail.)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "bad_response,expected_error",
+ [
+ pytest.param(
+ (200, {"Content-Type": "text/plain"}, {}),
+ r'failed to parse OpenID discovery document: unexpected content type "text/plain"',
+ id="not JSON",
+ ),
+ pytest.param(
+ (200, {}, {}),
+ r"failed to parse OpenID discovery document: no content type was provided",
+ id="no Content-Type",
+ ),
+ pytest.param(
+ (204, {}, None),
+ r"failed to fetch OpenID discovery document: unexpected response code 204",
+ id="no content",
+ ),
+ pytest.param(
+ (301, {"Location": "https://localhost/"}, None),
+ r"failed to fetch OpenID discovery document: unexpected response code 301",
+ id="redirection",
+ ),
+ pytest.param(
+ (404, {}),
+ r"failed to fetch OpenID discovery document: unexpected response code 404",
+ id="not found",
+ ),
+ pytest.param(
+ (200, RawResponse("blah\x00blah")),
+ r"failed to parse OpenID discovery document: response contains embedded NULLs",
+ id="NULL bytes in document",
+ ),
+ pytest.param(
+ (200, 123),
+ r"failed to parse OpenID discovery document: top-level element must be an object",
+ id="scalar at top level",
+ ),
+ pytest.param(
+ (200, []),
+ r"failed to parse OpenID discovery document: top-level element must be an object",
+ id="array at top level",
+ ),
+ pytest.param(
+ (200, RawResponse("{")),
+ r"failed to parse OpenID discovery document.* input string ended unexpectedly",
+ id="unclosed object",
+ ),
+ pytest.param(
+ (200, RawResponse(r'{ "hello": ] }')),
+ r"failed to parse OpenID discovery document.* Expected JSON value",
+ id="bad array",
+ ),
+ pytest.param(
+ (200, {"issuer": 123}),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="non-string issuer",
+ ),
+ pytest.param(
+ (200, {"issuer": ["something"]}),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="issuer array",
+ ),
+ pytest.param(
+ (200, {"issuer": {}}),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="issuer object",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": 123}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="scalar grant types field",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": {}}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="object grant types field",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": [123]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="non-string grant types",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": ["something", 123]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="non-string grant types later in the list",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": ["something", {}]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="object grant types later in the list",
+ ),
+ pytest.param(
+ (200, {"grant_types_supported": ["something", ["something"]]}),
+ r'failed to parse OpenID discovery document: field "grant_types_supported" must be an array of strings',
+ id="embedded array grant types later in the list",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "grant_types_supported": ["something"],
+ "token_endpoint": "https://example.com/",
+ "issuer": 123,
+ },
+ ),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="non-string issuer after other valid fields",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "ignored": {"grant_types_supported": 123, "token_endpoint": 123},
+ "issuer": 123,
+ },
+ ),
+ r'failed to parse OpenID discovery document: field "issuer" must be a string',
+ id="non-string issuer after other ignored fields",
+ ),
+ pytest.param(
+ (200, {"token_endpoint": "https://example.com/"}),
+ r'failed to parse OpenID discovery document: field "issuer" is missing',
+ id="missing issuer",
+ ),
+ pytest.param(
+ (200, {"issuer": "https://example.com/"}),
+ r'failed to parse OpenID discovery document: field "token_endpoint" is missing',
+ id="missing token endpoint",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "issuer": "https://example.com",
+ "token_endpoint": "https://example.com/token",
+ "device_authorization_endpoint": "https://example.com/dev",
+ },
+ ),
+ r'cannot run OAuth device authorization: issuer "https://example.com" does not support device code grants',
+ id="missing device code grants",
+ ),
+ pytest.param(
+ (
+ 200,
+ {
+ "issuer": "https://example.com",
+ "token_endpoint": "https://example.com/token",
+ "grant_types_supported": [
+ "urn:ietf:params:oauth:grant-type:device_code"
+ ],
+ },
+ ),
+ r'cannot run OAuth device authorization: issuer "https://example.com" does not provide a device authorization endpoint',
+ id="missing device_authorization_endpoint",
+ ),
+ #
+ # Exercise HTTP-level failures by breaking the protocol. Note that the
+ # error messages here are implementation-dependent.
+ #
+ pytest.param(
+ (1000, {}),
+ r"failed to fetch OpenID discovery document: Unsupported protocol \(.*\)",
+ id="invalid HTTP response code",
+ ),
+ pytest.param(
+ (200, {"Content-Length": -1}, {}),
+ r"failed to fetch OpenID discovery document: Weird server reply \(.*Content-Length.*\)",
+ id="bad HTTP Content-Length",
+ ),
+ ],
+)
+def test_oauth_discovery_provider_failure(
+ accept, openid_provider, bad_response, expected_error
+):
+ sock, client = accept(
+ oauth_issuer=openid_provider.issuer,
+ oauth_client_id=secrets.token_hex(),
+ )
+
+ def failing_discovery_handler(headers, params):
+ return bad_response
+
+ openid_provider.register_endpoint(
+ None,
+ "GET",
+ "/.well-known/openid-configuration",
+ failing_discovery_handler,
+ )
+
+ expect_disconnected_handshake(sock)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
+
+
+@pytest.mark.parametrize(
+ "sasl_err,resp_type,resp_payload,expected_error",
+ [
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.ErrorResponse,
+ dict(
+ fields=[b"SFATAL", b"C28000", b"Mexpected error message", b""],
+ ),
+ "expected error message",
+ id="standard server error: invalid_request",
+ ),
+ pytest.param(
+ {"status": "invalid_token"},
+ pq3.types.ErrorResponse,
+ dict(
+ fields=[b"SFATAL", b"C28000", b"Mexpected error message", b""],
+ ),
+ "expected error message",
+ id="standard server error: invalid_token without discovery URI",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASLContinue, body=b""),
+ "server sent additional OAuth data",
+ id="broken server: additional challenge after error",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASLFinal),
+ "server sent additional OAuth data",
+ id="broken server: SASL success after error",
+ ),
+ pytest.param(
+ {"status": "invalid_request"},
+ pq3.types.AuthnRequest,
+ dict(type=pq3.authn.SASL, body=[b"OAUTHBEARER", b""]),
+ "duplicate SASL authentication request",
+ id="broken server: SASL reinitialization after error",
+ ),
+ ],
+)
+def test_oauth_server_error(accept, sasl_err, resp_type, resp_payload, expected_error):
+ sock, client = accept()
+
+ with sock:
+ with pq3.wrap(sock, debug_stream=sys.stdout) as conn:
+ start_oauth_handshake(conn)
+
+ # Ignore the client data. Return an error "challenge".
+ resp = json.dumps(sasl_err)
+ resp = resp.encode("utf-8")
+
+ pq3.send(
+ conn, pq3.types.AuthnRequest, type=pq3.authn.SASLContinue, body=resp
+ )
+
+ # Per RFC, the client is required to send a dummy ^A response.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.PasswordMessage
+ assert pkt.payload == b"\x01"
+
+ # Now fail the SASL exchange (in either a valid way, or an invalid
+ # one, depending on the test).
+ pq3.send(conn, resp_type, **resp_payload)
+
+ with pytest.raises(psycopg2.OperationalError, match=expected_error):
+ client.check_completed()
diff --git a/src/test/python/conftest.py b/src/test/python/conftest.py
new file mode 100644
index 0000000000..1a73865ee4
--- /dev/null
+++ b/src/test/python/conftest.py
@@ -0,0 +1,34 @@
+#
+# Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import os
+
+import pytest
+
+
+def pytest_addoption(parser):
+ """
+ Adds custom command line options to py.test. We add one to signal temporary
+ Postgres instance creation for the server tests.
+
+ Per pytest documentation, this must live in the top level test directory.
+ """
+ parser.addoption(
+ "--temp-instance",
+ metavar="DIR",
+ help="create a temporary Postgres instance in DIR",
+ )
+
+
+@pytest.fixture(scope="session", autouse=True)
+def _check_PG_TEST_EXTRA(request):
+ """
+ Automatically skips the whole suite if PG_TEST_EXTRA doesn't contain
+ 'python'. pytestmark doesn't seem to work in a top-level conftest.py, so
+ I've made this an autoused fixture instead.
+ """
+ extra_tests = os.getenv("PG_TEST_EXTRA", "").split()
+ if "python" not in extra_tests:
+ pytest.skip("Potentially unsafe test 'python' not enabled in PG_TEST_EXTRA")
diff --git a/src/test/python/meson.build b/src/test/python/meson.build
new file mode 100644
index 0000000000..045526a43a
--- /dev/null
+++ b/src/test/python/meson.build
@@ -0,0 +1,46 @@
+# Copyright (c) 2023, PostgreSQL Global Development Group
+
+subdir('server')
+
+pytest_env = {
+ 'with_oauth': oauth_library,
+
+ # Point to the default database; the tests will create their own databases as
+ # needed.
+ 'PGDATABASE': 'postgres',
+
+ # Avoid the need for a Rust compiler on platforms without prebuilt wheels for
+ # pyca/cryptography.
+ 'CRYPTOGRAPHY_DONT_BUILD_RUST': '1',
+}
+
+# Some modules (psycopg2) need OpenSSL at compile time; for platforms where we
+# might have multiple implementations installed (macOS+brew), try to use the
+# same one that libpq is using.
+if ssl.found()
+ pytest_incdir = ssl.get_variable(pkgconfig: 'includedir', default_value: '')
+ if pytest_incdir != ''
+ pytest_env += { 'CPPFLAGS': '-I@0@'.format(pytest_incdir) }
+ endif
+
+ pytest_libdir = ssl.get_variable(pkgconfig: 'libdir', default_value: '')
+ if pytest_libdir != ''
+ pytest_env += { 'LDFLAGS': '-L@0@'.format(pytest_libdir) }
+ endif
+endif
+
+tests += {
+ 'name': 'python',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'pytest': {
+ 'requirements': meson.current_source_dir() / 'requirements.txt',
+ 'tests': [
+ './client',
+ './server',
+ './test_internals.py',
+ './test_pq3.py',
+ ],
+ 'env': pytest_env,
+ },
+}
diff --git a/src/test/python/pq3.py b/src/test/python/pq3.py
new file mode 100644
index 0000000000..ef809e288a
--- /dev/null
+++ b/src/test/python/pq3.py
@@ -0,0 +1,740 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import getpass
+import io
+import os
+import platform
+import ssl
+import sys
+import textwrap
+
+from construct import *
+
+import tls
+
+
+def protocol(major, minor):
+ """
+ Returns the protocol version, in integer format, corresponding to the given
+ major and minor version numbers.
+ """
+ return (major << 16) | minor
+
+
+# Startup
+
+StringList = GreedyRange(NullTerminated(GreedyBytes))
+
+
+class KeyValueAdapter(Adapter):
+ """
+ Turns a key-value store into a null-terminated list of null-terminated
+ strings, as presented on the wire in the startup packet.
+ """
+
+ def _encode(self, obj, context, path):
+ if isinstance(obj, list):
+ return obj
+
+ l = []
+
+ for k, v in obj.items():
+ if isinstance(k, str):
+ k = k.encode("utf-8")
+ l.append(k)
+
+ if isinstance(v, str):
+ v = v.encode("utf-8")
+ l.append(v)
+
+ l.append(b"")
+ return l
+
+ def _decode(self, obj, context, path):
+ # TODO: turn a list back into a dict
+ return obj
+
+
+KeyValues = KeyValueAdapter(StringList)
+
+_startup_payload = Switch(
+ this.proto,
+ {
+ protocol(3, 0): KeyValues,
+ },
+ default=GreedyBytes,
+)
+
+
+def _default_protocol(this):
+ try:
+ if isinstance(this.payload, (list, dict)):
+ return protocol(3, 0)
+ except AttributeError:
+ pass # no payload passed during build
+
+ return 0
+
+
+def _startup_payload_len(this):
+ """
+ The payload field has a fixed size based on the length of the packet. But
+ if the caller hasn't supplied an explicit length at build time, we have to
+ build the payload to figure out how long it is, which requires us to know
+ the length first... This function exists solely to break the cycle.
+ """
+ assert this._building, "_startup_payload_len() cannot be called during parsing"
+
+ try:
+ payload = this.payload
+ except AttributeError:
+ return 0 # no payload
+
+ if isinstance(payload, bytes):
+ # already serialized; just use the given length
+ return len(payload)
+
+ try:
+ proto = this.proto
+ except AttributeError:
+ proto = _default_protocol(this)
+
+ data = _startup_payload.build(payload, proto=proto)
+ return len(data)
+
+
+Startup = Struct(
+ "len" / Default(Int32sb, lambda this: _startup_payload_len(this) + 8),
+ "proto" / Default(Hex(Int32sb), _default_protocol),
+ "payload" / FixedSized(this.len - 8, Default(_startup_payload, b"")),
+)
+
+# Pq3
+
+
+# Adapted from construct.core.EnumIntegerString
+class EnumNamedByte:
+ def __init__(self, val, name):
+ self._val = val
+ self._name = name
+
+ def __int__(self):
+ return ord(self._val)
+
+ def __str__(self):
+ return "(enum) %s %r" % (self._name, self._val)
+
+ def __repr__(self):
+ return "EnumNamedByte(%r)" % self._val
+
+ def __eq__(self, other):
+ if isinstance(other, EnumNamedByte):
+ other = other._val
+ if not isinstance(other, bytes):
+ return NotImplemented
+
+ return self._val == other
+
+ def __hash__(self):
+ return hash(self._val)
+
+
+# Adapted from construct.core.Enum
+class ByteEnum(Adapter):
+ def __init__(self, **mapping):
+ super(ByteEnum, self).__init__(Byte)
+ self.namemapping = {k: EnumNamedByte(v, k) for k, v in mapping.items()}
+ self.decmapping = {v: EnumNamedByte(v, k) for k, v in mapping.items()}
+
+ def __getattr__(self, name):
+ if name in self.namemapping:
+ return self.decmapping[self.namemapping[name]]
+ raise AttributeError
+
+ def _decode(self, obj, context, path):
+ b = bytes([obj])
+ try:
+ return self.decmapping[b]
+ except KeyError:
+ return EnumNamedByte(b, "(unknown)")
+
+ def _encode(self, obj, context, path):
+ if isinstance(obj, int):
+ return obj
+ elif isinstance(obj, bytes):
+ return ord(obj)
+ return int(obj)
+
+
+types = ByteEnum(
+ ErrorResponse=b"E",
+ ReadyForQuery=b"Z",
+ Query=b"Q",
+ EmptyQueryResponse=b"I",
+ AuthnRequest=b"R",
+ PasswordMessage=b"p",
+ BackendKeyData=b"K",
+ CommandComplete=b"C",
+ ParameterStatus=b"S",
+ DataRow=b"D",
+ Terminate=b"X",
+)
+
+
+authn = Enum(
+ Int32ub,
+ OK=0,
+ SASL=10,
+ SASLContinue=11,
+ SASLFinal=12,
+)
+
+
+_authn_body = Switch(
+ this.type,
+ {
+ authn.OK: Terminated,
+ authn.SASL: StringList,
+ },
+ default=GreedyBytes,
+)
+
+
+def _data_len(this):
+ assert this._building, "_data_len() cannot be called during parsing"
+
+ if not hasattr(this, "data") or this.data is None:
+ return -1
+
+ return len(this.data)
+
+
+# The protocol reuses the PasswordMessage for several authentication response
+# types, and there's no good way to figure out which is which without keeping
+# state for the entire stream. So this is a separate Construct that can be
+# explicitly parsed/built by code that knows it's needed.
+SASLInitialResponse = Struct(
+ "name" / NullTerminated(GreedyBytes),
+ "len" / Default(Int32sb, lambda this: _data_len(this)),
+ "data"
+ / IfThenElse(
+ # Allow tests to explicitly pass an incorrect length during testing, by
+ # not enforcing a FixedSized during build. (The len calculation above
+ # defaults to the correct size.)
+ this._building,
+ Optional(GreedyBytes),
+ If(this.len != -1, Default(FixedSized(this.len, GreedyBytes), b"")),
+ ),
+ Terminated, # make sure the entire response is consumed
+)
+
+
+_column = FocusedSeq(
+ "data",
+ "len" / Default(Int32sb, lambda this: _data_len(this)),
+ "data" / If(this.len != -1, FixedSized(this.len, GreedyBytes)),
+)
+
+
+_payload_map = {
+ types.ErrorResponse: Struct("fields" / StringList),
+ types.ReadyForQuery: Struct("status" / Bytes(1)),
+ types.Query: Struct("query" / NullTerminated(GreedyBytes)),
+ types.EmptyQueryResponse: Terminated,
+ types.AuthnRequest: Struct("type" / authn, "body" / Default(_authn_body, b"")),
+ types.BackendKeyData: Struct("pid" / Int32ub, "key" / Hex(Int32ub)),
+ types.CommandComplete: Struct("tag" / NullTerminated(GreedyBytes)),
+ types.ParameterStatus: Struct(
+ "name" / NullTerminated(GreedyBytes), "value" / NullTerminated(GreedyBytes)
+ ),
+ types.DataRow: Struct("columns" / Default(PrefixedArray(Int16sb, _column), b"")),
+ types.Terminate: Terminated,
+}
+
+
+_payload = FocusedSeq(
+ "_payload",
+ "_payload"
+ / Switch(
+ this._.type,
+ _payload_map,
+ default=GreedyBytes,
+ ),
+ Terminated, # make sure every payload consumes the entire packet
+)
+
+
+def _payload_len(this):
+ """
+ See _startup_payload_len() for an explanation.
+ """
+ assert this._building, "_payload_len() cannot be called during parsing"
+
+ try:
+ payload = this.payload
+ except AttributeError:
+ return 0 # no payload
+
+ if isinstance(payload, bytes):
+ # already serialized; just use the given length
+ return len(payload)
+
+ data = _payload.build(payload, type=this.type)
+ return len(data)
+
+
+Pq3 = Struct(
+ "type" / types,
+ "len" / Default(Int32ub, lambda this: _payload_len(this) + 4),
+ "payload"
+ / IfThenElse(
+ # Allow tests to explicitly pass an incorrect length during testing, by
+ # not enforcing a FixedSized during build. (The len calculation above
+ # defaults to the correct size.)
+ this._building,
+ Optional(_payload),
+ FixedSized(this.len - 4, Default(_payload, b"")),
+ ),
+)
+
+
+# Environment
+
+
+def pghost():
+ return os.environ.get("PGHOST", default="localhost")
+
+
+def pgport():
+ return int(os.environ.get("PGPORT", default=5432))
+
+
+def pguser():
+ try:
+ return os.environ["PGUSER"]
+ except KeyError:
+ if platform.system() == "Windows":
+ # libpq defaults to GetUserName() on Windows.
+ return os.getlogin()
+ return getpass.getuser()
+
+
+def pgdatabase():
+ return os.environ.get("PGDATABASE", default="postgres")
+
+
+# Connections
+
+
+def _hexdump_translation_map():
+ """
+ For hexdumps. Translates any unprintable or non-ASCII bytes into '.'.
+ """
+ input = bytearray()
+
+ for i in range(128):
+ c = chr(i)
+
+ if not c.isprintable():
+ input += bytes([i])
+
+ input += bytes(range(128, 256))
+
+ return bytes.maketrans(input, b"." * len(input))
+
+
+class _DebugStream(object):
+ """
+ Wraps a file-like object and adds hexdumps of the read and write data. Call
+ end_packet() on a _DebugStream to write the accumulated hexdumps to the
+ output stream, along with the packet that was sent.
+ """
+
+ _translation_map = _hexdump_translation_map()
+
+ def __init__(self, stream, out=sys.stdout):
+ """
+ Creates a new _DebugStream wrapping the given stream (which must have
+ been created by wrap()). All attributes not provided by the _DebugStream
+ are delegated to the wrapped stream. out is the text stream to which
+ hexdumps are written.
+ """
+ self.raw = stream
+ self._out = out
+ self._rbuf = io.BytesIO()
+ self._wbuf = io.BytesIO()
+
+ def __getattr__(self, name):
+ return getattr(self.raw, name)
+
+ def __setattr__(self, name, value):
+ if name in ("raw", "_out", "_rbuf", "_wbuf"):
+ return object.__setattr__(self, name, value)
+
+ setattr(self.raw, name, value)
+
+ def read(self, *args, **kwargs):
+ buf = self.raw.read(*args, **kwargs)
+
+ self._rbuf.write(buf)
+ return buf
+
+ def write(self, b):
+ self._wbuf.write(b)
+ return self.raw.write(b)
+
+ def recv(self, *args):
+ buf = self.raw.recv(*args)
+
+ self._rbuf.write(buf)
+ return buf
+
+ def _flush(self, buf, prefix):
+ width = 16
+ hexwidth = width * 3 - 1
+
+ count = 0
+ buf.seek(0)
+
+ while True:
+ line = buf.read(16)
+
+ if not line:
+ if count:
+ self._out.write("\n") # separate the output block with a newline
+ return
+
+ self._out.write("%s %04X:\t" % (prefix, count))
+ self._out.write("%*s\t" % (-hexwidth, line.hex(" ")))
+ self._out.write(line.translate(self._translation_map).decode("ascii"))
+ self._out.write("\n")
+
+ count += 16
+
+ def print_debug(self, obj, *, prefix=""):
+ contents = ""
+ if obj is not None:
+ contents = str(obj)
+
+ for line in contents.splitlines():
+ self._out.write("%s%s\n" % (prefix, line))
+
+ self._out.write("\n")
+
+ def flush_debug(self, *, prefix=""):
+ self._flush(self._rbuf, prefix + "<")
+ self._rbuf = io.BytesIO()
+
+ self._flush(self._wbuf, prefix + ">")
+ self._wbuf = io.BytesIO()
+
+ def end_packet(self, pkt, *, read=False, prefix="", indent=" "):
+ """
+ Marks the end of a logical "packet" of data. A string representation of
+ pkt will be printed, and the debug buffers will be flushed with an
+ indent. All lines can be optionally prefixed.
+
+ If read is True, the packet representation is written after the debug
+ buffers; otherwise the default of False (meaning write) causes the
+ packet representation to be dumped first. This is meant to capture the
+ logical flow of layer translation.
+ """
+ write = not read
+
+ if write:
+ self.print_debug(pkt, prefix=prefix + "> ")
+
+ self.flush_debug(prefix=prefix + indent)
+
+ if read:
+ self.print_debug(pkt, prefix=prefix + "< ")
+
+
+@contextlib.contextmanager
+def wrap(socket, *, debug_stream=None):
+ """
+ Transforms a raw socket into a connection that can be used for Construct
+ building and parsing. The return value is a context manager and can be used
+ in a with statement.
+ """
+ # It is critical that buffering be disabled here, so that we can still
+ # manipulate the raw socket without desyncing the stream.
+ with socket.makefile("rwb", buffering=0) as sfile:
+ # Expose the original socket's recv() on the SocketIO object we return.
+ def recv(self, *args):
+ return socket.recv(*args)
+
+ sfile.recv = recv.__get__(sfile)
+
+ conn = sfile
+ if debug_stream:
+ conn = _DebugStream(conn, debug_stream)
+
+ try:
+ yield conn
+ finally:
+ if debug_stream:
+ conn.flush_debug(prefix="? ")
+
+
+def _send(stream, cls, obj):
+ debugging = hasattr(stream, "flush_debug")
+ out = io.BytesIO()
+
+ # Ideally we would build directly to the passed stream, but because we need
+ # to reparse the generated output for the debugging case, build to an
+ # intermediate BytesIO and send it instead.
+ cls.build_stream(obj, out)
+ buf = out.getvalue()
+
+ stream.write(buf)
+ if debugging:
+ pkt = cls.parse(buf)
+ stream.end_packet(pkt)
+
+ stream.flush()
+
+
+def send(stream, packet_type, payload_data=None, **payloadkw):
+ """
+ Sends a packet on the given pq3 connection. type is the pq3.types member
+ that should be assigned to the packet. If payload_data is given, it will be
+ used as the packet payload; otherwise the key/value pairs in payloadkw will
+ be the payload contents.
+ """
+ data = payloadkw
+
+ if payload_data is not None:
+ if payloadkw:
+ raise ValueError(
+ "payload_data and payload keywords may not be used simultaneously"
+ )
+
+ data = payload_data
+
+ _send(stream, Pq3, dict(type=packet_type, payload=data))
+
+
+def send_startup(stream, proto=None, **kwargs):
+ """
+ Sends a startup packet on the given pq3 connection. In most cases you should
+ use the handshake functions instead, which will do this for you.
+
+ By default, a protocol version 3 packet will be sent. This can be overridden
+ with the proto parameter.
+ """
+ pkt = {}
+
+ if proto is not None:
+ pkt["proto"] = proto
+ if kwargs:
+ pkt["payload"] = kwargs
+
+ _send(stream, Startup, pkt)
+
+
+def recv1(stream, *, cls=Pq3):
+ """
+ Receives a single pq3 packet from the given stream and returns it.
+ """
+ resp = cls.parse_stream(stream)
+
+ debugging = hasattr(stream, "flush_debug")
+ if debugging:
+ stream.end_packet(resp, read=True)
+
+ return resp
+
+
+def handshake(stream, **kwargs):
+ """
+ Performs a libpq v3 startup handshake. kwargs should contain the key/value
+ parameters to send to the server in the startup packet.
+ """
+ # Send our startup parameters.
+ send_startup(stream, **kwargs)
+
+ # Receive and dump packets until the server indicates it's ready for our
+ # first query.
+ while True:
+ resp = recv1(stream)
+ if resp is None:
+ raise RuntimeError("server closed connection during handshake")
+
+ if resp.type == types.ReadyForQuery:
+ return
+ elif resp.type == types.ErrorResponse:
+ raise RuntimeError(
+ f"received error response from peer: {resp.payload.fields!r}"
+ )
+
+
+# TLS
+
+
+class _TLSStream(object):
+ """
+ A file-like object that performs TLS encryption/decryption on a wrapped
+ stream. Differs from ssl.SSLSocket in that we have full visibility and
+ control over the TLS layer.
+ """
+
+ def __init__(self, stream, context):
+ self._stream = stream
+ self._debugging = hasattr(stream, "flush_debug")
+
+ self._in = ssl.MemoryBIO()
+ self._out = ssl.MemoryBIO()
+ self._ssl = context.wrap_bio(self._in, self._out)
+
+ def handshake(self):
+ try:
+ self._pump(lambda: self._ssl.do_handshake())
+ finally:
+ self._flush_debug(prefix="? ")
+
+ def read(self, *args):
+ return self._pump(lambda: self._ssl.read(*args))
+
+ def write(self, *args):
+ return self._pump(lambda: self._ssl.write(*args))
+
+ def _decode(self, buf):
+ """
+ Attempts to decode a buffer of TLS data into a packet representation
+ that can be printed.
+
+ TODO: handle buffers (and record fragments) that don't align with packet
+ boundaries.
+ """
+ end = len(buf)
+ bio = io.BytesIO(buf)
+
+ ret = io.StringIO()
+
+ while bio.tell() < end:
+ record = tls.Plaintext.parse_stream(bio)
+
+ if ret.tell() > 0:
+ ret.write("\n")
+ ret.write("[Record] ")
+ ret.write(str(record))
+ ret.write("\n")
+
+ if record.type == tls.ContentType.handshake:
+ record_cls = tls.Handshake
+ else:
+ continue
+
+ innerlen = len(record.fragment)
+ inner = io.BytesIO(record.fragment)
+
+ while inner.tell() < innerlen:
+ msg = record_cls.parse_stream(inner)
+
+ indented = "[Message] " + str(msg)
+ indented = textwrap.indent(indented, " ")
+
+ ret.write("\n")
+ ret.write(indented)
+ ret.write("\n")
+
+ return ret.getvalue()
+
+ def flush(self):
+ if not self._out.pending:
+ self._stream.flush()
+ return
+
+ buf = self._out.read()
+ self._stream.write(buf)
+
+ if self._debugging:
+ pkt = self._decode(buf)
+ self._stream.end_packet(pkt, prefix=" ")
+
+ self._stream.flush()
+
+ def _pump(self, operation):
+ while True:
+ try:
+ return operation()
+ except (ssl.SSLWantReadError, ssl.SSLWantWriteError) as e:
+ want = e
+ self._read_write(want)
+
+ def _recv(self, maxsize):
+ buf = self._stream.recv(4096)
+ if not buf:
+ self._in.write_eof()
+ return
+
+ self._in.write(buf)
+
+ if not self._debugging:
+ return
+
+ pkt = self._decode(buf)
+ self._stream.end_packet(pkt, read=True, prefix=" ")
+
+ def _read_write(self, want):
+ # XXX This needs work. So many corner cases yet to handle. For one,
+ # doing blocking writes in flush may lead to distributed deadlock if the
+ # peer is already blocking on its writes.
+
+ if isinstance(want, ssl.SSLWantWriteError):
+ assert self._out.pending, "SSL backend wants write without data"
+
+ self.flush()
+
+ if isinstance(want, ssl.SSLWantReadError):
+ self._recv(4096)
+
+ def _flush_debug(self, prefix):
+ if not self._debugging:
+ return
+
+ self._stream.flush_debug(prefix=prefix)
+
+
+@contextlib.contextmanager
+def tls_handshake(stream, context):
+ """
+ Performs a TLS handshake over the given stream (which must have been created
+ via a call to wrap()), and returns a new stream which transparently tunnels
+ data over the TLS connection.
+
+ If the passed stream has debugging enabled, the returned stream will also
+ have debugging, using the same output IO.
+ """
+ debugging = hasattr(stream, "flush_debug")
+
+ # Send our startup parameters.
+ send_startup(stream, proto=protocol(1234, 5679))
+
+ # Look at the SSL response.
+ resp = stream.read(1)
+ if debugging:
+ stream.flush_debug(prefix=" ")
+
+ if resp == b"N":
+ raise RuntimeError("server does not support SSLRequest")
+ if resp != b"S":
+ raise RuntimeError(f"unexpected response of type {resp!r} during TLS startup")
+
+ tls = _TLSStream(stream, context)
+ tls.handshake()
+
+ if debugging:
+ tls = _DebugStream(tls, stream._out)
+
+ try:
+ yield tls
+ # TODO: teardown/unwrap the connection?
+ finally:
+ if debugging:
+ tls.flush_debug(prefix="? ")
diff --git a/src/test/python/pytest.ini b/src/test/python/pytest.ini
new file mode 100644
index 0000000000..ab7a6e7fb9
--- /dev/null
+++ b/src/test/python/pytest.ini
@@ -0,0 +1,4 @@
+[pytest]
+
+markers =
+ slow: mark test as slow
diff --git a/src/test/python/requirements.txt b/src/test/python/requirements.txt
new file mode 100644
index 0000000000..57ba1ced94
--- /dev/null
+++ b/src/test/python/requirements.txt
@@ -0,0 +1,9 @@
+black
+# cryptography 35.x and later add many platform/toolchain restrictions, beware
+cryptography~=3.4.8
+construct~=2.10.61
+isort~=5.6
+# TODO: update to psycopg[c] 3.1
+psycopg2~=2.9.7
+pytest~=7.3
+pytest-asyncio~=0.21.0
diff --git a/src/test/python/server/__init__.py b/src/test/python/server/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/src/test/python/server/conftest.py b/src/test/python/server/conftest.py
new file mode 100644
index 0000000000..c89f0756ae
--- /dev/null
+++ b/src/test/python/server/conftest.py
@@ -0,0 +1,141 @@
+#
+# Portions Copyright 2021 VMware, Inc.
+# Portions Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import collections
+import contextlib
+import os
+import shutil
+import socket
+import subprocess
+import sys
+
+import pytest
+
+import pq3
+
+BLOCKING_TIMEOUT = 2 # the number of seconds to wait for blocking calls
+
+
+def cleanup_prior_instance(datadir):
+ """
+ Clean up an existing data directory, but make sure it actually looks like a
+ data directory first. (Empty folders will remain untouched, since initdb can
+ populate them.)
+ """
+ required_entries = set(["base", "PG_VERSION", "postgresql.conf"])
+ empty = True
+
+ try:
+ with os.scandir(datadir) as entries:
+ for e in entries:
+ empty = False
+ required_entries.discard(e.name)
+
+ except FileNotFoundError:
+ return # nothing to clean up
+
+ if empty:
+ return # initdb can handle an empty datadir
+
+ if required_entries:
+ pytest.fail(
+ f"--temp-instance directory \"{datadir}\" is not empty and doesn't look like a data directory (missing {', '.join(required_entries)})"
+ )
+
+ # Okay, seems safe enough now.
+ shutil.rmtree(datadir)
+
+
+@pytest.fixture(scope="session")
+def postgres_instance(pytestconfig, unused_tcp_port_factory):
+ """
+ If --temp-instance has been passed to pytest, this fixture runs a temporary
+ Postgres instance on an available port. Otherwise, the fixture will attempt
+ to contact a running Postgres server on (PGHOST, PGPORT); dependent tests
+ will be skipped if the connection fails.
+
+ Yields a (host, port) tuple for connecting to the server.
+ """
+ PGInstance = collections.namedtuple("PGInstance", ["addr", "temporary"])
+
+ datadir = pytestconfig.getoption("temp_instance")
+ if datadir:
+ # We were told to create a temporary instance. Use pg_ctl to set it up
+ # on an unused port.
+ cleanup_prior_instance(datadir)
+ subprocess.run(["pg_ctl", "-D", datadir, "init"], check=True)
+
+ # The CI looks for *.log files to upload, so the file name here isn't
+ # completely arbitrary.
+ log = os.path.join(datadir, "postmaster.log")
+ port = unused_tcp_port_factory()
+
+ subprocess.run(
+ [
+ "pg_ctl",
+ "-D",
+ datadir,
+ "-l",
+ log,
+ "-o",
+ " ".join(
+ [
+ f"-c port={port}",
+ "-c listen_addresses=localhost",
+ "-c log_connections=on",
+ "-c shared_preload_libraries=oauthtest",
+ "-c oauth_validator_library=oauthtest",
+ ]
+ ),
+ "start",
+ ],
+ check=True,
+ )
+
+ yield ("localhost", port)
+
+ subprocess.run(["pg_ctl", "-D", datadir, "stop"], check=True)
+
+ else:
+ # Try to contact an already running server; skip the suite if we can't
+ # find one.
+ addr = (pq3.pghost(), pq3.pgport())
+
+ try:
+ with socket.create_connection(addr, timeout=BLOCKING_TIMEOUT):
+ pass
+ except ConnectionError as e:
+ pytest.skip(f"unable to connect to Postgres server at {addr}: {e}")
+
+ yield addr
+
+
+@pytest.fixture
+def connect(postgres_instance):
+ """
+ A factory fixture that, when called, returns a socket connected to a
+ Postgres server, wrapped in a pq3 connection. Dependent tests will be
+ skipped if no server is available.
+ """
+ addr = postgres_instance
+
+ # Set up an ExitStack to handle safe cleanup of all of the moving pieces.
+ with contextlib.ExitStack() as stack:
+
+ def conn_factory():
+ sock = socket.create_connection(addr, timeout=BLOCKING_TIMEOUT)
+
+ # Have ExitStack close our socket.
+ stack.enter_context(sock)
+
+ # Wrap the connection in a pq3 layer and have ExitStack clean it up
+ # too.
+ wrap_ctx = pq3.wrap(sock, debug_stream=sys.stdout)
+ conn = stack.enter_context(wrap_ctx)
+
+ return conn
+
+ yield conn_factory
diff --git a/src/test/python/server/meson.build b/src/test/python/server/meson.build
new file mode 100644
index 0000000000..85534b9cc9
--- /dev/null
+++ b/src/test/python/server/meson.build
@@ -0,0 +1,18 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+oauthtest_sources = files(
+ 'oauthtest.c',
+)
+
+if host_system == 'windows'
+ oauthtest_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'oauthtest',
+ '--FILEDESC', 'passthrough module to validate OAuth tests',
+ ])
+endif
+
+oauthtest = shared_module('oauthtest',
+ oauthtest_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += oauthtest
diff --git a/src/test/python/server/oauthtest.c b/src/test/python/server/oauthtest.c
new file mode 100644
index 0000000000..dbb8b8823c
--- /dev/null
+++ b/src/test/python/server/oauthtest.c
@@ -0,0 +1,118 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauthtest.c
+ * Test module for serverside OAuth token validation callbacks
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/test/python/server/oauthtest.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "libpq/oauth.h"
+#include "utils/guc.h"
+#include "utils/memutils.h"
+
+PG_MODULE_MAGIC;
+
+static void test_startup(ValidatorModuleState *state);
+static void test_shutdown(ValidatorModuleState *state);
+static ValidatorModuleResult * test_validate(ValidatorModuleState *state,
+ const char *token,
+ const char *role);
+
+static const OAuthValidatorCallbacks callbacks = {
+ .startup_cb = test_startup,
+ .shutdown_cb = test_shutdown,
+ .validate_cb = test_validate,
+};
+
+static char *expected_bearer = "";
+static bool set_authn_id = false;
+static char *authn_id = "";
+static bool reflect_role = false;
+
+void
+_PG_init(void)
+{
+ DefineCustomStringVariable("oauthtest.expected_bearer",
+ "Expected Bearer token for future connections",
+ NULL,
+ &expected_bearer,
+ "",
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("oauthtest.set_authn_id",
+ "Whether to set an authenticated identity",
+ NULL,
+ &set_authn_id,
+ false,
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+ DefineCustomStringVariable("oauthtest.authn_id",
+ "Authenticated identity to use for future connections",
+ NULL,
+ &authn_id,
+ "",
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+
+ DefineCustomBoolVariable("oauthtest.reflect_role",
+ "Ignore the bearer token; use the requested role as the authn_id",
+ NULL,
+ &reflect_role,
+ false,
+ PGC_SIGHUP,
+ 0,
+ NULL, NULL, NULL);
+
+ MarkGUCPrefixReserved("oauthtest");
+}
+
+const OAuthValidatorCallbacks *
+_PG_oauth_validator_module_init(void)
+{
+ return &callbacks;
+}
+
+static void
+test_startup(ValidatorModuleState *state)
+{
+}
+
+static void
+test_shutdown(ValidatorModuleState *state)
+{
+}
+
+static ValidatorModuleResult *
+test_validate(ValidatorModuleState *state, const char *token, const char *role)
+{
+ ValidatorModuleResult *res;
+
+ res = palloc0(sizeof(ValidatorModuleResult)); /* TODO: palloc context? */
+
+ if (reflect_role)
+ {
+ res->authorized = true;
+ res->authn_id = pstrdup(role); /* TODO: constify? */
+ }
+ else
+ {
+ if (*expected_bearer && !strcmp(token, expected_bearer))
+ res->authorized = true;
+ if (set_authn_id)
+ res->authn_id = authn_id;
+ }
+
+ return res;
+}
diff --git a/src/test/python/server/test_oauth.py b/src/test/python/server/test_oauth.py
new file mode 100644
index 0000000000..427ab063e6
--- /dev/null
+++ b/src/test/python/server/test_oauth.py
@@ -0,0 +1,1074 @@
+#
+# Copyright 2021 VMware, Inc.
+# Portions Copyright 2023 Timescale, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import base64
+import contextlib
+import json
+import os
+import pathlib
+import platform
+import secrets
+import shlex
+import shutil
+import socket
+import struct
+from multiprocessing import shared_memory
+
+import psycopg2
+import pytest
+from construct import Container
+from psycopg2 import sql
+
+import pq3
+
+from .conftest import BLOCKING_TIMEOUT
+
+MAX_SASL_MESSAGE_LENGTH = 65535
+
+INVALID_AUTHORIZATION_ERRCODE = b"28000"
+PROTOCOL_VIOLATION_ERRCODE = b"08P01"
+FEATURE_NOT_SUPPORTED_ERRCODE = b"0A000"
+
+SHARED_MEM_NAME = "oauth-pytest"
+MAX_UINT16 = 2**16 - 1
+
+
+@contextlib.contextmanager
+def prepend_file(path, lines, *, suffix=".bak"):
+ """
+ A context manager that prepends a file on disk with the desired lines of
+ text. When the context manager is exited, the file will be restored to its
+ original contents.
+ """
+ # First make a backup of the original file.
+ bak = path + suffix
+ shutil.copy2(path, bak)
+
+ try:
+ # Write the new lines, followed by the original file content.
+ with open(path, "w") as new, open(bak, "r") as orig:
+ new.writelines(lines)
+ shutil.copyfileobj(orig, new)
+
+ # Return control to the calling code.
+ yield
+
+ finally:
+ # Put the backup back into place.
+ os.replace(bak, path)
+
+
+@pytest.fixture(scope="module")
+def oauth_ctx(postgres_instance):
+ """
+ Creates a database and user that use the oauth auth method. The context
+ object contains the dbname and user attributes as strings to be used during
+ connection, as well as the issuer and scope that have been set in the HBA
+ configuration.
+
+ This fixture assumes that the standard PG* environment variables point to a
+ server running on a local machine, and that the PGUSER has rights to create
+ databases and roles.
+ """
+ id = secrets.token_hex(4)
+
+ class Context:
+ dbname = "oauth_test_" + id
+
+ user = "oauth_user_" + id
+ punct_user = "oauth_\"'? ;&!_user_" + id # username w/ punctuation
+ map_user = "oauth_map_user_" + id
+ authz_user = "oauth_authz_user_" + id
+
+ issuer = "https://example.com/" + id
+ scope = "openid " + id
+
+ ctx = Context()
+ hba_lines = [
+ f'host {ctx.dbname} {ctx.map_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" map=oauth\n',
+ f'host {ctx.dbname} {ctx.authz_user} samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}" trust_validator_authz=1\n',
+ f'host {ctx.dbname} all samehost oauth issuer="{ctx.issuer}" scope="{ctx.scope}"\n',
+ ]
+ ident_lines = [r"oauth /^(.*)@example\.com$ \1"]
+
+ if platform.system() == "Windows":
+ # XXX why is 'samehost' not behaving as expected on Windows?
+ for l in list(hba_lines):
+ hba_lines.append(l.replace("samehost", "::1/128"))
+
+ host, port = postgres_instance
+ conn = psycopg2.connect(host=host, port=port)
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+
+ # Create our roles and database.
+ user = sql.Identifier(ctx.user)
+ punct_user = sql.Identifier(ctx.punct_user)
+ map_user = sql.Identifier(ctx.map_user)
+ authz_user = sql.Identifier(ctx.authz_user)
+ dbname = sql.Identifier(ctx.dbname)
+
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(punct_user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(map_user))
+ c.execute(sql.SQL("CREATE ROLE {} LOGIN;").format(authz_user))
+ c.execute(sql.SQL("CREATE DATABASE {};").format(dbname))
+
+ # Replace pg_hba and pg_ident.
+ c.execute("SHOW hba_file;")
+ hba = c.fetchone()[0]
+
+ c.execute("SHOW ident_file;")
+ ident = c.fetchone()[0]
+
+ with prepend_file(hba, hba_lines), prepend_file(ident, ident_lines):
+ c.execute("SELECT pg_reload_conf();")
+
+ # Use the new database and user.
+ yield ctx
+
+ # Put things back the way they were.
+ c.execute("SELECT pg_reload_conf();")
+
+ c.execute(sql.SQL("DROP DATABASE {};").format(dbname))
+ c.execute(sql.SQL("DROP ROLE {};").format(authz_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(map_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(punct_user))
+ c.execute(sql.SQL("DROP ROLE {};").format(user))
+
+
+@pytest.fixture()
+def conn(oauth_ctx, connect):
+ """
+ A convenience wrapper for connect(). The main purpose of this fixture is to
+ make sure oauth_ctx runs its setup code before the connection is made.
+ """
+ return connect()
+
+
+def bearer_token(*, size=16):
+ """
+ Generates a Bearer token using secrets.token_urlsafe(). The generated token
+ size in bytes may be specified; if unset, a small 16-byte token will be
+ generated.
+ """
+
+ if size % 4:
+ raise ValueError(f"requested token size {size} is not a multiple of 4")
+
+ token = secrets.token_urlsafe(size // 4 * 3)
+ assert len(token) == size
+
+ return token
+
+
+def begin_oauth_handshake(conn, oauth_ctx, *, user=None):
+ if user is None:
+ user = oauth_ctx.authz_user
+
+ pq3.send_startup(conn, user=user, database=oauth_ctx.dbname)
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.AuthnRequest
+
+ # The server should advertise exactly one mechanism.
+ assert resp.payload.type == pq3.authn.SASL
+ assert resp.payload.body == [b"OAUTHBEARER", b""]
+
+
+def send_initial_response(conn, *, auth=None, bearer=None):
+ """
+ Sends the OAUTHBEARER initial response on the connection, using the given
+ bearer token. Alternatively to a bearer token, the initial response's auth
+ field may be explicitly specified to test corner cases.
+ """
+ if bearer is not None and auth is not None:
+ raise ValueError("exactly one of the auth and bearer kwargs must be set")
+
+ if bearer is not None:
+ auth = b"Bearer " + bearer
+
+ if auth is None:
+ raise ValueError("exactly one of the auth and bearer kwargs must be set")
+
+ initial = pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"n,,\x01auth=" + auth + b"\x01\x01",
+ )
+ )
+ pq3.send(conn, pq3.types.PasswordMessage, initial)
+
+
+def expect_handshake_success(conn):
+ """
+ Validates that the server responds with an AuthnOK message, and then drains
+ the connection until a ReadyForQuery message is received.
+ """
+ resp = pq3.recv1(conn)
+
+ assert resp.type == pq3.types.AuthnRequest
+ assert resp.payload.type == pq3.authn.OK
+ assert not resp.payload.body
+
+ receive_until(conn, pq3.types.ReadyForQuery)
+
+
+def expect_handshake_failure(conn, oauth_ctx):
+ """
+ Performs the OAUTHBEARER SASL failure "handshake" and validates the server's
+ side of the conversation, including the final ErrorResponse.
+ """
+
+ # We expect a discovery "challenge" back from the server before the authn
+ # failure message.
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.AuthnRequest
+
+ req = resp.payload
+ assert req.type == pq3.authn.SASLContinue
+
+ body = json.loads(req.body)
+ assert body["status"] == "invalid_token"
+ assert body["scope"] == oauth_ctx.scope
+
+ expected_config = oauth_ctx.issuer + "/.well-known/openid-configuration"
+ assert body["openid-configuration"] == expected_config
+
+ # Send the dummy response to complete the failed handshake.
+ pq3.send(conn, pq3.types.PasswordMessage, b"\x01")
+ resp = pq3.recv1(conn)
+
+ err = ExpectedError(INVALID_AUTHORIZATION_ERRCODE, "bearer authentication failed")
+ err.match(resp)
+
+
+def receive_until(conn, type):
+ """
+ receive_until pulls packets off the pq3 connection until a packet with the
+ desired type is found, or an error response is received.
+ """
+ while True:
+ pkt = pq3.recv1(conn)
+
+ if pkt.type == type:
+ return pkt
+ elif pkt.type == pq3.types.ErrorResponse:
+ raise RuntimeError(
+ f"received error response from peer: {pkt.payload.fields!r}"
+ )
+
+
+@pytest.fixture()
+def setup_validator(postgres_instance):
+ """
+ A per-test fixture that sets up the test validator with expected behavior.
+ The setting will be reverted during teardown.
+ """
+ host, port = postgres_instance
+ conn = psycopg2.connect(host=host, port=port)
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+ prev = dict()
+
+ def setter(**gucs):
+ for guc, val in gucs.items():
+ # Save the previous value.
+ c.execute(sql.SQL("SHOW oauthtest.{};").format(sql.Identifier(guc)))
+ prev[guc] = c.fetchone()[0]
+
+ c.execute(
+ sql.SQL("ALTER SYSTEM SET oauthtest.{} TO %s;").format(
+ sql.Identifier(guc)
+ ),
+ (val,),
+ )
+ c.execute("SELECT pg_reload_conf();")
+
+ yield setter
+
+ # Restore the previous values.
+ for guc, val in prev.items():
+ c.execute(
+ sql.SQL("ALTER SYSTEM SET oauthtest.{} TO %s;").format(
+ sql.Identifier(guc)
+ ),
+ (val,),
+ )
+ c.execute("SELECT pg_reload_conf();")
+
+
+@pytest.mark.parametrize("token_len", [16, 1024, 4096])
+@pytest.mark.parametrize(
+ "auth_prefix",
+ [
+ b"Bearer ",
+ b"bearer ",
+ b"Bearer ",
+ ],
+)
+def test_oauth(setup_validator, connect, oauth_ctx, auth_prefix, token_len):
+ # Generate our bearer token with the desired length.
+ token = bearer_token(size=token_len)
+ setup_validator(expected_bearer=token)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ auth = auth_prefix + token.encode("ascii")
+ send_initial_response(conn, auth=auth)
+ expect_handshake_success(conn)
+
+ # Make sure that the server has not set an authenticated ID.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT system_user;")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ assert row.columns == [None]
+
+
+@pytest.mark.parametrize(
+ "token_value",
+ [
+ "abcdzA==",
+ "123456M=",
+ "x-._~+/x",
+ ],
+)
+def test_oauth_bearer_corner_cases(setup_validator, connect, oauth_ctx, token_value):
+ setup_validator(expected_bearer=token_value)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ send_initial_response(conn, bearer=token_value.encode("ascii"))
+
+ expect_handshake_success(conn)
+
+
+@pytest.mark.parametrize(
+ "user,authn_id,should_succeed",
+ [
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: ctx.user,
+ True,
+ id="validator authn: succeeds when authn_id == username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: None,
+ False,
+ id="validator authn: fails when authn_id is not set",
+ ),
+ pytest.param(
+ lambda ctx: ctx.user,
+ lambda ctx: ctx.authz_user,
+ False,
+ id="validator authn: fails when authn_id != username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: ctx.map_user + "@example.com",
+ True,
+ id="validator with map: succeeds when authn_id matches map",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: None,
+ False,
+ id="validator with map: fails when authn_id is not set",
+ ),
+ pytest.param(
+ lambda ctx: ctx.map_user,
+ lambda ctx: ctx.map_user + "@example.net",
+ False,
+ id="validator with map: fails when authn_id doesn't match map",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: None,
+ True,
+ id="validator authz: succeeds with no authn_id",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "",
+ True,
+ id="validator authz: succeeds with empty authn_id",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "postgres",
+ True,
+ id="validator authz: succeeds with basic username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.authz_user,
+ lambda ctx: "me@example.com",
+ True,
+ id="validator authz: succeeds with email address",
+ ),
+ ],
+)
+def test_oauth_authn_id(
+ setup_validator, connect, oauth_ctx, user, authn_id, should_succeed
+):
+ token = bearer_token()
+ authn_id = authn_id(oauth_ctx)
+
+ # Set up the validator appropriately.
+ gucs = dict(expected_bearer=token)
+ if authn_id is not None:
+ gucs["set_authn_id"] = True
+ gucs["authn_id"] = authn_id
+ setup_validator(**gucs)
+
+ conn = connect()
+ username = user(oauth_ctx)
+ begin_oauth_handshake(conn, oauth_ctx, user=username)
+ send_initial_response(conn, bearer=token.encode("ascii"))
+
+ if not should_succeed:
+ expect_handshake_failure(conn, oauth_ctx)
+ return
+
+ expect_handshake_success(conn)
+
+ # Check the reported authn_id.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT system_user;")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ expected = authn_id
+ if expected is not None:
+ expected = b"oauth:" + expected.encode("ascii")
+
+ row = resp.payload
+ assert row.columns == [expected]
+
+
+class ExpectedError(object):
+ def __init__(self, code, msg=None, detail=None):
+ self.code = code
+ self.msg = msg
+ self.detail = detail
+
+ # Protect against the footgun of an accidental empty string, which will
+ # "match" anything. If you don't want to match message or detail, just
+ # don't pass them.
+ if self.msg == "":
+ raise ValueError("msg must be non-empty or None")
+ if self.detail == "":
+ raise ValueError("detail must be non-empty or None")
+
+ def _getfield(self, resp, type):
+ """
+ Searches an ErrorResponse for a single field of the given type (e.g.
+ "M", "C", "D") and returns its value. Asserts if it doesn't find exactly
+ one field.
+ """
+ prefix = type.encode("ascii")
+ fields = [f for f in resp.payload.fields if f.startswith(prefix)]
+
+ assert len(fields) == 1
+ return fields[0][1:] # strip off the type byte
+
+ def match(self, resp):
+ """
+ Checks that the given response matches the expected code, message, and
+ detail (if given). The error code must match exactly. The expected
+ message and detail must be contained within the actual strings.
+ """
+ assert resp.type == pq3.types.ErrorResponse
+
+ code = self._getfield(resp, "C")
+ assert code == self.code
+
+ if self.msg:
+ msg = self._getfield(resp, "M")
+ expected = self.msg.encode("utf-8")
+ assert expected in msg
+
+ if self.detail:
+ detail = self._getfield(resp, "D")
+ expected = self.detail.encode("utf-8")
+ assert expected in detail
+
+
+def test_oauth_rejected_bearer(conn, oauth_ctx):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send a bearer token that doesn't match what the validator expects. It
+ # should fail the connection.
+ send_initial_response(conn, bearer=b"xxxxxx")
+
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.parametrize(
+ "bad_bearer",
+ [
+ b"Bearer ",
+ b"Bearer a===b",
+ b"Bearer hello!",
+ b"Bearer trailingspace ",
+ b"Bearer trailingtab\t",
+ b"Bearer me@example.com",
+ b"Beare abcd",
+ b" Bearer leadingspace",
+ b'OAuth realm="Example"',
+ b"",
+ ],
+)
+def test_oauth_invalid_bearer(setup_validator, connect, oauth_ctx, bad_bearer):
+ # Tell the validator to accept any token. This ensures that the invalid
+ # bearer tokens are rejected before the validation step.
+ setup_validator(reflect_role=True)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+ send_initial_response(conn, auth=bad_bearer)
+
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.slow
+@pytest.mark.parametrize(
+ "resp_type,resp,err",
+ [
+ pytest.param(
+ None,
+ None,
+ None,
+ marks=pytest.mark.slow,
+ id="no response (expect timeout)",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"hello",
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not send a kvsep response",
+ ),
+ id="bad dummy response",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ b"\x01\x01",
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not send a kvsep response",
+ ),
+ id="multiple kvseps",
+ ),
+ pytest.param(
+ pq3.types.Query,
+ dict(query=b""),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "expected SASL response"),
+ id="bad response message type",
+ ),
+ ],
+)
+def test_oauth_bad_response_to_error_challenge(conn, oauth_ctx, resp_type, resp, err):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send an empty auth initial response, which will force an authn failure.
+ send_initial_response(conn, auth=b"")
+
+ # We expect a discovery "challenge" back from the server before the authn
+ # failure message.
+ pkt = pq3.recv1(conn)
+ assert pkt.type == pq3.types.AuthnRequest
+
+ req = pkt.payload
+ assert req.type == pq3.authn.SASLContinue
+
+ body = json.loads(req.body)
+ assert body["status"] == "invalid_token"
+
+ if resp_type is None:
+ # Do not send the dummy response. We should time out and not get a
+ # response from the server.
+ with pytest.raises(socket.timeout):
+ conn.read(1)
+
+ # Done with the test.
+ return
+
+ # Send the bad response.
+ pq3.send(conn, resp_type, resp)
+
+ # Make sure the server fails the connection correctly.
+ pkt = pq3.recv1(conn)
+ err.match(pkt)
+
+
+@pytest.mark.parametrize(
+ "type,payload,err",
+ [
+ pytest.param(
+ pq3.types.ErrorResponse,
+ dict(fields=[b""]),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "expected SASL response"),
+ id="error response in initial message",
+ ),
+ pytest.param(
+ None,
+ # Sending an actual 65k packet results in ECONNRESET on Windows, and
+ # it floods the tests' connection log uselessly, so just fake the
+ # length and send a smaller number of bytes.
+ dict(
+ type=pq3.types.PasswordMessage,
+ len=MAX_SASL_MESSAGE_LENGTH + 1,
+ payload=b"x" * 512,
+ ),
+ ExpectedError(
+ INVALID_AUTHORIZATION_ERRCODE, "bearer authentication failed"
+ ),
+ id="overlong initial response data",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"SCRAM-SHA-256")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE, "invalid SASL authentication mechanism"
+ ),
+ id="bad SASL mechanism selection",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", len=2, data=b"x")),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "insufficient data"),
+ id="SASL data underflow",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", len=0, data=b"x")),
+ ExpectedError(PROTOCOL_VIOLATION_ERRCODE, "invalid message format"),
+ id="SASL data overflow",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "message is empty",
+ ),
+ id="empty",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"n,,\x01auth=\x01\x01\0")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "length does not match input length",
+ ),
+ id="contains null byte",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected channel-binding flag", # XXX this is a bit strange
+ ),
+ id="initial error response",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"p=tls-server-end-point,,\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "server does not support channel binding",
+ ),
+ id="uses channel binding",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"x,,\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected channel-binding flag",
+ ),
+ id="invalid channel binding specifier",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Comma expected",
+ ),
+ id="bad GS2 header: missing channel binding terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,a")),
+ ExpectedError(
+ FEATURE_NOT_SUPPORTED_ERRCODE,
+ "client uses authorization identity",
+ ),
+ id="bad GS2 header: authzid in use",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,b,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected attribute",
+ ),
+ id="bad GS2 header: extra attribute",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Unexpected attribute 0x00", # XXX this is a bit strange
+ ),
+ id="bad GS2 header: missing authzid terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Key-value separator expected",
+ ),
+ id="missing initial kvsep",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "Key-value separator expected",
+ ),
+ id="missing initial kvsep",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "does not contain an auth value",
+ ),
+ id="missing auth value: empty key-value list",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01host=example.com\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "does not contain an auth value",
+ ),
+ id="missing auth value: other keys present",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01host=example.com")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "unterminated key/value pair",
+ ),
+ id="missing value terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER", data=b"y,,\x01")),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not contain a final terminator",
+ ),
+ id="missing list terminator: empty list",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01auth=Bearer 0\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "did not contain a final terminator",
+ ),
+ id="missing list terminator: with auth value",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01auth=Bearer 0\x01\x01blah")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "additional data after the final terminator",
+ ),
+ id="additional key after terminator",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(name=b"OAUTHBEARER", data=b"y,,\x01key\x01\x01")
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "key without a value",
+ ),
+ id="key without value",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"y,,\x01auth=Bearer 0\x01auth=Bearer 1\x01\x01",
+ )
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "contains multiple auth values",
+ ),
+ id="multiple auth values",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"y,,\x01=\x01\x01",
+ )
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "empty key name",
+ ),
+ id="empty key",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"y,,\x01my key= \x01\x01",
+ )
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "invalid key name",
+ ),
+ id="whitespace in key name",
+ ),
+ pytest.param(
+ pq3.types.PasswordMessage,
+ pq3.SASLInitialResponse.build(
+ dict(
+ name=b"OAUTHBEARER",
+ data=b"y,,\x01key=a\x05b\x01\x01",
+ )
+ ),
+ ExpectedError(
+ PROTOCOL_VIOLATION_ERRCODE,
+ "malformed OAUTHBEARER message",
+ "invalid value",
+ ),
+ id="junk in value",
+ ),
+ ],
+)
+def test_oauth_bad_initial_response(conn, oauth_ctx, type, payload, err):
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # The server expects a SASL response; give it something else instead.
+ if type is not None:
+ # Build a new packet of the desired type.
+ if not isinstance(payload, dict):
+ payload = dict(payload_data=payload)
+ pq3.send(conn, type, **payload)
+ else:
+ # The test has a custom packet to send. (The only reason to do this is
+ # if the packet is corrupt or otherwise unbuildable/unparsable, so we
+ # don't use the standard pq3.send().)
+ conn.write(pq3.Pq3.build(payload))
+ conn.end_packet(Container(payload))
+
+ resp = pq3.recv1(conn)
+ err.match(resp)
+
+
+def test_oauth_empty_initial_response(setup_validator, connect, oauth_ctx):
+ token = bearer_token()
+ setup_validator(expected_bearer=token)
+
+ conn = connect()
+ begin_oauth_handshake(conn, oauth_ctx)
+
+ # Send an initial response without data.
+ initial = pq3.SASLInitialResponse.build(dict(name=b"OAUTHBEARER"))
+ pq3.send(conn, pq3.types.PasswordMessage, initial)
+
+ # The server should respond with an empty challenge so we can send the data
+ # it wants.
+ pkt = pq3.recv1(conn)
+
+ assert pkt.type == pq3.types.AuthnRequest
+ assert pkt.payload.type == pq3.authn.SASLContinue
+ assert not pkt.payload.body
+
+ # Now send the initial data.
+ data = b"n,,\x01auth=Bearer " + token.encode("ascii") + b"\x01\x01"
+ pq3.send(conn, pq3.types.PasswordMessage, data)
+
+ # Server should now complete the handshake.
+ expect_handshake_success(conn)
+
+
+# TODO: see if there's a way to test this easily after the API switch
+def xtest_oauth_no_validator(setup_validator, oauth_ctx, connect):
+ # Clear out our validator command, then establish a new connection.
+ set_validator("")
+ conn = connect()
+
+ begin_oauth_handshake(conn, oauth_ctx)
+ send_initial_response(conn, bearer=bearer_token())
+
+ # The server should fail the connection.
+ expect_handshake_failure(conn, oauth_ctx)
+
+
+@pytest.mark.parametrize(
+ "user",
+ [
+ pytest.param(
+ lambda ctx: ctx.user,
+ id="basic username",
+ ),
+ pytest.param(
+ lambda ctx: ctx.punct_user,
+ id="'unsafe' characters are passed through correctly",
+ ),
+ ],
+)
+def test_oauth_validator_role(setup_validator, oauth_ctx, connect, user):
+ username = user(oauth_ctx)
+
+ # Tell the validator to reflect the PGUSER as the authenticated identity.
+ setup_validator(reflect_role=True)
+ conn = connect()
+
+ # Log in. Note that reflection ignores the bearer token.
+ begin_oauth_handshake(conn, oauth_ctx, user=username)
+ send_initial_response(conn, bearer=b"dontcare")
+ expect_handshake_success(conn)
+
+ # Check the user identity.
+ pq3.send(conn, pq3.types.Query, query=b"SELECT system_user;")
+ resp = receive_until(conn, pq3.types.DataRow)
+
+ row = resp.payload
+ expected = b"oauth:" + username.encode("utf-8")
+ assert row.columns == [expected]
+
+
+@pytest.fixture
+def odd_oauth_ctx(postgres_instance, oauth_ctx):
+ """
+ Adds an HBA entry with messed up issuer/scope settings, to pin the server
+ behavior.
+
+ TODO: these should really be rejected in the HBA rather than passed through
+ by the server.
+ """
+ id = secrets.token_hex(4)
+
+ class Context:
+ user = oauth_ctx.user
+ dbname = oauth_ctx.dbname
+
+ # Both of these embedded double-quotes are invalid; they're prohibited
+ # in both URLs and OAuth scope identifiers.
+ issuer = oauth_ctx.issuer + '/"/'
+ scope = oauth_ctx.scope + ' quo"ted'
+
+ ctx = Context()
+ hba_issuer = ctx.issuer.replace('"', '""')
+ hba_scope = ctx.scope.replace('"', '""')
+ hba_lines = [
+ f'host {ctx.dbname} {ctx.user} samehost oauth issuer="{hba_issuer}" scope="{hba_scope}"\n',
+ ]
+
+ if platform.system() == "Windows":
+ # XXX why is 'samehost' not behaving as expected on Windows?
+ for l in list(hba_lines):
+ hba_lines.append(l.replace("samehost", "::1/128"))
+
+ host, port = postgres_instance
+ conn = psycopg2.connect(host=host, port=port)
+ conn.autocommit = True
+
+ with contextlib.closing(conn):
+ c = conn.cursor()
+
+ # Replace pg_hba. Note that it's already been replaced once by
+ # oauth_ctx, so use a different backup prefix in prepend_file().
+ c.execute("SHOW hba_file;")
+ hba = c.fetchone()[0]
+
+ with prepend_file(hba, hba_lines, suffix=".bak2"):
+ c.execute("SELECT pg_reload_conf();")
+
+ yield ctx
+
+ # Put things back the way they were.
+ c.execute("SELECT pg_reload_conf();")
+
+
+def test_odd_server_response(odd_oauth_ctx, connect):
+ """
+ Verifies that the server is correctly escaping the JSON in its failure
+ response.
+ """
+ conn = connect()
+ begin_oauth_handshake(conn, odd_oauth_ctx, user=odd_oauth_ctx.user)
+
+ # Send an empty auth initial response, which will force an authn failure.
+ send_initial_response(conn, auth=b"")
+
+ expect_handshake_failure(conn, odd_oauth_ctx)
diff --git a/src/test/python/server/test_server.py b/src/test/python/server/test_server.py
new file mode 100644
index 0000000000..02126dba79
--- /dev/null
+++ b/src/test/python/server/test_server.py
@@ -0,0 +1,21 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import pq3
+
+
+def test_handshake(connect):
+ """Basic sanity check."""
+ conn = connect()
+
+ pq3.handshake(conn, user=pq3.pguser(), database=pq3.pgdatabase())
+
+ pq3.send(conn, pq3.types.Query, query=b"")
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.EmptyQueryResponse
+
+ resp = pq3.recv1(conn)
+ assert resp.type == pq3.types.ReadyForQuery
diff --git a/src/test/python/test_internals.py b/src/test/python/test_internals.py
new file mode 100644
index 0000000000..dee4855fc0
--- /dev/null
+++ b/src/test/python/test_internals.py
@@ -0,0 +1,138 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import io
+
+from pq3 import _DebugStream
+
+
+def test_DebugStream_read():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+
+ stream = _DebugStream(under, out)
+
+ res = stream.read(5)
+ assert res == b"abcde"
+
+ res = stream.read(16)
+ assert res == b"fghijklmnopqrstu"
+
+ stream.flush_debug()
+
+ res = stream.read()
+ assert res == b"vwxyz"
+
+ stream.flush_debug()
+
+ expected = (
+ "< 0000:\t61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f 70\tabcdefghijklmnop\n"
+ "< 0010:\t71 72 73 74 75 \tqrstu\n"
+ "\n"
+ "< 0000:\t76 77 78 79 7a \tvwxyz\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_write():
+ under = io.BytesIO()
+ out = io.StringIO()
+
+ stream = _DebugStream(under, out)
+
+ stream.write(b"\x00\x01\x02")
+ stream.flush()
+
+ assert under.getvalue() == b"\x00\x01\x02"
+
+ stream.write(b"\xc0\xc1\xc2")
+ stream.flush()
+
+ assert under.getvalue() == b"\x00\x01\x02\xc0\xc1\xc2"
+
+ stream.flush_debug()
+
+ expected = "> 0000:\t00 01 02 c0 c1 c2 \t......\n\n"
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_read_write():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+ stream = _DebugStream(under, out)
+
+ res = stream.read(5)
+ assert res == b"abcde"
+
+ stream.write(b"xxxxx")
+ stream.flush()
+
+ assert under.getvalue() == b"abcdexxxxxklmnopqrstuvwxyz"
+
+ res = stream.read(5)
+ assert res == b"klmno"
+
+ stream.write(b"xxxxx")
+ stream.flush()
+
+ assert under.getvalue() == b"abcdexxxxxklmnoxxxxxuvwxyz"
+
+ stream.flush_debug()
+
+ expected = (
+ "< 0000:\t61 62 63 64 65 6b 6c 6d 6e 6f \tabcdeklmno\n"
+ "\n"
+ "> 0000:\t78 78 78 78 78 78 78 78 78 78 \txxxxxxxxxx\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
+
+
+def test_DebugStream_end_packet():
+ under = io.BytesIO(b"abcdefghijklmnopqrstuvwxyz")
+ out = io.StringIO()
+ stream = _DebugStream(under, out)
+
+ stream.read(5)
+ stream.end_packet("read description", read=True, indent=" ")
+
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("write description", indent=" ")
+
+ stream.read(5)
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("read/write combo for read", read=True, indent=" ")
+
+ stream.read(5)
+ stream.write(b"xxxxx")
+ stream.flush()
+ stream.end_packet("read/write combo for write", indent=" ")
+
+ expected = (
+ " < 0000:\t61 62 63 64 65 \tabcde\n"
+ "\n"
+ "< read description\n"
+ "\n"
+ "> write description\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ " < 0000:\t6b 6c 6d 6e 6f \tklmno\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ "< read/write combo for read\n"
+ "\n"
+ "> read/write combo for write\n"
+ "\n"
+ " < 0000:\t75 76 77 78 79 \tuvwxy\n"
+ "\n"
+ " > 0000:\t78 78 78 78 78 \txxxxx\n"
+ "\n"
+ )
+ assert out.getvalue() == expected
diff --git a/src/test/python/test_pq3.py b/src/test/python/test_pq3.py
new file mode 100644
index 0000000000..7c6817de31
--- /dev/null
+++ b/src/test/python/test_pq3.py
@@ -0,0 +1,574 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+import contextlib
+import getpass
+import io
+import os
+import platform
+import struct
+import sys
+
+import pytest
+from construct import Container, PaddingError, StreamError, TerminatedError
+
+import pq3
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"\x00\x00\x00\x10\x00\x04\x00\x00abcdefgh",
+ Container(len=16, proto=0x40000, payload=b"abcdefgh"),
+ b"",
+ id="8-byte payload",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x08\x00\x04\x00\x00",
+ Container(len=8, proto=0x40000, payload=b""),
+ b"",
+ id="no payload",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x09\x00\x04\x00\x00abcde",
+ Container(len=9, proto=0x40000, payload=b"a"),
+ b"bcde",
+ id="1-byte payload and extra padding",
+ ),
+ pytest.param(
+ b"\x00\x00\x00\x0B\x00\x03\x00\x00hi\x00",
+ Container(len=11, proto=pq3.protocol(3, 0), payload=[b"hi"]),
+ b"",
+ id="implied parameter list when using proto version 3.0",
+ ),
+ ],
+)
+def test_Startup_parse(raw, expected, extra):
+ with io.BytesIO(raw) as stream:
+ actual = pq3.Startup.parse_stream(stream)
+
+ assert actual == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "packet,expected_bytes",
+ [
+ pytest.param(
+ dict(),
+ b"\x00\x00\x00\x08\x00\x00\x00\x00",
+ id="nothing set",
+ ),
+ pytest.param(
+ dict(len=10, proto=0x12345678),
+ b"\x00\x00\x00\x0A\x12\x34\x56\x78\x00\x00",
+ id="len and proto set explicitly",
+ ),
+ pytest.param(
+ dict(proto=0x12345678),
+ b"\x00\x00\x00\x08\x12\x34\x56\x78",
+ id="implied len with no payload",
+ ),
+ pytest.param(
+ dict(proto=0x12345678, payload=b"abcd"),
+ b"\x00\x00\x00\x0C\x12\x34\x56\x78abcd",
+ id="implied len with payload",
+ ),
+ pytest.param(
+ dict(payload=[b""]),
+ b"\x00\x00\x00\x09\x00\x03\x00\x00\x00",
+ id="implied proto version 3 when sending parameters",
+ ),
+ pytest.param(
+ dict(payload=[b"hi", b""]),
+ b"\x00\x00\x00\x0C\x00\x03\x00\x00hi\x00\x00",
+ id="implied proto version 3 and len when sending more than one parameter",
+ ),
+ pytest.param(
+ dict(payload=dict(user="jsmith", database="postgres")),
+ b"\x00\x00\x00\x27\x00\x03\x00\x00user\x00jsmith\x00database\x00postgres\x00\x00",
+ id="auto-serialization of dict parameters",
+ ),
+ ],
+)
+def test_Startup_build(packet, expected_bytes):
+ actual = pq3.Startup.build(packet)
+ assert actual == expected_bytes
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"*\x00\x00\x00\x08abcd",
+ dict(type=b"*", len=8, payload=b"abcd"),
+ b"",
+ id="4-byte payload",
+ ),
+ pytest.param(
+ b"*\x00\x00\x00\x04",
+ dict(type=b"*", len=4, payload=b""),
+ b"",
+ id="no payload",
+ ),
+ pytest.param(
+ b"*\x00\x00\x00\x05xabcd",
+ dict(type=b"*", len=5, payload=b"x"),
+ b"abcd",
+ id="1-byte payload with extra padding",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x08\x00\x00\x00\x00",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=8,
+ payload=dict(type=pq3.authn.OK, body=None),
+ ),
+ b"",
+ id="AuthenticationOk",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x12\x00\x00\x00\x0AEXTERNAL\x00\x00",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=18,
+ payload=dict(type=pq3.authn.SASL, body=[b"EXTERNAL", b""]),
+ ),
+ b"",
+ id="AuthenticationSASL",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0B12345",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=13,
+ payload=dict(type=pq3.authn.SASLContinue, body=b"12345"),
+ ),
+ b"",
+ id="AuthenticationSASLContinue",
+ ),
+ pytest.param(
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0C12345",
+ dict(
+ type=pq3.types.AuthnRequest,
+ len=13,
+ payload=dict(type=pq3.authn.SASLFinal, body=b"12345"),
+ ),
+ b"",
+ id="AuthenticationSASLFinal",
+ ),
+ pytest.param(
+ b"p\x00\x00\x00\x0Bhunter2",
+ dict(
+ type=pq3.types.PasswordMessage,
+ len=11,
+ payload=b"hunter2",
+ ),
+ b"",
+ id="PasswordMessage",
+ ),
+ pytest.param(
+ b"K\x00\x00\x00\x0C\x00\x00\x00\x00\x12\x34\x56\x78",
+ dict(
+ type=pq3.types.BackendKeyData,
+ len=12,
+ payload=dict(pid=0, key=0x12345678),
+ ),
+ b"",
+ id="BackendKeyData",
+ ),
+ pytest.param(
+ b"C\x00\x00\x00\x08SET\x00",
+ dict(
+ type=pq3.types.CommandComplete,
+ len=8,
+ payload=dict(tag=b"SET"),
+ ),
+ b"",
+ id="CommandComplete",
+ ),
+ pytest.param(
+ b"E\x00\x00\x00\x11Mbad!\x00Mdog!\x00\x00",
+ dict(type=b"E", len=17, payload=dict(fields=[b"Mbad!", b"Mdog!", b""])),
+ b"",
+ id="ErrorResponse",
+ ),
+ pytest.param(
+ b"S\x00\x00\x00\x08a\x00b\x00",
+ dict(
+ type=pq3.types.ParameterStatus,
+ len=8,
+ payload=dict(name=b"a", value=b"b"),
+ ),
+ b"",
+ id="ParameterStatus",
+ ),
+ pytest.param(
+ b"Z\x00\x00\x00\x05x",
+ dict(type=b"Z", len=5, payload=dict(status=b"x")),
+ b"",
+ id="ReadyForQuery",
+ ),
+ pytest.param(
+ b"Q\x00\x00\x00\x06!\x00",
+ dict(type=pq3.types.Query, len=6, payload=dict(query=b"!")),
+ b"",
+ id="Query",
+ ),
+ pytest.param(
+ b"D\x00\x00\x00\x0B\x00\x01\x00\x00\x00\x01!",
+ dict(type=pq3.types.DataRow, len=11, payload=dict(columns=[b"!"])),
+ b"",
+ id="DataRow",
+ ),
+ pytest.param(
+ b"D\x00\x00\x00\x06\x00\x00extra",
+ dict(type=pq3.types.DataRow, len=6, payload=dict(columns=[])),
+ b"extra",
+ id="DataRow with extra data",
+ ),
+ pytest.param(
+ b"I\x00\x00\x00\x04",
+ dict(type=pq3.types.EmptyQueryResponse, len=4, payload=None),
+ b"",
+ id="EmptyQueryResponse",
+ ),
+ pytest.param(
+ b"I\x00\x00\x00\x04\xFF",
+ dict(type=b"I", len=4, payload=None),
+ b"\xFF",
+ id="EmptyQueryResponse with extra bytes",
+ ),
+ pytest.param(
+ b"X\x00\x00\x00\x04",
+ dict(type=pq3.types.Terminate, len=4, payload=None),
+ b"",
+ id="Terminate",
+ ),
+ ],
+)
+def test_Pq3_parse(raw, expected, extra):
+ with io.BytesIO(raw) as stream:
+ actual = pq3.Pq3.parse_stream(stream)
+
+ assert actual == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(type=b"*", len=5),
+ b"*\x00\x00\x00\x05",
+ id="type and len set explicitly",
+ ),
+ pytest.param(
+ dict(type=b"*"),
+ b"*\x00\x00\x00\x04",
+ id="implied len with no payload",
+ ),
+ pytest.param(
+ dict(type=b"*", payload=b"1234"),
+ b"*\x00\x00\x00\x081234",
+ id="implied len with payload",
+ ),
+ pytest.param(
+ dict(type=b"*", len=12, payload=b"1234"),
+ b"*\x00\x00\x00\x0C1234",
+ id="overridden len (payload underflow)",
+ ),
+ pytest.param(
+ dict(type=b"*", len=5, payload=b"1234"),
+ b"*\x00\x00\x00\x051234",
+ id="overridden len (payload overflow)",
+ ),
+ pytest.param(
+ dict(type=pq3.types.AuthnRequest, payload=dict(type=pq3.authn.OK)),
+ b"R\x00\x00\x00\x08\x00\x00\x00\x00",
+ id="implied len/type for AuthenticationOK",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(
+ type=pq3.authn.SASL,
+ body=[b"SCRAM-SHA-256-PLUS", b"SCRAM-SHA-256", b""],
+ ),
+ ),
+ b"R\x00\x00\x00\x2A\x00\x00\x00\x0ASCRAM-SHA-256-PLUS\x00SCRAM-SHA-256\x00\x00",
+ id="implied len/type for AuthenticationSASL",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(type=pq3.authn.SASLContinue, body=b"12345"),
+ ),
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0B12345",
+ id="implied len/type for AuthenticationSASLContinue",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.AuthnRequest,
+ payload=dict(type=pq3.authn.SASLFinal, body=b"12345"),
+ ),
+ b"R\x00\x00\x00\x0D\x00\x00\x00\x0C12345",
+ id="implied len/type for AuthenticationSASLFinal",
+ ),
+ pytest.param(
+ dict(
+ type=pq3.types.PasswordMessage,
+ payload=b"hunter2",
+ ),
+ b"p\x00\x00\x00\x0Bhunter2",
+ id="implied len/type for PasswordMessage",
+ ),
+ pytest.param(
+ dict(type=pq3.types.BackendKeyData, payload=dict(pid=1, key=7)),
+ b"K\x00\x00\x00\x0C\x00\x00\x00\x01\x00\x00\x00\x07",
+ id="implied len/type for BackendKeyData",
+ ),
+ pytest.param(
+ dict(type=pq3.types.CommandComplete, payload=dict(tag=b"SET")),
+ b"C\x00\x00\x00\x08SET\x00",
+ id="implied len/type for CommandComplete",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ErrorResponse, payload=dict(fields=[b"error", b""])),
+ b"E\x00\x00\x00\x0Berror\x00\x00",
+ id="implied len/type for ErrorResponse",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ParameterStatus, payload=dict(name=b"a", value=b"b")),
+ b"S\x00\x00\x00\x08a\x00b\x00",
+ id="implied len/type for ParameterStatus",
+ ),
+ pytest.param(
+ dict(type=pq3.types.ReadyForQuery, payload=dict(status=b"I")),
+ b"Z\x00\x00\x00\x05I",
+ id="implied len/type for ReadyForQuery",
+ ),
+ pytest.param(
+ dict(type=pq3.types.Query, payload=dict(query=b"SELECT 1;")),
+ b"Q\x00\x00\x00\x0eSELECT 1;\x00",
+ id="implied len/type for Query",
+ ),
+ pytest.param(
+ dict(type=pq3.types.DataRow, payload=dict(columns=[b"abcd"])),
+ b"D\x00\x00\x00\x0E\x00\x01\x00\x00\x00\x04abcd",
+ id="implied len/type for DataRow",
+ ),
+ pytest.param(
+ dict(type=pq3.types.EmptyQueryResponse),
+ b"I\x00\x00\x00\x04",
+ id="implied len for EmptyQueryResponse",
+ ),
+ pytest.param(
+ dict(type=pq3.types.Terminate),
+ b"X\x00\x00\x00\x04",
+ id="implied len for Terminate",
+ ),
+ ],
+)
+def test_Pq3_build(fields, expected):
+ actual = pq3.Pq3.build(fields)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "raw,expected,extra",
+ [
+ pytest.param(
+ b"\x00\x00",
+ dict(columns=[]),
+ b"",
+ id="no columns",
+ ),
+ pytest.param(
+ b"\x00\x01\x00\x00\x00\x04abcd",
+ dict(columns=[b"abcd"]),
+ b"",
+ id="one column",
+ ),
+ pytest.param(
+ b"\x00\x02\x00\x00\x00\x04abcd\x00\x00\x00\x01x",
+ dict(columns=[b"abcd", b"x"]),
+ b"",
+ id="multiple columns",
+ ),
+ pytest.param(
+ b"\x00\x02\x00\x00\x00\x00\x00\x00\x00\x01x",
+ dict(columns=[b"", b"x"]),
+ b"",
+ id="empty column value",
+ ),
+ pytest.param(
+ b"\x00\x02\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF",
+ dict(columns=[None, None]),
+ b"",
+ id="null columns",
+ ),
+ ],
+)
+def test_DataRow_parse(raw, expected, extra):
+ pkt = b"D" + struct.pack("!i", len(raw) + 4) + raw
+ with io.BytesIO(pkt) as stream:
+ actual = pq3.Pq3.parse_stream(stream)
+
+ assert actual.type == pq3.types.DataRow
+ assert actual.payload == expected
+ assert stream.read() == extra
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(),
+ b"\x00\x00",
+ id="no columns",
+ ),
+ pytest.param(
+ dict(columns=[None, None]),
+ b"\x00\x02\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF",
+ id="null columns",
+ ),
+ ],
+)
+def test_DataRow_build(fields, expected):
+ actual = pq3.Pq3.build(dict(type=pq3.types.DataRow, payload=fields))
+
+ expected = b"D" + struct.pack("!i", len(expected) + 4) + expected
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "raw,expected,exception",
+ [
+ pytest.param(
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ dict(name=b"EXTERNAL", len=-1, data=None),
+ None,
+ id="no initial response",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\x02me",
+ dict(name=b"EXTERNAL", len=2, data=b"me"),
+ None,
+ id="initial response",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\x02meextra",
+ None,
+ TerminatedError,
+ id="extra data",
+ ),
+ pytest.param(
+ b"EXTERNAL\x00\x00\x00\x00\xFFme",
+ None,
+ StreamError,
+ id="underflow",
+ ),
+ ],
+)
+def test_SASLInitialResponse_parse(raw, expected, exception):
+ ctx = contextlib.nullcontext()
+ if exception:
+ ctx = pytest.raises(exception)
+
+ with ctx:
+ actual = pq3.SASLInitialResponse.parse(raw)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "fields,expected",
+ [
+ pytest.param(
+ dict(name=b"EXTERNAL"),
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ id="no initial response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=None),
+ b"EXTERNAL\x00\xFF\xFF\xFF\xFF",
+ id="no initial response (explicit None)",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=b""),
+ b"EXTERNAL\x00\x00\x00\x00\x00",
+ id="empty response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", data=b"me@example.com"),
+ b"EXTERNAL\x00\x00\x00\x00\x0Eme@example.com",
+ id="initial response",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", len=2, data=b"me@example.com"),
+ b"EXTERNAL\x00\x00\x00\x00\x02me@example.com",
+ id="data overflow",
+ ),
+ pytest.param(
+ dict(name=b"EXTERNAL", len=14, data=b"me"),
+ b"EXTERNAL\x00\x00\x00\x00\x0Eme",
+ id="data underflow",
+ ),
+ ],
+)
+def test_SASLInitialResponse_build(fields, expected):
+ actual = pq3.SASLInitialResponse.build(fields)
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "version,expected_bytes",
+ [
+ pytest.param((3, 0), b"\x00\x03\x00\x00", id="version 3"),
+ pytest.param((1234, 5679), b"\x04\xd2\x16\x2f", id="SSLRequest"),
+ ],
+)
+def test_protocol(version, expected_bytes):
+ # Make sure the integer returned by protocol is correctly serialized on the
+ # wire.
+ assert struct.pack("!i", pq3.protocol(*version)) == expected_bytes
+
+
+@pytest.mark.parametrize(
+ "envvar,func,expected",
+ [
+ ("PGHOST", pq3.pghost, "localhost"),
+ ("PGPORT", pq3.pgport, 5432),
+ (
+ "PGUSER",
+ pq3.pguser,
+ os.getlogin() if platform.system() == "Windows" else getpass.getuser(),
+ ),
+ ("PGDATABASE", pq3.pgdatabase, "postgres"),
+ ],
+)
+def test_env_defaults(monkeypatch, envvar, func, expected):
+ monkeypatch.delenv(envvar, raising=False)
+
+ actual = func()
+ assert actual == expected
+
+
+@pytest.mark.parametrize(
+ "envvars,func,expected",
+ [
+ (dict(PGHOST="otherhost"), pq3.pghost, "otherhost"),
+ (dict(PGPORT="6789"), pq3.pgport, 6789),
+ (dict(PGUSER="postgres"), pq3.pguser, "postgres"),
+ (dict(PGDATABASE="template1"), pq3.pgdatabase, "template1"),
+ ],
+)
+def test_env(monkeypatch, envvars, func, expected):
+ for k, v in envvars.items():
+ monkeypatch.setenv(k, v)
+
+ actual = func()
+ assert actual == expected
diff --git a/src/test/python/tls.py b/src/test/python/tls.py
new file mode 100644
index 0000000000..075c02c1ca
--- /dev/null
+++ b/src/test/python/tls.py
@@ -0,0 +1,195 @@
+#
+# Copyright 2021 VMware, Inc.
+# SPDX-License-Identifier: PostgreSQL
+#
+
+from construct import *
+
+#
+# TLS 1.3
+#
+# Most of the types below are transcribed from RFC 8446:
+#
+# https://tools.ietf.org/html/rfc8446
+#
+
+
+def _Vector(size_field, element):
+ return Prefixed(size_field, GreedyRange(element))
+
+
+# Alerts
+
+AlertLevel = Enum(
+ Byte,
+ warning=1,
+ fatal=2,
+)
+
+AlertDescription = Enum(
+ Byte,
+ close_notify=0,
+ unexpected_message=10,
+ bad_record_mac=20,
+ decryption_failed_RESERVED=21,
+ record_overflow=22,
+ decompression_failure=30,
+ handshake_failure=40,
+ no_certificate_RESERVED=41,
+ bad_certificate=42,
+ unsupported_certificate=43,
+ certificate_revoked=44,
+ certificate_expired=45,
+ certificate_unknown=46,
+ illegal_parameter=47,
+ unknown_ca=48,
+ access_denied=49,
+ decode_error=50,
+ decrypt_error=51,
+ export_restriction_RESERVED=60,
+ protocol_version=70,
+ insufficient_security=71,
+ internal_error=80,
+ user_canceled=90,
+ no_renegotiation=100,
+ unsupported_extension=110,
+)
+
+Alert = Struct(
+ "level" / AlertLevel,
+ "description" / AlertDescription,
+)
+
+
+# Extensions
+
+ExtensionType = Enum(
+ Int16ub,
+ server_name=0,
+ max_fragment_length=1,
+ status_request=5,
+ supported_groups=10,
+ signature_algorithms=13,
+ use_srtp=14,
+ heartbeat=15,
+ application_layer_protocol_negotiation=16,
+ signed_certificate_timestamp=18,
+ client_certificate_type=19,
+ server_certificate_type=20,
+ padding=21,
+ pre_shared_key=41,
+ early_data=42,
+ supported_versions=43,
+ cookie=44,
+ psk_key_exchange_modes=45,
+ certificate_authorities=47,
+ oid_filters=48,
+ post_handshake_auth=49,
+ signature_algorithms_cert=50,
+ key_share=51,
+)
+
+Extension = Struct(
+ "extension_type" / ExtensionType,
+ "extension_data" / Prefixed(Int16ub, GreedyBytes),
+)
+
+
+# ClientHello
+
+
+class _CipherSuiteAdapter(Adapter):
+ class _hextuple(tuple):
+ def __repr__(self):
+ return f"(0x{self[0]:02X}, 0x{self[1]:02X})"
+
+ def _encode(self, obj, context, path):
+ return bytes(obj)
+
+ def _decode(self, obj, context, path):
+ assert len(obj) == 2
+ return self._hextuple(obj)
+
+
+ProtocolVersion = Hex(Int16ub)
+
+Random = Hex(Bytes(32))
+
+CipherSuite = _CipherSuiteAdapter(Byte[2])
+
+ClientHello = Struct(
+ "legacy_version" / ProtocolVersion,
+ "random" / Random,
+ "legacy_session_id" / Prefixed(Byte, Hex(GreedyBytes)),
+ "cipher_suites" / _Vector(Int16ub, CipherSuite),
+ "legacy_compression_methods" / Prefixed(Byte, GreedyBytes),
+ "extensions" / _Vector(Int16ub, Extension),
+)
+
+# ServerHello
+
+ServerHello = Struct(
+ "legacy_version" / ProtocolVersion,
+ "random" / Random,
+ "legacy_session_id_echo" / Prefixed(Byte, Hex(GreedyBytes)),
+ "cipher_suite" / CipherSuite,
+ "legacy_compression_method" / Hex(Byte),
+ "extensions" / _Vector(Int16ub, Extension),
+)
+
+# Handshake
+
+HandshakeType = Enum(
+ Byte,
+ client_hello=1,
+ server_hello=2,
+ new_session_ticket=4,
+ end_of_early_data=5,
+ encrypted_extensions=8,
+ certificate=11,
+ certificate_request=13,
+ certificate_verify=15,
+ finished=20,
+ key_update=24,
+ message_hash=254,
+)
+
+Handshake = Struct(
+ "msg_type" / HandshakeType,
+ "length" / Int24ub,
+ "payload"
+ / Switch(
+ this.msg_type,
+ {
+ HandshakeType.client_hello: ClientHello,
+ HandshakeType.server_hello: ServerHello,
+ # HandshakeType.end_of_early_data: EndOfEarlyData,
+ # HandshakeType.encrypted_extensions: EncryptedExtensions,
+ # HandshakeType.certificate_request: CertificateRequest,
+ # HandshakeType.certificate: Certificate,
+ # HandshakeType.certificate_verify: CertificateVerify,
+ # HandshakeType.finished: Finished,
+ # HandshakeType.new_session_ticket: NewSessionTicket,
+ # HandshakeType.key_update: KeyUpdate,
+ },
+ default=FixedSized(this.length, GreedyBytes),
+ ),
+)
+
+# Records
+
+ContentType = Enum(
+ Byte,
+ invalid=0,
+ change_cipher_spec=20,
+ alert=21,
+ handshake=22,
+ application_data=23,
+)
+
+Plaintext = Struct(
+ "type" / ContentType,
+ "legacy_record_version" / ProtocolVersion,
+ "length" / Int16ub,
+ "fragment" / FixedSized(this.length, GreedyBytes),
+)
diff --git a/src/tools/make_venv b/src/tools/make_venv
new file mode 100755
index 0000000000..804307ee12
--- /dev/null
+++ b/src/tools/make_venv
@@ -0,0 +1,56 @@
+#!/usr/bin/env python3
+
+import argparse
+import subprocess
+import os
+import platform
+import sys
+
+parser = argparse.ArgumentParser()
+
+parser.add_argument('--requirements', help='path to pip requirements file', type=str)
+parser.add_argument('--privatedir', help='private directory for target', type=str)
+parser.add_argument('venv_path', help='desired venv location')
+
+args = parser.parse_args()
+
+# Decide whether or not to capture stdout into a log file. We only do this if
+# we've been given our own private directory.
+#
+# FIXME Unfortunately this interferes with debugging on Cirrus, because the
+# private directory isn't uploaded in the sanity check's artifacts. When we
+# don't capture the log file, it gets spammed to stdout during build... Is there
+# a way to push this into the meson-log somehow? For now, the capture
+# implementation is commented out.
+logfile = None
+
+if args.privatedir:
+ if not os.path.isdir(args.privatedir):
+ os.mkdir(args.privatedir)
+
+ # FIXME see above comment
+ # logpath = os.path.join(args.privatedir, 'stdout.txt')
+ # logfile = open(logpath, 'w')
+
+def run(*args):
+ kwargs = dict(check=True)
+ if logfile:
+ kwargs.update(stdout=logfile)
+
+ subprocess.run(args, **kwargs)
+
+# Create the virtualenv first.
+run(sys.executable, '-m', 'venv', args.venv_path)
+
+# Update pip next. This helps avoid old pip bugs; the version inside system
+# Pythons tends to be pretty out of date.
+bindir = 'Scripts' if platform.system() == 'Windows' else 'bin'
+python = os.path.join(args.venv_path, bindir, 'python3')
+run(python, '-m', 'pip', 'install', '-U', 'pip')
+
+# Finally, install the test's requirements. We need pytest and pytest-tap, no
+# matter what the test needs.
+pip = os.path.join(args.venv_path, bindir, 'pip')
+run(pip, 'install', 'pytest', 'pytest-tap')
+if args.requirements:
+ run(pip, 'install', '-r', args.requirements)
--
2.34.1
v21-0006-XXX-temporary-patches-to-build-and-test.patch (application/octet-stream, 1.7K) download
From e6c5d94682d601002e029cd28fbdb9d94f2dac0d Mon Sep 17 00:00:00 2001
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Tue, 20 Feb 2024 11:35:29 -0800
Subject: [PATCH v21 6/7] XXX temporary patches to build and test
- construct 2.10.70 has some incompatibilities with the current tests
- temporarily skip the exit check (from Daniel Gustafsson); this needs
to be turned into an exception for curl rather than a plain exit call
---
src/interfaces/libpq/Makefile | 2 +-
src/test/python/requirements.txt | 4 +++-
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 2618c293af..e86d4803ff 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -124,7 +124,7 @@ libpq-refs-stamp: $(shlib)
ifneq ($(enable_coverage), yes)
ifeq (,$(filter solaris,$(PORTNAME)))
@if nm -A -u $< 2>/dev/null | grep -v -e __cxa_atexit -e __tsan_func_exit | grep exit; then \
- echo 'libpq must not be calling any function which invokes exit'; exit 1; \
+ echo 'libpq must not be calling any function which invokes exit'; \
fi
endif
endif
diff --git a/src/test/python/requirements.txt b/src/test/python/requirements.txt
index 57ba1ced94..0dfcffb83e 100644
--- a/src/test/python/requirements.txt
+++ b/src/test/python/requirements.txt
@@ -1,7 +1,9 @@
black
# cryptography 35.x and later add many platform/toolchain restrictions, beware
cryptography~=3.4.8
-construct~=2.10.61
+# TODO: figure out why 2.10.70 broke things
+# (probably https://github.com/construct/construct/pull/1015)
+construct==2.10.69
isort~=5.6
# TODO: update to psycopg[c] 3.1
psycopg2~=2.9.7
--
2.34.1
v21-0007-WIP-Python-OAuth-provider-implementation.patch (application/octet-stream, 10.8K) download
From decc90579a73dd3073ddb479c317d52f31f26d67 Mon Sep 17 00:00:00 2001
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Mon, 26 Feb 2024 16:24:32 -0800
Subject: [PATCH v21 7/7] WIP: Python OAuth provider implementation
---
src/test/modules/oauth_validator/Makefile | 2 +
src/test/modules/oauth_validator/meson.build | 3 +
.../modules/oauth_validator/t/001_server.pl | 16 +-
.../modules/oauth_validator/t/oauth_server.py | 114 ++++++++++++
src/test/perl/PostgreSQL/Test/OAuthServer.pm | 166 +++---------------
5 files changed, 149 insertions(+), 152 deletions(-)
create mode 100755 src/test/modules/oauth_validator/t/oauth_server.py
diff --git a/src/test/modules/oauth_validator/Makefile b/src/test/modules/oauth_validator/Makefile
index 1f874cd7f2..e93e01455a 100644
--- a/src/test/modules/oauth_validator/Makefile
+++ b/src/test/modules/oauth_validator/Makefile
@@ -1,3 +1,5 @@
+export PYTHON
+
MODULES = validator
PGFILEDESC = "validator - test OAuth validator module"
diff --git a/src/test/modules/oauth_validator/meson.build b/src/test/modules/oauth_validator/meson.build
index d9c1d1d577..3feba6f826 100644
--- a/src/test/modules/oauth_validator/meson.build
+++ b/src/test/modules/oauth_validator/meson.build
@@ -29,5 +29,8 @@ tests += {
'tests': [
't/001_server.pl',
],
+ 'env': {
+ 'PYTHON': python.path(),
+ },
},
}
diff --git a/src/test/modules/oauth_validator/t/001_server.pl b/src/test/modules/oauth_validator/t/001_server.pl
index 14c7778298..ea610fcd28 100644
--- a/src/test/modules/oauth_validator/t/001_server.pl
+++ b/src/test/modules/oauth_validator/t/001_server.pl
@@ -19,7 +19,11 @@ $node->start;
$node->safe_psql('postgres', 'CREATE USER test;');
$node->safe_psql('postgres', 'CREATE USER testalt;');
-my $issuer = "127.0.0.1:18080";
+my $webserver = PostgreSQL::Test::OAuthServer->new();
+$webserver->run();
+
+my $port = $webserver->port();
+my $issuer = "127.0.0.1:$port";
unlink($node->data_dir . '/pg_hba.conf');
$node->append_conf('pg_hba.conf', qq{
@@ -28,15 +32,6 @@ local all testalt oauth issuer="$issuer/alternate" scope="openid postgres alt"
});
$node->reload;
-my $webserver = PostgreSQL::Test::OAuthServer->new(18080);
-
-my $port = $webserver->port();
-
-is($port, 18080, "Port is 18080");
-
-$webserver->setup();
-$webserver->run();
-
my ($log_start, $log_end);
$log_start = $node->wait_for_log(qr/reloading configuration files/);
@@ -73,6 +68,7 @@ $node->log_check("user $user: validator sets authenticated identity", $log_start
]);
$log_start = $log_end;
+$webserver->stop();
$node->stop;
done_testing();
diff --git a/src/test/modules/oauth_validator/t/oauth_server.py b/src/test/modules/oauth_validator/t/oauth_server.py
new file mode 100755
index 0000000000..77e3883a81
--- /dev/null
+++ b/src/test/modules/oauth_validator/t/oauth_server.py
@@ -0,0 +1,114 @@
+#! /usr/bin/env python3
+
+import http.server
+import json
+import os
+import sys
+
+
+class OAuthHandler(http.server.BaseHTTPRequestHandler):
+ JsonObject = dict[str, object] # TypeAlias is not available until 3.10
+
+ def _check_issuer(self):
+ """
+ Switches the behavior of the provider depending on the issuer URI.
+ """
+ self._alt_issuer = self.path.startswith("/alternate/")
+ if self._alt_issuer:
+ self.path = self.path.removeprefix("/alternate")
+
+ def do_GET(self):
+ self._check_issuer()
+
+ if self.path == "/.well-known/openid-configuration":
+ resp = self.config()
+ else:
+ self.send_error(404, "Not Found")
+ return
+
+ self._send_json(resp)
+
+ def do_POST(self):
+ self._check_issuer()
+
+ if self.path == "/authorize":
+ resp = self.authorization()
+ elif self.path == "/token":
+ resp = self.token()
+ else:
+ self.send_error(404, "Not Found")
+ return
+
+ self._send_json(resp)
+
+ def _send_json(self, js: JsonObject) -> None:
+ """
+ Sends the provided JSON dict as an application/json response.
+ """
+
+ resp = json.dumps(js).encode("ascii")
+
+ self.send_response(200, "OK")
+ self.send_header("Content-Type", "application/json")
+ self.send_header("Content-Length", str(len(resp)))
+ self.end_headers()
+
+ self.wfile.write(resp)
+
+ def config(self) -> JsonObject:
+ port = self.server.socket.getsockname()[1]
+ issuer = f"http://localhost:{port}"
+ if self._alt_issuer:
+ issuer += "/alternate"
+
+ return {
+ "issuer": issuer,
+ "token_endpoint": issuer + "/token",
+ "device_authorization_endpoint": issuer + "/authorize",
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ "grant_types_supported": ["urn:ietf:params:oauth:grant-type:device_code"],
+ }
+
+ def authorization(self) -> JsonObject:
+ uri = "https://example.com/"
+ if self._alt_issuer:
+ uri = "https://example.org/"
+
+ return {
+ "device_code": "postgres",
+ "user_code": "postgresuser",
+ "interval": 0,
+ "verification_uri": uri,
+ "expires-in": 5,
+ }
+
+ def token(self) -> JsonObject:
+ token = "9243959234"
+ if self._alt_issuer:
+ token += "-alt"
+
+ return {
+ "access_token": token,
+ "token_type": "bearer",
+ }
+
+
+def main():
+ s = http.server.HTTPServer(("127.0.0.1", 0), OAuthHandler)
+
+ # Give the parent the port number to contact (this is also the signal that
+ # we're ready to receive requests).
+ port = s.socket.getsockname()[1]
+ print(port)
+
+ stdout = sys.stdout.fileno()
+ sys.stdout.close()
+ os.close(stdout)
+
+ s.serve_forever() # we expect our parent to send a termination signal
+
+
+if __name__ == "__main__":
+ main()
diff --git a/src/test/perl/PostgreSQL/Test/OAuthServer.pm b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
index 5c195efb79..d96733f531 100644
--- a/src/test/perl/PostgreSQL/Test/OAuthServer.pm
+++ b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
@@ -5,6 +5,7 @@ package PostgreSQL::Test::OAuthServer;
use warnings;
use strict;
use threads;
+use Scalar::Util;
use Socket;
use IO::Select;
@@ -13,27 +14,13 @@ local *server_socket;
sub new
{
my $class = shift;
- my $port = shift;
my $self = {};
bless($self, $class);
- $self->{'port'} = $port;
-
return $self;
}
-sub setup
-{
- my $self = shift;
- my $tcp = getprotobyname('tcp');
-
- socket($self->{'socket'}, PF_INET, SOCK_STREAM, $tcp)
- or die "no socket";
- setsockopt($self->{'socket'}, SOL_SOCKET, SO_REUSEADDR, pack("l", 1));
- bind($self->{'socket'}, sockaddr_in($self->{'port'}, INADDR_ANY));
-}
-
sub port
{
my $self = shift;
@@ -44,140 +31,35 @@ sub port
sub run
{
my $self = shift;
+ my $port;
- my $server_thread = threads->create(\&_listen, $self);
- $server_thread->detach();
+ my $pid = open(my $read_fh, "-|", $ENV{PYTHON}, "t/oauth_server.py")
+ // die "failed to start OAuth server: $!";
+
+ read($read_fh, $port, 7) // die "failed to read port number: $!";
+ chomp $port;
+ die "server did not advertise a valid port"
+ unless Scalar::Util::looks_like_number($port);
+
+ $self->{'pid'} = $pid;
+ $self->{'port'} = $port;
+ $self->{'child'} = $read_fh;
+
+ print("# OAuth provider (PID $pid) is listening on port $port\n");
}
-sub _listen
+sub stop
{
my $self = shift;
- listen($self->{'socket'}, SOMAXCONN) or die "fail to listen: $!";
-
- while (1)
- {
- my $fh;
- my %request;
- my $remote = accept($fh, $self->{'socket'});
- binmode $fh;
-
- my ($method, $object, $prot) = split(/ /, <$fh>);
- $request{'method'} = $method;
- $request{'object'} = $object;
- chomp($request{'object'});
-
- local $/ = Socket::CRLF;
- my $c = 0;
- while(<$fh>)
- {
- chomp;
- # Headers
- if (/:/)
- {
- my ($field, $value) = split(/:/, $_, 2);
- $value =~ s/^\s+//;
- $request{'headers'}{lc $field} = $value;
- }
- # POST data
- elsif (/^$/)
- {
- read($fh, $request{'content'}, $request{'headers'}{'content-length'})
- if defined $request{'headers'}{'content-length'};
- last;
- }
- }
-
- # Debug printing
- # print ": read ".$request{'method'} . ";" . $request{'object'}.";\n";
- # foreach my $h (keys(%{$request{'headers'}}))
- #{
- # printf ": headers: " . $request{'headers'}{$h} . "\n";
- #}
- #printf ": POST: " . $request{'content'} . "\n" if defined $request{'content'};
-
- my $alternate = 0;
- if ($request{'object'} =~ qr|^/alternate(/.*)$|)
- {
- $alternate = 1;
- $request{'object'} = $1;
- }
-
- if ($request{'object'} eq '/.well-known/openid-configuration')
- {
- my $issuer = "http://localhost:$self->{'port'}";
- if ($alternate)
- {
- $issuer .= "/alternate";
- }
-
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: application/json\r\n";
- print $fh "\r\n";
- print $fh <<EOR;
- {
- "issuer": "$issuer",
- "token_endpoint": "$issuer/token",
- "device_authorization_endpoint": "$issuer/authorize",
- "response_types_supported": ["token"],
- "subject_types_supported": ["public"],
- "id_token_signing_alg_values_supported": ["RS256"],
- "grant_types_supported": ["urn:ietf:params:oauth:grant-type:device_code"]
- }
-EOR
- }
- elsif ($request{'object'} eq '/authorize')
- {
- my $uri = "https://example.com/";
- if ($alternate)
- {
- $uri = "https://example.org/";
- }
-
- print ": returning device_code\n";
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: application/json\r\n";
- print $fh "\r\n";
- print $fh <<EOR;
- {
- "device_code": "postgres",
- "user_code" : "postgresuser",
- "interval" : 0,
- "verification_uri" : "$uri",
- "expires-in": 5
- }
-EOR
- }
- elsif ($request{'object'} eq '/token')
- {
- my $token = "9243959234";
- if ($alternate)
- {
- $token .= "-alt";
- }
-
- print ": returning token\n";
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: application/json\r\n";
- print $fh "\r\n";
- print $fh <<EOR;
- {
- "access_token": "$token",
- "token_type": "bearer"
- }
-EOR
- }
- else
- {
- print ": returning default\n";
- print $fh "HTTP/1.0 200 OK\r\nServer: Postgres Regress\r\n";
- print $fh "Content-Type: text/html\r\n";
- print $fh "\r\n";
- print $fh "Ok\n";
- }
-
- close($fh);
- }
+ print("# Sending SIGTERM to OAuth provider PID: $self->{'pid'}\n");
+
+ kill(15, $self->{'pid'});
+ $self->{'pid'} = undef;
+
+ # Closing the popen() handle waits for the process to exit.
+ close($self->{'child'});
+ $self->{'child'} = undef;
}
1;
--
2.34.1
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-03-28 22:34 Daniel Gustafsson <daniel@yesql.se>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 1 reply; 424+ messages in thread
From: Daniel Gustafsson @ 2024-03-28 22:34 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: pgsql-hackers; Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>; Andrew Dunstan <andrew@dunslane.net>
> On 22 Mar 2024, at 19:21, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
>
> v21 is a quick rebase over HEAD, which has adopted a few pieces of
> v20. I've also fixed a race condition in the tests.
Thanks for the rebase, I have a few comments from working with it a bit:
In jsonapi.c, makeJsonLexContextCstringLen initialize a JsonLexContext with
palloc0 which would need to be ported over to use ALLOC for frontend code. On
that note, the errorhandling in parse_oauth_json() for content-type checks
attempts to free the JsonLexContext even before it has been created. Here we
can just return false.
- echo 'libpq must not be calling any function which invokes exit'; exit 1; \
+ echo 'libpq must not be calling any function which invokes exit'; \
The offending codepath in libcurl was in the NTLM_WB module, a very old and
obscure form of NTLM support which was replaced (yet remained in the tree) a
long time ago by a full NTLM implementatin. Based on the findings in this
thread it was deprecated with a removal date set to April 2024 [0]. A bug in
the 8.4.0 release however disconnected NTLM_WB from the build and given the
lack of complaints it was decided to leave as is, so we can base our libcurl
requirements on 8.4.0 while keeping the exit() check intact.
+ else if (strcasecmp(content_type, "application/json") != 0)
This needs to handle parameters as well since it will now fail if the charset
parameter is appended (which undoubtedly will be pretty common). The easiest
way is probably to just verify the mediatype and skip the parameters since we
know it can only be charset?
+ /* TODO investigate using conn->Pfdebug and CURLOPT_DEBUGFUNCTION here */
+ CHECK_SETOPT(actx, CURLOPT_VERBOSE, 1L, return false);
+ CHECK_SETOPT(actx, CURLOPT_ERRORBUFFER, actx->curl_err, return false);
CURLOPT_ERRORBUFFER is the old and finicky way of extracting error messages, we
should absolutely move to using CURLOPT_DEBUGFUNCTION instead.
+ /* && response_code != 401 TODO */ )
Why is this marked with a TODO, do you remember?
+ print("# OAuth provider (PID $pid) is listening on port $port\n");
Code running under Test::More need to use diag() for printing non-test output
like this.
Another issue I have is the sheer size and the fact that so much code is
replaced by subsequent commits, so I took the liberty to squash some of this
down into something less daunting. The attached v22 retains the 0001 and then
condenses the rest into two commits for frontent and backend parts. I did drop
the Python pytest patch since I feel that it's unlikely to go in from this
thread (adding pytest seems worthy of its own thread and discussion), and the
weight of it makes this seem scarier than it is. For using it, it can be
easily applied from the v21 patchset independently. I did tweak the commit
message to match reality a bit better, but there is a lot of work left there.
The final patch contains fixes for all of the above review comments as well as
a some refactoring, smaller clean-ups and TODO fixing. If these fixes are
accepted I'll incorporate them into the two commits.
Next I intend to work on writing documentation for this.
--
Daniel Gustafsson
[0] https://curl.se/dev/deprecate.html
[1] https://github.com/curl/curl/pull/12479
Attachments: download all (4 files, .tar.gz)
v22-0004-Review-comments.patch (application/octet-stream, 35.1K) download
From 136930c0d55469a92f9d34a25a6d3fd5b195c6b2 Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Thu, 28 Mar 2024 21:59:02 +0100
Subject: [PATCH v22 4/4] Review comments
Fixes and tidy-ups following a review of v21, a few items
are (listed in no specific order):
* Implement a version check for libcurl in autoconf, the
equivalent check for Meson is still a TODO.
* Address a few TODOs in the code
* libpq JSON support memory management fixups
---
config/programs.m4 | 21 ++
configure | 34 +++
configure.ac | 1 +
src/backend/libpq/auth-oauth.c | 28 +-
src/common/jsonapi.c | 20 +-
src/common/parse_manifest.c | 2 +
src/include/common/oauth-common.h | 2 +-
src/interfaces/libpq/Makefile | 4 +-
src/interfaces/libpq/fe-auth-oauth-curl.c | 275 +++++++++++--------
src/interfaces/libpq/fe-auth-oauth.c | 9 +-
src/interfaces/libpq/fe-auth-oauth.h | 2 +-
src/test/modules/oauth_validator/validator.c | 2 +-
src/test/perl/PostgreSQL/Test/OAuthServer.pm | 10 +-
src/tools/pgindent/typedefs.list | 1 +
14 files changed, 274 insertions(+), 137 deletions(-)
diff --git a/config/programs.m4 b/config/programs.m4
index 490ec9fe9f..157da7eec5 100644
--- a/config/programs.m4
+++ b/config/programs.m4
@@ -142,7 +142,28 @@ if test "$pgac_cv_ldap_safe" != yes; then
*** also uses LDAP will crash on exit.])
fi])
+# PGAC_CHECK_LIBCURL
+# ------------------
+# Check for libcurl 8.4.0 or higher since earlier versions can be compiled
+# with a codepatch containing exit(), and PostgreSQL does not allow any lib
+# linked to libpq which can call exit.
+# PGAC_CHECK_LIBCURL
+AC_DEFUN([PGAC_CHECK_LIBCURL],
+[AC_CACHE_CHECK([for compatible libcurl], [pgac_cv_check_libcurl],
+[AC_COMPILE_IFELSE([AC_LANG_PROGRAM(
+[#include <curl/curlver.h>
+#if LIBCURL_VERSION_MAJOR <= 8 && LIBCURL_VERSION_MINOR < 4
+choke me
+#endif], [])],
+[pgac_cv_check_libcurl=yes],
+[pgac_cv_check_libcurl=no])])
+
+if test "$pgac_cv_check_libcurl" != yes; then
+ AC_MSG_ERROR([
+*** The installed version of libcurl is too old to use with PostgreSQL.
+*** libcurl version 8.4.0 or later is required.])
+fi])
# PGAC_CHECK_READLINE
# -------------------
diff --git a/configure b/configure
index d785ca846b..d2585e8ed0 100755
--- a/configure
+++ b/configure
@@ -13134,6 +13134,40 @@ else
as_fn_error $? "library 'curl' is required for --with-oauth=curl" "$LINENO" 5
fi
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for compatible libcurl" >&5
+$as_echo_n "checking for compatible libcurl... " >&6; }
+if ${pgac_cv_check_libcurl+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+#include <curl/curlver.h>
+#if LIBCURL_VERSION_MAJOR <= 8 && LIBCURL_VERSION_MINOR < 4
+choke me
+#endif
+int
+main ()
+{
+
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_compile "$LINENO"; then :
+ pgac_cv_check_libcurl=yes
+else
+ pgac_cv_check_libcurl=no
+fi
+rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_check_libcurl" >&5
+$as_echo "$pgac_cv_check_libcurl" >&6; }
+
+if test "$pgac_cv_check_libcurl" != yes; then
+ as_fn_error $? "
+*** The installed version of libcurl is too old to use with PostgreSQL.
+*** libcurl version 8.4.0 or later is required." "$LINENO" 5
+fi
fi
# for contrib/sepgsql
diff --git a/configure.ac b/configure.ac
index 1ec0712ed5..ab0949b8c4 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1445,6 +1445,7 @@ AC_SUBST(LDAP_LIBS_BE)
if test "$with_oauth" = curl ; then
AC_CHECK_LIB(curl, curl_multi_init, [], [AC_MSG_ERROR([library 'curl' is required for --with-oauth=curl])])
+ PGAC_CHECK_LIBCURL
fi
# for contrib/sepgsql
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
index 024f304e4d..ec1418c3fc 100644
--- a/src/backend/libpq/auth-oauth.c
+++ b/src/backend/libpq/auth-oauth.c
@@ -476,9 +476,9 @@ generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
initStringInfo(&buf);
/*
- * TODO: note that escaping here should be belt-and-suspenders, since
- * escapable characters aren't valid in either the issuer URI or the scope
- * list, but the HBA doesn't enforce that yet.
+ * Escaping the string here is belt-and-suspenders defensive programming
+ * since escapable characters aren't valid in either the issuer URI or the
+ * scope list, but the HBA doesn't enforce that yet.
*/
appendStringInfoString(&buf, "{ \"status\": \"invalid_token\", ");
@@ -533,7 +533,9 @@ validate_token_format(const char *header)
if (!header || strlen(header) <= 7)
{
ereport(COMMERROR,
- (errmsg("malformed OAuth bearer token 1")));
+ errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAuth bearer token"),
+ errdetail_log("Bearer token is less than 8 bytes."));
return NULL;
}
@@ -551,9 +553,9 @@ validate_token_format(const char *header)
if (!*token)
{
ereport(COMMERROR,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("malformed OAuth bearer token 2"),
- errdetail("Bearer token is empty.")));
+ errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAuth bearer token"),
+ errdetail_log("Bearer token is empty."));
return NULL;
}
@@ -573,9 +575,9 @@ validate_token_format(const char *header)
* of someone's password into the logs.
*/
ereport(COMMERROR,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("malformed OAuth bearer token 3"),
- errdetail("Bearer token is not in the correct format.")));
+ errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAuth bearer token"),
+ errdetail_log("Bearer token is not in the correct format."));
return NULL;
}
@@ -617,10 +619,10 @@ validate(Port *port, const char *auth)
/* Make sure the validator authenticated the user. */
if (ret->authn_id == NULL || ret->authn_id[0] == '\0')
{
- /* TODO: use logdetail; reduce message duplication */
ereport(LOG,
- (errmsg("OAuth bearer authentication failed for user \"%s\": validator provided no identity",
- port->user_name)));
+ errmsg("OAuth bearer authentication failed for user \"%s\"",
+ port->user_name),
+ errdetail_log("Validator provided no identity"));
return false;
}
diff --git a/src/common/jsonapi.c b/src/common/jsonapi.c
index 4aeedc0bc5..758be913be 100644
--- a/src/common/jsonapi.c
+++ b/src/common/jsonapi.c
@@ -29,13 +29,15 @@
#endif
/*
- * In backend, we will use palloc/pfree along with StringInfo. In frontend, use
- * malloc and PQExpBuffer, and return JSON_OUT_OF_MEMORY on out-of-memory.
+ * In backend, we will use palloc/pfree along with StringInfo. In frontend,
+ * use malloc and PQExpBuffer, and return JSON_OUT_OF_MEMORY on out-of-memory.
*/
#ifdef FRONTEND
#define STRDUP(s) strdup(s)
#define ALLOC(size) malloc(size)
+#define ALLOC0(size) calloc(1, size)
+#define FREE(s) free(s)
#define appendStrVal appendPQExpBuffer
#define appendBinaryStrVal appendBinaryPQExpBuffer
@@ -48,6 +50,8 @@
#define STRDUP(s) pstrdup(s)
#define ALLOC(size) palloc(size)
+#define ALLOC0(size) palloc0(size)
+#define FREE(s) pfree(s)
#define appendStrVal appendStringInfo
#define appendBinaryStrVal appendBinaryStringInfo
@@ -181,6 +185,9 @@ IsValidJsonNumber(const char *str, int len)
* responsible for freeing the returned struct, either by calling
* freeJsonLexContext() or (in backend environment) via memory context
* cleanup.
+ *
+ * In frontend code this can return NULL on OOM, so callers must inspect the
+ * returned pointer.
*/
JsonLexContext *
makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
@@ -188,7 +195,9 @@ makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
{
if (lex == NULL)
{
- lex = palloc0(sizeof(JsonLexContext));
+ lex = ALLOC0(sizeof(JsonLexContext));
+ if (!lex)
+ return NULL;
lex->flags |= JSONLEX_FREE_STRUCT;
}
else
@@ -227,6 +236,9 @@ freeJsonLexContext(JsonLexContext *lex)
{
static const JsonLexContext empty = {0};
+ if (!lex)
+ return;
+
if (lex->flags & JSONLEX_FREE_STRVAL)
destroyStrVal(lex->strval);
@@ -234,7 +246,7 @@ freeJsonLexContext(JsonLexContext *lex)
destroyStrVal(lex->errormsg);
if (lex->flags & JSONLEX_FREE_STRUCT)
- pfree(lex);
+ FREE(lex);
else
*lex = empty;
}
diff --git a/src/common/parse_manifest.c b/src/common/parse_manifest.c
index 40ec3b4f58..3d08e3177f 100644
--- a/src/common/parse_manifest.c
+++ b/src/common/parse_manifest.c
@@ -136,6 +136,8 @@ json_parse_manifest(JsonManifestParseContext *context, char *buffer,
/* Create a JSON lexing context. */
lex = makeJsonLexContextCstringLen(NULL, buffer, size, PG_UTF8, true);
+ if (!lex)
+ json_manifest_parse_failure(context, "out of memory");
/* Set up semantic actions. */
sem.semstate = &parse;
diff --git a/src/include/common/oauth-common.h b/src/include/common/oauth-common.h
index 5ff3488bfb..8fe5626778 100644
--- a/src/include/common/oauth-common.h
+++ b/src/include/common/oauth-common.h
@@ -3,7 +3,7 @@
* oauth-common.h
* Declarations for helper functions used for OAuth/OIDC authentication
*
- * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* src/include/common/oauth-common.h
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 2618c293af..c532cef4d7 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -116,6 +116,8 @@ backend_src = $(top_srcdir)/src/backend
# which seems to insert references to that even in pure C code. Excluding
# __tsan_func_exit is necessary when using ThreadSanitizer data race detector
# which use this function for instrumentation of function exit.
+# libcurl registers an exit handler in the memory debugging code when running
+# with LeakSanitizer.
# Skip the test when profiling, as gcc may insert exit() calls for that.
# Also skip the test on platforms where libpq infrastructure may be provided
# by statically-linked libraries, as we can't expect them to honor this
@@ -123,7 +125,7 @@ backend_src = $(top_srcdir)/src/backend
libpq-refs-stamp: $(shlib)
ifneq ($(enable_coverage), yes)
ifeq (,$(filter solaris,$(PORTNAME)))
- @if nm -A -u $< 2>/dev/null | grep -v -e __cxa_atexit -e __tsan_func_exit | grep exit; then \
+ @if nm -A -u $< 2>/dev/null | grep -v -e __cxa_atexit -e __tsan_func_exit -e _atexit | grep exit; then \
echo 'libpq must not be calling any function which invokes exit'; exit 1; \
fi
endif
diff --git a/src/interfaces/libpq/fe-auth-oauth-curl.c b/src/interfaces/libpq/fe-auth-oauth-curl.c
index 0504f96e4e..9dd8454cac 100644
--- a/src/interfaces/libpq/fe-auth-oauth-curl.c
+++ b/src/interfaces/libpq/fe-auth-oauth-curl.c
@@ -3,7 +3,7 @@
* fe-auth-oauth-curl.c
* The libcurl implementation of OAuth/OIDC authentication.
*
- * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* IDENTIFICATION
@@ -31,6 +31,8 @@
#include "libpq-int.h"
#include "mb/pg_wchar.h"
+#define MAX_OAUTH_RESPONSE_SIZE (1024 * 1024)
+
/*
* Parsed JSON Representations
*
@@ -143,7 +145,7 @@ free_token(struct token *tok)
/* States for the overall async machine. */
typedef enum
{
- OAUTH_STEP_INIT,
+ OAUTH_STEP_INIT = 0,
OAUTH_STEP_DISCOVERY,
OAUTH_STEP_DEVICE_AUTHORIZATION,
OAUTH_STEP_TOKEN_REQUEST,
@@ -151,8 +153,9 @@ typedef enum
} OAuthStep;
/*
- * The async_ctx holds onto state that needs to persist across multiple calls to
- * pg_fe_run_oauth_flow(). Almost everything interacts with this in some way.
+ * The async_ctx holds onto state that needs to persist across multiple calls
+ * to pg_fe_run_oauth_flow(). Almost everything interacts with this in some
+ * way.
*/
struct async_ctx
{
@@ -162,9 +165,10 @@ struct async_ctx
int timerfd; /* a timerfd for signaling async timeouts */
#endif
pgsocket mux; /* the multiplexer socket containing all
- * descriptors tracked by cURL, plus the
+ * descriptors tracked by libcurl, plus the
* timerfd */
- CURLM *curlm; /* top-level multi handle for cURL operations */
+ CURLM *curlm; /* top-level multi handle for libcurl
+ * operations */
CURL *curl; /* the (single) easy handle for serial
* requests */
@@ -183,7 +187,7 @@ struct async_ctx
* actx_error[_str] to manipulate this. This must be filled
* with something useful on an error.
*
- * - curl_err: an optional static error buffer used by cURL to put
+ * - curl_err: an optional static error buffer used by libcurl to put
* detailed information about failures. Unfortunately
* untranslatable.
*
@@ -195,7 +199,7 @@ struct async_ctx
*/
const char *errctx; /* not freed; must point to static allocation */
PQExpBufferData errbuf;
- char curl_err[CURL_ERROR_SIZE];
+ PQExpBufferData curl_err;
/*
* These documents need to survive over multiple calls, and are therefore
@@ -205,6 +209,8 @@ struct async_ctx
struct device_authz authz;
bool user_prompted; /* have we already sent the authz prompt? */
+
+ int running;
};
/*
@@ -238,7 +244,7 @@ free_curl_async_ctx(PGconn *conn, void *ctx)
if (err)
libpq_append_conn_error(conn,
- "cURL easy handle removal failed: %s",
+ "libcurl easy handle removal failed: %s",
curl_multi_strerror(err));
}
@@ -258,7 +264,7 @@ free_curl_async_ctx(PGconn *conn, void *ctx)
if (err)
libpq_append_conn_error(conn,
- "cURL multi handle cleanup failed: %s",
+ "libcurl multi handle cleanup failed: %s",
curl_multi_strerror(err));
}
@@ -292,8 +298,8 @@ free_curl_async_ctx(PGconn *conn, void *ctx)
appendPQExpBufferStr(&(ACTX)->errbuf, S)
/*
- * Macros for getting and setting state for the connection's two cURL handles,
- * so you don't have to write out the error handling every time.
+ * Macros for getting and setting state for the connection's two libcurl
+ * handles, so you don't have to write out the error handling every time.
*/
#define CHECK_MSETOPT(ACTX, OPT, VAL, FAILACTION) \
@@ -622,19 +628,28 @@ parse_oauth_json(struct async_ctx *actx, const struct json_field *fields)
actx_error(actx, "no content type was provided");
goto cleanup;
}
- else if (strcasecmp(content_type, "application/json") != 0)
+
+ /*
+ * We only check the media-type and not the parameters, so we need to
+ * perform a length limited comparison and not compare the whole string.
+ */
+ if (pg_strncasecmp(content_type, "application/json", strlen("application/json")) != 0)
{
- actx_error(actx, "unexpected content type \"%s\"", content_type);
- goto cleanup;
+ actx_error(actx, "unexpected content type: \"%s\"", content_type);
+ return false;
}
if (strlen(resp->data) != resp->len)
{
actx_error(actx, "response contains embedded NULLs");
- goto cleanup;
+ return false;
}
- makeJsonLexContextCstringLen(&lex, resp->data, resp->len, PG_UTF8, true);
+ if (!makeJsonLexContextCstringLen(&lex, resp->data, resp->len, PG_UTF8, true))
+ {
+ actx_error(actx, "out of memory");
+ return false;
+ }
ctx.errbuf = &actx->errbuf;
ctx.fields = fields;
@@ -787,7 +802,11 @@ parse_device_authz(struct async_ctx *actx, struct device_authz *authz)
authz->interval = parse_interval(authz->interval_str);
else
{
- /* TODO: handle default interval of 5 seconds */
+ /*
+ * RFC 8628 specify 5 seconds as the default value if the server
+ * doesn't provide an interval.
+ */
+ authz->interval = 5;
}
return true;
@@ -838,7 +857,7 @@ parse_access_token(struct async_ctx *actx, struct token *tok)
}
/*
- * cURL Multi Setup/Callbacks
+ * libcurl Multi Setup/Callbacks
*/
/*
@@ -894,7 +913,7 @@ setup_multiplexer(struct async_ctx *actx)
/*
* Adds and removes sockets from the multiplexer set, as directed by the
- * cURL multi handle.
+ * libcurl multi handle.
*/
static int
register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
@@ -925,7 +944,7 @@ register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
break;
default:
- actx_error(actx, "unknown cURL socket operation (%d)", what);
+ actx_error(actx, "unknown libcurl socket operation: %d", what);
return -1;
}
@@ -997,7 +1016,7 @@ register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
break;
default:
- actx_error(actx, "unknown cURL socket operation (%d)", what);
+ actx_error(actx, "unknown libcurl socket operation: %d", what);
return -1;
}
@@ -1018,7 +1037,7 @@ register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
/*
* EV_RECEIPT should guarantee one EV_ERROR result for every change,
* whether successful or not. Failed entries contain a non-zero errno
- * in the `data` field.
+ * in the data field.
*/
Assert(ev_out[i].flags & EV_ERROR);
@@ -1043,9 +1062,8 @@ register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
/*
* Adds or removes timeouts from the multiplexer set, as directed by the
- * cURL multi handle. Rather than continually adding and removing the timer,
- * we keep it in the set at all times and just disarm it when it's not
- * needed.
+ * libcurl multi handle. Rather than continually adding and removing the timer,
+ * we keep it in the set at all times and just disarm it when it's not needed.
*/
static int
register_timer(CURLM *curlm, long timeout, void *ctx)
@@ -1061,9 +1079,9 @@ register_timer(CURLM *curlm, long timeout, void *ctx)
else if (timeout == 0)
{
/*
- * A zero timeout means cURL wants us to call back immediately. That's
- * not technically an option for timerfd, but we can make the timeout
- * ridiculously short.
+ * A zero timeout means libcurl wants us to call back immediately.
+ * That's not technically an option for timerfd, but we can make the
+ * timeout ridiculously short.
*
* TODO: maybe just signal drive_request() to immediately call back in
* this case?
@@ -1098,8 +1116,21 @@ register_timer(CURLM *curlm, long timeout, void *ctx)
return 0;
}
+static int
+debug_callback(CURL *handle, curl_infotype *type, char *data, size_t size,
+ void *clientp)
+{
+ struct async_ctx *actx = (struct async_ctx *) clientp;
+
+ /* For now we only store TEXT debug information, extending is a TODO */
+ if (type == CURLINFO_TEXT)
+ appendBinaryPQExpBuffer(&actx->curl_err, data, size);
+
+ return 0;
+}
+
/*
- * Initializes the two cURL handles in the async_ctx. The multi handle,
+ * Initializes the two libcurl handles in the async_ctx. The multi handle,
* actx->curlm, is what drives the asynchronous engine and tells us what to do
* next. The easy handle, actx->curl, encapsulates the state for a single
* request/response. It's added to the multi handle as needed, during
@@ -1108,17 +1139,17 @@ register_timer(CURLM *curlm, long timeout, void *ctx)
static bool
setup_curl_handles(struct async_ctx *actx)
{
- curl_version_info_data *curl_info;
+ curl_version_info_data *curl_info;
/*
* Create our multi handle. This encapsulates the entire conversation with
- * cURL for this connection.
+ * libcurl for this connection.
*/
actx->curlm = curl_multi_init();
if (!actx->curlm)
{
/* We don't get a lot of feedback on the failure reason. */
- actx_error(actx, "failed to create cURL multi handle");
+ actx_error(actx, "failed to create libcurl multi handle");
return false;
}
@@ -1143,7 +1174,7 @@ setup_curl_handles(struct async_ctx *actx)
actx->curl = curl_easy_init();
if (!actx->curl)
{
- actx_error(actx, "failed to create cURL handle");
+ actx_error(actx, "failed to create libcurl handle");
return false;
}
@@ -1160,9 +1191,14 @@ setup_curl_handles(struct async_ctx *actx)
/* No alternative resolver, TODO: warn about timeouts */
}
- /* TODO investigate using conn->Pfdebug and CURLOPT_DEBUGFUNCTION here */
+ /*
+ * Set a callback for retrieving error information from libcurl, the
+ * function only takes effect when CURLOPT_VERBOSE has been set so make
+ * sure the order is kept.
+ */
+ CHECK_SETOPT(actx, CURLOPT_DEBUGDATA, actx, return false);
+ CHECK_SETOPT(actx, CURLOPT_DEBUGFUNCTION, debug_callback, return false);
CHECK_SETOPT(actx, CURLOPT_VERBOSE, 1L, return false);
- CHECK_SETOPT(actx, CURLOPT_ERRORBUFFER, actx->curl_err, return false);
/*
* Only HTTP[S] is allowed. TODO: disallow HTTP without user opt-in
@@ -1175,7 +1211,12 @@ setup_curl_handles(struct async_ctx *actx)
* pretty strict when it comes to provider behavior, so we have to check
* what comes back anyway.)
*/
- actx->headers = curl_slist_append(actx->headers, "Accept:"); /* TODO: check result */
+ actx->headers = curl_slist_append(actx->headers, "Accept:");
+ if (actx->headers == NULL)
+ {
+ actx_error(actx, "out of memory");
+ return false;
+ }
CHECK_SETOPT(actx, CURLOPT_HTTPHEADER, actx->headers, return false);
return true;
@@ -1186,8 +1227,10 @@ setup_curl_handles(struct async_ctx *actx)
*/
/*
- * Response callback from cURL; appends the response body into actx->work_data.
- * See start_request().
+ * Response callback from libcurl which appends the response body into
+ * actx->work_data (see start_request()). The maximum size of the data is
+ * defined by CURL_MAX_WRITE_SIZE which by default is 16kb (and can only be
+ * changed by recompiling libcurl).
*/
static size_t
append_data(char *buf, size_t size, size_t nmemb, void *userdata)
@@ -1195,9 +1238,19 @@ append_data(char *buf, size_t size, size_t nmemb, void *userdata)
PQExpBuffer resp = userdata;
size_t len = size * nmemb;
- /* TODO: cap the maximum size */
+ /* In case we receive data over the threshold, abort the transfer */
+ if ((resp->len + len) > MAX_OAUTH_RESPONSE_SIZE)
+ return 0;
+
+ /* The data passed from libcurl is not null-terminated */
appendBinaryPQExpBuffer(resp, buf, len);
- /* TODO: check for broken buffer */
+
+ /*
+ * Signal an error in order to abort the transfer in case we ran out of
+ * memory in accepting the data.
+ */
+ if (PQExpBufferBroken(resp))
+ return 0;
return len;
}
@@ -1214,7 +1267,6 @@ static bool
start_request(struct async_ctx *actx)
{
CURLMcode err;
- int running;
resetPQExpBuffer(&actx->work_data);
CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data, return false);
@@ -1228,7 +1280,7 @@ start_request(struct async_ctx *actx)
return false;
}
- err = curl_multi_socket_action(actx->curlm, CURL_SOCKET_TIMEOUT, 0, &running);
+ err = curl_multi_socket_action(actx->curlm, CURL_SOCKET_TIMEOUT, 0, &actx->running);
if (err)
{
actx_error(actx, "asynchronous HTTP request failed: %s",
@@ -1237,19 +1289,11 @@ start_request(struct async_ctx *actx)
}
/*
- * Sanity check.
- *
- * TODO: even though this is nominally an asynchronous process, there are
- * apparently operations that can synchronously fail by this point, such
- * as connections to closed local ports. Maybe we need to let this case
- * fall through to drive_request instead, or else perform a
- * curl_multi_info_read immediately.
+ * Even though this is nominally an asynchronous process, there are some
+ * operations that can synchronously fail by this point like connections
+ * to closed local ports. Fall through and leave the sanity check for the
+ * next state consuming actx.
*/
- if (running != 1)
- {
- actx_error(actx, "failed to queue HTTP request");
- return false;
- }
return true;
}
@@ -1262,12 +1306,18 @@ static PostgresPollingStatusType
drive_request(struct async_ctx *actx)
{
CURLMcode err;
- int running;
CURLMsg *msg;
int msgs_left;
bool done;
- err = curl_multi_socket_all(actx->curlm, &running);
+ /* Sanity check the previous operation */
+ if (actx->running != 1)
+ {
+ actx_error(actx, "failed to queue HTTP request");
+ return false;
+ }
+
+ err = curl_multi_socket_all(actx->curlm, &actx->running);
if (err)
{
actx_error(actx, "asynchronous HTTP request failed: %s",
@@ -1275,7 +1325,7 @@ drive_request(struct async_ctx *actx)
return PGRES_POLLING_FAILED;
}
- if (running)
+ if (actx->running)
{
/* We'll come back again. */
return PGRES_POLLING_READING;
@@ -1287,7 +1337,7 @@ drive_request(struct async_ctx *actx)
if (msg->msg != CURLMSG_DONE)
{
/*
- * Future cURL versions may define new message types; we don't
+ * Future libcurl versions may define new message types; we don't
* know how to handle them, so we'll ignore them.
*/
continue;
@@ -1304,7 +1354,7 @@ drive_request(struct async_ctx *actx)
err = curl_multi_remove_handle(actx->curlm, msg->easy_handle);
if (err)
{
- actx_error(actx, "cURL easy handle removal failed: %s",
+ actx_error(actx, "libcurl easy handle removal failed: %s",
curl_multi_strerror(err));
return PGRES_POLLING_FAILED;
}
@@ -1489,7 +1539,12 @@ start_device_authz(struct async_ctx *actx, PGconn *conn)
appendPQExpBuffer(work_buffer, "client_id=%s", conn->oauth_client_id);
if (conn->oauth_scope)
appendPQExpBuffer(work_buffer, "&scope=%s", conn->oauth_scope);
- /* TODO check for broken buffer */
+
+ if (PQExpBufferBroken(work_buffer))
+ {
+ actx_error(actx, "out of memory");
+ return false;
+ }
/* Make our request. */
CHECK_SETOPT(actx, CURLOPT_URL, device_authz_uri, return false);
@@ -1631,37 +1686,40 @@ finish_token_request(struct async_ctx *actx, struct token *tok)
CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
/*
- * Per RFC 6749, Section 5, a successful response uses 200 OK. An error
- * response uses either 400 Bad Request or 401 Unauthorized.
- *
- * TODO: there are references online to 403 appearing in the wild...
+ * Per RFC 6749, Section 5, a successful response uses 200 OK.
*/
- if (response_code != 200
- && response_code != 400
- /* && response_code != 401 TODO */ )
+ if (response_code == 200)
{
- actx_error(actx, "unexpected response code %ld", response_code);
- return false;
+ actx->errctx = "failed to parse access token response";
+ if (!parse_access_token(actx, tok))
+ return false; /* error message already set */
+
+ return true;
}
/*
- * Pull the fields we care about from the document.
+ * An error response uses either 400 Bad Request or 401 Unauthorized.
+ * There are references online to implementations using 403 for error
+ * return which would violate the specification. For now we stick to the
+ * specification but we might have to revisit this.
*/
- if (response_code == 200)
+ if (response_code == 400 || response_code == 401)
{
- actx->errctx = "failed to parse access token response";
- if (!parse_access_token(actx, tok))
- return false; /* error message already set */
+ if (!parse_token_error(actx, &tok->err))
+ return false;
+
+ return true;
}
- else if (!parse_token_error(actx, &tok->err))
- return false;
- return true;
+ /* Any other response codes are considered invalid */
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
}
+
/*
- * The top-level, nonblocking entry point for the cURL implementation. This will
- * be called several times to pump the async engine.
+ * The top-level, nonblocking entry point for the libcurl implementation. This
+ * will be called several times to pump the async engine.
*
* The architecture is based on PQconnectPoll(). The first half drives the
* connection state forward as necessary, returning if we're not ready to
@@ -1682,7 +1740,7 @@ pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
struct token tok = {0};
/*
- * XXX This is not safe. cURL has stringent requirements for the thread
+ * XXX This is not safe. libcurl has stringent requirements for the thread
* context in which you call curl_global_init(), because it's going to try
* initializing a bunch of other libraries (OpenSSL, Winsock...). And we
* probably need to consider both the TLS backend libcurl is compiled
@@ -1691,16 +1749,16 @@ pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
* Recent versions of libcurl have improved the thread-safety situation,
* but you apparently can't check at compile time whether the
* implementation is thread-safe, and there's a chicken-and-egg problem
- * where you can't check the thread safety until you've initialized cURL,
- * which you can't do before you've made sure it's thread-safe...
+ * where you can't check the thread safety until you've initialized
+ * libcurl, which you can't do before you've made sure it's thread-safe...
*
* We know we've already initialized Winsock by this point, so we should
- * be able to safely skip that bit. But we have to tell cURL to initialize
- * everything else, because other pieces of our client executable may
- * already be using cURL for their own purposes. If we initialize libcurl
- * first, with only a subset of its features, we could break those other
- * clients nondeterministically, and that would probably be a nightmare to
- * debug.
+ * be able to safely skip that bit. But we have to tell libcurl to
+ * initialize everything else, because other pieces of our client
+ * executable may already be using libcurl for their own purposes. If we
+ * initialize libcurl first, with only a subset of its features, we could
+ * break those other clients nondeterministically, and that would probably
+ * be a nightmare to debug.
*/
curl_global_init(CURL_GLOBAL_ALL
& ~CURL_GLOBAL_WIN32); /* we already initialized Winsock */
@@ -1729,6 +1787,7 @@ pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
initPQExpBuffer(&actx->work_data);
initPQExpBuffer(&actx->errbuf);
+ initPQExpBuffer(&actx->curl_err);
if (!setup_multiplexer(actx))
goto error_return;
@@ -1873,16 +1932,20 @@ pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
* errors; anything else and we bail.
*/
err = &tok.err;
- if (!err->error || (strcmp(err->error, "authorization_pending")
- && strcmp(err->error, "slow_down")))
+ if (!err->error)
+ {
+ actx_error(actx, "unknown error");
+ goto error_return;
+ }
+
+ if (strcmp(err->error, "authorization_pending") != 0 &&
+ strcmp(err->error, "slow_down") != 0)
{
- /* TODO handle !err->error */
if (err->error_description)
appendPQExpBuffer(&actx->errbuf, "%s ",
err->error_description);
appendPQExpBuffer(&actx->errbuf, "(%s)", err->error);
-
goto error_return;
}
@@ -1892,7 +1955,14 @@ pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
*/
if (strcmp(err->error, "slow_down") == 0)
{
- actx->authz.interval += 5; /* TODO check for overflow? */
+ int prev_interval = actx->authz.interval;
+
+ actx->authz.interval += 5;
+ if (actx->authz.interval < prev_interval)
+ {
+ actx_error(actx, "slow_down interval overflow");
+ goto error_return;
+ }
}
/*
@@ -1959,21 +2029,8 @@ error_return:
else
appendPQExpBufferStr(&conn->errorMessage, actx->errbuf.data);
- if (actx->curl_err[0])
- {
- size_t len;
-
- appendPQExpBuffer(&conn->errorMessage, " (%s)", actx->curl_err);
-
- /* Sometimes libcurl adds a newline to the error buffer. :( */
- len = conn->errorMessage.len;
- if (len >= 2 && conn->errorMessage.data[len - 2] == '\n')
- {
- conn->errorMessage.data[len - 2] = ')';
- conn->errorMessage.data[len - 1] = '\0';
- conn->errorMessage.len--;
- }
- }
+ if (actx->curl_err.len > 0)
+ appendPQExpBuffer(&conn->errorMessage, " (%s)", actx->curl_err.data);
appendPQExpBufferStr(&conn->errorMessage, "\n");
diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
index 66ee8ff076..61de9ac451 100644
--- a/src/interfaces/libpq/fe-auth-oauth.c
+++ b/src/interfaces/libpq/fe-auth-oauth.c
@@ -3,7 +3,7 @@
* fe-auth-oauth.c
* The front-end (client) implementation of OAuth/OIDC authentication.
*
- * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* IDENTIFICATION
@@ -247,7 +247,12 @@ handle_oauth_sasl_error(PGconn *conn, char *msg, int msglen)
return false;
}
- makeJsonLexContextCstringLen(&lex, msg, msglen, PG_UTF8, true);
+ if (!makeJsonLexContextCstringLen(&lex, msg, msglen, PG_UTF8, true))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("out of memory"));
+ return false;
+ }
initPQExpBuffer(&ctx.errbuf);
sem.semstate = &ctx;
diff --git a/src/interfaces/libpq/fe-auth-oauth.h b/src/interfaces/libpq/fe-auth-oauth.h
index 8d4ea45aa8..6e5e946364 100644
--- a/src/interfaces/libpq/fe-auth-oauth.h
+++ b/src/interfaces/libpq/fe-auth-oauth.h
@@ -4,7 +4,7 @@
*
* Definitions for OAuth authentication implementations
*
- * Portions Copyright (c) 2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 2024, PostgreSQL Global Development Group
*
* src/interfaces/libpq/fe-auth-oauth.h
*
diff --git a/src/test/modules/oauth_validator/validator.c b/src/test/modules/oauth_validator/validator.c
index 09a4bf61d2..7b4dc9c494 100644
--- a/src/test/modules/oauth_validator/validator.c
+++ b/src/test/modules/oauth_validator/validator.c
@@ -66,7 +66,7 @@ validate_token(ValidatorModuleState *state, const char *token, const char *role)
/* Check to make sure our private state still exists. */
if (state->private_data != PRIVATE_COOKIE)
elog(ERROR, "oauth_validator: private state cookie changed to %p",
- state->private_data);
+ state->private_data);
res = palloc(sizeof(ValidatorModuleResult));
diff --git a/src/test/perl/PostgreSQL/Test/OAuthServer.pm b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
index d96733f531..9e18186f23 100644
--- a/src/test/perl/PostgreSQL/Test/OAuthServer.pm
+++ b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
@@ -4,10 +4,10 @@ package PostgreSQL::Test::OAuthServer;
use warnings;
use strict;
-use threads;
use Scalar::Util;
use Socket;
use IO::Select;
+use Test::More;
local *server_socket;
@@ -34,9 +34,9 @@ sub run
my $port;
my $pid = open(my $read_fh, "-|", $ENV{PYTHON}, "t/oauth_server.py")
- // die "failed to start OAuth server: $!";
+ or die "failed to start OAuth server: $!";
- read($read_fh, $port, 7) // die "failed to read port number: $!";
+ read($read_fh, $port, 7) or die "failed to read port number: $!";
chomp $port;
die "server did not advertise a valid port"
unless Scalar::Util::looks_like_number($port);
@@ -45,14 +45,14 @@ sub run
$self->{'port'} = $port;
$self->{'child'} = $read_fh;
- print("# OAuth provider (PID $pid) is listening on port $port\n");
+ diag("OAuth provider (PID $pid) is listening on port $port\n");
}
sub stop
{
my $self = shift;
- print("# Sending SIGTERM to OAuth provider PID: $self->{'pid'}\n");
+ diag("Sending SIGTERM to OAuth provider PID: $self->{'pid'}\n");
kill(15, $self->{'pid'});
$self->{'pid'} = undef;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 0eaa079261..0e95e7a0e0 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3002,6 +3002,7 @@ VacuumStmt
ValidIOData
ValidateIndexState
ValidatorModuleState
+ValidatorModuleResult
ValuesScan
ValuesScanState
Var
--
2.32.1 (Apple Git-133)
=
v22-0003-backend-add-OAUTHBEARER-SASL-mechanism.patch (application/octet-stream, 52.2K) download
From 96a896b37a1b4716e1ef2e788df5da7073af0887 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 4 May 2021 16:21:11 -0700
Subject: [PATCH v22 3/4] backend: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) on the server side. This adds a new
auth method, oauth, to pg_hba.
Because OAuth implementations vary so wildly, and bearer token
validation is heavily dependent on the issuing party, authn/z is done by
communicating with an external validator module using callbacks.
The module is responsible for:
1. Validate the bearer token. The correct way to do this depends on the
issuer, but it generally involves either cryptographic operations to
prove that the token was issued by a trusted party, or the
presentation of the bearer token to some other party so that _it_ can
perform validation.
The command MUST maintain confidentiality of the bearer token, since
in most cases it can be used just like a password. (There are ways to
cryptographically bind tokens to client certificates, but they are
way beyond the scope of this commit message.)
If the token cannot be validated, the authorized member of the
ValidatorModuleResult struct is used to indicate failure.
Further authentication/authorization is pointless if
the bearer token wasn't issued by someone you trust.
3. Authenticate the user, authorize the user, or both:
a. To authenticate the user, use the bearer token to retrieve some
trusted identifier string for the end user. The exact process for
this is, again, issuer-dependent. The module wull return the
authenticated identity in the authn_id member.
b. To optionally authorize the user, in combination with the HBA
option trust_validator_authz=1 (see below).
The hard part is in determining whether the given token truly
authorizes the client to use the given role, which must
unfortunately be left as an exercise to the reader.
This obviously requires some care, as a poorly implemented token
validator may silently open the entire database to anyone with a
bearer token. But it may be a more portable approach, since OAuth
is designed as an authorization framework, not an authentication
framework. For example, the user's bearer token could carry an
"allow_superuser_access" claim, which would authorize pseudonymous
database access as any role. It's then up to the OAuth system
administrators to ensure that allow_superuser_access is doled out
only to the proper users.
c. It's possible that the user can be successfully authenticated but
isn't authorized to connect. In this case, the validator module may
return the authenticated ID and then fail with false authorized
member. (This can make it easier to see what's going on in the
Postgres logs.)
The oauth method supports the following HBA options (but note that two
of them are not optional, since we have no way of choosing sensible
defaults):
issuer: Required. The URL of the OAuth issuing party, which the client
must contact to receive a bearer token.
Some real-world examples as of time of writing:
- https://accounts.google.com
- https://login.microsoft.com/[tenant-id]/v2.0
scope: Required. The OAuth scope(s) required for the server to
authenticate and/or authorize the user. This is heavily
deployment-specific, but a simple example is "openid email".
map: Optional. Specify a standard PostgreSQL user map; this works
the same as with other auth methods such as peer. If a map is
not specified, the user ID returned by the token validator
must exactly match the role that's being requested (but see
trust_validator_authz, below).
trust_validator_authz:
Optional. When set to 1, this allows the token validator to
take full control of the authorization process. Standard user
mapping is skipped: if the validator command succeeds, the
client is allowed to connect under its desired role and no
further checks are done.
Several TODOs:
- port to platforms other than "modern Linux/BSD"
- implement more helpful handling of HBA misconfigurations
- use logdetail during auth failures
- allow passing the configured issuer to the oauth_validator_command, to
deal with multi-issuer setups
- ...and more.
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
---
src/backend/libpq/Makefile | 1 +
src/backend/libpq/auth-oauth.c | 666 ++++++++++++++++++
src/backend/libpq/auth-sasl.c | 10 +-
src/backend/libpq/auth-scram.c | 4 +-
src/backend/libpq/auth.c | 26 +-
src/backend/libpq/hba.c | 31 +-
src/backend/libpq/meson.build | 1 +
src/backend/utils/misc/guc_tables.c | 12 +
src/common/Makefile | 2 +-
src/include/libpq/auth.h | 17 +
src/include/libpq/hba.h | 6 +-
src/include/libpq/oauth.h | 49 ++
src/include/libpq/sasl.h | 11 +
src/test/modules/meson.build | 1 +
src/test/modules/oauth_validator/.gitignore | 4 +
src/test/modules/oauth_validator/Makefile | 21 +
.../oauth_validator/expected/validator.out | 6 +
src/test/modules/oauth_validator/meson.build | 36 +
.../modules/oauth_validator/sql/validator.sql | 1 +
.../modules/oauth_validator/t/001_server.pl | 74 ++
.../modules/oauth_validator/t/oauth_server.py | 114 +++
src/test/modules/oauth_validator/validator.c | 82 +++
src/test/perl/PostgreSQL/Test/Cluster.pm | 14 +-
src/test/perl/PostgreSQL/Test/OAuthServer.pm | 65 ++
src/tools/pgindent/typedefs.list | 3 +
25 files changed, 1223 insertions(+), 34 deletions(-)
create mode 100644 src/backend/libpq/auth-oauth.c
create mode 100644 src/include/libpq/oauth.h
create mode 100644 src/test/modules/oauth_validator/.gitignore
create mode 100644 src/test/modules/oauth_validator/Makefile
create mode 100644 src/test/modules/oauth_validator/expected/validator.out
create mode 100644 src/test/modules/oauth_validator/meson.build
create mode 100644 src/test/modules/oauth_validator/sql/validator.sql
create mode 100644 src/test/modules/oauth_validator/t/001_server.pl
create mode 100755 src/test/modules/oauth_validator/t/oauth_server.py
create mode 100644 src/test/modules/oauth_validator/validator.c
create mode 100644 src/test/perl/PostgreSQL/Test/OAuthServer.pm
diff --git a/src/backend/libpq/Makefile b/src/backend/libpq/Makefile
index 6d385fd6a4..98eb2a8242 100644
--- a/src/backend/libpq/Makefile
+++ b/src/backend/libpq/Makefile
@@ -15,6 +15,7 @@ include $(top_builddir)/src/Makefile.global
# be-fsstubs is here for historical reasons, probably belongs elsewhere
OBJS = \
+ auth-oauth.o \
auth-sasl.o \
auth-scram.o \
auth.o \
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
new file mode 100644
index 0000000000..024f304e4d
--- /dev/null
+++ b/src/backend/libpq/auth-oauth.c
@@ -0,0 +1,666 @@
+/*-------------------------------------------------------------------------
+ *
+ * auth-oauth.c
+ * Server-side implementation of the SASL OAUTHBEARER mechanism.
+ *
+ * See the following RFC for more details:
+ * - RFC 7628: https://tools.ietf.org/html/rfc7628
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/backend/libpq/auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <unistd.h>
+#include <fcntl.h>
+
+#include "common/oauth-common.h"
+#include "fmgr.h"
+#include "lib/stringinfo.h"
+#include "libpq/auth.h"
+#include "libpq/hba.h"
+#include "libpq/oauth.h"
+#include "libpq/sasl.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "utils/json.h"
+
+/* GUC */
+char *OAuthValidatorLibrary = "";
+
+static void oauth_get_mechanisms(Port *port, StringInfo buf);
+static void *oauth_init(Port *port, const char *selected_mech, const char *shadow_pass);
+static int oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, const char **logdetail);
+
+static void load_validator_library(void);
+static void shutdown_validator_library(int code, Datum arg);
+
+static ValidatorModuleState *validator_module_state;
+static const OAuthValidatorCallbacks *ValidatorCallbacks;
+
+/* Mechanism declaration */
+const pg_be_sasl_mech pg_be_oauth_mech = {
+ oauth_get_mechanisms,
+ oauth_init,
+ oauth_exchange,
+
+ PG_MAX_AUTH_TOKEN_LENGTH,
+};
+
+
+typedef enum
+{
+ OAUTH_STATE_INIT = 0,
+ OAUTH_STATE_ERROR,
+ OAUTH_STATE_FINISHED,
+} oauth_state;
+
+struct oauth_ctx
+{
+ oauth_state state;
+ Port *port;
+ const char *issuer;
+ const char *scope;
+};
+
+static char *sanitize_char(char c);
+static char *parse_kvpairs_for_auth(char **input);
+static void generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen);
+static bool validate(Port *port, const char *auth);
+
+#define KVSEP 0x01
+#define AUTH_KEY "auth"
+#define BEARER_SCHEME "Bearer "
+
+static void
+oauth_get_mechanisms(Port *port, StringInfo buf)
+{
+ /* Only OAUTHBEARER is supported. */
+ appendStringInfoString(buf, OAUTHBEARER_NAME);
+ appendStringInfoChar(buf, '\0');
+}
+
+static void *
+oauth_init(Port *port, const char *selected_mech, const char *shadow_pass)
+{
+ struct oauth_ctx *ctx;
+
+ if (strcmp(selected_mech, OAUTHBEARER_NAME))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("client selected an invalid SASL authentication mechanism")));
+
+ ctx = palloc0(sizeof(*ctx));
+
+ ctx->state = OAUTH_STATE_INIT;
+ ctx->port = port;
+
+ Assert(port->hba);
+ ctx->issuer = port->hba->oauth_issuer;
+ ctx->scope = port->hba->oauth_scope;
+
+ load_validator_library();
+
+ return ctx;
+}
+
+static int
+oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, const char **logdetail)
+{
+ char *p;
+ char cbind_flag;
+ char *auth;
+
+ struct oauth_ctx *ctx = opaq;
+
+ *output = NULL;
+ *outputlen = -1;
+
+ /*
+ * If the client didn't include an "Initial Client Response" in the
+ * SASLInitialResponse message, send an empty challenge, to which the
+ * client will respond with the same data that usually comes in the
+ * Initial Client Response.
+ */
+ if (input == NULL)
+ {
+ Assert(ctx->state == OAUTH_STATE_INIT);
+
+ *output = pstrdup("");
+ *outputlen = 0;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ /*
+ * Check that the input length agrees with the string length of the input.
+ */
+ if (inputlen == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The message is empty.")));
+ if (inputlen != strlen(input))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message length does not match input length.")));
+
+ switch (ctx->state)
+ {
+ case OAUTH_STATE_INIT:
+ /* Handle this case below. */
+ break;
+
+ case OAUTH_STATE_ERROR:
+
+ /*
+ * Only one response is valid for the client during authentication
+ * failure: a single kvsep.
+ */
+ if (inputlen != 1 || *input != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Client did not send a kvsep response.")));
+
+ /* The (failed) handshake is now complete. */
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_FAILURE;
+
+ default:
+ elog(ERROR, "invalid OAUTHBEARER exchange state");
+ return PG_SASL_EXCHANGE_FAILURE;
+ }
+
+ /* Handle the client's initial message. */
+ p = pstrdup(input);
+
+ /*
+ * OAUTHBEARER does not currently define a channel binding (so there is no
+ * OAUTHBEARER-PLUS, and we do not accept a 'p' specifier). We accept a
+ * 'y' specifier purely for the remote chance that a future specification
+ * could define one; then future clients can still interoperate with this
+ * server implementation. 'n' is the expected case.
+ */
+ cbind_flag = *p;
+ switch (cbind_flag)
+ {
+ case 'p':
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The server does not support channel binding for OAuth, but the client message includes channel binding data.")));
+ break;
+
+ case 'y': /* fall through */
+ case 'n':
+ p++;
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Comma expected, but found character \"%s\".",
+ sanitize_char(*p))));
+ p++;
+ break;
+
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected channel-binding flag %s.",
+ sanitize_char(cbind_flag))));
+ }
+
+ /*
+ * Forbid optional authzid (authorization identity). We don't support it.
+ */
+ if (*p == 'a')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("client uses authorization identity, but it is not supported")));
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected attribute %s in client-first-message.",
+ sanitize_char(*p))));
+ p++;
+
+ /* All remaining fields are separated by the RFC's kvsep (\x01). */
+ if (*p != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Key-value separator expected, but found character %s.",
+ sanitize_char(*p))));
+ p++;
+
+ auth = parse_kvpairs_for_auth(&p);
+ if (!auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message does not contain an auth value.")));
+
+ /* We should be at the end of our message. */
+ if (*p)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains additional data after the final terminator.")));
+
+ if (!validate(ctx->port, auth))
+ {
+ generate_error_response(ctx, output, outputlen);
+
+ ctx->state = OAUTH_STATE_ERROR;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_SUCCESS;
+}
+
+/*
+ * Convert an arbitrary byte to printable form. For error messages.
+ *
+ * If it's a printable ASCII character, print it as a single character.
+ * otherwise, print it in hex.
+ *
+ * The returned pointer points to a static buffer.
+ */
+static char *
+sanitize_char(char c)
+{
+ static char buf[5];
+
+ if (c >= 0x21 && c <= 0x7E)
+ snprintf(buf, sizeof(buf), "'%c'", c);
+ else
+ snprintf(buf, sizeof(buf), "0x%02x", (unsigned char) c);
+ return buf;
+}
+
+/*
+ * Performs syntactic validation of a key and value from the initial client
+ * response. (Semantic validation of interesting values must be performed
+ * later.)
+ */
+static void
+validate_kvpair(const char *key, const char *val)
+{
+ /*-----
+ * From Sec 3.1:
+ * key = 1*(ALPHA)
+ */
+ static const char *key_allowed_set =
+ "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
+
+ size_t span;
+
+ if (!key[0])
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an empty key name.")));
+
+ span = strspn(key, key_allowed_set);
+ if (key[span] != '\0')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an invalid key name.")));
+
+ /*-----
+ * From Sec 3.1:
+ * value = *(VCHAR / SP / HTAB / CR / LF )
+ *
+ * The VCHAR (visible character) class is large; a loop is more
+ * straightforward than strspn().
+ */
+ for (; *val; ++val)
+ {
+ if (0x21 <= *val && *val <= 0x7E)
+ continue; /* VCHAR */
+
+ switch (*val)
+ {
+ case ' ':
+ case '\t':
+ case '\r':
+ case '\n':
+ continue; /* SP, HTAB, CR, LF */
+
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an invalid value.")));
+ }
+ }
+}
+
+/*
+ * Consumes all kvpairs in an OAUTHBEARER exchange message. If the "auth" key is
+ * found, its value is returned.
+ */
+static char *
+parse_kvpairs_for_auth(char **input)
+{
+ char *pos = *input;
+ char *auth = NULL;
+
+ /*----
+ * The relevant ABNF, from Sec. 3.1:
+ *
+ * kvsep = %x01
+ * key = 1*(ALPHA)
+ * value = *(VCHAR / SP / HTAB / CR / LF )
+ * kvpair = key "=" value kvsep
+ * ;;gs2-header = See RFC 5801
+ * client-resp = (gs2-header kvsep *kvpair kvsep) / kvsep
+ *
+ * By the time we reach this code, the gs2-header and initial kvsep have
+ * already been validated. We start at the beginning of the first kvpair.
+ */
+
+ while (*pos)
+ {
+ char *end;
+ char *sep;
+ char *key;
+ char *value;
+
+ /*
+ * Find the end of this kvpair. Note that input is null-terminated by
+ * the SASL code, so the strchr() is bounded.
+ */
+ end = strchr(pos, KVSEP);
+ if (!end)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an unterminated key/value pair.")));
+ *end = '\0';
+
+ if (pos == end)
+ {
+ /* Empty kvpair, signifying the end of the list. */
+ *input = pos + 1;
+ return auth;
+ }
+
+ /*
+ * Find the end of the key name.
+ */
+ sep = strchr(pos, '=');
+ if (!sep)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains a key without a value.")));
+ *sep = '\0';
+
+ /* Both key and value are now safely terminated. */
+ key = pos;
+ value = sep + 1;
+ validate_kvpair(key, value);
+
+ if (!strcmp(key, AUTH_KEY))
+ {
+ if (auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains multiple auth values.")));
+
+ auth = value;
+ }
+ else
+ {
+ /*
+ * The RFC also defines the host and port keys, but they are not
+ * required for OAUTHBEARER and we do not use them. Also, per Sec.
+ * 3.1, any key/value pairs we don't recognize must be ignored.
+ */
+ }
+
+ /* Move to the next pair. */
+ pos = end + 1;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message did not contain a final terminator.")));
+
+ return NULL; /* unreachable */
+}
+
+static void
+generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
+{
+ StringInfoData buf;
+ StringInfoData issuer;
+
+ /*
+ * The admin needs to set an issuer and scope for OAuth to work. There's
+ * not really a way to hide this from the user, either, because we can't
+ * choose a "default" issuer, so be honest in the failure message.
+ *
+ * TODO: see if there's a better place to fail, earlier than this.
+ */
+ if (!ctx->issuer || !ctx->scope)
+ ereport(FATAL,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("OAuth is not properly configured for this user"),
+ errdetail_log("The issuer and scope parameters must be set in pg_hba.conf.")));
+
+ /*------
+ * Build the .well-known URI based on our issuer.
+ * TODO: RFC 8414 defines a competing well-known URI, so we'll probably
+ * have to make this configurable too.
+ */
+ initStringInfo(&issuer);
+ appendStringInfoString(&issuer, ctx->issuer);
+ appendStringInfoString(&issuer, "/.well-known/openid-configuration");
+
+ initStringInfo(&buf);
+
+ /*
+ * TODO: note that escaping here should be belt-and-suspenders, since
+ * escapable characters aren't valid in either the issuer URI or the scope
+ * list, but the HBA doesn't enforce that yet.
+ */
+ appendStringInfoString(&buf, "{ \"status\": \"invalid_token\", ");
+
+ appendStringInfoString(&buf, "\"openid-configuration\": ");
+ escape_json(&buf, issuer.data);
+ pfree(issuer.data);
+
+ appendStringInfoString(&buf, ", \"scope\": ");
+ escape_json(&buf, ctx->scope);
+
+ appendStringInfoString(&buf, " }");
+
+ *output = buf.data;
+ *outputlen = buf.len;
+}
+
+/*-----
+ * Validates the provided Authorization header and returns the token from
+ * within it. NULL is returned on validation failure.
+ *
+ * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
+ * 2.1:
+ *
+ * b64token = 1*( ALPHA / DIGIT /
+ * "-" / "." / "_" / "~" / "+" / "/" ) *"="
+ * credentials = "Bearer" 1*SP b64token
+ *
+ * The "credentials" construction is what we receive in our auth value.
+ *
+ * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
+ * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
+ * compared case-insensitively. (This is not mentioned in RFC 6750, but
+ * it's pointed out in RFC 7628 Sec. 4.)
+ *
+ * Invalid formats are technically a protocol violation, but we shouldn't
+ * reflect any information about the sensitive Bearer token back to the
+ * client; log at COMMERROR instead.
+ *
+ * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
+ */
+static const char *
+validate_token_format(const char *header)
+{
+ size_t span;
+ const char *token;
+ static const char *const b64token_allowed_set =
+ "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-._~+/";
+
+ /* If the token is empty or simply too short to be correct */
+ if (!header || strlen(header) <= 7)
+ {
+ ereport(COMMERROR,
+ (errmsg("malformed OAuth bearer token 1")));
+ return NULL;
+ }
+
+ if (pg_strncasecmp(header, BEARER_SCHEME, strlen(BEARER_SCHEME)))
+ return NULL;
+
+ /* Pull the bearer token out of the auth value. */
+ token = header + strlen(BEARER_SCHEME);
+
+ /* Swallow any additional spaces. */
+ while (*token == ' ')
+ token++;
+
+ /* Tokens must not be empty. */
+ if (!*token)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAuth bearer token 2"),
+ errdetail("Bearer token is empty.")));
+ return NULL;
+ }
+
+ /*
+ * Make sure the token contains only allowed characters. Tokens may end
+ * with any number of '=' characters.
+ */
+ span = strspn(token, b64token_allowed_set);
+ while (token[span] == '=')
+ span++;
+
+ if (token[span] != '\0')
+ {
+ /*
+ * This error message could be more helpful by printing the
+ * problematic character(s), but that'd be a bit like printing a piece
+ * of someone's password into the logs.
+ */
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAuth bearer token 3"),
+ errdetail("Bearer token is not in the correct format.")));
+ return NULL;
+ }
+
+ return token;
+}
+
+static bool
+validate(Port *port, const char *auth)
+{
+ int map_status;
+ ValidatorModuleResult *ret;
+ const char *token;
+
+ /* Ensure that we have a correct token to validate */
+ if (!(token = validate_token_format(auth)))
+ return false;
+
+ /* Call the validation function from the validator module */
+ ret = ValidatorCallbacks->validate_cb(validator_module_state,
+ token, port->user_name);
+
+ if (!ret->authorized)
+ return false;
+
+ if (ret->authn_id)
+ set_authn_id(port, ret->authn_id);
+
+ if (port->hba->oauth_skip_usermap)
+ {
+ /*
+ * If the validator is our authorization authority, we're done.
+ * Authentication may or may not have been performed depending on the
+ * validator implementation; all that matters is that the validator
+ * says the user can log in with the target role.
+ */
+ return true;
+ }
+
+ /* Make sure the validator authenticated the user. */
+ if (ret->authn_id == NULL || ret->authn_id[0] == '\0')
+ {
+ /* TODO: use logdetail; reduce message duplication */
+ ereport(LOG,
+ (errmsg("OAuth bearer authentication failed for user \"%s\": validator provided no identity",
+ port->user_name)));
+ return false;
+ }
+
+ /* Finally, check the user map. */
+ map_status = check_usermap(port->hba->usermap, port->user_name,
+ MyClientConnectionInfo.authn_id, false);
+ return (map_status == STATUS_OK);
+}
+
+static void
+load_validator_library(void)
+{
+ OAuthValidatorModuleInit validator_init;
+
+ if (OAuthValidatorLibrary[0] == '\0')
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("oauth_validator_library is not set")));
+
+ validator_init = (OAuthValidatorModuleInit)
+ load_external_function(OAuthValidatorLibrary,
+ "_PG_oauth_validator_module_init", false, NULL);
+
+ if (validator_init == NULL)
+ ereport(ERROR,
+ (errmsg("%s module \"%s\" have to define the symbol %s",
+ "OAuth validator", OAuthValidatorLibrary, "_PG_oauth_validator_module_init")));
+
+ ValidatorCallbacks = (*validator_init) ();
+
+ validator_module_state = (ValidatorModuleState *) palloc0(sizeof(ValidatorModuleState));
+ if (ValidatorCallbacks->startup_cb != NULL)
+ ValidatorCallbacks->startup_cb(validator_module_state);
+
+ before_shmem_exit(shutdown_validator_library, 0);
+}
+
+static void
+shutdown_validator_library(int code, Datum arg)
+{
+ if (ValidatorCallbacks->shutdown_cb != NULL)
+ ValidatorCallbacks->shutdown_cb(validator_module_state);
+}
diff --git a/src/backend/libpq/auth-sasl.c b/src/backend/libpq/auth-sasl.c
index 08b24d90b4..4039e7fa3e 100644
--- a/src/backend/libpq/auth-sasl.c
+++ b/src/backend/libpq/auth-sasl.c
@@ -20,14 +20,6 @@
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
-/*
- * Maximum accepted size of SASL messages.
- *
- * The messages that the server or libpq generate are much smaller than this,
- * but have some headroom.
- */
-#define PG_MAX_SASL_MESSAGE_LENGTH 1024
-
/*
* Perform a SASL exchange with a libpq client, using a specific mechanism
* implementation.
@@ -103,7 +95,7 @@ CheckSASLAuth(const pg_be_sasl_mech *mech, Port *port, char *shadow_pass,
/* Get the actual SASL message */
initStringInfo(&buf);
- if (pq_getmessage(&buf, PG_MAX_SASL_MESSAGE_LENGTH))
+ if (pq_getmessage(&buf, mech->max_message_length))
{
/* EOF - pq_getmessage already logged error */
pfree(buf.data);
diff --git a/src/backend/libpq/auth-scram.c b/src/backend/libpq/auth-scram.c
index 4161959914..486a34e719 100644
--- a/src/backend/libpq/auth-scram.c
+++ b/src/backend/libpq/auth-scram.c
@@ -113,7 +113,9 @@ static int scram_exchange(void *opaq, const char *input, int inputlen,
const pg_be_sasl_mech pg_be_scram_mech = {
scram_get_mechanisms,
scram_init,
- scram_exchange
+ scram_exchange,
+
+ PG_MAX_SASL_MESSAGE_LENGTH
};
/*
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 2b607c5270..0a5a8640fc 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -29,6 +29,7 @@
#include "libpq/auth.h"
#include "libpq/crypt.h"
#include "libpq/libpq.h"
+#include "libpq/oauth.h"
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
#include "libpq/scram.h"
@@ -45,7 +46,6 @@
*/
static void auth_failed(Port *port, int status, const char *logdetail);
static char *recv_password_packet(Port *port);
-static void set_authn_id(Port *port, const char *id);
/*----------------------------------------------------------------
@@ -201,22 +201,6 @@ static int CheckRADIUSAuth(Port *port);
static int PerformRadiusTransaction(const char *server, const char *secret, const char *portstr, const char *identifier, const char *user_name, const char *passwd);
-/*
- * Maximum accepted size of GSS and SSPI authentication tokens.
- * We also use this as a limit on ordinary password packet lengths.
- *
- * Kerberos tickets are usually quite small, but the TGTs issued by Windows
- * domain controllers include an authorization field known as the Privilege
- * Attribute Certificate (PAC), which contains the user's Windows permissions
- * (group memberships etc.). The PAC is copied into all tickets obtained on
- * the basis of this TGT (even those issued by Unix realms which the Windows
- * realm trusts), and can be several kB in size. The maximum token size
- * accepted by Windows systems is determined by the MaxAuthToken Windows
- * registry setting. Microsoft recommends that it is not set higher than
- * 65535 bytes, so that seems like a reasonable limit for us as well.
- */
-#define PG_MAX_AUTH_TOKEN_LENGTH 65535
-
/*----------------------------------------------------------------
* Global authentication functions
*----------------------------------------------------------------
@@ -305,6 +289,9 @@ auth_failed(Port *port, int status, const char *logdetail)
case uaRADIUS:
errstr = gettext_noop("RADIUS authentication failed for user \"%s\"");
break;
+ case uaOAuth:
+ errstr = gettext_noop("OAuth bearer authentication failed for user \"%s\"");
+ break;
default:
errstr = gettext_noop("authentication failed for user \"%s\": invalid authentication method");
break;
@@ -340,7 +327,7 @@ auth_failed(Port *port, int status, const char *logdetail)
* lifetime of MyClientConnectionInfo, so it is safe to pass a string that is
* managed by an external library.
*/
-static void
+void
set_authn_id(Port *port, const char *id)
{
Assert(id);
@@ -627,6 +614,9 @@ ClientAuthentication(Port *port)
case uaTrust:
status = STATUS_OK;
break;
+ case uaOAuth:
+ status = CheckSASLAuth(&pg_be_oauth_mech, port, NULL, NULL);
+ break;
}
if ((status == STATUS_OK && port->hba->clientcert == clientCertFull)
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index d506c3c0b7..e592bedf9f 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -114,7 +114,8 @@ static const char *const UserAuthName[] =
"ldap",
"cert",
"radius",
- "peer"
+ "peer",
+ "oauth",
};
/*
@@ -1743,6 +1744,8 @@ parse_hba_line(TokenizedAuthLine *tok_line, int elevel)
#endif
else if (strcmp(token->string, "radius") == 0)
parsedline->auth_method = uaRADIUS;
+ else if (strcmp(token->string, "oauth") == 0)
+ parsedline->auth_method = uaOAuth;
else
{
ereport(elevel,
@@ -2062,8 +2065,9 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->auth_method != uaPeer &&
hbaline->auth_method != uaGSS &&
hbaline->auth_method != uaSSPI &&
- hbaline->auth_method != uaCert)
- INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, and cert"));
+ hbaline->auth_method != uaCert &&
+ hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, cert, and oauth"));
hbaline->usermap = pstrdup(val);
}
else if (strcmp(name, "clientcert") == 0)
@@ -2446,6 +2450,27 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->radiusidentifiers = parsed_identifiers;
hbaline->radiusidentifiers_s = pstrdup(val);
}
+ else if (strcmp(name, "issuer") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("issuer", gettext_noop("oauth"));
+ hbaline->oauth_issuer = pstrdup(val);
+ }
+ else if (strcmp(name, "scope") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("scope", gettext_noop("oauth"));
+ hbaline->oauth_scope = pstrdup(val);
+ }
+ else if (strcmp(name, "trust_validator_authz") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("trust_validator_authz", gettext_noop("oauth"));
+ if (strcmp(val, "1") == 0)
+ hbaline->oauth_skip_usermap = true;
+ else
+ hbaline->oauth_skip_usermap = false;
+ }
else
{
ereport(elevel,
diff --git a/src/backend/libpq/meson.build b/src/backend/libpq/meson.build
index 7c65314512..c85527fb01 100644
--- a/src/backend/libpq/meson.build
+++ b/src/backend/libpq/meson.build
@@ -1,6 +1,7 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
backend_sources += files(
+ 'auth-oauth.c',
'auth-sasl.c',
'auth-scram.c',
'auth.c',
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 1e71e7db4a..62d4f15d2b 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -48,6 +48,7 @@
#include "jit/jit.h"
#include "libpq/auth.h"
#include "libpq/libpq.h"
+#include "libpq/oauth.h"
#include "libpq/scram.h"
#include "nodes/queryjumble.h"
#include "optimizer/cost.h"
@@ -4684,6 +4685,17 @@ struct config_string ConfigureNamesString[] =
check_standby_slot_names, assign_standby_slot_names, NULL
},
+ {
+ {"oauth_validator_library", PGC_SIGHUP, CONN_AUTH_AUTH,
+ gettext_noop("Sets the library that will be called to validate OAuth v2 bearer tokens."),
+ NULL,
+ GUC_SUPERUSER_ONLY | GUC_NOT_IN_SAMPLE
+ },
+ &OAuthValidatorLibrary,
+ "",
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, NULL, NULL, NULL, NULL
diff --git a/src/common/Makefile b/src/common/Makefile
index c4f4448e2f..98b8dac0b4 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -41,7 +41,7 @@ override CPPFLAGS += -DVAL_LDFLAGS_SL="\"$(LDFLAGS_SL)\""
override CPPFLAGS += -DVAL_LIBS="\"$(LIBS)\""
override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common -I$(libpq_srcdir) $(CPPFLAGS)
-LIBS += $(PTHREAD_LIBS)
+LIBS += $(PTHREAD_LIBS) $(libpq_pgport)
OBJS_COMMON = \
archive.o \
diff --git a/src/include/libpq/auth.h b/src/include/libpq/auth.h
index 227b41daf6..22f6ab9f1d 100644
--- a/src/include/libpq/auth.h
+++ b/src/include/libpq/auth.h
@@ -16,6 +16,22 @@
#include "libpq/libpq-be.h"
+/*
+ * Maximum accepted size of GSS and SSPI authentication tokens.
+ * We also use this as a limit on ordinary password packet lengths.
+ *
+ * Kerberos tickets are usually quite small, but the TGTs issued by Windows
+ * domain controllers include an authorization field known as the Privilege
+ * Attribute Certificate (PAC), which contains the user's Windows permissions
+ * (group memberships etc.). The PAC is copied into all tickets obtained on
+ * the basis of this TGT (even those issued by Unix realms which the Windows
+ * realm trusts), and can be several kB in size. The maximum token size
+ * accepted by Windows systems is determined by the MaxAuthToken Windows
+ * registry setting. Microsoft recommends that it is not set higher than
+ * 65535 bytes, so that seems like a reasonable limit for us as well.
+ */
+#define PG_MAX_AUTH_TOKEN_LENGTH 65535
+
extern PGDLLIMPORT char *pg_krb_server_keyfile;
extern PGDLLIMPORT bool pg_krb_caseins_users;
extern PGDLLIMPORT bool pg_gss_accept_delegation;
@@ -23,6 +39,7 @@ extern PGDLLIMPORT bool pg_gss_accept_delegation;
extern void ClientAuthentication(Port *port);
extern void sendAuthRequest(Port *port, AuthRequest areq, const char *extradata,
int extralen);
+extern void set_authn_id(Port *port, const char *id);
/* Hook for plugins to get control in ClientAuthentication() */
typedef void (*ClientAuthentication_hook_type) (Port *, int);
diff --git a/src/include/libpq/hba.h b/src/include/libpq/hba.h
index 8ea837ae82..53c999a69f 100644
--- a/src/include/libpq/hba.h
+++ b/src/include/libpq/hba.h
@@ -39,7 +39,8 @@ typedef enum UserAuth
uaCert,
uaRADIUS,
uaPeer,
-#define USER_AUTH_LAST uaPeer /* Must be last value of this enum */
+ uaOAuth,
+#define USER_AUTH_LAST uaOAuth /* Must be last value of this enum */
} UserAuth;
/*
@@ -135,6 +136,9 @@ typedef struct HbaLine
char *radiusidentifiers_s;
List *radiusports;
char *radiusports_s;
+ char *oauth_issuer;
+ char *oauth_scope;
+ bool oauth_skip_usermap;
} HbaLine;
typedef struct IdentLine
diff --git a/src/include/libpq/oauth.h b/src/include/libpq/oauth.h
new file mode 100644
index 0000000000..6f98e84cc9
--- /dev/null
+++ b/src/include/libpq/oauth.h
@@ -0,0 +1,49 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth.h
+ * Interface to libpq/auth-oauth.c
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/libpq/oauth.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_OAUTH_H
+#define PG_OAUTH_H
+
+#include "libpq/libpq-be.h"
+#include "libpq/sasl.h"
+
+extern PGDLLIMPORT char *OAuthValidatorLibrary;
+
+typedef struct ValidatorModuleState
+{
+ void *private_data;
+} ValidatorModuleState;
+
+typedef struct ValidatorModuleResult
+{
+ bool authorized;
+ char *authn_id;
+} ValidatorModuleResult;
+
+typedef void (*ValidatorStartupCB) (ValidatorModuleState *state);
+typedef void (*ValidatorShutdownCB) (ValidatorModuleState *state);
+typedef ValidatorModuleResult *(*ValidatorValidateCB) (ValidatorModuleState *state, const char *token, const char *role);
+
+typedef struct OAuthValidatorCallbacks
+{
+ ValidatorStartupCB startup_cb;
+ ValidatorShutdownCB shutdown_cb;
+ ValidatorValidateCB validate_cb;
+} OAuthValidatorCallbacks;
+
+typedef const OAuthValidatorCallbacks *(*OAuthValidatorModuleInit) (void);
+extern PGDLLEXPORT const OAuthValidatorCallbacks *_PG_oauth_validator_module_init(void);
+
+/* Implementation */
+extern const pg_be_sasl_mech pg_be_oauth_mech;
+
+#endif /* PG_OAUTH_H */
diff --git a/src/include/libpq/sasl.h b/src/include/libpq/sasl.h
index 7a1f970cca..3f2c02b8f2 100644
--- a/src/include/libpq/sasl.h
+++ b/src/include/libpq/sasl.h
@@ -26,6 +26,14 @@
#define PG_SASL_EXCHANGE_SUCCESS 1
#define PG_SASL_EXCHANGE_FAILURE 2
+/*
+ * Maximum accepted size of SASL messages.
+ *
+ * The messages that the server or libpq generate are much smaller than this,
+ * but have some headroom.
+ */
+#define PG_MAX_SASL_MESSAGE_LENGTH 1024
+
/*
* Backend SASL mechanism callbacks.
*
@@ -127,6 +135,9 @@ typedef struct pg_be_sasl_mech
const char *input, int inputlen,
char **output, int *outputlen,
const char **logdetail);
+
+ /* The maximum size allowed for client SASLResponses. */
+ int max_message_length;
} pg_be_sasl_mech;
/* Common implementation for auth.c */
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index 7c11fb97f2..a08105e0c2 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -9,6 +9,7 @@ subdir('gin')
subdir('injection_points')
subdir('ldap_password_func')
subdir('libpq_pipeline')
+subdir('oauth_validator')
subdir('plsample')
subdir('spgist_name_ops')
subdir('ssl_passphrase_callback')
diff --git a/src/test/modules/oauth_validator/.gitignore b/src/test/modules/oauth_validator/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/src/test/modules/oauth_validator/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/oauth_validator/Makefile b/src/test/modules/oauth_validator/Makefile
new file mode 100644
index 0000000000..e93e01455a
--- /dev/null
+++ b/src/test/modules/oauth_validator/Makefile
@@ -0,0 +1,21 @@
+export PYTHON
+
+MODULES = validator
+PGFILEDESC = "validator - test OAuth validator module"
+
+NO_INSTALLCHECK = 1
+
+TAP_TESTS = 1
+
+REGRESS = validator
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/oauth_validator
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/oauth_validator/expected/validator.out b/src/test/modules/oauth_validator/expected/validator.out
new file mode 100644
index 0000000000..360caa2cb3
--- /dev/null
+++ b/src/test/modules/oauth_validator/expected/validator.out
@@ -0,0 +1,6 @@
+SELECT 1;
+ ?column?
+----------
+ 1
+(1 row)
+
diff --git a/src/test/modules/oauth_validator/meson.build b/src/test/modules/oauth_validator/meson.build
new file mode 100644
index 0000000000..3feba6f826
--- /dev/null
+++ b/src/test/modules/oauth_validator/meson.build
@@ -0,0 +1,36 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+validator_sources = files(
+ 'validator.c',
+)
+
+if host_system == 'windows'
+ validator_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'validator',
+ '--FILEDESC', 'validator - test OAuth validator module',])
+endif
+
+validator = shared_module('validator',
+ validator_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += validator
+
+tests += {
+ 'name': 'oauth_validator',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'regress': {
+ 'sql': [
+ 'validator',
+ ],
+ },
+ 'tap': {
+ 'tests': [
+ 't/001_server.pl',
+ ],
+ 'env': {
+ 'PYTHON': python.path(),
+ },
+ },
+}
diff --git a/src/test/modules/oauth_validator/sql/validator.sql b/src/test/modules/oauth_validator/sql/validator.sql
new file mode 100644
index 0000000000..e0ac49d1ec
--- /dev/null
+++ b/src/test/modules/oauth_validator/sql/validator.sql
@@ -0,0 +1 @@
+SELECT 1;
diff --git a/src/test/modules/oauth_validator/t/001_server.pl b/src/test/modules/oauth_validator/t/001_server.pl
new file mode 100644
index 0000000000..ea610fcd28
--- /dev/null
+++ b/src/test/modules/oauth_validator/t/001_server.pl
@@ -0,0 +1,74 @@
+
+# Copyright (c) 2021-2024, PostgreSQL Global Development Group
+
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::OAuthServer;
+use Test::More;
+
+my $node = PostgreSQL::Test::Cluster->new('primary');
+$node->init;
+$node->append_conf('postgresql.conf', "log_connections = on\n");
+$node->append_conf('postgresql.conf', "shared_preload_libraries = 'validator'\n");
+$node->append_conf('postgresql.conf', "oauth_validator_library = 'validator'\n");
+$node->start;
+
+$node->safe_psql('postgres', 'CREATE USER test;');
+$node->safe_psql('postgres', 'CREATE USER testalt;');
+
+my $webserver = PostgreSQL::Test::OAuthServer->new();
+$webserver->run();
+
+my $port = $webserver->port();
+my $issuer = "127.0.0.1:$port";
+
+unlink($node->data_dir . '/pg_hba.conf');
+$node->append_conf('pg_hba.conf', qq{
+local all test oauth issuer="$issuer" scope="openid postgres"
+local all testalt oauth issuer="$issuer/alternate" scope="openid postgres alt"
+});
+$node->reload;
+
+my ($log_start, $log_end);
+$log_start = $node->wait_for_log(qr/reloading configuration files/);
+
+my $user = "test";
+$node->connect_ok("user=$user dbname=postgres oauth_client_id=f02c6361-0635", "connect",
+ expected_stderr => qr@Visit https://example\.com/ and enter the code: postgresuser@);
+
+$log_end = $node->wait_for_log(qr/connection authorized/, $log_start);
+$node->log_check("user $user: validator receives correct parameters", $log_start,
+ log_like => [
+ qr/oauth_validator: token="9243959234", role="$user"/,
+ qr/oauth_validator: issuer="\Q$issuer\E", scope="openid postgres"/,
+ ]);
+$node->log_check("user $user: validator sets authenticated identity", $log_start,
+ log_like => [
+ qr/connection authenticated: identity="test" method=oauth/,
+ ]);
+$log_start = $log_end;
+
+# The /alternate issuer uses slightly different parameters.
+$user = "testalt";
+$node->connect_ok("user=$user dbname=postgres oauth_client_id=f02c6361-0636", "connect",
+ expected_stderr => qr@Visit https://example\.org/ and enter the code: postgresuser@);
+
+$log_end = $node->wait_for_log(qr/connection authorized/, $log_start);
+$node->log_check("user $user: validator receives correct parameters", $log_start,
+ log_like => [
+ qr/oauth_validator: token="9243959234-alt", role="$user"/,
+ qr|oauth_validator: issuer="\Q$issuer/alternate\E", scope="openid postgres alt"|,
+ ]);
+$node->log_check("user $user: validator sets authenticated identity", $log_start,
+ log_like => [
+ qr/connection authenticated: identity="testalt" method=oauth/,
+ ]);
+$log_start = $log_end;
+
+$webserver->stop();
+$node->stop;
+
+done_testing();
diff --git a/src/test/modules/oauth_validator/t/oauth_server.py b/src/test/modules/oauth_validator/t/oauth_server.py
new file mode 100755
index 0000000000..77e3883a81
--- /dev/null
+++ b/src/test/modules/oauth_validator/t/oauth_server.py
@@ -0,0 +1,114 @@
+#! /usr/bin/env python3
+
+import http.server
+import json
+import os
+import sys
+
+
+class OAuthHandler(http.server.BaseHTTPRequestHandler):
+ JsonObject = dict[str, object] # TypeAlias is not available until 3.10
+
+ def _check_issuer(self):
+ """
+ Switches the behavior of the provider depending on the issuer URI.
+ """
+ self._alt_issuer = self.path.startswith("/alternate/")
+ if self._alt_issuer:
+ self.path = self.path.removeprefix("/alternate")
+
+ def do_GET(self):
+ self._check_issuer()
+
+ if self.path == "/.well-known/openid-configuration":
+ resp = self.config()
+ else:
+ self.send_error(404, "Not Found")
+ return
+
+ self._send_json(resp)
+
+ def do_POST(self):
+ self._check_issuer()
+
+ if self.path == "/authorize":
+ resp = self.authorization()
+ elif self.path == "/token":
+ resp = self.token()
+ else:
+ self.send_error(404, "Not Found")
+ return
+
+ self._send_json(resp)
+
+ def _send_json(self, js: JsonObject) -> None:
+ """
+ Sends the provided JSON dict as an application/json response.
+ """
+
+ resp = json.dumps(js).encode("ascii")
+
+ self.send_response(200, "OK")
+ self.send_header("Content-Type", "application/json")
+ self.send_header("Content-Length", str(len(resp)))
+ self.end_headers()
+
+ self.wfile.write(resp)
+
+ def config(self) -> JsonObject:
+ port = self.server.socket.getsockname()[1]
+ issuer = f"http://localhost:{port}"
+ if self._alt_issuer:
+ issuer += "/alternate"
+
+ return {
+ "issuer": issuer,
+ "token_endpoint": issuer + "/token",
+ "device_authorization_endpoint": issuer + "/authorize",
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ "grant_types_supported": ["urn:ietf:params:oauth:grant-type:device_code"],
+ }
+
+ def authorization(self) -> JsonObject:
+ uri = "https://example.com/"
+ if self._alt_issuer:
+ uri = "https://example.org/"
+
+ return {
+ "device_code": "postgres",
+ "user_code": "postgresuser",
+ "interval": 0,
+ "verification_uri": uri,
+ "expires-in": 5,
+ }
+
+ def token(self) -> JsonObject:
+ token = "9243959234"
+ if self._alt_issuer:
+ token += "-alt"
+
+ return {
+ "access_token": token,
+ "token_type": "bearer",
+ }
+
+
+def main():
+ s = http.server.HTTPServer(("127.0.0.1", 0), OAuthHandler)
+
+ # Give the parent the port number to contact (this is also the signal that
+ # we're ready to receive requests).
+ port = s.socket.getsockname()[1]
+ print(port)
+
+ stdout = sys.stdout.fileno()
+ sys.stdout.close()
+ os.close(stdout)
+
+ s.serve_forever() # we expect our parent to send a termination signal
+
+
+if __name__ == "__main__":
+ main()
diff --git a/src/test/modules/oauth_validator/validator.c b/src/test/modules/oauth_validator/validator.c
new file mode 100644
index 0000000000..09a4bf61d2
--- /dev/null
+++ b/src/test/modules/oauth_validator/validator.c
@@ -0,0 +1,82 @@
+/*-------------------------------------------------------------------------
+ *
+ * validator.c
+ * Test module for serverside OAuth token validation callbacks
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/test/modules/oauth_validator/validator.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "libpq/oauth.h"
+#include "miscadmin.h"
+#include "utils/memutils.h"
+
+PG_MODULE_MAGIC;
+
+static void validator_startup(ValidatorModuleState *state);
+static void validator_shutdown(ValidatorModuleState *state);
+static ValidatorModuleResult * validate_token(ValidatorModuleState *state,
+ const char *token,
+ const char *role);
+
+static const OAuthValidatorCallbacks validator_callbacks = {
+ .startup_cb = validator_startup,
+ .shutdown_cb = validator_shutdown,
+ .validate_cb = validate_token
+};
+
+void
+_PG_init(void)
+{
+ /* no-op */
+}
+
+const OAuthValidatorCallbacks *
+_PG_oauth_validator_module_init(void)
+{
+ return &validator_callbacks;
+}
+
+#define PRIVATE_COOKIE ((void *) 13579)
+
+static void
+validator_startup(ValidatorModuleState *state)
+{
+ state->private_data = PRIVATE_COOKIE;
+}
+
+static void
+validator_shutdown(ValidatorModuleState *state)
+{
+ /* do nothing */
+}
+
+static ValidatorModuleResult *
+validate_token(ValidatorModuleState *state, const char *token, const char *role)
+{
+ ValidatorModuleResult *res;
+
+ /* Check to make sure our private state still exists. */
+ if (state->private_data != PRIVATE_COOKIE)
+ elog(ERROR, "oauth_validator: private state cookie changed to %p",
+ state->private_data);
+
+ res = palloc(sizeof(ValidatorModuleResult));
+
+ elog(LOG, "oauth_validator: token=\"%s\", role=\"%s\"", token, role);
+ elog(LOG, "oauth_validator: issuer=\"%s\", scope=\"%s\"",
+ MyProcPort->hba->oauth_issuer,
+ MyProcPort->hba->oauth_scope);
+
+ res->authorized = true;
+ res->authn_id = pstrdup(role);
+
+ return res;
+}
diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm
index b08296605c..4b7c93f19d 100644
--- a/src/test/perl/PostgreSQL/Test/Cluster.pm
+++ b/src/test/perl/PostgreSQL/Test/Cluster.pm
@@ -2299,6 +2299,11 @@ instead of the default.
If this regular expression is set, matches it with the output generated.
+=item expected_stderr => B<value>
+
+If this regular expression is set, matches it against the standard error
+stream; otherwise the stderr must be empty.
+
=item log_like => [ qr/required message/ ]
=item log_unlike => [ qr/prohibited message/ ]
@@ -2342,7 +2347,14 @@ sub connect_ok
like($stdout, $params{expected_stdout}, "$test_name: stdout matches");
}
- is($stderr, "", "$test_name: no stderr");
+ if (defined($params{expected_stderr}))
+ {
+ like($stderr, $params{expected_stderr}, "$test_name: stderr matches");
+ }
+ else
+ {
+ is($stderr, "", "$test_name: no stderr");
+ }
$self->log_check($test_name, $log_location, %params);
}
diff --git a/src/test/perl/PostgreSQL/Test/OAuthServer.pm b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
new file mode 100644
index 0000000000..d96733f531
--- /dev/null
+++ b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
@@ -0,0 +1,65 @@
+#!/usr/bin/perl
+
+package PostgreSQL::Test::OAuthServer;
+
+use warnings;
+use strict;
+use threads;
+use Scalar::Util;
+use Socket;
+use IO::Select;
+
+local *server_socket;
+
+sub new
+{
+ my $class = shift;
+
+ my $self = {};
+ bless($self, $class);
+
+ return $self;
+}
+
+sub port
+{
+ my $self = shift;
+
+ return $self->{'port'};
+}
+
+sub run
+{
+ my $self = shift;
+ my $port;
+
+ my $pid = open(my $read_fh, "-|", $ENV{PYTHON}, "t/oauth_server.py")
+ // die "failed to start OAuth server: $!";
+
+ read($read_fh, $port, 7) // die "failed to read port number: $!";
+ chomp $port;
+ die "server did not advertise a valid port"
+ unless Scalar::Util::looks_like_number($port);
+
+ $self->{'pid'} = $pid;
+ $self->{'port'} = $port;
+ $self->{'child'} = $read_fh;
+
+ print("# OAuth provider (PID $pid) is listening on port $port\n");
+}
+
+sub stop
+{
+ my $self = shift;
+
+ print("# Sending SIGTERM to OAuth provider PID: $self->{'pid'}\n");
+
+ kill(15, $self->{'pid'});
+ $self->{'pid'} = undef;
+
+ # Closing the popen() handle waits for the process to exit.
+ close($self->{'child'});
+ $self->{'child'} = undef;
+}
+
+1;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index f535fbbd5c..0eaa079261 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1673,6 +1673,7 @@ NumericSortSupport
NumericSumAccum
NumericVar
OAuthStep
+OAuthValidatorCallbacks
OM_uint32
OP
OSAPerGroupState
@@ -3000,6 +3001,7 @@ VacuumRelation
VacuumStmt
ValidIOData
ValidateIndexState
+ValidatorModuleState
ValuesScan
ValuesScanState
Var
@@ -3564,6 +3566,7 @@ normal_rand_fctx
nsphash_hash
ntile_context
numeric
+oauth_state
object_access_hook_type
object_access_hook_type_str
off_t
--
2.32.1 (Apple Git-133)
=
v22-0002-libpq-add-OAUTHBEARER-SASL-mechanism.patch (application/octet-stream, 106.4K) download
From 01f9bbae380abc2385f8425680d6815a4375fa5f Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 13 Apr 2021 10:27:27 -0700
Subject: [PATCH v22 2/4] libpq: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) and OAuth 2.0 Device Authorization
Grants (RFC 8628) on the client side. When speaking to a OAuth-enabled
server, it looks a bit like this:
$ psql 'host=example.org oauth_client_id=f02c6361-0635-...'
Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG
The OAuth issuer must support device authorization. No other OAuth flows
are currently implemented (but clients may provide their own flows; see
below).
The client implementation requires libcurl and its development headers.
Pass `curl` to --with-oauth/-Doauth during configuration.
Thomas Munro wrote the kqueue() implementation for oauth-curl; thanks!
= PQauthDataHook =
Clients may override two pieces of OAuth handling using the new
PQsetAuthDataHook():
- PQAUTHDATA_PROMPT_OAUTH_DEVICE: replaces the default user prompt to
standard error when using the builtin device authorization flow
- PQAUTHDATA_OAUTH_BEARER_TOKEN: replaces the entire OAuth flow with a
custom asynchronous implementation
In general, a hook implementation should examine the incoming `type` to
decide whether or not to handle a specific piece of authdata; if not, it
should delegate to the previous hook in the chain (retrievable via
PQgetAuthDataHook()). Otherwise, it should return an integer > 0 and
follow the authdata-specific instructions. Returning an integer < 0
signals an error condition and abandons the connection attempt.
== PQAUTHDATA_PROMPT_OAUTH_DEVICE ==
The hook should display the device prompt (URL + code) using whatever
method it prefers.
== PQAUTHDATA_OAUTH_BEARER_TOKEN ==
The hook should either directly return a Bearer token for the current
user/issuer/scope combination, if one is available without blocking, or
else set up an asynchronous callback to retrieve one. See the
documentation for PQoauthBearerRequest.
Several TODOs:
- don't retry forever if the server won't accept our token
- perform several sanity checks on the OAuth issuer's responses
- handle cases where the client has been set up with an issuer and
scope, but the Postgres server wants to use something different
- improve error debuggability during the OAuth handshake
- fix libcurl initialization thread-safety
- harden the libcurl flow implementation
- figure out pgsocket/int difference on Windows
- fix intermittent failure in the cleanup callback tests (race
condition?)
- support require_auth
- ...and more.
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
---
configure | 110 ++
configure.ac | 28 +
meson.build | 29 +
meson_options.txt | 4 +
src/Makefile.global.in | 1 +
src/include/common/oauth-common.h | 19 +
src/include/pg_config.h.in | 9 +
src/interfaces/libpq/Makefile | 10 +-
src/interfaces/libpq/exports.txt | 3 +
src/interfaces/libpq/fe-auth-oauth-curl.c | 1982 +++++++++++++++++++++
src/interfaces/libpq/fe-auth-oauth.c | 659 +++++++
src/interfaces/libpq/fe-auth-oauth.h | 42 +
src/interfaces/libpq/fe-auth-sasl.h | 10 +-
src/interfaces/libpq/fe-auth-scram.c | 6 +-
src/interfaces/libpq/fe-auth.c | 105 +-
src/interfaces/libpq/fe-auth.h | 9 +-
src/interfaces/libpq/fe-connect.c | 85 +-
src/interfaces/libpq/fe-misc.c | 7 +-
src/interfaces/libpq/libpq-fe.h | 77 +-
src/interfaces/libpq/libpq-int.h | 14 +
src/interfaces/libpq/meson.build | 7 +
src/makefiles/meson.build | 1 +
src/tools/pgindent/typedefs.list | 10 +
23 files changed, 3200 insertions(+), 27 deletions(-)
create mode 100644 src/include/common/oauth-common.h
create mode 100644 src/interfaces/libpq/fe-auth-oauth-curl.c
create mode 100644 src/interfaces/libpq/fe-auth-oauth.c
create mode 100644 src/interfaces/libpq/fe-auth-oauth.h
diff --git a/configure b/configure
index 36feeafbb2..d785ca846b 100755
--- a/configure
+++ b/configure
@@ -712,6 +712,7 @@ with_uuid
with_readline
with_systemd
with_selinux
+with_oauth
with_ldap
with_krb_srvnam
krb_srvtab
@@ -858,6 +859,7 @@ with_krb_srvnam
with_pam
with_bsd_auth
with_ldap
+with_oauth
with_bonjour
with_selinux
with_systemd
@@ -1568,6 +1570,7 @@ Optional Packages:
--with-pam build with PAM support
--with-bsd-auth build with BSD Authentication support
--with-ldap build with LDAP support
+ --with-oauth=LIB use LIB for OAuth 2.0 support (curl)
--with-bonjour build with Bonjour support
--with-selinux build with SELinux support
--with-systemd build with systemd support
@@ -8485,6 +8488,52 @@ $as_echo "$with_ldap" >&6; }
+#
+# OAuth 2.0
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with OAuth support" >&5
+$as_echo_n "checking whether to build with OAuth support... " >&6; }
+
+
+
+# Check whether --with-oauth was given.
+if test "${with_oauth+set}" = set; then :
+ withval=$with_oauth;
+ case $withval in
+ yes)
+ as_fn_error $? "argument required for --with-oauth option" "$LINENO" 5
+ ;;
+ no)
+ as_fn_error $? "argument required for --with-oauth option" "$LINENO" 5
+ ;;
+ *)
+
+ ;;
+ esac
+
+fi
+
+
+if test x"$with_oauth" = x"" ; then
+ with_oauth=no
+fi
+
+if test x"$with_oauth" = x"curl"; then
+
+$as_echo "#define USE_OAUTH 1" >>confdefs.h
+
+
+$as_echo "#define USE_OAUTH_CURL 1" >>confdefs.h
+
+elif test x"$with_oauth" != x"no"; then
+ as_fn_error $? "--with-oauth must specify curl" "$LINENO" 5
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_oauth" >&5
+$as_echo "$with_oauth" >&6; }
+
+
+
#
# Bonjour
#
@@ -13037,6 +13086,56 @@ fi
+if test "$with_oauth" = curl ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for curl_multi_init in -lcurl" >&5
+$as_echo_n "checking for curl_multi_init in -lcurl... " >&6; }
+if ${ac_cv_lib_curl_curl_multi_init+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lcurl $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char curl_multi_init ();
+int
+main ()
+{
+return curl_multi_init ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_curl_curl_multi_init=yes
+else
+ ac_cv_lib_curl_curl_multi_init=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_curl_curl_multi_init" >&5
+$as_echo "$ac_cv_lib_curl_curl_multi_init" >&6; }
+if test "x$ac_cv_lib_curl_curl_multi_init" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBCURL 1
+_ACEOF
+
+ LIBS="-lcurl $LIBS"
+
+else
+ as_fn_error $? "library 'curl' is required for --with-oauth=curl" "$LINENO" 5
+fi
+
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for security_compute_create_name in -lselinux" >&5
@@ -14062,6 +14161,17 @@ fi
done
+fi
+
+if test "$with_oauth" = curl; then
+ ac_fn_c_check_header_mongrel "$LINENO" "curl/curl.h" "ac_cv_header_curl_curl_h" "$ac_includes_default"
+if test "x$ac_cv_header_curl_curl_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <curl/curl.h> is required for OAuth" "$LINENO" 5
+fi
+
+
fi
if test "$PORTNAME" = "win32" ; then
diff --git a/configure.ac b/configure.ac
index 57f734879e..1ec0712ed5 100644
--- a/configure.ac
+++ b/configure.ac
@@ -927,6 +927,26 @@ AC_MSG_RESULT([$with_ldap])
AC_SUBST(with_ldap)
+#
+# OAuth 2.0
+#
+AC_MSG_CHECKING([whether to build with OAuth support])
+PGAC_ARG_REQ(with, oauth, [LIB], [use LIB for OAuth 2.0 support (curl)])
+if test x"$with_oauth" = x"" ; then
+ with_oauth=no
+fi
+
+if test x"$with_oauth" = x"curl"; then
+ AC_DEFINE([USE_OAUTH], 1, [Define to 1 to build with OAuth 2.0 support. (--with-oauth)])
+ AC_DEFINE([USE_OAUTH_CURL], 1, [Define to 1 to use libcurl for OAuth support.])
+elif test x"$with_oauth" != x"no"; then
+ AC_MSG_ERROR([--with-oauth must specify curl])
+fi
+
+AC_MSG_RESULT([$with_oauth])
+AC_SUBST(with_oauth)
+
+
#
# Bonjour
#
@@ -1423,6 +1443,10 @@ fi
AC_SUBST(LDAP_LIBS_FE)
AC_SUBST(LDAP_LIBS_BE)
+if test "$with_oauth" = curl ; then
+ AC_CHECK_LIB(curl, curl_multi_init, [], [AC_MSG_ERROR([library 'curl' is required for --with-oauth=curl])])
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
AC_CHECK_LIB(selinux, security_compute_create_name, [],
@@ -1614,6 +1638,10 @@ elif test "$with_uuid" = ossp ; then
[AC_MSG_ERROR([header file <ossp/uuid.h> or <uuid.h> is required for OSSP UUID])])])
fi
+if test "$with_oauth" = curl; then
+ AC_CHECK_HEADER(curl/curl.h, [], [AC_MSG_ERROR([header file <curl/curl.h> is required for OAuth])])
+fi
+
if test "$PORTNAME" = "win32" ; then
AC_CHECK_HEADERS(crtdefs.h)
fi
diff --git a/meson.build b/meson.build
index 18b5be842e..99e29513e4 100644
--- a/meson.build
+++ b/meson.build
@@ -840,6 +840,33 @@ endif
+###############################################################
+# Library: oauth
+###############################################################
+
+oauth = not_found_dep
+oauth_library = 'none'
+oauthopt = get_option('oauth')
+
+if oauthopt == 'auto' and auto_features.disabled()
+ oauthopt = 'none'
+endif
+
+if oauthopt in ['auto', 'curl']
+ oauth = dependency('libcurl', required: (oauthopt == 'curl'))
+
+ if oauth.found()
+ oauth_library = 'curl'
+ cdata.set('USE_OAUTH', 1)
+ cdata.set('USE_OAUTH_CURL', 1)
+ endif
+endif
+
+if oauthopt == 'auto' and auto_features.enabled() and not oauth.found()
+ error('no OAuth implementation library found')
+endif
+
+
###############################################################
# Library: Tcl (for pltcl)
#
@@ -2845,6 +2872,7 @@ libpq_deps += [
gssapi,
ldap_r,
libintl,
+ oauth,
ssl,
]
@@ -3511,6 +3539,7 @@ if meson.version().version_compare('>=0.57')
'llvm': llvm,
'lz4': lz4,
'nls': libintl,
+ 'oauth': oauth,
'openssl': ssl,
'pam': pam,
'plperl': perl_dep,
diff --git a/meson_options.txt b/meson_options.txt
index 249ecc5ffd..3248b9cc1c 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -121,6 +121,10 @@ option('lz4', type: 'feature', value: 'auto',
option('nls', type: 'feature', value: 'auto',
description: 'Native language support')
+option('oauth', type : 'combo', choices : ['auto', 'none', 'curl'],
+ value: 'auto',
+ description: 'use LIB for OAuth 2.0 support (curl)')
+
option('pam', type: 'feature', value: 'auto',
description: 'PAM support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 8b3f8c24e0..79b3647834 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -193,6 +193,7 @@ with_ldap = @with_ldap@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
+with_oauth = @with_oauth@
with_system_tzdata = @with_system_tzdata@
with_uuid = @with_uuid@
with_zlib = @with_zlib@
diff --git a/src/include/common/oauth-common.h b/src/include/common/oauth-common.h
new file mode 100644
index 0000000000..5ff3488bfb
--- /dev/null
+++ b/src/include/common/oauth-common.h
@@ -0,0 +1,19 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth-common.h
+ * Declarations for helper functions used for OAuth/OIDC authentication
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/common/oauth-common.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef OAUTH_COMMON_H
+#define OAUTH_COMMON_H
+
+/* Name of SASL mechanism per IANA */
+#define OAUTHBEARER_NAME "OAUTHBEARER"
+
+#endif /* OAUTH_COMMON_H */
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 591e1ca3df..399578cebb 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -246,6 +246,9 @@
/* Define to 1 if you have the `crypto' library (-lcrypto). */
#undef HAVE_LIBCRYPTO
+/* Define to 1 if you have the `curl' library (-lcurl). */
+#undef HAVE_LIBCURL
+
/* Define to 1 if you have the `ldap' library (-lldap). */
#undef HAVE_LIBLDAP
@@ -714,6 +717,12 @@
/* Define to select named POSIX semaphores. */
#undef USE_NAMED_POSIX_SEMAPHORES
+/* Define to 1 to build with OAuth 2.0 support. (--with-oauth) */
+#undef USE_OAUTH
+
+/* Define to 1 to use libcurl for OAuth support. */
+#undef USE_OAUTH_CURL
+
/* Define to 1 to build with OpenSSL support. (--with-ssl=openssl) */
#undef USE_OPENSSL
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index fe2af575c5..2618c293af 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -61,6 +61,14 @@ OBJS += \
fe-secure-gssapi.o
endif
+ifneq ($(with_oauth),no)
+OBJS += fe-auth-oauth.o
+
+ifeq ($(with_oauth),curl)
+OBJS += fe-auth-oauth-curl.o
+endif
+endif
+
ifeq ($(PORTNAME), cygwin)
override shlib = cyg$(NAME)$(DLSUFFIX)
endif
@@ -79,7 +87,7 @@ endif
# that are built correctly for use in a shlib.
SHLIB_LINK_INTERNAL = -lpgcommon_shlib -lpgport_shlib
ifneq ($(PORTNAME), win32)
-SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
+SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lcurl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
else
SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi32 -lssl -lsocket -lnsl -lresolv -lintl -lm $(PTHREAD_LIBS), $(LIBS)) $(LDAP_LIBS_FE)
endif
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 9fbd3d3407..9946764c4a 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -202,3 +202,6 @@ PQcancelSocket 199
PQcancelErrorMessage 200
PQcancelReset 201
PQcancelFinish 202
+PQsetAuthDataHook 203
+PQgetAuthDataHook 204
+PQdefaultAuthDataHook 205
diff --git a/src/interfaces/libpq/fe-auth-oauth-curl.c b/src/interfaces/libpq/fe-auth-oauth-curl.c
new file mode 100644
index 0000000000..0504f96e4e
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth-curl.c
@@ -0,0 +1,1982 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth-curl.c
+ * The libcurl implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth-curl.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <curl/curl.h>
+#include <math.h>
+#ifdef HAVE_SYS_EPOLL_H
+#include <sys/epoll.h>
+#include <sys/timerfd.h>
+#endif
+#ifdef HAVE_SYS_EVENT_H
+#include <sys/event.h>
+#endif
+#include <unistd.h>
+
+#include "common/jsonapi.h"
+#include "fe-auth.h"
+#include "fe-auth-oauth.h"
+#include "libpq-int.h"
+#include "mb/pg_wchar.h"
+
+/*
+ * Parsed JSON Representations
+ *
+ * As a general rule, we parse and cache only the fields we're currently using.
+ * When adding new fields, ensure the corresponding free_*() function is updated
+ * too.
+ */
+
+/*
+ * The OpenID Provider configuration (alternatively named "authorization server
+ * metadata") jointly described by OpenID Connect Discovery 1.0 and RFC 8414:
+ *
+ * https://openid.net/specs/openid-connect-discovery-1_0.html
+ * https://www.rfc-editor.org/rfc/rfc8414#section-3.2
+ */
+struct provider
+{
+ char *issuer;
+ char *token_endpoint;
+ char *device_authorization_endpoint;
+ struct curl_slist *grant_types_supported;
+};
+
+static void
+free_provider(struct provider *provider)
+{
+ free(provider->issuer);
+ free(provider->token_endpoint);
+ free(provider->device_authorization_endpoint);
+ curl_slist_free_all(provider->grant_types_supported);
+}
+
+/*
+ * The Device Authorization response, described by RFC 8628:
+ *
+ * https://www.rfc-editor.org/rfc/rfc8628#section-3.2
+ */
+struct device_authz
+{
+ char *device_code;
+ char *user_code;
+ char *verification_uri;
+ char *interval_str;
+
+ /* Fields below are parsed from the corresponding string above. */
+ int interval;
+};
+
+static void
+free_device_authz(struct device_authz *authz)
+{
+ free(authz->device_code);
+ free(authz->user_code);
+ free(authz->verification_uri);
+ free(authz->interval_str);
+}
+
+/*
+ * The Token Endpoint error response, as described by RFC 6749:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-5.2
+ *
+ * Note that this response type can also be returned from the Device
+ * Authorization Endpoint.
+ */
+struct token_error
+{
+ char *error;
+ char *error_description;
+};
+
+static void
+free_token_error(struct token_error *err)
+{
+ free(err->error);
+ free(err->error_description);
+}
+
+/*
+ * The Access Token response, as described by RFC 6749:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-4.1.4
+ *
+ * During the Device Authorization flow, several temporary errors are expected
+ * as part of normal operation. To make it easy to handle these in the happy
+ * path, this contains an embedded token_error that is filled in if needed.
+ */
+struct token
+{
+ /* for successful responses */
+ char *access_token;
+ char *token_type;
+
+ /* for error responses */
+ struct token_error err;
+};
+
+static void
+free_token(struct token *tok)
+{
+ free(tok->access_token);
+ free(tok->token_type);
+ free_token_error(&tok->err);
+}
+
+/*
+ * Asynchronous State
+ */
+
+/* States for the overall async machine. */
+typedef enum
+{
+ OAUTH_STEP_INIT,
+ OAUTH_STEP_DISCOVERY,
+ OAUTH_STEP_DEVICE_AUTHORIZATION,
+ OAUTH_STEP_TOKEN_REQUEST,
+ OAUTH_STEP_WAIT_INTERVAL,
+} OAuthStep;
+
+/*
+ * The async_ctx holds onto state that needs to persist across multiple calls to
+ * pg_fe_run_oauth_flow(). Almost everything interacts with this in some way.
+ */
+struct async_ctx
+{
+ OAuthStep step; /* where are we in the flow? */
+
+#ifdef HAVE_SYS_EPOLL_H
+ int timerfd; /* a timerfd for signaling async timeouts */
+#endif
+ pgsocket mux; /* the multiplexer socket containing all
+ * descriptors tracked by cURL, plus the
+ * timerfd */
+ CURLM *curlm; /* top-level multi handle for cURL operations */
+ CURL *curl; /* the (single) easy handle for serial
+ * requests */
+
+ struct curl_slist *headers; /* common headers for all requests */
+ PQExpBufferData work_data; /* scratch buffer for general use (remember to
+ * clear out prior contents first!) */
+
+ /*------
+ * Since a single logical operation may stretch across multiple calls to
+ * our entry point, errors have three parts:
+ *
+ * - errctx: an optional static string, describing the global operation
+ * currently in progress. It'll be translated for you.
+ *
+ * - errbuf: contains the actual error message. Generally speaking, use
+ * actx_error[_str] to manipulate this. This must be filled
+ * with something useful on an error.
+ *
+ * - curl_err: an optional static error buffer used by cURL to put
+ * detailed information about failures. Unfortunately
+ * untranslatable.
+ *
+ * These pieces will be combined into a single error message looking
+ * something like the following, with errctx and/or curl_err omitted when
+ * absent:
+ *
+ * connection to server ... failed: errctx: errbuf (curl_err)
+ */
+ const char *errctx; /* not freed; must point to static allocation */
+ PQExpBufferData errbuf;
+ char curl_err[CURL_ERROR_SIZE];
+
+ /*
+ * These documents need to survive over multiple calls, and are therefore
+ * cached directly in the async_ctx.
+ */
+ struct provider provider;
+ struct device_authz authz;
+
+ bool user_prompted; /* have we already sent the authz prompt? */
+};
+
+/*
+ * Frees the async_ctx, which is stored directly on the PGconn. This is called
+ * during pqDropConnection() so that we don't leak resources even if
+ * PQconnectPoll() never calls us back.
+ *
+ * TODO: we should probably call this at the end of a successful authentication,
+ * too, to proactively free up resources.
+ */
+static void
+free_curl_async_ctx(PGconn *conn, void *ctx)
+{
+ struct async_ctx *actx = ctx;
+
+ Assert(actx); /* oauth_free() shouldn't call us otherwise */
+
+ /*
+ * TODO: in general, none of the error cases below should ever happen if
+ * we have no bugs above. But if we do hit them, surfacing those errors
+ * somehow might be the only way to have a chance to debug them. What's
+ * the best way to do that? Assertions? Spraying messages on stderr?
+ * Bubbling an error code to the top? Appending to the connection's error
+ * message only helps if the bug caused a connection failure; otherwise
+ * it'll be buried...
+ */
+
+ if (actx->curlm && actx->curl)
+ {
+ CURLMcode err = curl_multi_remove_handle(actx->curlm, actx->curl);
+
+ if (err)
+ libpq_append_conn_error(conn,
+ "cURL easy handle removal failed: %s",
+ curl_multi_strerror(err));
+ }
+
+ if (actx->curl)
+ {
+ /*
+ * curl_multi_cleanup() doesn't free any associated easy handles; we
+ * need to do that separately. We only ever have one easy handle per
+ * multi handle.
+ */
+ curl_easy_cleanup(actx->curl);
+ }
+
+ if (actx->curlm)
+ {
+ CURLMcode err = curl_multi_cleanup(actx->curlm);
+
+ if (err)
+ libpq_append_conn_error(conn,
+ "cURL multi handle cleanup failed: %s",
+ curl_multi_strerror(err));
+ }
+
+ free_provider(&actx->provider);
+ free_device_authz(&actx->authz);
+
+ curl_slist_free_all(actx->headers);
+ termPQExpBuffer(&actx->work_data);
+ termPQExpBuffer(&actx->errbuf);
+
+ if (actx->mux != PGINVALID_SOCKET)
+ close(actx->mux);
+#ifdef HAVE_SYS_EPOLL_H
+ if (actx->timerfd >= 0)
+ close(actx->timerfd);
+#endif
+
+ free(actx);
+}
+
+/*
+ * Macros for manipulating actx->errbuf. actx_error() translates and formats a
+ * string for you; actx_error_str() appends a string directly without
+ * translation.
+ */
+
+#define actx_error(ACTX, FMT, ...) \
+ appendPQExpBuffer(&(ACTX)->errbuf, libpq_gettext(FMT), ##__VA_ARGS__)
+
+#define actx_error_str(ACTX, S) \
+ appendPQExpBufferStr(&(ACTX)->errbuf, S)
+
+/*
+ * Macros for getting and setting state for the connection's two cURL handles,
+ * so you don't have to write out the error handling every time.
+ */
+
+#define CHECK_MSETOPT(ACTX, OPT, VAL, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLMcode _setopterr = curl_multi_setopt(_actx->curlm, OPT, VAL); \
+ if (_setopterr) { \
+ actx_error(_actx, "failed to set %s on OAuth connection: %s",\
+ #OPT, curl_multi_strerror(_setopterr)); \
+ FAILACTION; \
+ } \
+ } while (0)
+
+#define CHECK_SETOPT(ACTX, OPT, VAL, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLcode _setopterr = curl_easy_setopt(_actx->curl, OPT, VAL); \
+ if (_setopterr) { \
+ actx_error(_actx, "failed to set %s on OAuth connection: %s",\
+ #OPT, curl_easy_strerror(_setopterr)); \
+ FAILACTION; \
+ } \
+ } while (0)
+
+#define CHECK_GETINFO(ACTX, INFO, OUT, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLcode _getinfoerr = curl_easy_getinfo(_actx->curl, INFO, OUT); \
+ if (_getinfoerr) { \
+ actx_error(_actx, "failed to get %s from OAuth response: %s",\
+ #INFO, curl_easy_strerror(_getinfoerr)); \
+ FAILACTION; \
+ } \
+ } while (0)
+
+/*
+ * General JSON Parsing for OAuth Responses
+ */
+
+/*
+ * Represents a single name/value pair in a JSON object. This is the primary
+ * interface to parse_oauth_json().
+ *
+ * All fields are stored internally as strings or lists of strings, so clients
+ * have to explicitly parse other scalar types (though they will have gone
+ * through basic lexical validation). Storing nested objects is not currently
+ * supported, nor is parsing arrays of anything other than strings.
+ */
+struct json_field
+{
+ const char *name; /* name (key) of the member */
+
+ JsonTokenType type; /* currently supports JSON_TOKEN_STRING,
+ * JSON_TOKEN_NUMBER, and
+ * JSON_TOKEN_ARRAY_START */
+ union
+ {
+ char **scalar; /* for all scalar types */
+ struct curl_slist **array; /* for type == JSON_TOKEN_ARRAY_START */
+ };
+
+ bool required; /* REQUIRED field, or just OPTIONAL? */
+};
+
+/* Documentation macros for json_field.required. */
+#define REQUIRED true
+#define OPTIONAL false
+
+/* Parse state for parse_oauth_json(). */
+struct oauth_parse
+{
+ PQExpBuffer errbuf; /* detail message for JSON_SEM_ACTION_FAILED */
+ int nested; /* nesting level (zero is the top) */
+
+ const struct json_field *fields; /* field definition array */
+ const struct json_field *active; /* points inside the fields array */
+};
+
+#define oauth_parse_set_error(ctx, fmt, ...) \
+ appendPQExpBuffer((ctx)->errbuf, libpq_gettext(fmt), ##__VA_ARGS__)
+
+static void
+report_type_mismatch(struct oauth_parse *ctx)
+{
+ char *msgfmt;
+
+ Assert(ctx->active);
+
+ /*
+ * At the moment, the only fields we're interested in are strings,
+ * numbers, and arrays of strings.
+ */
+ switch (ctx->active->type)
+ {
+ case JSON_TOKEN_STRING:
+ msgfmt = "field \"%s\" must be a string";
+ break;
+
+ case JSON_TOKEN_NUMBER:
+ msgfmt = "field \"%s\" must be a number";
+ break;
+
+ case JSON_TOKEN_ARRAY_START:
+ msgfmt = "field \"%s\" must be an array of strings";
+ break;
+
+ default:
+ Assert(false);
+ msgfmt = "field \"%s\" has unexpected type";
+ }
+
+ oauth_parse_set_error(ctx, msgfmt, ctx->active->name);
+}
+
+static JsonParseErrorType
+oauth_json_object_start(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (ctx->active)
+ {
+ /*
+ * Currently, none of the fields we're interested in can be or contain
+ * objects, so we can reject this case outright.
+ */
+ report_type_mismatch(ctx);
+ return JSON_SEM_ACTION_FAILED;
+ }
+
+ ++ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_field_start(void *state, char *name, bool isnull)
+{
+ struct oauth_parse *ctx = state;
+
+ /* We care only about the top-level fields. */
+ if (ctx->nested == 1)
+ {
+ const struct json_field *field = ctx->fields;
+
+ /*
+ * We should never start parsing a new field while a previous one is
+ * still active.
+ *
+ * TODO: this code relies on assertions too much. We need to exit
+ * sanely on internal logic errors, to avoid turning bugs into
+ * vulnerabilities.
+ */
+ Assert(!ctx->active);
+
+ while (field->name)
+ {
+ if (strcmp(name, field->name) == 0)
+ {
+ ctx->active = field;
+ break;
+ }
+
+ ++field;
+ }
+ }
+
+ free(name);
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_end(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_start(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (!ctx->nested)
+ {
+ oauth_parse_set_error(ctx, "top-level element must be an object");
+ return JSON_SEM_ACTION_FAILED;
+ }
+
+ if (ctx->active)
+ {
+ if (ctx->active->type != JSON_TOKEN_ARRAY_START
+ /* The arrays we care about must not have arrays as values. */
+ || ctx->nested > 1)
+ {
+ report_type_mismatch(ctx);
+ return JSON_SEM_ACTION_FAILED;
+ }
+ }
+
+ ++ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_end(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (ctx->active)
+ {
+ /*
+ * This assumes that no target arrays can contain other arrays, which
+ * we check in the array_start callback.
+ */
+ Assert(ctx->nested == 2);
+ Assert(ctx->active->type == JSON_TOKEN_ARRAY_START);
+
+ ctx->active = NULL;
+ }
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_scalar(void *state, char *token, JsonTokenType type)
+{
+ struct oauth_parse *ctx = state;
+ JsonParseErrorType result = JSON_SUCCESS;
+
+ if (!ctx->nested)
+ {
+ oauth_parse_set_error(ctx, "top-level element must be an object");
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ if (ctx->active)
+ {
+ JsonTokenType expected;
+
+ /*
+ * Make sure this matches what the active field expects. Arrays must
+ * contain only strings with the current implementation.
+ */
+ if (ctx->active->type == JSON_TOKEN_ARRAY_START)
+ expected = JSON_TOKEN_STRING;
+ else
+ expected = ctx->active->type;
+
+ if (type != expected)
+ {
+ report_type_mismatch(ctx);
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ /*
+ * FIXME if the JSON field is duplicated, we'll leak the prior value.
+ * Error out in that case instead.
+ */
+ if (ctx->active->type != JSON_TOKEN_ARRAY_START)
+ {
+ Assert(ctx->nested == 1);
+
+ *ctx->active->scalar = token;
+ ctx->active = NULL;
+
+ return JSON_SUCCESS; /* don't free the token */
+ }
+ else /* ctx->target_array */
+ {
+ struct curl_slist *temp;
+
+ Assert(ctx->nested == 2);
+
+ temp = curl_slist_append(*ctx->active->array, token);
+ if (!temp)
+ {
+ oauth_parse_set_error(ctx, "out of memory");
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ *ctx->active->array = temp;
+
+ /*
+ * Note that curl_slist_append() makes a copy of the token, so we
+ * can free it below.
+ */
+ }
+ }
+ else
+ {
+ /* otherwise we just ignore it */
+ }
+
+cleanup:
+ free(token);
+ return result;
+}
+
+/*
+ * A helper function for general JSON parsing. fields is the array of field
+ * definitions with their backing pointers. The response will be parsed from
+ * actx->curl and actx->work_data (as set up by start_request()), and any
+ * parsing errors will be placed into actx->errbuf.
+ */
+static bool
+parse_oauth_json(struct async_ctx *actx, const struct json_field *fields)
+{
+ PQExpBuffer resp = &actx->work_data;
+ char *content_type;
+ JsonLexContext lex = {0};
+ JsonSemAction sem = {0};
+ JsonParseErrorType err;
+ struct oauth_parse ctx = {0};
+ bool success = false;
+
+ /* Make sure the server thinks it's given us JSON. */
+ CHECK_GETINFO(actx, CURLINFO_CONTENT_TYPE, &content_type, return false);
+
+ if (!content_type)
+ {
+ actx_error(actx, "no content type was provided");
+ goto cleanup;
+ }
+ else if (strcasecmp(content_type, "application/json") != 0)
+ {
+ actx_error(actx, "unexpected content type \"%s\"", content_type);
+ goto cleanup;
+ }
+
+ if (strlen(resp->data) != resp->len)
+ {
+ actx_error(actx, "response contains embedded NULLs");
+ goto cleanup;
+ }
+
+ makeJsonLexContextCstringLen(&lex, resp->data, resp->len, PG_UTF8, true);
+
+ ctx.errbuf = &actx->errbuf;
+ ctx.fields = fields;
+ sem.semstate = &ctx;
+
+ sem.object_start = oauth_json_object_start;
+ sem.object_field_start = oauth_json_object_field_start;
+ sem.object_end = oauth_json_object_end;
+ sem.array_start = oauth_json_array_start;
+ sem.array_end = oauth_json_array_end;
+ sem.scalar = oauth_json_scalar;
+
+ err = pg_parse_json(&lex, &sem);
+
+ if (err != JSON_SUCCESS)
+ {
+ /*
+ * For JSON_SEM_ACTION_FAILED, we've already written the error
+ * message. Other errors come directly from pg_parse_json(), already
+ * translated.
+ */
+ if (err != JSON_SEM_ACTION_FAILED)
+ actx_error_str(actx, json_errdetail(err, &lex));
+
+ goto cleanup;
+ }
+
+ /* Check all required fields. */
+ while (fields->name)
+ {
+ if (fields->required && !*fields->scalar && !*fields->array)
+ {
+ actx_error(actx, "field \"%s\" is missing", fields->name);
+ goto cleanup;
+ }
+
+ fields++;
+ }
+
+ success = true;
+
+cleanup:
+ freeJsonLexContext(&lex);
+ return success;
+}
+
+/*
+ * JSON Parser Definitions
+ */
+
+static bool
+parse_provider(struct async_ctx *actx, struct provider *provider)
+{
+ struct json_field fields[] = {
+ {"issuer", JSON_TOKEN_STRING, {&provider->issuer}, REQUIRED},
+ {"token_endpoint", JSON_TOKEN_STRING, {&provider->token_endpoint}, REQUIRED},
+
+ /*----
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - jwks_uri
+ * - response_types_supported
+ * - subject_types_supported
+ * - id_token_signing_alg_values_supported
+ */
+
+ {"device_authorization_endpoint", JSON_TOKEN_STRING, {&provider->device_authorization_endpoint}, OPTIONAL},
+ {"grant_types_supported", JSON_TOKEN_ARRAY_START, {.array = &provider->grant_types_supported}, OPTIONAL},
+
+ {0},
+ };
+
+ return parse_oauth_json(actx, fields);
+}
+
+/*
+ * Parses the "interval" JSON number, corresponding to the number of seconds to
+ * wait between token endpoint requests.
+ *
+ * RFC 8628 is pretty silent on sanity checks for the interval. As a matter of
+ * practicality, round any fractional intervals up to the next second, and clamp
+ * the result at a minimum of one. (Zero-second intervals would result in an
+ * expensive network polling loop.)
+ *
+ * TODO: maybe clamp the upper bound too, based on the libpq timeout and/or the
+ * code expiration time?
+ */
+static int
+parse_interval(const char *interval_str)
+{
+ double parsed;
+ int cnt;
+
+ /*
+ * The JSON lexer has already validated the number, which is stricter than
+ * the %f format, so we should be good to use sscanf().
+ */
+ cnt = sscanf(interval_str, "%lf", &parsed);
+
+ if (cnt != 1)
+ {
+ /*
+ * Either the lexer screwed up or our assumption above isn't true, and
+ * either way a developer needs to take a look.
+ */
+ Assert(cnt == 1);
+ return 1; /* don't fall through in release builds */
+ }
+
+ parsed = ceil(parsed);
+
+ if (parsed < 1)
+ return 1; /* TODO this slows down the tests
+ * considerably... */
+ else if (INT_MAX <= parsed)
+ return INT_MAX;
+
+ return parsed;
+}
+
+static bool
+parse_device_authz(struct async_ctx *actx, struct device_authz *authz)
+{
+ struct json_field fields[] = {
+ {"device_code", JSON_TOKEN_STRING, {&authz->device_code}, REQUIRED},
+ {"user_code", JSON_TOKEN_STRING, {&authz->user_code}, REQUIRED},
+ {"verification_uri", JSON_TOKEN_STRING, {&authz->verification_uri}, REQUIRED},
+
+ /*
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - expires_in
+ */
+
+ {"interval", JSON_TOKEN_NUMBER, {&authz->interval_str}, OPTIONAL},
+
+ {0},
+ };
+
+ if (!parse_oauth_json(actx, fields))
+ return false;
+
+ /*
+ * Parse our numeric fields. Lexing has already completed by this time, so
+ * we at least know they're valid JSON numbers.
+ */
+ if (authz->interval_str)
+ authz->interval = parse_interval(authz->interval_str);
+ else
+ {
+ /* TODO: handle default interval of 5 seconds */
+ }
+
+ return true;
+}
+
+static bool
+parse_token_error(struct async_ctx *actx, struct token_error *err)
+{
+ bool result;
+ struct json_field fields[] = {
+ {"error", JSON_TOKEN_STRING, {&err->error}, REQUIRED},
+
+ {"error_description", JSON_TOKEN_STRING, {&err->error_description}, OPTIONAL},
+
+ {0},
+ };
+
+ result = parse_oauth_json(actx, fields);
+
+ /*
+ * Since token errors are parsed during other active error paths, only
+ * override the errctx if parsing explicitly fails.
+ */
+ if (!result)
+ actx->errctx = "failed to parse token error response";
+
+ return result;
+}
+
+static bool
+parse_access_token(struct async_ctx *actx, struct token *tok)
+{
+ struct json_field fields[] = {
+ {"access_token", JSON_TOKEN_STRING, {&tok->access_token}, REQUIRED},
+ {"token_type", JSON_TOKEN_STRING, {&tok->token_type}, REQUIRED},
+
+ /*
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - scope (only required if different than requested -- TODO check)
+ */
+
+ {0},
+ };
+
+ return parse_oauth_json(actx, fields);
+}
+
+/*
+ * cURL Multi Setup/Callbacks
+ */
+
+/*
+ * Sets up the actx->mux, which is the altsock that PQconnectPoll clients will
+ * select() on instead of the Postgres socket during OAuth negotiation.
+ *
+ * This is just an epoll set or kqueue abstracting multiple other descriptors.
+ * A timerfd is always part of the set when using epoll; it's just disabled
+ * when we're not using it.
+ */
+static bool
+setup_multiplexer(struct async_ctx *actx)
+{
+#ifdef HAVE_SYS_EPOLL_H
+ struct epoll_event ev = {.events = EPOLLIN};
+
+ actx->mux = epoll_create1(EPOLL_CLOEXEC);
+ if (actx->mux < 0)
+ {
+ actx_error(actx, "failed to create epoll set: %m");
+ return false;
+ }
+
+ actx->timerfd = timerfd_create(CLOCK_MONOTONIC, TFD_CLOEXEC);
+ if (actx->timerfd < 0)
+ {
+ actx_error(actx, "failed to create timerfd: %m");
+ return false;
+ }
+
+ if (epoll_ctl(actx->mux, EPOLL_CTL_ADD, actx->timerfd, &ev) < 0)
+ {
+ actx_error(actx, "failed to add timerfd to epoll set: %m");
+ return false;
+ }
+
+ return true;
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ actx->mux = kqueue();
+ if (actx->mux < 0)
+ {
+ actx_error(actx, "failed to create kqueue: %m");
+ return false;
+ }
+
+ return true;
+#endif
+
+ actx_error(actx, "here's a nickel kid, get yourself a better computer");
+ return false;
+}
+
+/*
+ * Adds and removes sockets from the multiplexer set, as directed by the
+ * cURL multi handle.
+ */
+static int
+register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
+ void *socketp)
+{
+#ifdef HAVE_SYS_EPOLL_H
+ struct async_ctx *actx = ctx;
+ struct epoll_event ev = {0};
+ int res;
+ int op = EPOLL_CTL_ADD;
+
+ switch (what)
+ {
+ case CURL_POLL_IN:
+ ev.events = EPOLLIN;
+ break;
+
+ case CURL_POLL_OUT:
+ ev.events = EPOLLOUT;
+ break;
+
+ case CURL_POLL_INOUT:
+ ev.events = EPOLLIN | EPOLLOUT;
+ break;
+
+ case CURL_POLL_REMOVE:
+ op = EPOLL_CTL_DEL;
+ break;
+
+ default:
+ actx_error(actx, "unknown cURL socket operation (%d)", what);
+ return -1;
+ }
+
+ res = epoll_ctl(actx->mux, op, socket, &ev);
+ if (res < 0 && errno == EEXIST)
+ {
+ /* We already had this socket in the pollset. */
+ op = EPOLL_CTL_MOD;
+ res = epoll_ctl(actx->mux, op, socket, &ev);
+ }
+
+ if (res < 0)
+ {
+ switch (op)
+ {
+ case EPOLL_CTL_ADD:
+ actx_error(actx, "could not add to epoll set: %m");
+ break;
+
+ case EPOLL_CTL_DEL:
+ actx_error(actx, "could not delete from epoll set: %m");
+ break;
+
+ default:
+ actx_error(actx, "could not update epoll set: %m");
+ }
+
+ return -1;
+ }
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct async_ctx *actx = ctx;
+ struct kevent ev[2] = {{0}};
+ struct kevent ev_out[2];
+ struct timespec timeout = {0};
+ int nev = 0;
+ int res;
+
+ switch (what)
+ {
+ case CURL_POLL_IN:
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_OUT:
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_INOUT:
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_REMOVE:
+
+ /*
+ * We don't know which of these is currently registered, perhaps
+ * both, so we try to remove both. This means we need to tolerate
+ * ENOENT below.
+ */
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_DELETE | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_DELETE | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ default:
+ actx_error(actx, "unknown cURL socket operation (%d)", what);
+ return -1;
+ }
+
+ res = kevent(actx->mux, ev, nev, ev_out, lengthof(ev_out), &timeout);
+ if (res < 0)
+ {
+ actx_error(actx, "could not modify kqueue: %m");
+ return -1;
+ }
+
+ /*
+ * We can't use the simple errno version of kevent, because we need to
+ * skip over ENOENT while still allowing a second change to be processed.
+ * So we need a longer-form error checking loop.
+ */
+ for (int i = 0; i < res; ++i)
+ {
+ /*
+ * EV_RECEIPT should guarantee one EV_ERROR result for every change,
+ * whether successful or not. Failed entries contain a non-zero errno
+ * in the `data` field.
+ */
+ Assert(ev_out[i].flags & EV_ERROR);
+
+ errno = ev_out[i].data;
+ if (errno && errno != ENOENT)
+ {
+ switch (what)
+ {
+ case CURL_POLL_REMOVE:
+ actx_error(actx, "could not delete from kqueue: %m");
+ break;
+ default:
+ actx_error(actx, "could not add to kqueue: %m");
+ }
+ return -1;
+ }
+ }
+#endif
+
+ return 0;
+}
+
+/*
+ * Adds or removes timeouts from the multiplexer set, as directed by the
+ * cURL multi handle. Rather than continually adding and removing the timer,
+ * we keep it in the set at all times and just disarm it when it's not
+ * needed.
+ */
+static int
+register_timer(CURLM *curlm, long timeout, void *ctx)
+{
+#if HAVE_SYS_EPOLL_H
+ struct async_ctx *actx = ctx;
+ struct itimerspec spec = {0};
+
+ if (timeout < 0)
+ {
+ /* the zero itimerspec will disarm the timer below */
+ }
+ else if (timeout == 0)
+ {
+ /*
+ * A zero timeout means cURL wants us to call back immediately. That's
+ * not technically an option for timerfd, but we can make the timeout
+ * ridiculously short.
+ *
+ * TODO: maybe just signal drive_request() to immediately call back in
+ * this case?
+ */
+ spec.it_value.tv_nsec = 1;
+ }
+ else
+ {
+ spec.it_value.tv_sec = timeout / 1000;
+ spec.it_value.tv_nsec = (timeout % 1000) * 1000000;
+ }
+
+ if (timerfd_settime(actx->timerfd, 0 /* no flags */ , &spec, NULL) < 0)
+ {
+ actx_error(actx, "setting timerfd to %ld: %m", timeout);
+ return -1;
+ }
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct async_ctx *actx = ctx;
+ struct kevent ev;
+
+ EV_SET(&ev, 1, EVFILT_TIMER, timeout < 0 ? EV_DELETE : EV_ADD,
+ 0, timeout, 0);
+ if (kevent(actx->mux, &ev, 1, NULL, 0, NULL) < 0 && errno != ENOENT)
+ {
+ actx_error(actx, "setting kqueue timer to %ld: %m", timeout);
+ return -1;
+ }
+#endif
+
+ return 0;
+}
+
+/*
+ * Initializes the two cURL handles in the async_ctx. The multi handle,
+ * actx->curlm, is what drives the asynchronous engine and tells us what to do
+ * next. The easy handle, actx->curl, encapsulates the state for a single
+ * request/response. It's added to the multi handle as needed, during
+ * start_request().
+ */
+static bool
+setup_curl_handles(struct async_ctx *actx)
+{
+ curl_version_info_data *curl_info;
+
+ /*
+ * Create our multi handle. This encapsulates the entire conversation with
+ * cURL for this connection.
+ */
+ actx->curlm = curl_multi_init();
+ if (!actx->curlm)
+ {
+ /* We don't get a lot of feedback on the failure reason. */
+ actx_error(actx, "failed to create cURL multi handle");
+ return false;
+ }
+
+ /*
+ * Extract information about the libcurl we are linked against.
+ */
+ curl_info = curl_version_info(CURLVERSION_NOW);
+
+ /*
+ * The multi handle tells us what to wait on using two callbacks. These
+ * will manipulate actx->mux as needed.
+ */
+ CHECK_MSETOPT(actx, CURLMOPT_SOCKETFUNCTION, register_socket, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_SOCKETDATA, actx, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_TIMERFUNCTION, register_timer, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_TIMERDATA, actx, return false);
+
+ /*
+ * Set up an easy handle. All of our requests are made serially, so we
+ * only ever need to keep track of one.
+ */
+ actx->curl = curl_easy_init();
+ if (!actx->curl)
+ {
+ actx_error(actx, "failed to create cURL handle");
+ return false;
+ }
+
+ /*
+ * Multi-threaded applications must set CURLOPT_NOSIGNAL. This requires us
+ * to handle the possibility of SIGPIPE ourselves.
+ *
+ * TODO: handle SIGPIPE via pq_block_sigpipe(), or via a
+ * CURLOPT_SOCKOPTFUNCTION maybe...
+ */
+ CHECK_SETOPT(actx, CURLOPT_NOSIGNAL, 1L, return false);
+ if (!curl_info->ares_num)
+ {
+ /* No alternative resolver, TODO: warn about timeouts */
+ }
+
+ /* TODO investigate using conn->Pfdebug and CURLOPT_DEBUGFUNCTION here */
+ CHECK_SETOPT(actx, CURLOPT_VERBOSE, 1L, return false);
+ CHECK_SETOPT(actx, CURLOPT_ERRORBUFFER, actx->curl_err, return false);
+
+ /*
+ * Only HTTP[S] is allowed. TODO: disallow HTTP without user opt-in
+ */
+ CHECK_SETOPT(actx, CURLOPT_PROTOCOLS, CURLPROTO_HTTP | CURLPROTO_HTTPS, return false);
+
+ /*
+ * Suppress the Accept header to make our request as minimal as possible.
+ * (Ideally we would set it to "application/json" instead, but OpenID is
+ * pretty strict when it comes to provider behavior, so we have to check
+ * what comes back anyway.)
+ */
+ actx->headers = curl_slist_append(actx->headers, "Accept:"); /* TODO: check result */
+ CHECK_SETOPT(actx, CURLOPT_HTTPHEADER, actx->headers, return false);
+
+ return true;
+}
+
+/*
+ * Generic HTTP Request Handlers
+ */
+
+/*
+ * Response callback from cURL; appends the response body into actx->work_data.
+ * See start_request().
+ */
+static size_t
+append_data(char *buf, size_t size, size_t nmemb, void *userdata)
+{
+ PQExpBuffer resp = userdata;
+ size_t len = size * nmemb;
+
+ /* TODO: cap the maximum size */
+ appendBinaryPQExpBuffer(resp, buf, len);
+ /* TODO: check for broken buffer */
+
+ return len;
+}
+
+/*
+ * Begins an HTTP request on the multi handle. The caller should have set up all
+ * request-specific options on actx->curl first. The server's response body will
+ * be accumulated in actx->work_data (which will be reset, so don't store
+ * anything important there across this call).
+ *
+ * Once a request is queued, it can be driven to completion via drive_request().
+ */
+static bool
+start_request(struct async_ctx *actx)
+{
+ CURLMcode err;
+ int running;
+
+ resetPQExpBuffer(&actx->work_data);
+ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data, return false);
+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, &actx->work_data, return false);
+
+ err = curl_multi_add_handle(actx->curlm, actx->curl);
+ if (err)
+ {
+ actx_error(actx, "failed to queue HTTP request: %s",
+ curl_multi_strerror(err));
+ return false;
+ }
+
+ err = curl_multi_socket_action(actx->curlm, CURL_SOCKET_TIMEOUT, 0, &running);
+ if (err)
+ {
+ actx_error(actx, "asynchronous HTTP request failed: %s",
+ curl_multi_strerror(err));
+ return false;
+ }
+
+ /*
+ * Sanity check.
+ *
+ * TODO: even though this is nominally an asynchronous process, there are
+ * apparently operations that can synchronously fail by this point, such
+ * as connections to closed local ports. Maybe we need to let this case
+ * fall through to drive_request instead, or else perform a
+ * curl_multi_info_read immediately.
+ */
+ if (running != 1)
+ {
+ actx_error(actx, "failed to queue HTTP request");
+ return false;
+ }
+
+ return true;
+}
+
+/*
+ * Drives the multi handle towards completion. The caller should have already
+ * set up an asynchronous request via start_request().
+ */
+static PostgresPollingStatusType
+drive_request(struct async_ctx *actx)
+{
+ CURLMcode err;
+ int running;
+ CURLMsg *msg;
+ int msgs_left;
+ bool done;
+
+ err = curl_multi_socket_all(actx->curlm, &running);
+ if (err)
+ {
+ actx_error(actx, "asynchronous HTTP request failed: %s",
+ curl_multi_strerror(err));
+ return PGRES_POLLING_FAILED;
+ }
+
+ if (running)
+ {
+ /* We'll come back again. */
+ return PGRES_POLLING_READING;
+ }
+
+ done = false;
+ while ((msg = curl_multi_info_read(actx->curlm, &msgs_left)) != NULL)
+ {
+ if (msg->msg != CURLMSG_DONE)
+ {
+ /*
+ * Future cURL versions may define new message types; we don't
+ * know how to handle them, so we'll ignore them.
+ */
+ continue;
+ }
+
+ /* First check the status of the request itself. */
+ if (msg->data.result != CURLE_OK)
+ {
+ actx_error_str(actx, curl_easy_strerror(msg->data.result));
+ return PGRES_POLLING_FAILED;
+ }
+
+ /* Now remove the finished handle; we'll add it back later if needed. */
+ err = curl_multi_remove_handle(actx->curlm, msg->easy_handle);
+ if (err)
+ {
+ actx_error(actx, "cURL easy handle removal failed: %s",
+ curl_multi_strerror(err));
+ return PGRES_POLLING_FAILED;
+ }
+
+ done = true;
+ }
+
+ /* Sanity check. */
+ if (!done)
+ {
+ actx_error(actx, "no result was retrieved for the finished handle");
+ return PGRES_POLLING_FAILED;
+ }
+
+ return PGRES_POLLING_OK;
+}
+
+/*
+ * Specific HTTP Request Handlers
+ *
+ * This is finally the beginning of the actual application logic. Generally
+ * speaking, a single request consists of a start_* and a finish_* step, with
+ * drive_request() pumping the machine in between.
+ */
+
+/*
+ * Queue an OpenID Provider Configuration Request:
+ *
+ * https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderConfigurationRequest
+ * https://www.rfc-editor.org/rfc/rfc8414#section-3.1
+ *
+ * This is done first to get the endpoint URIs we need to contact and to make
+ * sure the provider provides a device authorization flow. finish_discovery()
+ * will fill in actx->provider.
+ */
+static bool
+start_discovery(struct async_ctx *actx, const char *discovery_uri)
+{
+ CHECK_SETOPT(actx, CURLOPT_HTTPGET, 1L, return false);
+ CHECK_SETOPT(actx, CURLOPT_URL, discovery_uri, return false);
+
+ return start_request(actx);
+}
+
+static bool
+finish_discovery(struct async_ctx *actx)
+{
+ long response_code;
+
+ /*----
+ * Now check the response. OIDC Discovery 1.0 is pretty strict:
+ *
+ * A successful response MUST use the 200 OK HTTP status code and
+ * return a JSON object using the application/json content type that
+ * contains a set of Claims as its members that are a subset of the
+ * Metadata values defined in Section 3.
+ *
+ * Compared to standard HTTP semantics, this makes life easy -- we don't
+ * need to worry about redirections (which would call the Issuer host
+ * validation into question), or non-authoritative responses, or any other
+ * complications.
+ */
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ if (response_code != 200)
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ actx->errctx = "failed to parse OpenID discovery document";
+ if (!parse_provider(actx, &actx->provider))
+ return false; /* error message already set */
+
+ /*
+ * Fill in any defaults for OPTIONAL/RECOMMENDED fields we care about.
+ */
+ if (!actx->provider.grant_types_supported)
+ {
+ /*
+ * Per Section 3, the default is ["authorization_code", "implicit"].
+ */
+ struct curl_slist *temp = actx->provider.grant_types_supported;
+
+ temp = curl_slist_append(temp, "authorization_code");
+ if (temp)
+ {
+ temp = curl_slist_append(temp, "implicit");
+ }
+
+ if (!temp)
+ {
+ actx_error(actx, "out of memory");
+ return false;
+ }
+
+ actx->provider.grant_types_supported = temp;
+ }
+
+ return true;
+}
+
+#define OAUTH_GRANT_TYPE_DEVICE_CODE "urn:ietf:params:oauth:grant-type:device_code"
+
+/*
+ * Ensure that the provider supports the Device Authorization flow (i.e. it
+ * accepts the device_code grant type and provides an authorization endpoint).
+ */
+static bool
+check_for_device_flow(struct async_ctx *actx)
+{
+ const struct provider *provider = &actx->provider;
+ const struct curl_slist *grant;
+ bool device_grant_found = false;
+
+ Assert(provider->issuer); /* ensured by get_discovery_document() */
+
+ /*------
+ * First, sanity checks for discovery contents that are OPTIONAL in the
+ * spec but required for our flow:
+ * - the issuer must support the device_code grant
+ * - the issuer must have actually given us a
+ * device_authorization_endpoint
+ */
+
+ grant = provider->grant_types_supported;
+ while (grant)
+ {
+ if (strcmp(grant->data, OAUTH_GRANT_TYPE_DEVICE_CODE) == 0)
+ {
+ device_grant_found = true;
+ break;
+ }
+
+ grant = grant->next;
+ }
+
+ if (!device_grant_found)
+ {
+ actx_error(actx, "issuer \"%s\" does not support device code grants",
+ provider->issuer);
+ return false;
+ }
+
+ if (!provider->device_authorization_endpoint)
+ {
+ actx_error(actx,
+ "issuer \"%s\" does not provide a device authorization endpoint",
+ provider->issuer);
+ return false;
+ }
+
+ /* TODO: check that the endpoint uses HTTPS */
+
+ return true;
+}
+
+/*
+ * Queue a Device Authorization Request:
+ *
+ * https://www.rfc-editor.org/rfc/rfc8628#section-3.1
+ *
+ * This is the second step. We ask the provider to verify the end user out of
+ * band and authorize us to act on their behalf; it will give us the required
+ * nonces for us to later poll the request status, which we'll grab in
+ * finish_device_authz().
+ */
+static bool
+start_device_authz(struct async_ctx *actx, PGconn *conn)
+{
+ const char *device_authz_uri = actx->provider.device_authorization_endpoint;
+ PQExpBuffer work_buffer = &actx->work_data;
+
+ Assert(conn->oauth_client_id); /* ensured by get_auth_token() */
+ Assert(device_authz_uri); /* ensured by check_for_device_flow() */
+
+ /* Construct our request body. TODO: url-encode */
+ resetPQExpBuffer(work_buffer);
+ appendPQExpBuffer(work_buffer, "client_id=%s", conn->oauth_client_id);
+ if (conn->oauth_scope)
+ appendPQExpBuffer(work_buffer, "&scope=%s", conn->oauth_scope);
+ /* TODO check for broken buffer */
+
+ /* Make our request. */
+ CHECK_SETOPT(actx, CURLOPT_URL, device_authz_uri, return false);
+ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data, return false);
+
+ if (conn->oauth_client_secret)
+ {
+ /*----
+ * Use HTTP Basic auth to send the password. Per RFC 6749, Sec. 2.3.1,
+ *
+ * Including the client credentials in the request-body using the
+ * two parameters is NOT RECOMMENDED and SHOULD be limited to
+ * clients unable to directly utilize the HTTP Basic authentication
+ * scheme (or other password-based HTTP authentication schemes).
+ *
+ * TODO: should we omit client_id from the body in this case?
+ */
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC, return false);
+ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id, return false);
+ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret, return false);
+ }
+ else
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE, return false);
+
+ return start_request(actx);
+}
+
+static bool
+finish_device_authz(struct async_ctx *actx)
+{
+ long response_code;
+
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ /*
+ * The device authorization endpoint uses the same error response as the
+ * token endpoint, so we have to handle 400/401 here too.
+ */
+ if (response_code != 200
+ && response_code != 400
+ /* && response_code != 401 TODO */ )
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ if (response_code != 200)
+ {
+ struct token_error err = {0};
+
+ if (!parse_token_error(actx, &err))
+ {
+ free_token_error(&err);
+ return false;
+ }
+
+ if (err.error_description)
+ appendPQExpBuffer(&actx->errbuf, "%s ", err.error_description);
+
+ appendPQExpBuffer(&actx->errbuf, "(%s)", err.error);
+
+ free_token_error(&err);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ actx->errctx = "failed to parse device authorization";
+ if (!parse_device_authz(actx, &actx->authz))
+ return false; /* error message already set */
+
+ return true;
+}
+
+/*
+ * Queue an Access Token Request:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-4.1.3
+ *
+ * This is the final step. We continually poll the token endpoint to see if the
+ * user has authorized us yet. finish_token_request() will pull either the token
+ * or a (ideally temporary) error status from the provider.
+ */
+static bool
+start_token_request(struct async_ctx *actx, PGconn *conn)
+{
+ const char *token_uri = actx->provider.token_endpoint;
+ const char *device_code = actx->authz.device_code;
+ PQExpBuffer work_buffer = &actx->work_data;
+
+ Assert(conn->oauth_client_id); /* ensured by get_auth_token() */
+ Assert(token_uri); /* ensured by get_discovery_document() */
+ Assert(device_code); /* ensured by run_device_authz() */
+
+ /* Construct our request body. TODO: url-encode */
+ resetPQExpBuffer(work_buffer);
+ appendPQExpBuffer(work_buffer, "client_id=%s", conn->oauth_client_id);
+ appendPQExpBuffer(work_buffer, "&device_code=%s", device_code);
+ appendPQExpBuffer(work_buffer, "&grant_type=%s",
+ OAUTH_GRANT_TYPE_DEVICE_CODE);
+ /* TODO check for broken buffer */
+
+ /* Make our request. */
+ CHECK_SETOPT(actx, CURLOPT_URL, token_uri, return false);
+ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data, return false);
+
+ if (conn->oauth_client_secret)
+ {
+ /*----
+ * Use HTTP Basic auth to send the password. Per RFC 6749, Sec. 2.3.1,
+ *
+ * Including the client credentials in the request-body using the
+ * two parameters is NOT RECOMMENDED and SHOULD be limited to
+ * clients unable to directly utilize the HTTP Basic authentication
+ * scheme (or other password-based HTTP authentication schemes).
+ *
+ * TODO: should we omit client_id from the body in this case?
+ */
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC, return false);
+ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id, return false);
+ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret, return false);
+ }
+ else
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE, return false);
+
+ resetPQExpBuffer(work_buffer);
+ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data, return false);
+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, work_buffer, return false);
+
+ return start_request(actx);
+}
+
+static bool
+finish_token_request(struct async_ctx *actx, struct token *tok)
+{
+ long response_code;
+
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ /*
+ * Per RFC 6749, Section 5, a successful response uses 200 OK. An error
+ * response uses either 400 Bad Request or 401 Unauthorized.
+ *
+ * TODO: there are references online to 403 appearing in the wild...
+ */
+ if (response_code != 200
+ && response_code != 400
+ /* && response_code != 401 TODO */ )
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ if (response_code == 200)
+ {
+ actx->errctx = "failed to parse access token response";
+ if (!parse_access_token(actx, tok))
+ return false; /* error message already set */
+ }
+ else if (!parse_token_error(actx, &tok->err))
+ return false;
+
+ return true;
+}
+
+/*
+ * The top-level, nonblocking entry point for the cURL implementation. This will
+ * be called several times to pump the async engine.
+ *
+ * The architecture is based on PQconnectPoll(). The first half drives the
+ * connection state forward as necessary, returning if we're not ready to
+ * proceed to the next step yet. The second half performs the actual transition
+ * between states.
+ *
+ * You can trace the overall OAuth flow through the second half. It's linear
+ * until we get to the end, where we flip back and forth between
+ * OAUTH_STEP_TOKEN_REQUEST and OAUTH_STEP_WAIT_INTERVAL to regularly ping the
+ * provider.
+ */
+PostgresPollingStatusType
+pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
+{
+ fe_oauth_state *state = conn->sasl_state;
+ struct async_ctx *actx;
+
+ struct token tok = {0};
+
+ /*
+ * XXX This is not safe. cURL has stringent requirements for the thread
+ * context in which you call curl_global_init(), because it's going to try
+ * initializing a bunch of other libraries (OpenSSL, Winsock...). And we
+ * probably need to consider both the TLS backend libcurl is compiled
+ * against and what the user has asked us to do via PQinit[Open]SSL.
+ *
+ * Recent versions of libcurl have improved the thread-safety situation,
+ * but you apparently can't check at compile time whether the
+ * implementation is thread-safe, and there's a chicken-and-egg problem
+ * where you can't check the thread safety until you've initialized cURL,
+ * which you can't do before you've made sure it's thread-safe...
+ *
+ * We know we've already initialized Winsock by this point, so we should
+ * be able to safely skip that bit. But we have to tell cURL to initialize
+ * everything else, because other pieces of our client executable may
+ * already be using cURL for their own purposes. If we initialize libcurl
+ * first, with only a subset of its features, we could break those other
+ * clients nondeterministically, and that would probably be a nightmare to
+ * debug.
+ */
+ curl_global_init(CURL_GLOBAL_ALL
+ & ~CURL_GLOBAL_WIN32); /* we already initialized Winsock */
+
+ if (!state->async_ctx)
+ {
+ /*
+ * Create our asynchronous state, and hook it into the upper-level
+ * OAuth state immediately, so any failures below won't leak the
+ * context allocation.
+ */
+ actx = calloc(1, sizeof(*actx));
+ if (!actx)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return PGRES_POLLING_FAILED;
+ }
+
+ actx->mux = PGINVALID_SOCKET;
+#ifdef HAVE_SYS_EPOLL_H
+ actx->timerfd = -1;
+#endif
+
+ state->async_ctx = actx;
+ state->free_async_ctx = free_curl_async_ctx;
+
+ initPQExpBuffer(&actx->work_data);
+ initPQExpBuffer(&actx->errbuf);
+
+ if (!setup_multiplexer(actx))
+ goto error_return;
+
+ if (!setup_curl_handles(actx))
+ goto error_return;
+ }
+
+ actx = state->async_ctx;
+
+ /* By default, the multiplexer is the altsock. Reassign as desired. */
+ *altsock = actx->mux;
+
+ switch (actx->step)
+ {
+ case OAUTH_STEP_INIT:
+ break;
+
+ case OAUTH_STEP_DISCOVERY:
+ case OAUTH_STEP_DEVICE_AUTHORIZATION:
+ case OAUTH_STEP_TOKEN_REQUEST:
+ {
+ PostgresPollingStatusType status;
+
+ status = drive_request(actx);
+
+ if (status == PGRES_POLLING_FAILED)
+ goto error_return;
+ else if (status != PGRES_POLLING_OK)
+ {
+ /* not done yet */
+ free_token(&tok);
+ return status;
+ }
+ }
+
+ case OAUTH_STEP_WAIT_INTERVAL:
+ /* TODO check that the timer has expired */
+ break;
+ }
+
+ switch (actx->step)
+ {
+ case OAUTH_STEP_INIT:
+ actx->errctx = "failed to fetch OpenID discovery document";
+ if (!start_discovery(actx, conn->oauth_discovery_uri))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_DISCOVERY;
+ break;
+
+ case OAUTH_STEP_DISCOVERY:
+ if (!finish_discovery(actx))
+ goto error_return;
+
+ /* TODO: check issuer */
+
+ actx->errctx = "cannot run OAuth device authorization";
+ if (!check_for_device_flow(actx))
+ goto error_return;
+
+ actx->errctx = "failed to obtain device authorization";
+ if (!start_device_authz(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_DEVICE_AUTHORIZATION;
+ break;
+
+ case OAUTH_STEP_DEVICE_AUTHORIZATION:
+ if (!finish_device_authz(actx))
+ goto error_return;
+
+ actx->errctx = "failed to obtain access token";
+ if (!start_token_request(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_TOKEN_REQUEST;
+ break;
+
+ case OAUTH_STEP_TOKEN_REQUEST:
+ {
+ const struct token_error *err;
+#ifdef HAVE_SYS_EPOLL_H
+ struct itimerspec spec = {0};
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct kevent ev = {0};
+#endif
+
+ if (!finish_token_request(actx, &tok))
+ goto error_return;
+
+ if (!actx->user_prompted)
+ {
+ int res;
+ PQpromptOAuthDevice prompt = {
+ .verification_uri = actx->authz.verification_uri,
+ .user_code = actx->authz.user_code,
+ /* TODO: optional fields */
+ };
+
+ /*
+ * Now that we know the token endpoint isn't broken, give
+ * the user the login instructions.
+ */
+ res = PQauthDataHook(PQAUTHDATA_PROMPT_OAUTH_DEVICE, conn,
+ &prompt);
+
+ if (!res)
+ {
+ fprintf(stderr, "Visit %s and enter the code: %s",
+ prompt.verification_uri, prompt.user_code);
+ }
+ else if (res < 0)
+ {
+ actx_error(actx, "device prompt failed");
+ goto error_return;
+ }
+
+ actx->user_prompted = true;
+ }
+
+ if (tok.access_token)
+ {
+ /* Construct our Bearer token. */
+ resetPQExpBuffer(&actx->work_data);
+ appendPQExpBuffer(&actx->work_data, "Bearer %s",
+ tok.access_token);
+
+ if (PQExpBufferDataBroken(actx->work_data))
+ {
+ actx_error(actx, "out of memory");
+ goto error_return;
+ }
+
+ state->token = strdup(actx->work_data.data);
+ break;
+ }
+
+ /*
+ * authorization_pending and slow_down are the only acceptable
+ * errors; anything else and we bail.
+ */
+ err = &tok.err;
+ if (!err->error || (strcmp(err->error, "authorization_pending")
+ && strcmp(err->error, "slow_down")))
+ {
+ /* TODO handle !err->error */
+ if (err->error_description)
+ appendPQExpBuffer(&actx->errbuf, "%s ",
+ err->error_description);
+
+ appendPQExpBuffer(&actx->errbuf, "(%s)", err->error);
+
+ goto error_return;
+ }
+
+ /*
+ * A slow_down error requires us to permanently increase our
+ * retry interval by five seconds. RFC 8628, Sec. 3.5.
+ */
+ if (strcmp(err->error, "slow_down") == 0)
+ {
+ actx->authz.interval += 5; /* TODO check for overflow? */
+ }
+
+ /*
+ * Wait for the required interval before issuing the next
+ * request.
+ */
+ Assert(actx->authz.interval > 0);
+#ifdef HAVE_SYS_EPOLL_H
+ spec.it_value.tv_sec = actx->authz.interval;
+
+ if (timerfd_settime(actx->timerfd, 0 /* no flags */ , &spec, NULL) < 0)
+ {
+ actx_error(actx, "failed to set timerfd: %m");
+ goto error_return;
+ }
+
+ *altsock = actx->timerfd;
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ /* XXX: I guess this wants to be hidden in a routine */
+ EV_SET(&ev, 1, EVFILT_TIMER, EV_ADD, 0,
+ actx->authz.interval * 1000, 0);
+ if (kevent(actx->mux, &ev, 1, NULL, 0, NULL) < 0)
+ {
+ actx_error(actx, "failed to set kqueue timer: %m");
+ goto error_return;
+ }
+ /* XXX: why did we change the altsock in the epoll version? */
+#endif
+ actx->step = OAUTH_STEP_WAIT_INTERVAL;
+ break;
+ }
+
+ case OAUTH_STEP_WAIT_INTERVAL:
+ actx->errctx = "failed to obtain access token";
+ if (!start_token_request(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_TOKEN_REQUEST;
+ break;
+ }
+
+ free_token(&tok);
+
+ /* If we've stored a token, we're done. Otherwise come back later. */
+ return state->token ? PGRES_POLLING_OK : PGRES_POLLING_READING;
+
+error_return:
+
+ /*
+ * Assemble the three parts of our error: context, body, and detail. See
+ * also the documentation for struct async_ctx.
+ */
+ if (actx->errctx)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext(actx->errctx));
+ appendPQExpBufferStr(&conn->errorMessage, ": ");
+ }
+
+ if (PQExpBufferDataBroken(actx->errbuf))
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("out of memory"));
+ else
+ appendPQExpBufferStr(&conn->errorMessage, actx->errbuf.data);
+
+ if (actx->curl_err[0])
+ {
+ size_t len;
+
+ appendPQExpBuffer(&conn->errorMessage, " (%s)", actx->curl_err);
+
+ /* Sometimes libcurl adds a newline to the error buffer. :( */
+ len = conn->errorMessage.len;
+ if (len >= 2 && conn->errorMessage.data[len - 2] == '\n')
+ {
+ conn->errorMessage.data[len - 2] = ')';
+ conn->errorMessage.data[len - 1] = '\0';
+ conn->errorMessage.len--;
+ }
+ }
+
+ appendPQExpBufferStr(&conn->errorMessage, "\n");
+
+ free_token(&tok);
+ return PGRES_POLLING_FAILED;
+}
diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
new file mode 100644
index 0000000000..66ee8ff076
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth.c
@@ -0,0 +1,659 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth.c
+ * The front-end (client) implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "common/base64.h"
+#include "common/hmac.h"
+#include "common/jsonapi.h"
+#include "common/oauth-common.h"
+#include "fe-auth.h"
+#include "fe-auth-oauth.h"
+#include "mb/pg_wchar.h"
+
+/* The exported OAuth callback mechanism. */
+static void *oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism);
+static SASLStatus oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen);
+static bool oauth_channel_bound(void *opaq);
+static void oauth_free(void *opaq);
+
+const pg_fe_sasl_mech pg_oauth_mech = {
+ oauth_init,
+ oauth_exchange,
+ oauth_channel_bound,
+ oauth_free,
+};
+
+static void *
+oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism)
+{
+ fe_oauth_state *state;
+
+ /*
+ * We only support one SASL mechanism here; anything else is programmer
+ * error.
+ */
+ Assert(sasl_mechanism != NULL);
+ Assert(strcmp(sasl_mechanism, OAUTHBEARER_NAME) == 0);
+
+ state = calloc(1, sizeof(*state));
+ if (!state)
+ return NULL;
+
+ state->state = FE_OAUTH_INIT;
+ state->conn = conn;
+
+ return state;
+}
+
+#define kvsep "\x01"
+
+static char *
+client_initial_response(PGconn *conn, const char *token)
+{
+ static const char *const resp_format = "n,," kvsep "auth=%s" kvsep kvsep;
+
+ PQExpBufferData buf;
+ char *response = NULL;
+
+ if (!token)
+ {
+ /*
+ * Either programmer error, or something went badly wrong during the
+ * asynchronous fetch.
+ *
+ * TODO: users shouldn't see this; what action should they take if
+ * they do?
+ */
+ libpq_append_conn_error(conn, "no OAuth token was set for the connection");
+ return NULL;
+ }
+
+ initPQExpBuffer(&buf);
+ appendPQExpBuffer(&buf, resp_format, token);
+
+ if (!PQExpBufferDataBroken(buf))
+ response = strdup(buf.data);
+
+ termPQExpBuffer(&buf);
+ return response;
+}
+
+#define ERROR_STATUS_FIELD "status"
+#define ERROR_SCOPE_FIELD "scope"
+#define ERROR_OPENID_CONFIGURATION_FIELD "openid-configuration"
+
+struct json_ctx
+{
+ char *errmsg; /* any non-NULL value stops all processing */
+ PQExpBufferData errbuf; /* backing memory for errmsg */
+ int nested; /* nesting level (zero is the top) */
+
+ const char *target_field_name; /* points to a static allocation */
+ char **target_field; /* see below */
+
+ /* target_field, if set, points to one of the following: */
+ char *status;
+ char *scope;
+ char *discovery_uri;
+};
+
+#define oauth_json_has_error(ctx) \
+ (PQExpBufferDataBroken((ctx)->errbuf) || (ctx)->errmsg)
+
+#define oauth_json_set_error(ctx, ...) \
+ do { \
+ appendPQExpBuffer(&(ctx)->errbuf, __VA_ARGS__); \
+ (ctx)->errmsg = (ctx)->errbuf.data; \
+ } while (0)
+
+static JsonParseErrorType
+oauth_json_object_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ ++ctx->nested;
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_end(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_field_start(void *state, char *name, bool isnull)
+{
+ struct json_ctx *ctx = state;
+
+ if (ctx->nested == 1)
+ {
+ if (strcmp(name, ERROR_STATUS_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_STATUS_FIELD;
+ ctx->target_field = &ctx->status;
+ }
+ else if (strcmp(name, ERROR_SCOPE_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_SCOPE_FIELD;
+ ctx->target_field = &ctx->scope;
+ }
+ else if (strcmp(name, ERROR_OPENID_CONFIGURATION_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_OPENID_CONFIGURATION_FIELD;
+ ctx->target_field = &ctx->discovery_uri;
+ }
+ }
+
+ free(name);
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_scalar(void *state, char *token, JsonTokenType type)
+{
+ struct json_ctx *ctx = state;
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ if (type == JSON_TOKEN_STRING)
+ {
+ *ctx->target_field = token;
+
+ ctx->target_field = NULL;
+ ctx->target_field_name = NULL;
+
+ return JSON_SUCCESS; /* don't free the token we're using */
+ }
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ free(token);
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static bool
+handle_oauth_sasl_error(PGconn *conn, char *msg, int msglen)
+{
+ JsonLexContext lex = {0};
+ JsonSemAction sem = {0};
+ JsonParseErrorType err;
+ struct json_ctx ctx = {0};
+ char *errmsg = NULL;
+
+ /* Sanity check. */
+ if (strlen(msg) != msglen)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server's error message contained an embedded NULL, and was discarded"));
+ return false;
+ }
+
+ makeJsonLexContextCstringLen(&lex, msg, msglen, PG_UTF8, true);
+
+ initPQExpBuffer(&ctx.errbuf);
+ sem.semstate = &ctx;
+
+ sem.object_start = oauth_json_object_start;
+ sem.object_end = oauth_json_object_end;
+ sem.object_field_start = oauth_json_object_field_start;
+ sem.array_start = oauth_json_array_start;
+ sem.scalar = oauth_json_scalar;
+
+ err = pg_parse_json(&lex, &sem);
+
+ if (err == JSON_SEM_ACTION_FAILED)
+ {
+ if (PQExpBufferDataBroken(ctx.errbuf))
+ errmsg = libpq_gettext("out of memory");
+ else if (ctx.errmsg)
+ errmsg = ctx.errmsg;
+ else
+ {
+ /*
+ * Developer error: one of the action callbacks didn't call
+ * oauth_json_set_error() before erroring out.
+ */
+ Assert(oauth_json_has_error(&ctx));
+ errmsg = "<unexpected empty error>";
+ }
+ }
+ else if (err != JSON_SUCCESS)
+ errmsg = json_errdetail(err, &lex);
+
+ if (errmsg)
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("failed to parse server's error response: %s"),
+ errmsg);
+
+ /* Don't need the error buffer or the JSON lexer anymore. */
+ termPQExpBuffer(&ctx.errbuf);
+ freeJsonLexContext(&lex);
+
+ if (errmsg)
+ return false;
+
+ /* TODO: what if these override what the user already specified? */
+ if (ctx.discovery_uri)
+ {
+ if (conn->oauth_discovery_uri)
+ free(conn->oauth_discovery_uri);
+
+ conn->oauth_discovery_uri = ctx.discovery_uri;
+ }
+
+ if (ctx.scope)
+ {
+ if (conn->oauth_scope)
+ free(conn->oauth_scope);
+
+ conn->oauth_scope = ctx.scope;
+ }
+ /* TODO: missing error scope should clear any existing connection scope */
+
+ if (!ctx.status)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("server sent error response without a status"));
+ return false;
+ }
+
+ if (strcmp(ctx.status, "invalid_token") == 0)
+ {
+ /*
+ * invalid_token is the only error code we'll automatically retry for,
+ * but only if we have enough information to do so.
+ */
+ if (conn->oauth_discovery_uri)
+ conn->oauth_want_retry = true;
+ }
+ /* TODO: include status in hard failure message */
+
+ return true;
+}
+
+static void
+free_request(PGconn *conn, void *vreq)
+{
+ PQoauthBearerRequest *request = vreq;
+
+ if (request->cleanup)
+ request->cleanup(conn, request);
+
+ free(request);
+}
+
+static PostgresPollingStatusType
+run_user_oauth_flow(PGconn *conn, pgsocket *altsock)
+{
+ fe_oauth_state *state = conn->sasl_state;
+ PQoauthBearerRequest *request = state->async_ctx;
+ PostgresPollingStatusType status;
+
+ if (!request->async)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow provided neither a token nor an async callback");
+ return PGRES_POLLING_FAILED;
+ }
+
+ status = request->async(conn, request, altsock);
+ if (status == PGRES_POLLING_FAILED)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow failed");
+ return status;
+ }
+ else if (status == PGRES_POLLING_OK)
+ {
+ /*
+ * We already have a token, so copy it into the state. (We can't hold
+ * onto the original string, since it may not be safe for us to free()
+ * it.)
+ */
+ PQExpBufferData token;
+
+ if (!request->token)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow did not provide a token");
+ return PGRES_POLLING_FAILED;
+ }
+
+ initPQExpBuffer(&token);
+ appendPQExpBuffer(&token, "Bearer %s", request->token);
+
+ if (PQExpBufferDataBroken(token))
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return PGRES_POLLING_FAILED;
+ }
+
+ state->token = token.data;
+ return PGRES_POLLING_OK;
+ }
+
+ /* TODO: what if no altsock was set? */
+ return status;
+}
+
+static bool
+setup_token_request(PGconn *conn, fe_oauth_state *state)
+{
+ int res;
+ PQoauthBearerRequest request = {
+ .openid_configuration = conn->oauth_discovery_uri,
+ .scope = conn->oauth_scope,
+ };
+
+ Assert(request.openid_configuration);
+
+ /* The client may have overridden the OAuth flow. */
+ res = PQauthDataHook(PQAUTHDATA_OAUTH_BEARER_TOKEN, conn, &request);
+ if (res > 0)
+ {
+ PQoauthBearerRequest *request_copy;
+
+ if (request.token)
+ {
+ /*
+ * We already have a token, so copy it into the state. (We can't
+ * hold onto the original string, since it may not be safe for us
+ * to free() it.)
+ */
+ PQExpBufferData token;
+
+ initPQExpBuffer(&token);
+ appendPQExpBuffer(&token, "Bearer %s", request.token);
+
+ if (PQExpBufferDataBroken(token))
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ goto fail;
+ }
+
+ state->token = token.data;
+
+ /* short-circuit */
+ if (request.cleanup)
+ request.cleanup(conn, &request);
+ return true;
+ }
+
+ request_copy = malloc(sizeof(*request_copy));
+ if (!request_copy)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ goto fail;
+ }
+
+ memcpy(request_copy, &request, sizeof(request));
+
+ conn->async_auth = run_user_oauth_flow;
+ state->async_ctx = request_copy;
+ state->free_async_ctx = free_request;
+ }
+ else if (res < 0)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow failed");
+ goto fail;
+ }
+ else
+ {
+ /* Use our built-in OAuth flow. */
+ conn->async_auth = pg_fe_run_oauth_flow;
+ }
+
+ return true;
+
+fail:
+ if (request.cleanup)
+ request.cleanup(conn, &request);
+ return false;
+}
+
+static bool
+derive_discovery_uri(PGconn *conn)
+{
+ PQExpBufferData discovery_buf;
+
+ if (conn->oauth_discovery_uri || !conn->oauth_issuer)
+ {
+ /*
+ * Either we already have one, or we aren't able to derive one
+ * ourselves. The latter case is not an error condition; we'll just
+ * ask the server to provide one for us.
+ */
+ return true;
+ }
+
+ initPQExpBuffer(&discovery_buf);
+
+ Assert(!conn->oauth_discovery_uri);
+ Assert(conn->oauth_issuer);
+
+ /*
+ * If we don't yet have a discovery URI, but the user gave us an explicit
+ * issuer, use the .well-known discovery URI for that issuer.
+ */
+ appendPQExpBufferStr(&discovery_buf, conn->oauth_issuer);
+ appendPQExpBufferStr(&discovery_buf, "/.well-known/openid-configuration");
+
+ if (PQExpBufferDataBroken(discovery_buf))
+ goto cleanup;
+
+ conn->oauth_discovery_uri = strdup(discovery_buf.data);
+
+cleanup:
+ termPQExpBuffer(&discovery_buf);
+ return (conn->oauth_discovery_uri != NULL);
+}
+
+static SASLStatus
+oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen)
+{
+ fe_oauth_state *state = opaq;
+ PGconn *conn = state->conn;
+
+ *output = NULL;
+ *outputlen = 0;
+
+ switch (state->state)
+ {
+ case FE_OAUTH_INIT:
+ Assert(inputlen == -1);
+
+ if (!derive_discovery_uri(conn))
+ return SASL_FAILED;
+
+ if (conn->oauth_discovery_uri)
+ {
+ if (!conn->oauth_client_id)
+ {
+ /* We can't talk to a server without a client identifier. */
+ libpq_append_conn_error(conn, "no oauth_client_id is set for the connection");
+ return SASL_FAILED;
+ }
+
+ /*
+ * Decide whether we're using a user-provided OAuth flow, or
+ * the one we have built in.
+ */
+ if (!setup_token_request(conn, state))
+ return SASL_FAILED;
+
+ if (state->token)
+ {
+ /*
+ * A really smart user implementation may have already
+ * given us the token (e.g. if there was an unexpired copy
+ * already cached). In that case, we can just fall
+ * through.
+ */
+ }
+ else
+ {
+ /*
+ * Otherwise, we have to hand the connection over to our
+ * OAuth implementation. This involves a number of HTTP
+ * connections and timed waits, so we escape the
+ * synchronous auth processing and tell PQconnectPoll to
+ * transfer control to our async implementation.
+ */
+ Assert(conn->async_auth); /* should have been set
+ * already */
+ state->state = FE_OAUTH_REQUESTING_TOKEN;
+ return SASL_ASYNC;
+ }
+ }
+ else
+ {
+ /*
+ * If we don't have a discovery URI to be able to request a
+ * token, we ask the server for one explicitly with an empty
+ * token. This doesn't require any asynchronous work.
+ */
+ state->token = strdup("");
+ if (!state->token)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return SASL_FAILED;
+ }
+ }
+
+ /* fall through */
+
+ case FE_OAUTH_REQUESTING_TOKEN:
+ /* We should still be in the initial response phase. */
+ Assert(inputlen == -1);
+
+ *output = client_initial_response(conn, state->token);
+ if (!*output)
+ return SASL_FAILED;
+
+ *outputlen = strlen(*output);
+ state->state = FE_OAUTH_BEARER_SENT;
+
+ return SASL_CONTINUE;
+
+ case FE_OAUTH_BEARER_SENT:
+ if (final)
+ {
+ /* TODO: ensure there is no message content here. */
+ return SASL_COMPLETE;
+ }
+
+ /*
+ * Error message sent by the server.
+ */
+ if (!handle_oauth_sasl_error(conn, input, inputlen))
+ return SASL_FAILED;
+
+ /*
+ * Respond with the required dummy message (RFC 7628, sec. 3.2.3).
+ */
+ *output = strdup(kvsep);
+ *outputlen = strlen(*output); /* == 1 */
+
+ state->state = FE_OAUTH_SERVER_ERROR;
+ return SASL_CONTINUE;
+
+ case FE_OAUTH_SERVER_ERROR:
+
+ /*
+ * After an error, the server should send an error response to
+ * fail the SASL handshake, which is handled in higher layers.
+ *
+ * If we get here, the server either sent *another* challenge
+ * which isn't defined in the RFC, or completed the handshake
+ * successfully after telling us it was going to fail. Neither is
+ * acceptable.
+ */
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server sent additional OAuth data after error\n"));
+ return SASL_FAILED;
+
+ default:
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("invalid OAuth exchange state\n"));
+ break;
+ }
+
+ return SASL_FAILED;
+}
+
+static bool
+oauth_channel_bound(void *opaq)
+{
+ /* This mechanism does not support channel binding. */
+ return false;
+}
+
+static void
+oauth_free(void *opaq)
+{
+ fe_oauth_state *state = opaq;
+
+ free(state->token);
+ if (state->async_ctx)
+ state->free_async_ctx(state->conn, state->async_ctx);
+
+ free(state);
+}
diff --git a/src/interfaces/libpq/fe-auth-oauth.h b/src/interfaces/libpq/fe-auth-oauth.h
new file mode 100644
index 0000000000..8d4ea45aa8
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth.h
@@ -0,0 +1,42 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth.h
+ *
+ * Definitions for OAuth authentication implementations
+ *
+ * Portions Copyright (c) 2023, PostgreSQL Global Development Group
+ *
+ * src/interfaces/libpq/fe-auth-oauth.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FE_AUTH_OAUTH_H
+#define FE_AUTH_OAUTH_H
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+
+
+typedef enum
+{
+ FE_OAUTH_INIT,
+ FE_OAUTH_REQUESTING_TOKEN,
+ FE_OAUTH_BEARER_SENT,
+ FE_OAUTH_SERVER_ERROR,
+} fe_oauth_state_enum;
+
+typedef struct
+{
+ fe_oauth_state_enum state;
+
+ PGconn *conn;
+ char *token;
+
+ void *async_ctx;
+ void (*free_async_ctx) (PGconn *conn, void *ctx);
+} fe_oauth_state;
+
+extern PostgresPollingStatusType pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock);
+
+#endif /* FE_AUTH_OAUTH_H */
diff --git a/src/interfaces/libpq/fe-auth-sasl.h b/src/interfaces/libpq/fe-auth-sasl.h
index 4eecf53a15..83bdab1f40 100644
--- a/src/interfaces/libpq/fe-auth-sasl.h
+++ b/src/interfaces/libpq/fe-auth-sasl.h
@@ -30,6 +30,7 @@ typedef enum
SASL_COMPLETE = 0,
SASL_FAILED,
SASL_CONTINUE,
+ SASL_ASYNC,
} SASLStatus;
/*
@@ -77,6 +78,8 @@ typedef struct pg_fe_sasl_mech
*
* state: The opaque mechanism state returned by init()
*
+ * final: true if the server has sent a final exchange outcome
+ *
* input: The challenge data sent by the server, or NULL when
* generating a client-first initial response (that is, when
* the server expects the client to send a message to start
@@ -101,12 +104,17 @@ typedef struct pg_fe_sasl_mech
*
* SASL_CONTINUE: The output buffer is filled with a client response.
* Additional server challenge is expected
+ * SASL_ASYNC: Some asynchronous processing external to the
+ * connection needs to be done before a response can be
+ * generated. The mechanism is responsible for setting up
+ * conn->async_auth appropriately before returning.
* SASL_COMPLETE: The SASL exchange has completed successfully.
* SASL_FAILED: The exchange has failed and the connection should be
* dropped.
*--------
*/
- SASLStatus (*exchange) (void *state, char *input, int inputlen,
+ SASLStatus (*exchange) (void *state, bool final,
+ char *input, int inputlen,
char **output, int *outputlen);
/*--------
diff --git a/src/interfaces/libpq/fe-auth-scram.c b/src/interfaces/libpq/fe-auth-scram.c
index 0bb820e0d9..da168eb2f5 100644
--- a/src/interfaces/libpq/fe-auth-scram.c
+++ b/src/interfaces/libpq/fe-auth-scram.c
@@ -24,7 +24,8 @@
/* The exported SCRAM callback mechanism. */
static void *scram_init(PGconn *conn, const char *password,
const char *sasl_mechanism);
-static SASLStatus scram_exchange(void *opaq, char *input, int inputlen,
+static SASLStatus scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen);
static bool scram_channel_bound(void *opaq);
static void scram_free(void *opaq);
@@ -202,7 +203,8 @@ scram_free(void *opaq)
* Exchange a SCRAM message with backend.
*/
static SASLStatus
-scram_exchange(void *opaq, char *input, int inputlen,
+scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen)
{
fe_scram_state *state = (fe_scram_state *) opaq;
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index 81ec08485d..9cd5c8cfb1 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -39,6 +39,7 @@
#endif
#include "common/md5.h"
+#include "common/oauth-common.h"
#include "common/scram-common.h"
#include "fe-auth.h"
#include "fe-auth-sasl.h"
@@ -419,7 +420,7 @@ pg_SSPI_startup(PGconn *conn, int use_negotiate, int payloadlen)
* Initialize SASL authentication exchange.
*/
static int
-pg_SASL_init(PGconn *conn, int payloadlen)
+pg_SASL_init(PGconn *conn, int payloadlen, bool *async)
{
char *initialresponse = NULL;
int initialresponselen;
@@ -437,7 +438,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
goto error;
}
- if (conn->sasl_state)
+ if (conn->sasl_state && !conn->async_auth)
{
libpq_append_conn_error(conn, "duplicate SASL authentication request");
goto error;
@@ -524,6 +525,15 @@ pg_SASL_init(PGconn *conn, int payloadlen)
conn->sasl = &pg_scram_mech;
conn->password_needed = true;
}
+#ifdef USE_OAUTH
+ else if (strcmp(mechanism_buf.data, OAUTHBEARER_NAME) == 0 &&
+ !selected_mechanism)
+ {
+ selected_mechanism = OAUTHBEARER_NAME;
+ conn->sasl = &pg_oauth_mech;
+ conn->password_needed = false;
+ }
+#endif
}
if (!selected_mechanism)
@@ -563,26 +573,48 @@ pg_SASL_init(PGconn *conn, int payloadlen)
Assert(conn->sasl);
- /*
- * Initialize the SASL state information with all the information gathered
- * during the initial exchange.
- *
- * Note: Only tls-unique is supported for the moment.
- */
- conn->sasl_state = conn->sasl->init(conn,
- password,
- selected_mechanism);
if (!conn->sasl_state)
- goto oom_error;
+ {
+ /*
+ * Initialize the SASL state information with all the information
+ * gathered during the initial exchange.
+ *
+ * Note: Only tls-unique is supported for the moment.
+ */
+ conn->sasl_state = conn->sasl->init(conn,
+ password,
+ selected_mechanism);
+ if (!conn->sasl_state)
+ goto oom_error;
+ }
+ else
+ {
+ /*
+ * This is only possible if we're returning from an async loop.
+ * Disconnect it now.
+ */
+ Assert(conn->async_auth);
+ conn->async_auth = NULL;
+ }
/* Get the mechanism-specific Initial Client Response, if any */
- status = conn->sasl->exchange(conn->sasl_state,
+ status = conn->sasl->exchange(conn->sasl_state, false,
NULL, -1,
&initialresponse, &initialresponselen);
if (status == SASL_FAILED)
goto error;
+ if (status == SASL_ASYNC)
+ {
+ /*
+ * The mechanism should have set up the necessary callbacks; all we
+ * need to do is signal the caller.
+ */
+ *async = true;
+ return STATUS_OK;
+ }
+
/*
* Build a SASLInitialResponse message, and send it.
*/
@@ -625,7 +657,7 @@ oom_error:
* the protocol.
*/
static int
-pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
+pg_SASL_continue(PGconn *conn, int payloadlen, bool final, bool *async)
{
char *output;
int outputlen;
@@ -650,11 +682,21 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
/* For safety and convenience, ensure the buffer is NULL-terminated. */
challenge[payloadlen] = '\0';
- status = conn->sasl->exchange(conn->sasl_state,
+ status = conn->sasl->exchange(conn->sasl_state, final,
challenge, payloadlen,
&output, &outputlen);
free(challenge); /* don't need the input anymore */
+ if (status == SASL_ASYNC)
+ {
+ /*
+ * The mechanism should have set up the necessary callbacks; all we
+ * need to do is signal the caller.
+ */
+ *async = true;
+ return STATUS_OK;
+ }
+
if (final && status == SASL_CONTINUE)
{
if (outputlen != 0)
@@ -955,12 +997,18 @@ check_expected_areq(AuthRequest areq, PGconn *conn)
* it. We are responsible for reading any remaining extra data, specific
* to the authentication method. 'payloadlen' is the remaining length in
* the message.
+ *
+ * If *async is set to true on return, the client doesn't yet have enough
+ * information to respond, and the caller must temporarily switch to
+ * conn->async_auth() to continue driving the exchange.
*/
int
-pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
+pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn, bool *async)
{
int oldmsglen;
+ *async = false;
+
if (!check_expected_areq(areq, conn))
return STATUS_ERROR;
@@ -1118,7 +1166,7 @@ pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
* The request contains the name (as assigned by IANA) of the
* authentication mechanism.
*/
- if (pg_SASL_init(conn, payloadlen) != STATUS_OK)
+ if (pg_SASL_init(conn, payloadlen, async) != STATUS_OK)
{
/* pg_SASL_init already set the error message */
return STATUS_ERROR;
@@ -1135,7 +1183,8 @@ pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
}
oldmsglen = conn->errorMessage.len;
if (pg_SASL_continue(conn, payloadlen,
- (areq == AUTH_REQ_SASL_FIN)) != STATUS_OK)
+ (areq == AUTH_REQ_SASL_FIN),
+ async) != STATUS_OK)
{
/* Use this message if pg_SASL_continue didn't supply one */
if (conn->errorMessage.len == oldmsglen)
@@ -1451,3 +1500,23 @@ PQchangePassword(PGconn *conn, const char *user, const char *passwd)
}
}
}
+
+PQauthDataHook_type PQauthDataHook = PQdefaultAuthDataHook;
+
+PQauthDataHook_type
+PQgetAuthDataHook(void)
+{
+ return PQauthDataHook;
+}
+
+void
+PQsetAuthDataHook(PQauthDataHook_type hook)
+{
+ PQauthDataHook = hook ? hook : PQdefaultAuthDataHook;
+}
+
+int
+PQdefaultAuthDataHook(PGAuthData type, PGconn *conn, void *data)
+{
+ return 0; /* handle nothing */
+}
diff --git a/src/interfaces/libpq/fe-auth.h b/src/interfaces/libpq/fe-auth.h
index a18c508688..3e2bc1333f 100644
--- a/src/interfaces/libpq/fe-auth.h
+++ b/src/interfaces/libpq/fe-auth.h
@@ -18,8 +18,12 @@
#include "libpq-int.h"
+extern PQauthDataHook_type PQauthDataHook;
+
+
/* Prototypes for functions in fe-auth.c */
-extern int pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn);
+extern int pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn,
+ bool *async);
extern char *pg_fe_getusername(uid_t user_id, PQExpBuffer errorMessage);
extern char *pg_fe_getauthname(PQExpBuffer errorMessage);
@@ -29,4 +33,7 @@ extern char *pg_fe_scram_build_secret(const char *password,
int iterations,
const char **errstr);
+/* Mechanisms in fe-auth-oauth.c */
+extern const pg_fe_sasl_mech pg_oauth_mech;
+
#endif /* FE_AUTH_H */
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 01e49c6975..714594324e 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -359,6 +359,23 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"Load-Balance-Hosts", "", 8, /* sizeof("disable") = 8 */
offsetof(struct pg_conn, load_balance_hosts)},
+ /* OAuth v2 */
+ {"oauth_issuer", NULL, NULL, NULL,
+ "OAuth-Issuer", "", 40,
+ offsetof(struct pg_conn, oauth_issuer)},
+
+ {"oauth_client_id", NULL, NULL, NULL,
+ "OAuth-Client-ID", "", 40,
+ offsetof(struct pg_conn, oauth_client_id)},
+
+ {"oauth_client_secret", NULL, NULL, NULL,
+ "OAuth-Client-Secret", "", 40,
+ offsetof(struct pg_conn, oauth_client_secret)},
+
+ {"oauth_scope", NULL, NULL, NULL,
+ "OAuth-Scope", "", 15,
+ offsetof(struct pg_conn, oauth_scope)},
+
/* Terminating entry --- MUST BE LAST */
{NULL, NULL, NULL, NULL,
NULL, NULL, 0}
@@ -616,6 +633,7 @@ pqDropServerData(PGconn *conn)
conn->write_failed = false;
free(conn->write_err_msg);
conn->write_err_msg = NULL;
+ /* conn->oauth_want_retry = false; TODO */
/*
* Cancel connections need to retain their be_pid and be_key across
@@ -2595,6 +2613,7 @@ PQconnectPoll(PGconn *conn)
case CONNECTION_NEEDED:
case CONNECTION_GSS_STARTUP:
case CONNECTION_CHECK_TARGET:
+ case CONNECTION_AUTHENTICATING:
break;
default:
@@ -3603,6 +3622,7 @@ keep_going: /* We will come back to here until there is
int avail;
AuthRequest areq;
int res;
+ bool async;
/*
* Scan the message from current point (note that if we find
@@ -3758,6 +3778,16 @@ keep_going: /* We will come back to here until there is
/* Check to see if we should mention pgpassfile */
pgpassfileWarning(conn);
+#ifdef USE_OAUTH
+ if (conn->sasl == &pg_oauth_mech
+ && conn->oauth_want_retry)
+ {
+ /* TODO: only allow retry once */
+ need_new_connection = true;
+ goto keep_going;
+ }
+#endif
+
#ifdef ENABLE_GSS
/*
@@ -3839,7 +3869,17 @@ keep_going: /* We will come back to here until there is
* Note that conn->pghost must be non-NULL if we are going to
* avoid the Kerberos code doing a hostname look-up.
*/
- res = pg_fe_sendauth(areq, msgLength, conn);
+ res = pg_fe_sendauth(areq, msgLength, conn, &async);
+
+ if (async && (res == STATUS_OK))
+ {
+ /*
+ * We'll come back later once we're ready to respond.
+ * Don't consume the request yet.
+ */
+ conn->status = CONNECTION_AUTHENTICATING;
+ goto keep_going;
+ }
/* OK, we have processed the message; mark data consumed */
conn->inStart = conn->inCursor;
@@ -3872,6 +3912,41 @@ keep_going: /* We will come back to here until there is
goto keep_going;
}
+ case CONNECTION_AUTHENTICATING:
+ {
+ PostgresPollingStatusType status;
+ pgsocket altsock = PGINVALID_SOCKET;
+
+ if (!conn->async_auth)
+ {
+ /* programmer error; should not happen */
+ libpq_append_conn_error(conn, "async authentication has no handler");
+ goto error_return;
+ }
+
+ status = conn->async_auth(conn, &altsock);
+
+ if (status == PGRES_POLLING_FAILED)
+ goto error_return;
+
+ if (status == PGRES_POLLING_OK)
+ {
+ /*
+ * Reenter the authentication exchange with the server. We
+ * didn't consume the message that started external
+ * authentication, so it'll be reprocessed as if we just
+ * received it.
+ */
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ conn->altsock = PGINVALID_SOCKET; /* TODO: what frees
+ * this? */
+ goto keep_going;
+ }
+
+ conn->altsock = altsock;
+ return status;
+ }
+
case CONNECTION_AUTH_OK:
{
/*
@@ -4377,6 +4452,7 @@ pqMakeEmptyPGconn(void)
conn->verbosity = PQERRORS_DEFAULT;
conn->show_context = PQSHOW_CONTEXT_ERRORS;
conn->sock = PGINVALID_SOCKET;
+ conn->altsock = PGINVALID_SOCKET;
conn->Pfdebug = NULL;
/*
@@ -4493,6 +4569,11 @@ freePGconn(PGconn *conn)
free(conn->rowBuf);
free(conn->target_session_attrs);
free(conn->load_balance_hosts);
+ free(conn->oauth_issuer);
+ free(conn->oauth_discovery_uri);
+ free(conn->oauth_client_id);
+ free(conn->oauth_client_secret);
+ free(conn->oauth_scope);
termPQExpBuffer(&conn->errorMessage);
termPQExpBuffer(&conn->workBuffer);
@@ -6975,6 +7056,8 @@ PQsocket(const PGconn *conn)
{
if (!conn)
return -1;
+ if (conn->altsock != PGINVALID_SOCKET)
+ return conn->altsock;
return (conn->sock != PGINVALID_SOCKET) ? conn->sock : -1;
}
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index f2fc78a481..663b1c1acf 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -1039,10 +1039,13 @@ static int
pqSocketCheck(PGconn *conn, int forRead, int forWrite, time_t end_time)
{
int result;
+ pgsocket sock;
if (!conn)
return -1;
- if (conn->sock == PGINVALID_SOCKET)
+
+ sock = (conn->altsock != PGINVALID_SOCKET) ? conn->altsock : conn->sock;
+ if (sock == PGINVALID_SOCKET)
{
libpq_append_conn_error(conn, "invalid socket");
return -1;
@@ -1059,7 +1062,7 @@ pqSocketCheck(PGconn *conn, int forRead, int forWrite, time_t end_time)
/* We will retry as long as we get EINTR */
do
- result = pqSocketPoll(conn->sock, forRead, forWrite, end_time);
+ result = pqSocketPoll(sock, forRead, forWrite, end_time);
while (result < 0 && SOCK_ERRNO == EINTR);
if (result < 0)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 09b485bd2b..454ec8236b 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -38,6 +38,8 @@ extern "C"
#define LIBPQ_HAS_TRACE_FLAGS 1
/* Indicates that PQsslAttribute(NULL, "library") is useful */
#define LIBPQ_HAS_SSL_LIBRARY_DETECTION 1
+/* Indicates presence of the PQAUTHDATA_PROMPT_OAUTH_DEVICE authdata hook */
+#define LIBPQ_HAS_PROMPT_OAUTH_DEVICE 1
/*
* Option flags for PQcopyResult
@@ -80,8 +82,10 @@ typedef enum
CONNECTION_CHECK_TARGET, /* Internal state: checking target server
* properties. */
CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
- CONNECTION_ALLOCATED /* Waiting for connection attempt to be
+ CONNECTION_ALLOCATED, /* Waiting for connection attempt to be
* started. */
+ CONNECTION_AUTHENTICATING, /* Authentication is in progress with some
+ * external system. */
} ConnStatusType;
typedef enum
@@ -163,6 +167,13 @@ typedef enum
PQ_PIPELINE_ABORTED
} PGpipelineStatus;
+typedef enum
+{
+ PQAUTHDATA_PROMPT_OAUTH_DEVICE, /* user must visit a device-authorization
+ * URL */
+ PQAUTHDATA_OAUTH_BEARER_TOKEN, /* server requests an OAuth Bearer token */
+} PGAuthData;
+
/* PGconn encapsulates a connection to the backend.
* The contents of this struct are not supposed to be known to applications.
*/
@@ -684,10 +695,74 @@ extern int PQenv2encoding(void);
/* === in fe-auth.c === */
+typedef struct _PQpromptOAuthDevice
+{
+ const char *verification_uri; /* verification URI to visit */
+ const char *user_code; /* user code to enter */
+} PQpromptOAuthDevice;
+
+typedef struct _PQoauthBearerRequest
+{
+ /* Hook inputs (constant across all calls) */
+ const char *const openid_configuration; /* OIDC discovery URI */
+ const char *const scope; /* required scope(s), or NULL */
+
+ /* Hook outputs */
+
+ /*---------
+ * Callback implementing a custom asynchronous OAuth flow.
+ *
+ * The callback may return
+ * - PGRES_POLLING_READING/WRITING, to indicate that a file descriptor
+ * has been stored in *altsock and libpq should wait until it is
+ * readable or writable before calling back;
+ * - PGRES_POLLING_OK, to indicate that the flow is complete and
+ * request->token has been set; or
+ * - PGRES_POLLING_FAILED, to indicate that token retrieval has failed.
+ *
+ * This callback is optional. If the token can be obtained without
+ * blocking during the original call to the PQAUTHDATA_OAUTH_BEARER_TOKEN
+ * hook, it may be returned directly, but one of request->async or
+ * request->token must be set by the hook.
+ */
+ PostgresPollingStatusType (*async) (PGconn *conn,
+ struct _PQoauthBearerRequest *request,
+ int *altsock);
+
+ /*
+ * Callback to clean up custom allocations. A hook implementation may use
+ * this to free request->token and any resources in request->user.
+ *
+ * This is technically optional, but highly recommended, because there is
+ * no other indication as to when it is safe to free the token.
+ */
+ void (*cleanup) (PGconn *conn, struct _PQoauthBearerRequest *request);
+
+ /*
+ * The hook should set this to the Bearer token contents for the
+ * connection, once the flow is completed. The token contents must remain
+ * available to libpq until the hook's cleanup callback is called.
+ */
+ char *token;
+
+ /*
+ * Hook-defined data. libpq will not modify this pointer across calls to
+ * the async callback, so it can be used to keep track of
+ * application-specific state. Resources allocated here should be freed by
+ * the cleanup callback.
+ */
+ void *user;
+} PQoauthBearerRequest;
+
extern char *PQencryptPassword(const char *passwd, const char *user);
extern char *PQencryptPasswordConn(PGconn *conn, const char *passwd, const char *user, const char *algorithm);
extern PGresult *PQchangePassword(PGconn *conn, const char *user, const char *passwd);
+typedef int (*PQauthDataHook_type) (PGAuthData type, PGconn *conn, void *data);
+extern void PQsetAuthDataHook(PQauthDataHook_type hook);
+extern PQauthDataHook_type PQgetAuthDataHook(void);
+extern int PQdefaultAuthDataHook(PGAuthData type, PGconn *conn, void *data);
+
/* === in encnames.c === */
extern int pg_char_to_encoding(const char *name);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index 9c05f11a6e..5f3a0b00db 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -351,6 +351,8 @@ typedef struct pg_conn_host
* found in password file. */
} pg_conn_host;
+typedef PostgresPollingStatusType (*AsyncAuthFunc) (PGconn *conn, pgsocket *altsock);
+
/*
* PGconn stores all the state data associated with a single connection
* to a backend.
@@ -413,6 +415,15 @@ struct pg_conn
* cancel request, instead of being a normal
* connection that's used for queries */
+ /* OAuth v2 */
+ char *oauth_issuer; /* token issuer URL */
+ char *oauth_discovery_uri; /* URI of the issuer's discovery
+ * document */
+ char *oauth_client_id; /* client identifier */
+ char *oauth_client_secret; /* client secret */
+ char *oauth_scope; /* access token scope */
+ bool oauth_want_retry; /* should we retry on failure? */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -481,6 +492,9 @@ struct pg_conn
bool client_finished_auth; /* have we finished our half of the
* authentication exchange? */
+ AsyncAuthFunc async_auth; /* callback for external async authentication */
+ pgsocket altsock; /* alternative socket for client to poll */
+
/* Transient state needed while establishing connection */
PGTargetServerType target_server_type; /* desired session properties */
diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build
index be6fadaea2..0d4b7ac17d 100644
--- a/src/interfaces/libpq/meson.build
+++ b/src/interfaces/libpq/meson.build
@@ -40,6 +40,13 @@ if gssapi.found()
)
endif
+if oauth.found()
+ libpq_sources += files('fe-auth-oauth.c')
+ if oauth_library == 'curl'
+ libpq_sources += files('fe-auth-oauth-curl.c')
+ endif
+endif
+
export_file = custom_target('libpq.exports',
kwargs: gen_export_kwargs,
)
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index b0f4178b3d..f803c1200b 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -231,6 +231,7 @@ pgxs_deps = {
'llvm': llvm,
'lz4': lz4,
'nls': libintl,
+ 'oauth': oauth,
'pam': pam,
'perl': perl_dep,
'python': python3_dep,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index e2a0525dd4..f535fbbd5c 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -153,6 +153,7 @@ ArrayMetaState
ArraySubWorkspace
ArrayToken
ArrayType
+AsyncAuthFunc
AsyncQueueControl
AsyncQueueEntry
AsyncRequest
@@ -354,6 +355,8 @@ CState
CTECycleClause
CTEMaterialize
CTESearchClause
+CURL
+CURLM
CV
CachedExpression
CachedPlan
@@ -1669,6 +1672,7 @@ NumericDigit
NumericSortSupport
NumericSumAccum
NumericVar
+OAuthStep
OM_uint32
OP
OSAPerGroupState
@@ -1734,6 +1738,7 @@ PFN
PGAlignedBlock
PGAlignedXLogBlock
PGAsyncStatusType
+PGAuthData
PGCALL2
PGChecksummablePage
PGContextVisibility
@@ -1894,11 +1899,14 @@ PQArgBlock
PQEnvironmentOption
PQExpBuffer
PQExpBufferData
+PQauthDataHook_type
PQcommMethods
PQconninfoOption
PQnoticeProcessor
PQnoticeReceiver
+PQoauthBearerRequest
PQprintOpt
+PQpromptOAuthDevice
PQsslKeyPassHook_OpenSSL_type
PREDICATELOCK
PREDICATELOCKTAG
@@ -3364,6 +3372,8 @@ explain_get_index_name_hook_type
f_smgr
fasthash_state
fd_set
+fe_oauth_state
+fe_oauth_state_enum
fe_scram_state
fe_scram_state_enum
fetch_range_request
--
2.32.1 (Apple Git-133)
=
v22-0001-common-jsonapi-support-libpq-as-a-client.patch (application/octet-stream, 16.6K) download
From 95ed868179a09ffe374316d0356a128adcebfe71 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Mon, 3 May 2021 15:38:26 -0700
Subject: [PATCH v22 1/4] common/jsonapi: support libpq as a client
Based on a patch by Michael Paquier.
For frontend code, use PQExpBuffer instead of StringInfo. This requires
us to track allocation failures so that we can return JSON_OUT_OF_MEMORY
as needed rather than exit()ing.
---
src/bin/pg_combinebackup/Makefile | 4 +-
src/bin/pg_combinebackup/meson.build | 2 +-
src/bin/pg_verifybackup/Makefile | 2 +-
src/common/Makefile | 2 +-
src/common/jsonapi.c | 173 ++++++++++++++++++++-------
src/common/meson.build | 8 +-
src/include/common/jsonapi.h | 19 ++-
7 files changed, 158 insertions(+), 52 deletions(-)
diff --git a/src/bin/pg_combinebackup/Makefile b/src/bin/pg_combinebackup/Makefile
index c3729755ba..2f7dc1ed87 100644
--- a/src/bin/pg_combinebackup/Makefile
+++ b/src/bin/pg_combinebackup/Makefile
@@ -18,6 +18,8 @@ include $(top_builddir)/src/Makefile.global
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils
+# TODO: fix this properly
+LDFLAGS_INTERNAL += -lpgcommon $(libpq_pgport)
OBJS = \
$(WIN32RES) \
@@ -30,7 +32,7 @@ OBJS = \
all: pg_combinebackup
-pg_combinebackup: $(OBJS) | submake-libpgport submake-libpgfeutils
+pg_combinebackup: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
$(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
install: all installdirs
diff --git a/src/bin/pg_combinebackup/meson.build b/src/bin/pg_combinebackup/meson.build
index 1d4b9c218f..cab677b574 100644
--- a/src/bin/pg_combinebackup/meson.build
+++ b/src/bin/pg_combinebackup/meson.build
@@ -17,7 +17,7 @@ endif
pg_combinebackup = executable('pg_combinebackup',
pg_combinebackup_sources,
- dependencies: [frontend_code],
+ dependencies: [frontend_code, libpq],
kwargs: default_bin_args,
)
bin_targets += pg_combinebackup
diff --git a/src/bin/pg_verifybackup/Makefile b/src/bin/pg_verifybackup/Makefile
index 7c045f142e..3372fada01 100644
--- a/src/bin/pg_verifybackup/Makefile
+++ b/src/bin/pg_verifybackup/Makefile
@@ -17,7 +17,7 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
# We need libpq only because fe_utils does.
-LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils -lpgcommon $(libpq_pgport)
OBJS = \
$(WIN32RES) \
diff --git a/src/common/Makefile b/src/common/Makefile
index 3d83299432..c4f4448e2f 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -40,7 +40,7 @@ override CPPFLAGS += -DVAL_LDFLAGS_EX="\"$(LDFLAGS_EX)\""
override CPPFLAGS += -DVAL_LDFLAGS_SL="\"$(LDFLAGS_SL)\""
override CPPFLAGS += -DVAL_LIBS="\"$(LIBS)\""
-override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common $(CPPFLAGS)
+override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common -I$(libpq_srcdir) $(CPPFLAGS)
LIBS += $(PTHREAD_LIBS)
OBJS_COMMON = \
diff --git a/src/common/jsonapi.c b/src/common/jsonapi.c
index 98d6e66a21..4aeedc0bc5 100644
--- a/src/common/jsonapi.c
+++ b/src/common/jsonapi.c
@@ -21,10 +21,43 @@
#include "mb/pg_wchar.h"
#include "port/pg_lfind.h"
-#ifndef FRONTEND
+#ifdef FRONTEND
+#include "pqexpbuffer.h"
+#else
+#include "lib/stringinfo.h"
#include "miscadmin.h"
#endif
+/*
+ * In backend, we will use palloc/pfree along with StringInfo. In frontend, use
+ * malloc and PQExpBuffer, and return JSON_OUT_OF_MEMORY on out-of-memory.
+ */
+#ifdef FRONTEND
+
+#define STRDUP(s) strdup(s)
+#define ALLOC(size) malloc(size)
+
+#define appendStrVal appendPQExpBuffer
+#define appendBinaryStrVal appendBinaryPQExpBuffer
+#define appendStrValChar appendPQExpBufferChar
+#define createStrVal createPQExpBuffer
+#define resetStrVal resetPQExpBuffer
+#define destroyStrVal destroyPQExpBuffer
+
+#else /* !FRONTEND */
+
+#define STRDUP(s) pstrdup(s)
+#define ALLOC(size) palloc(size)
+
+#define appendStrVal appendStringInfo
+#define appendBinaryStrVal appendBinaryStringInfo
+#define appendStrValChar appendStringInfoChar
+#define createStrVal makeStringInfo
+#define resetStrVal resetStringInfo
+#define destroyStrVal destroyStringInfo
+
+#endif
+
/*
* The context of the parser is maintained by the recursive descent
* mechanism, but is passed explicitly to the error reporting routine
@@ -168,9 +201,16 @@ makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
lex->input_encoding = encoding;
if (need_escapes)
{
- lex->strval = makeStringInfo();
+ /*
+ * This call can fail in FRONTEND code. We defer error handling to
+ * time of use (json_lex_string()) since we might not need to parse
+ * any strings anyway.
+ */
+ lex->strval = createStrVal();
lex->flags |= JSONLEX_FREE_STRVAL;
+ lex->parse_strval = true;
}
+ lex->errormsg = NULL;
return lex;
}
@@ -185,14 +225,18 @@ makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
void
freeJsonLexContext(JsonLexContext *lex)
{
+ static const JsonLexContext empty = {0};
+
if (lex->flags & JSONLEX_FREE_STRVAL)
- destroyStringInfo(lex->strval);
+ destroyStrVal(lex->strval);
if (lex->errormsg)
- destroyStringInfo(lex->errormsg);
+ destroyStrVal(lex->errormsg);
if (lex->flags & JSONLEX_FREE_STRUCT)
pfree(lex);
+ else
+ *lex = empty;
}
/*
@@ -258,7 +302,7 @@ json_count_array_elements(JsonLexContext *lex, int *elements)
* etc, so doing this with a copy makes that safe.
*/
memcpy(©lex, lex, sizeof(JsonLexContext));
- copylex.strval = NULL; /* not interested in values here */
+ copylex.parse_strval = false; /* not interested in values here */
copylex.lex_level++;
count = 0;
@@ -320,14 +364,21 @@ parse_scalar(JsonLexContext *lex, JsonSemAction *sem)
/* extract the de-escaped string value, or the raw lexeme */
if (lex_peek(lex) == JSON_TOKEN_STRING)
{
- if (lex->strval != NULL)
- val = pstrdup(lex->strval->data);
+ if (lex->parse_strval)
+ {
+ val = STRDUP(lex->strval->data);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
}
else
{
int len = (lex->token_terminator - lex->token_start);
- val = palloc(len + 1);
+ val = ALLOC(len + 1);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+
memcpy(val, lex->token_start, len);
val[len] = '\0';
}
@@ -361,8 +412,12 @@ parse_object_field(JsonLexContext *lex, JsonSemAction *sem)
if (lex_peek(lex) != JSON_TOKEN_STRING)
return report_parse_error(JSON_PARSE_STRING, lex);
- if ((ostart != NULL || oend != NULL) && lex->strval != NULL)
- fname = pstrdup(lex->strval->data);
+ if ((ostart != NULL || oend != NULL) && lex->parse_strval)
+ {
+ fname = STRDUP(lex->strval->data);
+ if (fname == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
result = json_lex(lex);
if (result != JSON_SUCCESS)
return result;
@@ -418,6 +473,11 @@ parse_object(JsonLexContext *lex, JsonSemAction *sem)
JsonParseErrorType result;
#ifndef FRONTEND
+
+ /*
+ * TODO: clients need some way to put a bound on stack growth. Parse level
+ * limits maybe?
+ */
check_stack_depth();
#endif
@@ -766,8 +826,15 @@ json_lex_string(JsonLexContext *lex)
return code; \
} while (0)
- if (lex->strval != NULL)
- resetStringInfo(lex->strval);
+ if (lex->parse_strval)
+ {
+#ifdef FRONTEND
+ /* make sure initialization succeeded */
+ if (lex->strval == NULL)
+ return JSON_OUT_OF_MEMORY;
+#endif
+ resetStrVal(lex->strval);
+ }
Assert(lex->input_length > 0);
s = lex->token_start;
@@ -804,7 +871,7 @@ json_lex_string(JsonLexContext *lex)
else
FAIL_AT_CHAR_END(JSON_UNICODE_ESCAPE_FORMAT);
}
- if (lex->strval != NULL)
+ if (lex->parse_strval)
{
/*
* Combine surrogate pairs.
@@ -861,19 +928,19 @@ json_lex_string(JsonLexContext *lex)
unicode_to_utf8(ch, (unsigned char *) utf8str);
utf8len = pg_utf_mblen((unsigned char *) utf8str);
- appendBinaryStringInfo(lex->strval, utf8str, utf8len);
+ appendBinaryPQExpBuffer(lex->strval, utf8str, utf8len);
}
else if (ch <= 0x007f)
{
/* The ASCII range is the same in all encodings */
- appendStringInfoChar(lex->strval, (char) ch);
+ appendPQExpBufferChar(lex->strval, (char) ch);
}
else
FAIL_AT_CHAR_END(JSON_UNICODE_HIGH_ESCAPE);
#endif /* FRONTEND */
}
}
- else if (lex->strval != NULL)
+ else if (lex->parse_strval)
{
if (hi_surrogate != -1)
FAIL_AT_CHAR_END(JSON_UNICODE_LOW_SURROGATE);
@@ -883,22 +950,22 @@ json_lex_string(JsonLexContext *lex)
case '"':
case '\\':
case '/':
- appendStringInfoChar(lex->strval, *s);
+ appendStrValChar(lex->strval, *s);
break;
case 'b':
- appendStringInfoChar(lex->strval, '\b');
+ appendStrValChar(lex->strval, '\b');
break;
case 'f':
- appendStringInfoChar(lex->strval, '\f');
+ appendStrValChar(lex->strval, '\f');
break;
case 'n':
- appendStringInfoChar(lex->strval, '\n');
+ appendStrValChar(lex->strval, '\n');
break;
case 'r':
- appendStringInfoChar(lex->strval, '\r');
+ appendStrValChar(lex->strval, '\r');
break;
case 't':
- appendStringInfoChar(lex->strval, '\t');
+ appendStrValChar(lex->strval, '\t');
break;
default:
@@ -933,7 +1000,7 @@ json_lex_string(JsonLexContext *lex)
/*
* Skip to the first byte that requires special handling, so we
- * can batch calls to appendBinaryStringInfo.
+ * can batch calls to appendBinaryStrVal.
*/
while (p < end - sizeof(Vector8) &&
!pg_lfind8('\\', (uint8 *) p, sizeof(Vector8)) &&
@@ -957,8 +1024,8 @@ json_lex_string(JsonLexContext *lex)
}
}
- if (lex->strval != NULL)
- appendBinaryStringInfo(lex->strval, s, p - s);
+ if (lex->parse_strval)
+ appendBinaryStrVal(lex->strval, s, p - s);
/*
* s will be incremented at the top of the loop, so set it to just
@@ -974,6 +1041,11 @@ json_lex_string(JsonLexContext *lex)
return JSON_UNICODE_LOW_SURROGATE;
}
+#ifdef FRONTEND
+ if (lex->parse_strval && PQExpBufferBroken(lex->strval))
+ return JSON_OUT_OF_MEMORY;
+#endif
+
/* Hooray, we found the end of the string! */
lex->prev_token_terminator = lex->token_terminator;
lex->token_terminator = s + 1;
@@ -1158,19 +1230,25 @@ report_parse_error(JsonParseContext ctx, JsonLexContext *lex)
char *
json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
{
+ if (error == JSON_OUT_OF_MEMORY)
+ {
+ /* Short circuit. Allocating anything for this case is unhelpful. */
+ return _("out of memory");
+ }
+
if (lex->errormsg)
- resetStringInfo(lex->errormsg);
+ resetStrVal(lex->errormsg);
else
- lex->errormsg = makeStringInfo();
+ lex->errormsg = createStrVal();
/*
* A helper for error messages that should print the current token. The
* format must contain exactly one %.*s specifier.
*/
#define token_error(lex, format) \
- appendStringInfo((lex)->errormsg, _(format), \
- (int) ((lex)->token_terminator - (lex)->token_start), \
- (lex)->token_start);
+ appendStrVal((lex)->errormsg, _(format), \
+ (int) ((lex)->token_terminator - (lex)->token_start), \
+ (lex)->token_start);
switch (error)
{
@@ -1181,9 +1259,9 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
token_error(lex, "Escape sequence \"\\%.*s\" is invalid.");
break;
case JSON_ESCAPING_REQUIRED:
- appendStringInfo(lex->errormsg,
- _("Character with value 0x%02x must be escaped."),
- (unsigned char) *(lex->token_terminator));
+ appendStrVal(lex->errormsg,
+ _("Character with value 0x%02x must be escaped."),
+ (unsigned char) *(lex->token_terminator));
break;
case JSON_EXPECTED_END:
token_error(lex, "Expected end of input, but found \"%.*s\".");
@@ -1214,6 +1292,9 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
case JSON_INVALID_TOKEN:
token_error(lex, "Token \"%.*s\" is invalid.");
break;
+ case JSON_OUT_OF_MEMORY:
+ /* should have been handled above; use the error path */
+ break;
case JSON_UNICODE_CODE_POINT_ZERO:
return _("\\u0000 cannot be converted to text.");
case JSON_UNICODE_ESCAPE_FORMAT:
@@ -1245,15 +1326,23 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
}
#undef token_error
- /*
- * We don't use a default: case, so that the compiler will warn about
- * unhandled enum values. But this needs to be here anyway to cover the
- * possibility of an incorrect input.
- */
- if (lex->errormsg->len == 0)
- appendStringInfo(lex->errormsg,
- _("unexpected json parse error type: %d"),
- (int) error);
+ /* Note that lex->errormsg can be NULL in FRONTEND code. */
+ if (lex->errormsg && lex->errormsg->len == 0)
+ {
+ /*
+ * We don't use a default: case, so that the compiler will warn about
+ * unhandled enum values. But this needs to be here anyway to cover
+ * the possibility of an incorrect input.
+ */
+ appendStrVal(lex->errormsg,
+ _("unexpected json parse error type: %d"),
+ (int) error);
+ }
+
+#ifdef FRONTEND
+ if (PQExpBufferBroken(lex->errormsg))
+ return _("out of memory while constructing error description");
+#endif
return lex->errormsg->data;
}
diff --git a/src/common/meson.build b/src/common/meson.build
index de68e408fa..379b228d86 100644
--- a/src/common/meson.build
+++ b/src/common/meson.build
@@ -125,13 +125,18 @@ common_sources_frontend_static += files(
# least cryptohash_openssl.c, hmac_openssl.c depend on it.
# controldata_utils.c depends on wait_event_types_h. That's arguably a
# layering violation, but ...
+#
+# XXX Frontend builds need libpq's pqexpbuffer.h, so adjust the include paths
+# appropriately. This seems completely broken.
pgcommon = {}
pgcommon_variants = {
'_srv': internal_lib_args + {
+ 'include_directories': include_directories('.'),
'sources': common_sources + [lwlocknames_h] + [wait_event_types_h],
'dependencies': [backend_common_code],
},
'': default_lib_args + {
+ 'include_directories': include_directories('../interfaces/libpq', '.'),
'sources': common_sources_frontend_static,
'dependencies': [frontend_common_code],
# Files in libpgcommon.a should use/export the "xxx_private" versions
@@ -140,6 +145,7 @@ pgcommon_variants = {
},
'_shlib': default_lib_args + {
'pic': true,
+ 'include_directories': include_directories('../interfaces/libpq', '.'),
'sources': common_sources_frontend_shlib,
'dependencies': [frontend_common_code],
},
@@ -157,7 +163,6 @@ foreach name, opts : pgcommon_variants
c_args = opts.get('c_args', []) + common_cflags[cflagname]
cflag_libs += static_library('libpgcommon@0@_@1@'.format(name, cflagname),
c_pch: pch_c_h,
- include_directories: include_directories('.'),
kwargs: opts + {
'sources': sources,
'c_args': c_args,
@@ -170,7 +175,6 @@ foreach name, opts : pgcommon_variants
lib = static_library('libpgcommon@0@'.format(name),
link_with: cflag_libs,
c_pch: pch_c_h,
- include_directories: include_directories('.'),
kwargs: opts + {
'dependencies': opts['dependencies'] + [ssl],
}
diff --git a/src/include/common/jsonapi.h b/src/include/common/jsonapi.h
index 86a0fc2d00..75d444c17a 100644
--- a/src/include/common/jsonapi.h
+++ b/src/include/common/jsonapi.h
@@ -14,8 +14,6 @@
#ifndef JSONAPI_H
#define JSONAPI_H
-#include "lib/stringinfo.h"
-
typedef enum JsonTokenType
{
JSON_TOKEN_INVALID,
@@ -48,6 +46,7 @@ typedef enum JsonParseErrorType
JSON_EXPECTED_OBJECT_NEXT,
JSON_EXPECTED_STRING,
JSON_INVALID_TOKEN,
+ JSON_OUT_OF_MEMORY,
JSON_UNICODE_CODE_POINT_ZERO,
JSON_UNICODE_ESCAPE_FORMAT,
JSON_UNICODE_HIGH_ESCAPE,
@@ -57,6 +56,17 @@ typedef enum JsonParseErrorType
JSON_SEM_ACTION_FAILED, /* error should already be reported */
} JsonParseErrorType;
+/*
+ * Don't depend on the internal type header for strval; if callers need access
+ * then they can include the appropriate header themselves.
+ */
+#ifdef FRONTEND
+#define StrValType PQExpBufferData
+#else
+#define StrValType StringInfoData
+#endif
+
+typedef struct StrValType StrValType;
/*
* All the fields in this structure should be treated as read-only.
@@ -88,8 +98,9 @@ typedef struct JsonLexContext
bits32 flags;
int line_number; /* line number, starting from 1 */
char *line_start; /* where that line starts within input */
- StringInfo strval;
- StringInfo errormsg;
+ bool parse_strval;
+ StrValType *strval; /* only used if parse_strval == true */
+ StrValType *errormsg;
} JsonLexContext;
typedef JsonParseErrorType (*json_struct_action) (void *state);
--
2.32.1 (Apple Git-133)
=
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-04-01 22:07 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Daniel Gustafsson <daniel@yesql.se>
0 siblings, 2 replies; 424+ messages in thread
From: Jacob Champion @ 2024-04-01 22:07 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: pgsql-hackers; Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>; Andrew Dunstan <andrew@dunslane.net>
On Thu, Mar 28, 2024 at 3:34 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> In jsonapi.c, makeJsonLexContextCstringLen initialize a JsonLexContext with
> palloc0 which would need to be ported over to use ALLOC for frontend code.
Seems reasonable (but see below, too).
> On
> that note, the errorhandling in parse_oauth_json() for content-type checks
> attempts to free the JsonLexContext even before it has been created. Here we
> can just return false.
Agreed. They're zero-initialized, so freeJsonLexContext() is safe
IIUC, but it's clearer not to call the free function at all.
But for these additions:
> - makeJsonLexContextCstringLen(&lex, resp->data, resp->len, PG_UTF8, true);
> + if (!makeJsonLexContextCstringLen(&lex, resp->data, resp->len, PG_UTF8, true))
> + {
> + actx_error(actx, "out of memory");
> + return false;
> + }
...since we're using the stack-based API as opposed to the heap-based
API, they shouldn't be possible to hit. Any failures in createStrVal()
are deferred to parse time on purpose.
> - echo 'libpq must not be calling any function which invokes exit'; exit 1; \
> + echo 'libpq must not be calling any function which invokes exit'; \
> The offending codepath in libcurl was in the NTLM_WB module, a very old and
> obscure form of NTLM support which was replaced (yet remained in the tree) a
> long time ago by a full NTLM implementatin. Based on the findings in this
> thread it was deprecated with a removal date set to April 2024 [0]. A bug in
> the 8.4.0 release however disconnected NTLM_WB from the build and given the
> lack of complaints it was decided to leave as is, so we can base our libcurl
> requirements on 8.4.0 while keeping the exit() check intact.
Of the Cirrus machines, it looks like only FreeBSD has a new enough
libcurl for that. Ubuntu won't until 24.04, Debian Bookworm doesn't
have it unless you're running backports, RHEL 9 is still on 7.x... I
think requiring libcurl 8 is effectively saying no one will be able to
use this for a long time. Is there an alternative?
> + else if (strcasecmp(content_type, "application/json") != 0)
> This needs to handle parameters as well since it will now fail if the charset
> parameter is appended (which undoubtedly will be pretty common). The easiest
> way is probably to just verify the mediatype and skip the parameters since we
> know it can only be charset?
Good catch. application/json no longer defines charsets officially
[1], so I think we should be able to just ignore them. The new
strncasecmp needs to handle a spurious prefix, too; I have that on my
TODO list.
> + /* TODO investigate using conn->Pfdebug and CURLOPT_DEBUGFUNCTION here */
> + CHECK_SETOPT(actx, CURLOPT_VERBOSE, 1L, return false);
> + CHECK_SETOPT(actx, CURLOPT_ERRORBUFFER, actx->curl_err, return false);
> CURLOPT_ERRORBUFFER is the old and finicky way of extracting error messages, we
> should absolutely move to using CURLOPT_DEBUGFUNCTION instead.
This new way doesn't do the same thing. Here's a sample error:
connection to server at "127.0.0.1", port 56619 failed: failed to
fetch OpenID discovery document: Weird server reply ( Trying
127.0.0.1:36647...
Connected to localhost (127.0.0.1) port 36647 (#0)
Mark bundle as not supporting multiuse
HTTP 1.0, assume close after body
Invalid Content-Length: value
Closing connection 0
)
IMO that's too much noise. Prior to the change, the same error would have been
connection to server at "127.0.0.1", port 56619 failed: failed to
fetch OpenID discovery document: Weird server reply (Invalid
Content-Length: value)
The error buffer is finicky for sure, but it's also a generic one-line
explanation of what went wrong... Is there an alternative API for that
I'm missing?
> + /* && response_code != 401 TODO */ )
> Why is this marked with a TODO, do you remember?
Yeah -- I have a feeling that 401s coming back are going to need more
helpful hints to the user, since it implies that libpq itself hasn't
authenticated correctly as opposed to some user-related auth failure.
I was hoping to find some sample behaviors in the wild and record
those into the suite.
> + print("# OAuth provider (PID $pid) is listening on port $port\n");
> Code running under Test::More need to use diag() for printing non-test output
> like this.
Ah, thanks.
> +#if LIBCURL_VERSION_MAJOR <= 8 && LIBCURL_VERSION_MINOR < 4
I don't think this catches versions like 7.76, does it? Maybe
`LIBCURL_VERSION_MAJOR < 8 || (LIBCURL_VERSION_MAJOR == 8 &&
LIBCURL_VERSION_MINOR < 4)`, or else `LIBCURL_VERSION_NUM < 0x080400`?
> my $pid = open(my $read_fh, "-|", $ENV{PYTHON}, "t/oauth_server.py")
> - // die "failed to start OAuth server: $!";
> + or die "failed to start OAuth server: $!";
>
> - read($read_fh, $port, 7) // die "failed to read port number: $!";
> + read($read_fh, $port, 7) or die "failed to read port number: $!";
The first hunk here looks good (thanks for the catch!) but I think the
second is not correct behavior. $! doesn't get set unless undef is
returned, if I'm reading the docs correctly. Yay Perl.
> + /* Sanity check the previous operation */
> + if (actx->running != 1)
> + {
> + actx_error(actx, "failed to queue HTTP request");
> + return false;
> + }
`running` can be set to zero on success, too. I'm having trouble
forcing that code path in a test so far, but we're going to have to do
something special in that case.
> Another issue I have is the sheer size and the fact that so much code is
> replaced by subsequent commits, so I took the liberty to squash some of this
> down into something less daunting. The attached v22 retains the 0001 and then
> condenses the rest into two commits for frontent and backend parts.
Looks good.
> I did drop
> the Python pytest patch since I feel that it's unlikely to go in from this
> thread (adding pytest seems worthy of its own thread and discussion), and the
> weight of it makes this seem scarier than it is.
Until its coverage gets ported over, can we keep it as a `DO NOT
MERGE` patch? Otherwise there's not much to run in Cirrus.
> The final patch contains fixes for all of the above review comments as well as
> a some refactoring, smaller clean-ups and TODO fixing. If these fixes are
> accepted I'll incorporate them into the two commits.
>
> Next I intend to work on writing documentation for this.
Awesome, thank you! I will start adding coverage to the new code paths.
--Jacob
[1] https://datatracker.ietf.org/doc/html/rfc7159#section-11
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-07-03 17:02 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
1 sibling, 0 replies; 424+ messages in thread
From: Jacob Champion @ 2024-07-03 17:02 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: pgsql-hackers; Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>; Andrew Dunstan <andrew@dunslane.net>
On Mon, Apr 1, 2024 at 3:07 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
>
> Awesome, thank you! I will start adding coverage to the new code paths.
This patchset rotted more than I thought it would with the new
incremental JSON, and I got stuck in rebase hell. Rather than chip
away at that while the cfbot is red, here's a rebase of v22 to get the
CI up again, and I will port what I've been working on over that. (So,
for prior reviewers: recent upthread and offline feedback is not yet
incorporated, sorry, come back later.)
The big change in v23 is that I've removed fe_memutils.c from
libpgcommon_shlib completely, to try to reduce my own hair-pulling
when it comes to keeping exit() out of libpq. (It snuck in several
ways with incremental JSON.)
As far as I can tell, removing fe_memutils causes only one problem,
which is that Informix ECPG is relying on pnstrdup(). And I think that
may be a bug in itself? There's code in deccvasc() right after the
pnstrdup() call that takes care of a failed allocation, but the
frontend pnstrdup() is going to call exit() on failure. So my 0001
patch reverts that change, which was made in 0b9466fce. If that can go
in, and I'm not missing something that makes that call okay, maybe
0002 can be peeled off as well.
Thanks,
--Jacob
Attachments: download all (7 files, .tar.gz)
since-v22.diff.txt (text/plain, 38.0K) download
-: ---------- > 1: aa553b7700 Revert ECPG's use of pnstrdup()
-: ---------- > 2: d6cae9157e Remove fe_memutils from libpgcommon_shlib
1: 9efb0d64e5 ! 3: 5543539169 common/jsonapi: support libpq as a client
@@ Commit message
us to track allocation failures so that we can return JSON_OUT_OF_MEMORY
as needed rather than exit()ing.
+ Co-authored-by: Michael Paquier <michael@paquier.xyz>
+ Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
+
## src/bin/pg_combinebackup/Makefile ##
@@ src/bin/pg_combinebackup/Makefile: include $(top_builddir)/src/Makefile.global
@@ src/common/jsonapi.c
#endif
+/*
-+ * In backend, we will use palloc/pfree along with StringInfo. In frontend, use
-+ * malloc and PQExpBuffer, and return JSON_OUT_OF_MEMORY on out-of-memory.
++ * In backend, we will use palloc/pfree along with StringInfo. In frontend,
++ * use malloc and PQExpBuffer, and return JSON_OUT_OF_MEMORY on out-of-memory.
+ */
+#ifdef FRONTEND
+
+#define STRDUP(s) strdup(s)
+#define ALLOC(size) malloc(size)
++#define ALLOC0(size) calloc(1, size)
++#define REALLOC realloc
++#define FREE(s) free(s)
+
-+#define appendStrVal appendPQExpBuffer
-+#define appendBinaryStrVal appendBinaryPQExpBuffer
-+#define appendStrValChar appendPQExpBufferChar
-+#define createStrVal createPQExpBuffer
-+#define resetStrVal resetPQExpBuffer
-+#define destroyStrVal destroyPQExpBuffer
++#define appendStrVal appendPQExpBuffer
++#define appendBinaryStrVal appendBinaryPQExpBuffer
++#define appendStrValChar appendPQExpBufferChar
++/* XXX should we add a macro version to PQExpBuffer? */
++#define appendStrValCharMacro appendPQExpBufferChar
++#define createStrVal createPQExpBuffer
++#define initStrVal initPQExpBuffer
++#define resetStrVal resetPQExpBuffer
++#define termStrVal termPQExpBuffer
++#define destroyStrVal destroyPQExpBuffer
+
+#else /* !FRONTEND */
+
+#define STRDUP(s) pstrdup(s)
+#define ALLOC(size) palloc(size)
++#define ALLOC0(size) palloc0(size)
++#define REALLOC repalloc
+
-+#define appendStrVal appendStringInfo
-+#define appendBinaryStrVal appendBinaryStringInfo
-+#define appendStrValChar appendStringInfoChar
-+#define createStrVal makeStringInfo
-+#define resetStrVal resetStringInfo
-+#define destroyStrVal destroyStringInfo
++/*
++ * Backend pfree() doesn't handle NULL pointers like the frontend's does; smooth
++ * that over to reduce mental gymnastics. Avoid multiple evaluation of the macro
++ * argument to avoid future hair-pulling.
++ */
++#define FREE(s) do { \
++ void *__v = (s); \
++ if (__v) \
++ pfree(__v); \
++} while (0)
++
++#define appendStrVal appendStringInfo
++#define appendBinaryStrVal appendBinaryStringInfo
++#define appendStrValChar appendStringInfoChar
++#define appendStrValCharMacro appendStringInfoCharMacro
++#define createStrVal makeStringInfo
++#define initStrVal initStringInfo
++#define resetStrVal resetStringInfo
++#define termStrVal(s) pfree((s)->data)
++#define destroyStrVal destroyStringInfo
+
+#endif
+
/*
* The context of the parser is maintained by the recursive descent
* mechanism, but is passed explicitly to the error reporting routine
-@@ src/common/jsonapi.c: makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
+@@ src/common/jsonapi.c: struct JsonIncrementalState
+ {
+ bool is_last_chunk;
+ bool partial_completed;
+- StringInfoData partial_token;
++ StrValType partial_token;
+ };
+
+ /*
+@@ src/common/jsonapi.c: static JsonParseErrorType parse_object(JsonLexContext *lex, JsonSemAction *sem);
+ static JsonParseErrorType parse_array_element(JsonLexContext *lex, JsonSemAction *sem);
+ static JsonParseErrorType parse_array(JsonLexContext *lex, JsonSemAction *sem);
+ static JsonParseErrorType report_parse_error(JsonParseContext ctx, JsonLexContext *lex);
++static bool allocate_incremental_state(JsonLexContext *lex);
+
+ /* the null action object used for pure validation */
+ JsonSemAction nullSemAction =
+@@ src/common/jsonapi.c: IsValidJsonNumber(const char *str, size_t len)
+ {
+ bool numeric_error;
+ size_t total_len;
+- JsonLexContext dummy_lex;
++ JsonLexContext dummy_lex = {0};
+
+ if (len <= 0)
+ return false;
+
+- dummy_lex.incremental = false;
+- dummy_lex.inc_state = NULL;
+- dummy_lex.pstack = NULL;
+-
+ /*
+ * json_lex_number expects a leading '-' to have been eaten already.
+ *
+@@ src/common/jsonapi.c: IsValidJsonNumber(const char *str, size_t len)
+ * responsible for freeing the returned struct, either by calling
+ * freeJsonLexContext() or (in backend environment) via memory context
+ * cleanup.
++ *
++ * In frontend code this can return NULL on OOM, so callers must inspect the
++ * returned pointer.
+ */
+ JsonLexContext *
+ makeJsonLexContextCstringLen(JsonLexContext *lex, const char *json,
+@@ src/common/jsonapi.c: makeJsonLexContextCstringLen(JsonLexContext *lex, const char *json,
+ {
+ if (lex == NULL)
+ {
+- lex = palloc0(sizeof(JsonLexContext));
++ lex = ALLOC0(sizeof(JsonLexContext));
++ if (!lex)
++ return NULL;
+ lex->flags |= JSONLEX_FREE_STRUCT;
+ }
+ else
+@@ src/common/jsonapi.c: makeJsonLexContextCstringLen(JsonLexContext *lex, const char *json,
lex->input_encoding = encoding;
if (need_escapes)
{
@@ src/common/jsonapi.c: makeJsonLexContextCstringLen(JsonLexContext *lex, char *js
lex->flags |= JSONLEX_FREE_STRVAL;
+ lex->parse_strval = true;
}
-+ lex->errormsg = NULL;
return lex;
}
-@@ src/common/jsonapi.c: makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
+
++/*
++ * Allocates the internal bookkeeping structures for incremental parsing. This
++ * can only fail in-band with FRONTEND code.
++ */
++#define JS_STACK_CHUNK_SIZE 64
++#define JS_MAX_PROD_LEN 10 /* more than we need */
++#define JSON_TD_MAX_STACK 6400 /* hard coded for now - this is a REALLY high
++ * number */
++static bool
++allocate_incremental_state(JsonLexContext *lex)
++{
++ void *pstack,
++ *prediction,
++ *fnames,
++ *fnull;
++
++ lex->inc_state = ALLOC0(sizeof(JsonIncrementalState));
++ pstack = ALLOC(sizeof(JsonParserStack));
++ prediction = ALLOC(JS_STACK_CHUNK_SIZE * JS_MAX_PROD_LEN);
++ fnames = ALLOC(JS_STACK_CHUNK_SIZE * sizeof(char *));
++ fnull = ALLOC(JS_STACK_CHUNK_SIZE * sizeof(bool));
++
++#ifdef FRONTEND
++ if (!lex->inc_state
++ || !pstack
++ || !prediction
++ || !fnames
++ || !fnull)
++ {
++ FREE(lex->inc_state);
++ FREE(pstack);
++ FREE(prediction);
++ FREE(fnames);
++ FREE(fnull);
++
++ return false;
++ }
++#endif
++
++ initStrVal(&(lex->inc_state->partial_token));
++ lex->pstack = pstack;
++ lex->pstack->stack_size = JS_STACK_CHUNK_SIZE;
++ lex->pstack->prediction = prediction;
++ lex->pstack->pred_index = 0;
++ lex->pstack->fnames = fnames;
++ lex->pstack->fnull = fnull;
++
++ lex->incremental = true;
++ return true;
++}
++
+
+ /*
+ * makeJsonLexContextIncremental
+@@ src/common/jsonapi.c: makeJsonLexContextCstringLen(JsonLexContext *lex, const char *json,
+ * we don't need the input, that will be handed in bit by bit to the
+ * parse routine. We also need an accumulator for partial tokens in case
+ * the boundary between chunks happens to fall in the middle of a token.
++ *
++ * In frontend code this can return NULL on OOM, so callers must inspect the
++ * returned pointer.
+ */
+-#define JS_STACK_CHUNK_SIZE 64
+-#define JS_MAX_PROD_LEN 10 /* more than we need */
+-#define JSON_TD_MAX_STACK 6400 /* hard coded for now - this is a REALLY high
+- * number */
+-
+ JsonLexContext *
+ makeJsonLexContextIncremental(JsonLexContext *lex, int encoding,
+ bool need_escapes)
+ {
+ if (lex == NULL)
+ {
+- lex = palloc0(sizeof(JsonLexContext));
++ lex = ALLOC0(sizeof(JsonLexContext));
++ if (!lex)
++ return NULL;
++
+ lex->flags |= JSONLEX_FREE_STRUCT;
+ }
+ else
+@@ src/common/jsonapi.c: makeJsonLexContextIncremental(JsonLexContext *lex, int encoding,
+
+ lex->line_number = 1;
+ lex->input_encoding = encoding;
+- lex->incremental = true;
+- lex->inc_state = palloc0(sizeof(JsonIncrementalState));
+- initStringInfo(&(lex->inc_state->partial_token));
+- lex->pstack = palloc(sizeof(JsonParserStack));
+- lex->pstack->stack_size = JS_STACK_CHUNK_SIZE;
+- lex->pstack->prediction = palloc(JS_STACK_CHUNK_SIZE * JS_MAX_PROD_LEN);
+- lex->pstack->pred_index = 0;
+- lex->pstack->fnames = palloc(JS_STACK_CHUNK_SIZE * sizeof(char *));
+- lex->pstack->fnull = palloc(JS_STACK_CHUNK_SIZE * sizeof(bool));
++
++ if (!allocate_incremental_state(lex))
++ {
++ if (lex->flags & JSONLEX_FREE_STRUCT)
++ FREE(lex);
++ return NULL;
++ }
++
+ if (need_escapes)
+ {
+- lex->strval = makeStringInfo();
++ /*
++ * This call can fail in FRONTEND code. We defer error handling to
++ * time of use (json_lex_string()) since we might not need to parse
++ * any strings anyway.
++ */
++ lex->strval = createStrVal();
+ lex->flags |= JSONLEX_FREE_STRVAL;
++ lex->parse_strval = true;
+ }
++
+ return lex;
+ }
+
+-static inline void
++static inline bool
+ inc_lex_level(JsonLexContext *lex)
+ {
+- lex->lex_level += 1;
+-
+- if (lex->incremental && lex->lex_level >= lex->pstack->stack_size)
++ if (lex->incremental && (lex->lex_level + 1) >= lex->pstack->stack_size)
+ {
+- lex->pstack->stack_size += JS_STACK_CHUNK_SIZE;
+- lex->pstack->prediction =
+- repalloc(lex->pstack->prediction,
+- lex->pstack->stack_size * JS_MAX_PROD_LEN);
+- if (lex->pstack->fnames)
+- lex->pstack->fnames =
+- repalloc(lex->pstack->fnames,
+- lex->pstack->stack_size * sizeof(char *));
+- if (lex->pstack->fnull)
+- lex->pstack->fnull =
+- repalloc(lex->pstack->fnull, lex->pstack->stack_size * sizeof(bool));
++ size_t new_stack_size;
++ char *new_prediction;
++ char **new_fnames;
++ bool *new_fnull;
++
++ new_stack_size = lex->pstack->stack_size + JS_STACK_CHUNK_SIZE;
++
++ new_prediction = REALLOC(lex->pstack->prediction,
++ new_stack_size * JS_MAX_PROD_LEN);
++ new_fnames = REALLOC(lex->pstack->fnames,
++ new_stack_size * sizeof(char *));
++ new_fnull = REALLOC(lex->pstack->fnull, new_stack_size * sizeof(bool));
++
++#ifdef FRONTEND
++ if (!new_prediction || !new_fnames || !new_fnull)
++ return false;
++#endif
++
++ lex->pstack->stack_size = new_stack_size;
++ lex->pstack->prediction = new_prediction;
++ lex->pstack->fnames = new_fnames;
++ lex->pstack->fnull = new_fnull;
+ }
++
++ lex->lex_level += 1;
++ return true;
+ }
+
+ static inline void
+@@ src/common/jsonapi.c: get_fnull(JsonLexContext *lex)
void
freeJsonLexContext(JsonLexContext *lex)
{
+ static const JsonLexContext empty = {0};
++
++ if (!lex)
++ return;
+
if (lex->flags & JSONLEX_FREE_STRVAL)
- destroyStringInfo(lex->strval);
@@ src/common/jsonapi.c: makeJsonLexContextCstringLen(JsonLexContext *lex, char *js
- destroyStringInfo(lex->errormsg);
+ destroyStrVal(lex->errormsg);
+ if (lex->incremental)
+ {
+- pfree(lex->inc_state->partial_token.data);
+- pfree(lex->inc_state);
+- pfree(lex->pstack->prediction);
+- pfree(lex->pstack->fnames);
+- pfree(lex->pstack->fnull);
+- pfree(lex->pstack);
++ termStrVal(&lex->inc_state->partial_token);
++ FREE(lex->inc_state);
++ FREE(lex->pstack->prediction);
++ FREE(lex->pstack->fnames);
++ FREE(lex->pstack->fnull);
++ FREE(lex->pstack);
+ }
+
if (lex->flags & JSONLEX_FREE_STRUCT)
- pfree(lex);
+- pfree(lex);
++ FREE(lex);
+ else
+ *lex = empty;
}
/*
+@@ src/common/jsonapi.c: JsonParseErrorType
+ pg_parse_json(JsonLexContext *lex, JsonSemAction *sem)
+ {
+ #ifdef FORCE_JSON_PSTACK
+-
+- lex->incremental = true;
+- lex->inc_state = palloc0(sizeof(JsonIncrementalState));
+-
+ /*
+ * We don't need partial token processing, there is only one chunk. But we
+ * still need to init the partial token string so that freeJsonLexContext
+- * works.
++ * works, so perform the full incremental initialization.
+ */
+- initStringInfo(&(lex->inc_state->partial_token));
+- lex->pstack = palloc(sizeof(JsonParserStack));
+- lex->pstack->stack_size = JS_STACK_CHUNK_SIZE;
+- lex->pstack->prediction = palloc(JS_STACK_CHUNK_SIZE * JS_MAX_PROD_LEN);
+- lex->pstack->pred_index = 0;
+- lex->pstack->fnames = palloc(JS_STACK_CHUNK_SIZE * sizeof(char *));
+- lex->pstack->fnull = palloc(JS_STACK_CHUNK_SIZE * sizeof(bool));
++ if (!allocate_incremental_state(lex))
++ return JSON_OUT_OF_MEMORY;
+
+ return pg_parse_json_incremental(lex, sem, lex->input, lex->input_length, true);
+
@@ src/common/jsonapi.c: json_count_array_elements(JsonLexContext *lex, int *elements)
* etc, so doing this with a copy makes that safe.
*/
@@ src/common/jsonapi.c: json_count_array_elements(JsonLexContext *lex, int *elemen
copylex.lex_level++;
count = 0;
+@@ src/common/jsonapi.c: pg_parse_json_incremental(JsonLexContext *lex,
+ if (result != JSON_SUCCESS)
+ return result;
+ }
+- inc_lex_level(lex);
++
++ if (!inc_lex_level(lex))
++ return JSON_OUT_OF_MEMORY;
+ }
+ break;
+ case JSON_SEM_OEND:
+@@ src/common/jsonapi.c: pg_parse_json_incremental(JsonLexContext *lex,
+ if (result != JSON_SUCCESS)
+ return result;
+ }
+- inc_lex_level(lex);
++
++ if (!inc_lex_level(lex))
++ return JSON_OUT_OF_MEMORY;
+ }
+ break;
+ case JSON_SEM_AEND:
+@@ src/common/jsonapi.c: pg_parse_json_incremental(JsonLexContext *lex,
+ json_ofield_action ostart = sem->object_field_start;
+ json_ofield_action oend = sem->object_field_end;
+
+- if ((ostart != NULL || oend != NULL) && lex->strval != NULL)
++ if ((ostart != NULL || oend != NULL) && lex->parse_strval)
+ {
+- fname = pstrdup(lex->strval->data);
++ fname = STRDUP(lex->strval->data);
++ if (fname == NULL)
++ return JSON_OUT_OF_MEMORY;
+ }
+ set_fname(lex, fname);
+ }
+@@ src/common/jsonapi.c: pg_parse_json_incremental(JsonLexContext *lex,
+ */
+ if (tok == JSON_TOKEN_STRING)
+ {
+- if (lex->strval != NULL)
+- pstack->scalar_val = pstrdup(lex->strval->data);
++ if (lex->parse_strval)
++ {
++ pstack->scalar_val = STRDUP(lex->strval->data);
++ if (pstack->scalar_val == NULL)
++ return JSON_OUT_OF_MEMORY;
++ }
+ }
+ else
+ {
+ ptrdiff_t tlen = (lex->token_terminator - lex->token_start);
+
+- pstack->scalar_val = palloc(tlen + 1);
++ pstack->scalar_val = ALLOC(tlen + 1);
++ if (pstack->scalar_val == NULL)
++ return JSON_OUT_OF_MEMORY;
++
+ memcpy(pstack->scalar_val, lex->token_start, tlen);
+ pstack->scalar_val[tlen] = '\0';
+ }
@@ src/common/jsonapi.c: parse_scalar(JsonLexContext *lex, JsonSemAction *sem)
/* extract the de-escaped string value, or the raw lexeme */
if (lex_peek(lex) == JSON_TOKEN_STRING)
@@ src/common/jsonapi.c: parse_object(JsonLexContext *lex, JsonSemAction *sem)
check_stack_depth();
#endif
+@@ src/common/jsonapi.c: json_lex(JsonLexContext *lex)
+ const char *const end = lex->input + lex->input_length;
+ JsonParseErrorType result;
+
+- if (lex->incremental && lex->inc_state->partial_completed)
++ if (lex->incremental)
+ {
+- /*
+- * We just lexed a completed partial token on the last call, so reset
+- * everything
+- */
+- resetStringInfo(&(lex->inc_state->partial_token));
+- lex->token_terminator = lex->input;
+- lex->inc_state->partial_completed = false;
++ if (lex->inc_state->partial_completed)
++ {
++ /*
++ * We just lexed a completed partial token on the last call, so
++ * reset everything
++ */
++ resetStrVal(&(lex->inc_state->partial_token));
++ lex->token_terminator = lex->input;
++ lex->inc_state->partial_completed = false;
++ }
++
++#ifdef FRONTEND
++ /* Make sure our partial token buffer is valid before using it below. */
++ if (PQExpBufferDataBroken(lex->inc_state->partial_token))
++ return JSON_OUT_OF_MEMORY;
++#endif
+ }
+
+ s = lex->token_terminator;
+@@ src/common/jsonapi.c: json_lex(JsonLexContext *lex)
+ * We have a partial token. Extend it and if completed lex it by a
+ * recursive call
+ */
+- StringInfo ptok = &(lex->inc_state->partial_token);
++ StrValType *ptok = &(lex->inc_state->partial_token);
+ size_t added = 0;
+ bool tok_done = false;
+ JsonLexContext dummy_lex;
+@@ src/common/jsonapi.c: json_lex(JsonLexContext *lex)
+ {
+ char c = lex->input[i];
+
+- appendStringInfoCharMacro(ptok, c);
++ appendStrValCharMacro(ptok, c);
+ added++;
+ if (c == '"' && escapes % 2 == 0)
+ {
+@@ src/common/jsonapi.c: json_lex(JsonLexContext *lex)
+ case '8':
+ case '9':
+ {
+- appendStringInfoCharMacro(ptok, cc);
++ appendStrValCharMacro(ptok, cc);
+ added++;
+ }
+ break;
+@@ src/common/jsonapi.c: json_lex(JsonLexContext *lex)
+
+ if (JSON_ALPHANUMERIC_CHAR(cc))
+ {
+- appendStringInfoCharMacro(ptok, cc);
++ appendStrValCharMacro(ptok, cc);
+ added++;
+ }
+ else
+@@ src/common/jsonapi.c: json_lex(JsonLexContext *lex)
+ dummy_lex.input_length = ptok->len;
+ dummy_lex.input_encoding = lex->input_encoding;
+ dummy_lex.incremental = false;
++ dummy_lex.parse_strval = lex->parse_strval;
+ dummy_lex.strval = lex->strval;
+
+ partial_result = json_lex(&dummy_lex);
+@@ src/common/jsonapi.c: json_lex(JsonLexContext *lex)
+ if (lex->incremental && !lex->inc_state->is_last_chunk &&
+ p == lex->input + lex->input_length)
+ {
+- appendBinaryStringInfo(
+- &(lex->inc_state->partial_token), s, end - s);
++ appendBinaryStrVal(
++ &(lex->inc_state->partial_token), s, end - s);
+ return JSON_INCOMPLETE;
+ }
+
+@@ src/common/jsonapi.c: json_lex_string(JsonLexContext *lex)
+ do { \
+ if (lex->incremental && !lex->inc_state->is_last_chunk) \
+ { \
+- appendBinaryStringInfo(&lex->inc_state->partial_token, \
+- lex->token_start, end - lex->token_start); \
++ appendBinaryStrVal(&lex->inc_state->partial_token, \
++ lex->token_start, end - lex->token_start); \
+ return JSON_INCOMPLETE; \
+ } \
+ lex->token_terminator = s; \
@@ src/common/jsonapi.c: json_lex_string(JsonLexContext *lex)
return code; \
} while (0)
@@ src/common/jsonapi.c: json_lex_string(JsonLexContext *lex)
/* Hooray, we found the end of the string! */
lex->prev_token_terminator = lex->token_terminator;
lex->token_terminator = s + 1;
+@@ src/common/jsonapi.c: json_lex_number(JsonLexContext *lex, const char *s,
+ if (lex->incremental && !lex->inc_state->is_last_chunk &&
+ len >= lex->input_length)
+ {
+- appendBinaryStringInfo(&lex->inc_state->partial_token,
+- lex->token_start, s - lex->token_start);
++ appendBinaryStrVal(&lex->inc_state->partial_token,
++ lex->token_start, s - lex->token_start);
+ if (num_err != NULL)
+ *num_err = error;
+
@@ src/common/jsonapi.c: report_parse_error(JsonParseContext ctx, JsonLexContext *lex)
char *
json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
@@ src/common/jsonapi.c: report_parse_error(JsonParseContext ctx, JsonLexContext *l
* A helper for error messages that should print the current token. The
* format must contain exactly one %.*s specifier.
*/
- #define token_error(lex, format) \
+ #define json_token_error(lex, format) \
- appendStringInfo((lex)->errormsg, _(format), \
- (int) ((lex)->token_terminator - (lex)->token_start), \
- (lex)->token_start);
@@ src/common/jsonapi.c: report_parse_error(JsonParseContext ctx, JsonLexContext *l
switch (error)
{
@@ src/common/jsonapi.c: json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
- token_error(lex, "Escape sequence \"\\%.*s\" is invalid.");
+ json_token_error(lex, "Escape sequence \"\\%.*s\" is invalid.");
break;
case JSON_ESCAPING_REQUIRED:
- appendStringInfo(lex->errormsg,
@@ src/common/jsonapi.c: json_errdetail(JsonParseErrorType error, JsonLexContext *l
+ (unsigned char) *(lex->token_terminator));
break;
case JSON_EXPECTED_END:
- token_error(lex, "Expected end of input, but found \"%.*s\".");
+ json_token_error(lex, "Expected end of input, but found \"%.*s\".");
@@ src/common/jsonapi.c: json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
case JSON_INVALID_TOKEN:
- token_error(lex, "Token \"%.*s\" is invalid.");
+ json_token_error(lex, "Token \"%.*s\" is invalid.");
break;
+ case JSON_OUT_OF_MEMORY:
+ /* should have been handled above; use the error path */
@@ src/common/jsonapi.c: json_errdetail(JsonParseErrorType error, JsonLexContext *l
case JSON_UNICODE_ESCAPE_FORMAT:
@@ src/common/jsonapi.c: json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
}
- #undef token_error
+ #undef json_token_error
- /*
- * We don't use a default: case, so that the compiler will warn about
@@ src/common/jsonapi.c: json_errdetail(JsonParseErrorType error, JsonLexContext *l
- */
- if (lex->errormsg->len == 0)
- appendStringInfo(lex->errormsg,
-- _("unexpected json parse error type: %d"),
+- "unexpected json parse error type: %d",
- (int) error);
+ /* Note that lex->errormsg can be NULL in FRONTEND code. */
+ if (lex->errormsg && lex->errormsg->len == 0)
@@ src/common/jsonapi.c: json_errdetail(JsonParseErrorType error, JsonLexContext *l
+ * the possibility of an incorrect input.
+ */
+ appendStrVal(lex->errormsg,
-+ _("unexpected json parse error type: %d"),
++ "unexpected json parse error type: %d",
+ (int) error);
+ }
+
@@ src/common/meson.build: foreach name, opts : pgcommon_variants
'dependencies': opts['dependencies'] + [ssl],
}
+ ## src/common/parse_manifest.c ##
+@@ src/common/parse_manifest.c: json_parse_manifest_incremental_init(JsonManifestParseContext *context)
+ parse->state = JM_EXPECT_TOPLEVEL_START;
+ parse->saw_version_field = false;
+
+- makeJsonLexContextIncremental(&(incstate->lex), PG_UTF8, true);
++ if (!makeJsonLexContextIncremental(&(incstate->lex), PG_UTF8, true))
++ context->error_cb(context, "out of memory");
+
+ incstate->sem.semstate = parse;
+ incstate->sem.object_start = json_manifest_object_start;
+@@ src/common/parse_manifest.c: json_parse_manifest(JsonManifestParseContext *context, const char *buffer,
+
+ /* Create a JSON lexing context. */
+ lex = makeJsonLexContextCstringLen(NULL, buffer, size, PG_UTF8, true);
++ if (!lex)
++ json_manifest_parse_failure(context, "out of memory");
+
+ /* Set up semantic actions. */
+ sem.semstate = &parse;
+
## src/include/common/jsonapi.h ##
@@
#ifndef JSONAPI_H
@@ src/include/common/jsonapi.h: typedef enum JsonParseErrorType
JSON_UNICODE_ESCAPE_FORMAT,
JSON_UNICODE_HIGH_ESCAPE,
@@ src/include/common/jsonapi.h: typedef enum JsonParseErrorType
- JSON_SEM_ACTION_FAILED, /* error should already be reported */
- } JsonParseErrorType;
+ typedef struct JsonParserStack JsonParserStack;
+ typedef struct JsonIncrementalState JsonIncrementalState;
+/*
+ * Don't depend on the internal type header for strval; if callers need access
@@ src/include/common/jsonapi.h: typedef enum JsonParseErrorType
+#endif
+
+typedef struct StrValType StrValType;
-
++
/*
* All the fields in this structure should be treated as read-only.
+ *
@@ src/include/common/jsonapi.h: typedef struct JsonLexContext
- bits32 flags;
- int line_number; /* line number, starting from 1 */
- char *line_start; /* where that line starts within input */
+ const char *line_start; /* where that line starts within input */
+ JsonParserStack *pstack;
+ JsonIncrementalState *inc_state;
- StringInfo strval;
- StringInfo errormsg;
+ bool parse_strval;
@@ src/include/common/jsonapi.h: typedef struct JsonLexContext
} JsonLexContext;
typedef JsonParseErrorType (*json_struct_action) (void *state);
+
+ ## src/test/modules/test_json_parser/Makefile ##
+@@ src/test/modules/test_json_parser/Makefile: include $(top_builddir)/src/Makefile.global
+ include $(top_srcdir)/contrib/contrib-global.mk
+ endif
+
++# TODO: fix this properly
++LDFLAGS_INTERNAL += -lpgcommon $(libpq_pgport)
++
+ all: test_json_parser_incremental$(X) test_json_parser_perf$(X)
+
+ %.o: $(top_srcdir)/$(subdir)/%.c
+
+ ## src/test/modules/test_json_parser/meson.build ##
+@@ src/test/modules/test_json_parser/meson.build: endif
+
+ test_json_parser_incremental = executable('test_json_parser_incremental',
+ test_json_parser_incremental_sources,
+- dependencies: [frontend_code],
++ dependencies: [frontend_code, libpq],
+ kwargs: default_bin_args + {
+ 'install': false,
+ },
+@@ src/test/modules/test_json_parser/meson.build: endif
+
+ test_json_parser_perf = executable('test_json_parser_perf',
+ test_json_parser_perf_sources,
+- dependencies: [frontend_code],
++ dependencies: [frontend_code, libpq],
+ kwargs: default_bin_args + {
+ 'install': false,
+ },
2: 31f2ffdf0b ! 4: 1639f7eb9a libpq: add OAUTHBEARER SASL mechanism
@@ src/interfaces/libpq/Makefile: endif
endif
## src/interfaces/libpq/exports.txt ##
-@@ src/interfaces/libpq/exports.txt: PQcancelSocket 199
- PQcancelErrorMessage 200
- PQcancelReset 201
- PQcancelFinish 202
-+PQsetAuthDataHook 203
-+PQgetAuthDataHook 204
-+PQdefaultAuthDataHook 205
+@@ src/interfaces/libpq/exports.txt: PQcancelFinish 202
+ PQsocketPoll 203
+ PQsetChunkedRowsMode 204
+ PQgetCurrentTimeUSec 205
++PQsetAuthDataHook 206
++PQgetAuthDataHook 207
++PQdefaultAuthDataHook 208
## src/interfaces/libpq/fe-auth-oauth-curl.c (new) ##
@@
@@ src/interfaces/libpq/fe-connect.c: keep_going: /* We will come back to here
+ }
+#endif
+
- #ifdef ENABLE_GSS
-
- /*
+ CONNECTION_FAILED();
+ }
+ else if (beresp == PqMsg_NegotiateProtocolVersion)
@@ src/interfaces/libpq/fe-connect.c: keep_going: /* We will come back to here until there is
* Note that conn->pghost must be non-NULL if we are going to
* avoid the Kerberos code doing a hostname look-up.
@@ src/interfaces/libpq/fe-connect.c: PQsocket(const PGconn *conn)
## src/interfaces/libpq/fe-misc.c ##
@@ src/interfaces/libpq/fe-misc.c: static int
- pqSocketCheck(PGconn *conn, int forRead, int forWrite, time_t end_time)
+ pqSocketCheck(PGconn *conn, int forRead, int forWrite, pg_usec_time_t end_time)
{
int result;
+ pgsocket sock;
@@ src/interfaces/libpq/fe-misc.c: static int
{
libpq_append_conn_error(conn, "invalid socket");
return -1;
-@@ src/interfaces/libpq/fe-misc.c: pqSocketCheck(PGconn *conn, int forRead, int forWrite, time_t end_time)
+@@ src/interfaces/libpq/fe-misc.c: pqSocketCheck(PGconn *conn, int forRead, int forWrite, pg_usec_time_t end_time)
/* We will retry as long as we get EINTR */
do
-- result = pqSocketPoll(conn->sock, forRead, forWrite, end_time);
-+ result = pqSocketPoll(sock, forRead, forWrite, end_time);
+- result = PQsocketPoll(conn->sock, forRead, forWrite, end_time);
++ result = PQsocketPoll(sock, forRead, forWrite, end_time);
while (result < 0 && SOCK_ERRNO == EINTR);
if (result < 0)
@@ src/interfaces/libpq/libpq-fe.h: extern "C"
/*
* Option flags for PQcopyResult
@@ src/interfaces/libpq/libpq-fe.h: typedef enum
- CONNECTION_CHECK_TARGET, /* Internal state: checking target server
- * properties. */
CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
-- CONNECTION_ALLOCATED /* Waiting for connection attempt to be
-+ CONNECTION_ALLOCATED, /* Waiting for connection attempt to be
+ CONNECTION_ALLOCATED, /* Waiting for connection attempt to be
* started. */
+ CONNECTION_AUTHENTICATING, /* Authentication is in progress with some
+ * external system. */
3: 96892c6249 ! 5: 044d8b08e9 backend: add OAUTHBEARER SASL mechanism
@@ Commit message
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
+ ## .cirrus.tasks.yml ##
+@@ .cirrus.tasks.yml: task:
+ chown root:postgres /tmp/cores
+ sysctl kern.corefile='/tmp/cores/%N.%P.core'
+ setup_additional_packages_script: |
+- #pkg install -y ...
++ pkg install -y curl
+
+ # NB: Intentionally build without -Dllvm. The freebsd image size is already
+ # large enough to make VM startup slow, and even without llvm freebsd
+@@ .cirrus.tasks.yml: task:
+ -Dcassert=true -Dinjection_points=true \
+ -Duuid=bsd -Dtcl_version=tcl86 -Ddtrace=auto \
+ -DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
++ -Doauth=curl \
+ -Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \
+ build
+ EOF
+@@ .cirrus.tasks.yml: LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
+ --with-libxslt
+ --with-llvm
+ --with-lz4
++ --with-oauth=curl
+ --with-pam
+ --with-perl
+ --with-python
+@@ .cirrus.tasks.yml: LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
+
+ LINUX_MESON_FEATURES: &LINUX_MESON_FEATURES >-
+ -Dllvm=enabled
++ -Doauth=curl
+ -Duuid=e2fs
+
+
+@@ .cirrus.tasks.yml: task:
+ EOF
+
+ setup_additional_packages_script: |
+- #apt-get update
+- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
++ apt-get update
++ DEBIAN_FRONTEND=noninteractive apt-get -y install \
++ libcurl4-openssl-dev \
++ libcurl4-openssl-dev:i386 \
+
+ matrix:
+ - name: Linux - Debian Bullseye - Autoconf
+@@ .cirrus.tasks.yml: task:
+ folder: $CCACHE_DIR
+
+ setup_additional_packages_script: |
+- #apt-get update
+- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
++ apt-get update
++ DEBIAN_FRONTEND=noninteractive apt-get -y install libcurl4-openssl-dev
+
+ ###
+ # Test that code can be built with gcc/clang without warnings
+
## src/backend/libpq/Makefile ##
@@ src/backend/libpq/Makefile: include $(top_builddir)/src/Makefile.global
# be-fsstubs is here for historical reasons, probably belongs elsewhere
@@ src/backend/utils/misc/guc_tables.c
#include "nodes/queryjumble.h"
#include "optimizer/cost.h"
@@ src/backend/utils/misc/guc_tables.c: struct config_string ConfigureNamesString[] =
- check_standby_slot_names, assign_standby_slot_names, NULL
+ check_synchronized_standby_slots, assign_synchronized_standby_slots, NULL
},
+ {
@@ src/include/libpq/sasl.h: typedef struct pg_be_sasl_mech
/* Common implementation for auth.c */
+ ## src/test/modules/Makefile ##
+@@ src/test/modules/Makefile: SUBDIRS = \
+ dummy_index_am \
+ dummy_seclabel \
+ libpq_pipeline \
++ oauth_validator \
+ plsample \
+ spgist_name_ops \
+ test_bloomfilter \
+
## src/test/modules/meson.build ##
@@ src/test/modules/meson.build: subdir('gin')
subdir('injection_points')
@@ src/test/modules/oauth_validator/.gitignore (new)
## src/test/modules/oauth_validator/Makefile (new) ##
@@
+export PYTHON
++export with_oauth
+
+MODULES = validator
+PGFILEDESC = "validator - test OAuth validator module"
@@ src/test/modules/oauth_validator/meson.build (new)
+ ],
+ 'env': {
+ 'PYTHON': python.path(),
++ 'with_oauth': oauth_library,
+ },
+ },
+}
@@ src/test/modules/oauth_validator/t/001_server.pl (new)
+use PostgreSQL::Test::OAuthServer;
+use Test::More;
+
++if ($ENV{with_oauth} ne 'curl')
++{
++ plan skip_all => 'client-side OAuth not supported by this build';
++}
++
+my $node = PostgreSQL::Test::Cluster->new('primary');
+$node->init;
+$node->append_conf('postgresql.conf', "log_connections = on\n");
4: 5677e59152 ! 6: 84c6893325 Review comments
@@ Commit message
* Implement a version check for libcurl in autoconf, the
equivalent check for Meson is still a TODO.
* Address a few TODOs in the code
- * libpq JSON support memory management fixups
+ * libpq JSON support memory management fixups [ed: these have been moved
+ to an earlier commit]
## config/programs.m4 ##
@@ config/programs.m4: if test "$pgac_cv_ldap_safe" != yes; then
@@ src/backend/libpq/auth-oauth.c: validate(Port *port, const char *auth)
}
- ## src/common/jsonapi.c ##
-@@
- #endif
-
- /*
-- * In backend, we will use palloc/pfree along with StringInfo. In frontend, use
-- * malloc and PQExpBuffer, and return JSON_OUT_OF_MEMORY on out-of-memory.
-+ * In backend, we will use palloc/pfree along with StringInfo. In frontend,
-+ * use malloc and PQExpBuffer, and return JSON_OUT_OF_MEMORY on out-of-memory.
- */
- #ifdef FRONTEND
-
- #define STRDUP(s) strdup(s)
- #define ALLOC(size) malloc(size)
-+#define ALLOC0(size) calloc(1, size)
-+#define FREE(s) free(s)
-
- #define appendStrVal appendPQExpBuffer
- #define appendBinaryStrVal appendBinaryPQExpBuffer
-@@
-
- #define STRDUP(s) pstrdup(s)
- #define ALLOC(size) palloc(size)
-+#define ALLOC0(size) palloc0(size)
-+#define FREE(s) pfree(s)
-
- #define appendStrVal appendStringInfo
- #define appendBinaryStrVal appendBinaryStringInfo
-@@ src/common/jsonapi.c: IsValidJsonNumber(const char *str, int len)
- * responsible for freeing the returned struct, either by calling
- * freeJsonLexContext() or (in backend environment) via memory context
- * cleanup.
-+ *
-+ * In frontend code this can return NULL on OOM, so callers must inspect the
-+ * returned pointer.
- */
- JsonLexContext *
- makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
-@@ src/common/jsonapi.c: makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
- {
- if (lex == NULL)
- {
-- lex = palloc0(sizeof(JsonLexContext));
-+ lex = ALLOC0(sizeof(JsonLexContext));
-+ if (!lex)
-+ return NULL;
- lex->flags |= JSONLEX_FREE_STRUCT;
- }
- else
-@@ src/common/jsonapi.c: freeJsonLexContext(JsonLexContext *lex)
- {
- static const JsonLexContext empty = {0};
-
-+ if (!lex)
-+ return;
-+
- if (lex->flags & JSONLEX_FREE_STRVAL)
- destroyStrVal(lex->strval);
-
-@@ src/common/jsonapi.c: freeJsonLexContext(JsonLexContext *lex)
- destroyStrVal(lex->errormsg);
-
- if (lex->flags & JSONLEX_FREE_STRUCT)
-- pfree(lex);
-+ FREE(lex);
- else
- *lex = empty;
- }
-
- ## src/common/parse_manifest.c ##
-@@ src/common/parse_manifest.c: json_parse_manifest(JsonManifestParseContext *context, char *buffer,
-
- /* Create a JSON lexing context. */
- lex = makeJsonLexContextCstringLen(NULL, buffer, size, PG_UTF8, true);
-+ if (!lex)
-+ json_manifest_parse_failure(context, "out of memory");
-
- /* Set up semantic actions. */
- sem.semstate = &parse;
-
## src/include/common/oauth-common.h ##
@@
* oauth-common.h
v23-0002-Remove-fe_memutils-from-libpgcommon_shlib.patch (application/octet-stream, 1.4K) download
From d6cae9157e1571f5afb514d39494a3c6cb5d5fa8 Mon Sep 17 00:00:00 2001
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Mon, 1 Jul 2024 14:18:33 -0700
Subject: [PATCH v23 2/6] Remove fe_memutils from libpgcommon_shlib
libpq appears to have no need for this, and the exit() references cause
our libpq-refs-stamp test to fail if the linker doesn't strip out the
unused code.
---
src/common/Makefile | 2 +-
src/common/meson.build | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/common/Makefile b/src/common/Makefile
index 3d83299432..5712078dae 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -105,11 +105,11 @@ endif
# libraries such as libpq to report errors directly.
OBJS_FRONTEND_SHLIB = \
$(OBJS_COMMON) \
- fe_memutils.o \
restricted_token.o \
sprompt.o
OBJS_FRONTEND = \
$(OBJS_FRONTEND_SHLIB) \
+ fe_memutils.o \
logging.o
# foo.o, foo_shlib.o, and foo_srv.o are all built from foo.c
diff --git a/src/common/meson.build b/src/common/meson.build
index de68e408fa..7eb604c608 100644
--- a/src/common/meson.build
+++ b/src/common/meson.build
@@ -105,13 +105,13 @@ common_sources_cflags = {
common_sources_frontend_shlib = common_sources
common_sources_frontend_shlib += files(
- 'fe_memutils.c',
'restricted_token.c',
'sprompt.c',
)
common_sources_frontend_static = common_sources_frontend_shlib
common_sources_frontend_static += files(
+ 'fe_memutils.c',
'logging.c',
)
--
2.34.1
v23-0003-common-jsonapi-support-libpq-as-a-client.patch (application/octet-stream, 33.0K) download
From 5543539169dbe3a9e5cef4f0151cff4f8498c8cf Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Mon, 3 May 2021 15:38:26 -0700
Subject: [PATCH v23 3/6] common/jsonapi: support libpq as a client
Based on a patch by Michael Paquier.
For frontend code, use PQExpBuffer instead of StringInfo. This requires
us to track allocation failures so that we can return JSON_OUT_OF_MEMORY
as needed rather than exit()ing.
Co-authored-by: Michael Paquier <michael@paquier.xyz>
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
---
src/bin/pg_combinebackup/Makefile | 4 +-
src/bin/pg_combinebackup/meson.build | 2 +-
src/bin/pg_verifybackup/Makefile | 2 +-
src/common/Makefile | 2 +-
src/common/jsonapi.c | 448 +++++++++++++-----
src/common/meson.build | 8 +-
src/common/parse_manifest.c | 5 +-
src/include/common/jsonapi.h | 20 +-
src/test/modules/test_json_parser/Makefile | 3 +
src/test/modules/test_json_parser/meson.build | 4 +-
10 files changed, 361 insertions(+), 137 deletions(-)
diff --git a/src/bin/pg_combinebackup/Makefile b/src/bin/pg_combinebackup/Makefile
index c3729755ba..2f7dc1ed87 100644
--- a/src/bin/pg_combinebackup/Makefile
+++ b/src/bin/pg_combinebackup/Makefile
@@ -18,6 +18,8 @@ include $(top_builddir)/src/Makefile.global
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils
+# TODO: fix this properly
+LDFLAGS_INTERNAL += -lpgcommon $(libpq_pgport)
OBJS = \
$(WIN32RES) \
@@ -30,7 +32,7 @@ OBJS = \
all: pg_combinebackup
-pg_combinebackup: $(OBJS) | submake-libpgport submake-libpgfeutils
+pg_combinebackup: $(OBJS) | submake-libpq submake-libpgport submake-libpgfeutils
$(CC) $(CFLAGS) $^ $(LDFLAGS) $(LDFLAGS_EX) $(LIBS) -o $@$(X)
install: all installdirs
diff --git a/src/bin/pg_combinebackup/meson.build b/src/bin/pg_combinebackup/meson.build
index 1d4b9c218f..cab677b574 100644
--- a/src/bin/pg_combinebackup/meson.build
+++ b/src/bin/pg_combinebackup/meson.build
@@ -17,7 +17,7 @@ endif
pg_combinebackup = executable('pg_combinebackup',
pg_combinebackup_sources,
- dependencies: [frontend_code],
+ dependencies: [frontend_code, libpq],
kwargs: default_bin_args,
)
bin_targets += pg_combinebackup
diff --git a/src/bin/pg_verifybackup/Makefile b/src/bin/pg_verifybackup/Makefile
index 7c045f142e..3372fada01 100644
--- a/src/bin/pg_verifybackup/Makefile
+++ b/src/bin/pg_verifybackup/Makefile
@@ -17,7 +17,7 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
# We need libpq only because fe_utils does.
-LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils $(libpq_pgport)
+LDFLAGS_INTERNAL += -L$(top_builddir)/src/fe_utils -lpgfeutils -lpgcommon $(libpq_pgport)
OBJS = \
$(WIN32RES) \
diff --git a/src/common/Makefile b/src/common/Makefile
index 5712078dae..f1da2ed13d 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -40,7 +40,7 @@ override CPPFLAGS += -DVAL_LDFLAGS_EX="\"$(LDFLAGS_EX)\""
override CPPFLAGS += -DVAL_LDFLAGS_SL="\"$(LDFLAGS_SL)\""
override CPPFLAGS += -DVAL_LIBS="\"$(LIBS)\""
-override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common $(CPPFLAGS)
+override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common -I$(libpq_srcdir) $(CPPFLAGS)
LIBS += $(PTHREAD_LIBS)
OBJS_COMMON = \
diff --git a/src/common/jsonapi.c b/src/common/jsonapi.c
index 0c6374b0fc..e5fe512b8b 100644
--- a/src/common/jsonapi.c
+++ b/src/common/jsonapi.c
@@ -21,10 +21,66 @@
#include "mb/pg_wchar.h"
#include "port/pg_lfind.h"
-#ifndef FRONTEND
+#ifdef FRONTEND
+#include "pqexpbuffer.h"
+#else
+#include "lib/stringinfo.h"
#include "miscadmin.h"
#endif
+/*
+ * In backend, we will use palloc/pfree along with StringInfo. In frontend,
+ * use malloc and PQExpBuffer, and return JSON_OUT_OF_MEMORY on out-of-memory.
+ */
+#ifdef FRONTEND
+
+#define STRDUP(s) strdup(s)
+#define ALLOC(size) malloc(size)
+#define ALLOC0(size) calloc(1, size)
+#define REALLOC realloc
+#define FREE(s) free(s)
+
+#define appendStrVal appendPQExpBuffer
+#define appendBinaryStrVal appendBinaryPQExpBuffer
+#define appendStrValChar appendPQExpBufferChar
+/* XXX should we add a macro version to PQExpBuffer? */
+#define appendStrValCharMacro appendPQExpBufferChar
+#define createStrVal createPQExpBuffer
+#define initStrVal initPQExpBuffer
+#define resetStrVal resetPQExpBuffer
+#define termStrVal termPQExpBuffer
+#define destroyStrVal destroyPQExpBuffer
+
+#else /* !FRONTEND */
+
+#define STRDUP(s) pstrdup(s)
+#define ALLOC(size) palloc(size)
+#define ALLOC0(size) palloc0(size)
+#define REALLOC repalloc
+
+/*
+ * Backend pfree() doesn't handle NULL pointers like the frontend's does; smooth
+ * that over to reduce mental gymnastics. Avoid multiple evaluation of the macro
+ * argument to avoid future hair-pulling.
+ */
+#define FREE(s) do { \
+ void *__v = (s); \
+ if (__v) \
+ pfree(__v); \
+} while (0)
+
+#define appendStrVal appendStringInfo
+#define appendBinaryStrVal appendBinaryStringInfo
+#define appendStrValChar appendStringInfoChar
+#define appendStrValCharMacro appendStringInfoCharMacro
+#define createStrVal makeStringInfo
+#define initStrVal initStringInfo
+#define resetStrVal resetStringInfo
+#define termStrVal(s) pfree((s)->data)
+#define destroyStrVal destroyStringInfo
+
+#endif
+
/*
* The context of the parser is maintained by the recursive descent
* mechanism, but is passed explicitly to the error reporting routine
@@ -103,7 +159,7 @@ struct JsonIncrementalState
{
bool is_last_chunk;
bool partial_completed;
- StringInfoData partial_token;
+ StrValType partial_token;
};
/*
@@ -219,6 +275,7 @@ static JsonParseErrorType parse_object(JsonLexContext *lex, JsonSemAction *sem);
static JsonParseErrorType parse_array_element(JsonLexContext *lex, JsonSemAction *sem);
static JsonParseErrorType parse_array(JsonLexContext *lex, JsonSemAction *sem);
static JsonParseErrorType report_parse_error(JsonParseContext ctx, JsonLexContext *lex);
+static bool allocate_incremental_state(JsonLexContext *lex);
/* the null action object used for pure validation */
JsonSemAction nullSemAction =
@@ -273,15 +330,11 @@ IsValidJsonNumber(const char *str, size_t len)
{
bool numeric_error;
size_t total_len;
- JsonLexContext dummy_lex;
+ JsonLexContext dummy_lex = {0};
if (len <= 0)
return false;
- dummy_lex.incremental = false;
- dummy_lex.inc_state = NULL;
- dummy_lex.pstack = NULL;
-
/*
* json_lex_number expects a leading '-' to have been eaten already.
*
@@ -321,6 +374,9 @@ IsValidJsonNumber(const char *str, size_t len)
* responsible for freeing the returned struct, either by calling
* freeJsonLexContext() or (in backend environment) via memory context
* cleanup.
+ *
+ * In frontend code this can return NULL on OOM, so callers must inspect the
+ * returned pointer.
*/
JsonLexContext *
makeJsonLexContextCstringLen(JsonLexContext *lex, const char *json,
@@ -328,7 +384,9 @@ makeJsonLexContextCstringLen(JsonLexContext *lex, const char *json,
{
if (lex == NULL)
{
- lex = palloc0(sizeof(JsonLexContext));
+ lex = ALLOC0(sizeof(JsonLexContext));
+ if (!lex)
+ return NULL;
lex->flags |= JSONLEX_FREE_STRUCT;
}
else
@@ -341,13 +399,70 @@ makeJsonLexContextCstringLen(JsonLexContext *lex, const char *json,
lex->input_encoding = encoding;
if (need_escapes)
{
- lex->strval = makeStringInfo();
+ /*
+ * This call can fail in FRONTEND code. We defer error handling to
+ * time of use (json_lex_string()) since we might not need to parse
+ * any strings anyway.
+ */
+ lex->strval = createStrVal();
lex->flags |= JSONLEX_FREE_STRVAL;
+ lex->parse_strval = true;
}
return lex;
}
+/*
+ * Allocates the internal bookkeeping structures for incremental parsing. This
+ * can only fail in-band with FRONTEND code.
+ */
+#define JS_STACK_CHUNK_SIZE 64
+#define JS_MAX_PROD_LEN 10 /* more than we need */
+#define JSON_TD_MAX_STACK 6400 /* hard coded for now - this is a REALLY high
+ * number */
+static bool
+allocate_incremental_state(JsonLexContext *lex)
+{
+ void *pstack,
+ *prediction,
+ *fnames,
+ *fnull;
+
+ lex->inc_state = ALLOC0(sizeof(JsonIncrementalState));
+ pstack = ALLOC(sizeof(JsonParserStack));
+ prediction = ALLOC(JS_STACK_CHUNK_SIZE * JS_MAX_PROD_LEN);
+ fnames = ALLOC(JS_STACK_CHUNK_SIZE * sizeof(char *));
+ fnull = ALLOC(JS_STACK_CHUNK_SIZE * sizeof(bool));
+
+#ifdef FRONTEND
+ if (!lex->inc_state
+ || !pstack
+ || !prediction
+ || !fnames
+ || !fnull)
+ {
+ FREE(lex->inc_state);
+ FREE(pstack);
+ FREE(prediction);
+ FREE(fnames);
+ FREE(fnull);
+
+ return false;
+ }
+#endif
+
+ initStrVal(&(lex->inc_state->partial_token));
+ lex->pstack = pstack;
+ lex->pstack->stack_size = JS_STACK_CHUNK_SIZE;
+ lex->pstack->prediction = prediction;
+ lex->pstack->pred_index = 0;
+ lex->pstack->fnames = fnames;
+ lex->pstack->fnull = fnull;
+
+ lex->incremental = true;
+ return true;
+}
+
/*
* makeJsonLexContextIncremental
@@ -357,19 +472,20 @@ makeJsonLexContextCstringLen(JsonLexContext *lex, const char *json,
* we don't need the input, that will be handed in bit by bit to the
* parse routine. We also need an accumulator for partial tokens in case
* the boundary between chunks happens to fall in the middle of a token.
+ *
+ * In frontend code this can return NULL on OOM, so callers must inspect the
+ * returned pointer.
*/
-#define JS_STACK_CHUNK_SIZE 64
-#define JS_MAX_PROD_LEN 10 /* more than we need */
-#define JSON_TD_MAX_STACK 6400 /* hard coded for now - this is a REALLY high
- * number */
-
JsonLexContext *
makeJsonLexContextIncremental(JsonLexContext *lex, int encoding,
bool need_escapes)
{
if (lex == NULL)
{
- lex = palloc0(sizeof(JsonLexContext));
+ lex = ALLOC0(sizeof(JsonLexContext));
+ if (!lex)
+ return NULL;
+
lex->flags |= JSONLEX_FREE_STRUCT;
}
else
@@ -377,42 +493,60 @@ makeJsonLexContextIncremental(JsonLexContext *lex, int encoding,
lex->line_number = 1;
lex->input_encoding = encoding;
- lex->incremental = true;
- lex->inc_state = palloc0(sizeof(JsonIncrementalState));
- initStringInfo(&(lex->inc_state->partial_token));
- lex->pstack = palloc(sizeof(JsonParserStack));
- lex->pstack->stack_size = JS_STACK_CHUNK_SIZE;
- lex->pstack->prediction = palloc(JS_STACK_CHUNK_SIZE * JS_MAX_PROD_LEN);
- lex->pstack->pred_index = 0;
- lex->pstack->fnames = palloc(JS_STACK_CHUNK_SIZE * sizeof(char *));
- lex->pstack->fnull = palloc(JS_STACK_CHUNK_SIZE * sizeof(bool));
+
+ if (!allocate_incremental_state(lex))
+ {
+ if (lex->flags & JSONLEX_FREE_STRUCT)
+ FREE(lex);
+ return NULL;
+ }
+
if (need_escapes)
{
- lex->strval = makeStringInfo();
+ /*
+ * This call can fail in FRONTEND code. We defer error handling to
+ * time of use (json_lex_string()) since we might not need to parse
+ * any strings anyway.
+ */
+ lex->strval = createStrVal();
lex->flags |= JSONLEX_FREE_STRVAL;
+ lex->parse_strval = true;
}
+
return lex;
}
-static inline void
+static inline bool
inc_lex_level(JsonLexContext *lex)
{
- lex->lex_level += 1;
-
- if (lex->incremental && lex->lex_level >= lex->pstack->stack_size)
+ if (lex->incremental && (lex->lex_level + 1) >= lex->pstack->stack_size)
{
- lex->pstack->stack_size += JS_STACK_CHUNK_SIZE;
- lex->pstack->prediction =
- repalloc(lex->pstack->prediction,
- lex->pstack->stack_size * JS_MAX_PROD_LEN);
- if (lex->pstack->fnames)
- lex->pstack->fnames =
- repalloc(lex->pstack->fnames,
- lex->pstack->stack_size * sizeof(char *));
- if (lex->pstack->fnull)
- lex->pstack->fnull =
- repalloc(lex->pstack->fnull, lex->pstack->stack_size * sizeof(bool));
+ size_t new_stack_size;
+ char *new_prediction;
+ char **new_fnames;
+ bool *new_fnull;
+
+ new_stack_size = lex->pstack->stack_size + JS_STACK_CHUNK_SIZE;
+
+ new_prediction = REALLOC(lex->pstack->prediction,
+ new_stack_size * JS_MAX_PROD_LEN);
+ new_fnames = REALLOC(lex->pstack->fnames,
+ new_stack_size * sizeof(char *));
+ new_fnull = REALLOC(lex->pstack->fnull, new_stack_size * sizeof(bool));
+
+#ifdef FRONTEND
+ if (!new_prediction || !new_fnames || !new_fnull)
+ return false;
+#endif
+
+ lex->pstack->stack_size = new_stack_size;
+ lex->pstack->prediction = new_prediction;
+ lex->pstack->fnames = new_fnames;
+ lex->pstack->fnull = new_fnull;
}
+
+ lex->lex_level += 1;
+ return true;
}
static inline void
@@ -482,24 +616,31 @@ get_fnull(JsonLexContext *lex)
void
freeJsonLexContext(JsonLexContext *lex)
{
+ static const JsonLexContext empty = {0};
+
+ if (!lex)
+ return;
+
if (lex->flags & JSONLEX_FREE_STRVAL)
- destroyStringInfo(lex->strval);
+ destroyStrVal(lex->strval);
if (lex->errormsg)
- destroyStringInfo(lex->errormsg);
+ destroyStrVal(lex->errormsg);
if (lex->incremental)
{
- pfree(lex->inc_state->partial_token.data);
- pfree(lex->inc_state);
- pfree(lex->pstack->prediction);
- pfree(lex->pstack->fnames);
- pfree(lex->pstack->fnull);
- pfree(lex->pstack);
+ termStrVal(&lex->inc_state->partial_token);
+ FREE(lex->inc_state);
+ FREE(lex->pstack->prediction);
+ FREE(lex->pstack->fnames);
+ FREE(lex->pstack->fnull);
+ FREE(lex->pstack);
}
if (lex->flags & JSONLEX_FREE_STRUCT)
- pfree(lex);
+ FREE(lex);
+ else
+ *lex = empty;
}
/*
@@ -522,22 +663,13 @@ JsonParseErrorType
pg_parse_json(JsonLexContext *lex, JsonSemAction *sem)
{
#ifdef FORCE_JSON_PSTACK
-
- lex->incremental = true;
- lex->inc_state = palloc0(sizeof(JsonIncrementalState));
-
/*
* We don't need partial token processing, there is only one chunk. But we
* still need to init the partial token string so that freeJsonLexContext
- * works.
+ * works, so perform the full incremental initialization.
*/
- initStringInfo(&(lex->inc_state->partial_token));
- lex->pstack = palloc(sizeof(JsonParserStack));
- lex->pstack->stack_size = JS_STACK_CHUNK_SIZE;
- lex->pstack->prediction = palloc(JS_STACK_CHUNK_SIZE * JS_MAX_PROD_LEN);
- lex->pstack->pred_index = 0;
- lex->pstack->fnames = palloc(JS_STACK_CHUNK_SIZE * sizeof(char *));
- lex->pstack->fnull = palloc(JS_STACK_CHUNK_SIZE * sizeof(bool));
+ if (!allocate_incremental_state(lex))
+ return JSON_OUT_OF_MEMORY;
return pg_parse_json_incremental(lex, sem, lex->input, lex->input_length, true);
@@ -597,7 +729,7 @@ json_count_array_elements(JsonLexContext *lex, int *elements)
* etc, so doing this with a copy makes that safe.
*/
memcpy(©lex, lex, sizeof(JsonLexContext));
- copylex.strval = NULL; /* not interested in values here */
+ copylex.parse_strval = false; /* not interested in values here */
copylex.lex_level++;
count = 0;
@@ -737,7 +869,9 @@ pg_parse_json_incremental(JsonLexContext *lex,
if (result != JSON_SUCCESS)
return result;
}
- inc_lex_level(lex);
+
+ if (!inc_lex_level(lex))
+ return JSON_OUT_OF_MEMORY;
}
break;
case JSON_SEM_OEND:
@@ -766,7 +900,9 @@ pg_parse_json_incremental(JsonLexContext *lex,
if (result != JSON_SUCCESS)
return result;
}
- inc_lex_level(lex);
+
+ if (!inc_lex_level(lex))
+ return JSON_OUT_OF_MEMORY;
}
break;
case JSON_SEM_AEND:
@@ -793,9 +929,11 @@ pg_parse_json_incremental(JsonLexContext *lex,
json_ofield_action ostart = sem->object_field_start;
json_ofield_action oend = sem->object_field_end;
- if ((ostart != NULL || oend != NULL) && lex->strval != NULL)
+ if ((ostart != NULL || oend != NULL) && lex->parse_strval)
{
- fname = pstrdup(lex->strval->data);
+ fname = STRDUP(lex->strval->data);
+ if (fname == NULL)
+ return JSON_OUT_OF_MEMORY;
}
set_fname(lex, fname);
}
@@ -883,14 +1021,21 @@ pg_parse_json_incremental(JsonLexContext *lex,
*/
if (tok == JSON_TOKEN_STRING)
{
- if (lex->strval != NULL)
- pstack->scalar_val = pstrdup(lex->strval->data);
+ if (lex->parse_strval)
+ {
+ pstack->scalar_val = STRDUP(lex->strval->data);
+ if (pstack->scalar_val == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
}
else
{
ptrdiff_t tlen = (lex->token_terminator - lex->token_start);
- pstack->scalar_val = palloc(tlen + 1);
+ pstack->scalar_val = ALLOC(tlen + 1);
+ if (pstack->scalar_val == NULL)
+ return JSON_OUT_OF_MEMORY;
+
memcpy(pstack->scalar_val, lex->token_start, tlen);
pstack->scalar_val[tlen] = '\0';
}
@@ -1025,14 +1170,21 @@ parse_scalar(JsonLexContext *lex, JsonSemAction *sem)
/* extract the de-escaped string value, or the raw lexeme */
if (lex_peek(lex) == JSON_TOKEN_STRING)
{
- if (lex->strval != NULL)
- val = pstrdup(lex->strval->data);
+ if (lex->parse_strval)
+ {
+ val = STRDUP(lex->strval->data);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
}
else
{
int len = (lex->token_terminator - lex->token_start);
- val = palloc(len + 1);
+ val = ALLOC(len + 1);
+ if (val == NULL)
+ return JSON_OUT_OF_MEMORY;
+
memcpy(val, lex->token_start, len);
val[len] = '\0';
}
@@ -1066,8 +1218,12 @@ parse_object_field(JsonLexContext *lex, JsonSemAction *sem)
if (lex_peek(lex) != JSON_TOKEN_STRING)
return report_parse_error(JSON_PARSE_STRING, lex);
- if ((ostart != NULL || oend != NULL) && lex->strval != NULL)
- fname = pstrdup(lex->strval->data);
+ if ((ostart != NULL || oend != NULL) && lex->parse_strval)
+ {
+ fname = STRDUP(lex->strval->data);
+ if (fname == NULL)
+ return JSON_OUT_OF_MEMORY;
+ }
result = json_lex(lex);
if (result != JSON_SUCCESS)
return result;
@@ -1123,6 +1279,11 @@ parse_object(JsonLexContext *lex, JsonSemAction *sem)
JsonParseErrorType result;
#ifndef FRONTEND
+
+ /*
+ * TODO: clients need some way to put a bound on stack growth. Parse level
+ * limits maybe?
+ */
check_stack_depth();
#endif
@@ -1312,15 +1473,24 @@ json_lex(JsonLexContext *lex)
const char *const end = lex->input + lex->input_length;
JsonParseErrorType result;
- if (lex->incremental && lex->inc_state->partial_completed)
+ if (lex->incremental)
{
- /*
- * We just lexed a completed partial token on the last call, so reset
- * everything
- */
- resetStringInfo(&(lex->inc_state->partial_token));
- lex->token_terminator = lex->input;
- lex->inc_state->partial_completed = false;
+ if (lex->inc_state->partial_completed)
+ {
+ /*
+ * We just lexed a completed partial token on the last call, so
+ * reset everything
+ */
+ resetStrVal(&(lex->inc_state->partial_token));
+ lex->token_terminator = lex->input;
+ lex->inc_state->partial_completed = false;
+ }
+
+#ifdef FRONTEND
+ /* Make sure our partial token buffer is valid before using it below. */
+ if (PQExpBufferDataBroken(lex->inc_state->partial_token))
+ return JSON_OUT_OF_MEMORY;
+#endif
}
s = lex->token_terminator;
@@ -1331,7 +1501,7 @@ json_lex(JsonLexContext *lex)
* We have a partial token. Extend it and if completed lex it by a
* recursive call
*/
- StringInfo ptok = &(lex->inc_state->partial_token);
+ StrValType *ptok = &(lex->inc_state->partial_token);
size_t added = 0;
bool tok_done = false;
JsonLexContext dummy_lex;
@@ -1358,7 +1528,7 @@ json_lex(JsonLexContext *lex)
{
char c = lex->input[i];
- appendStringInfoCharMacro(ptok, c);
+ appendStrValCharMacro(ptok, c);
added++;
if (c == '"' && escapes % 2 == 0)
{
@@ -1403,7 +1573,7 @@ json_lex(JsonLexContext *lex)
case '8':
case '9':
{
- appendStringInfoCharMacro(ptok, cc);
+ appendStrValCharMacro(ptok, cc);
added++;
}
break;
@@ -1424,7 +1594,7 @@ json_lex(JsonLexContext *lex)
if (JSON_ALPHANUMERIC_CHAR(cc))
{
- appendStringInfoCharMacro(ptok, cc);
+ appendStrValCharMacro(ptok, cc);
added++;
}
else
@@ -1467,6 +1637,7 @@ json_lex(JsonLexContext *lex)
dummy_lex.input_length = ptok->len;
dummy_lex.input_encoding = lex->input_encoding;
dummy_lex.incremental = false;
+ dummy_lex.parse_strval = lex->parse_strval;
dummy_lex.strval = lex->strval;
partial_result = json_lex(&dummy_lex);
@@ -1622,8 +1793,8 @@ json_lex(JsonLexContext *lex)
if (lex->incremental && !lex->inc_state->is_last_chunk &&
p == lex->input + lex->input_length)
{
- appendBinaryStringInfo(
- &(lex->inc_state->partial_token), s, end - s);
+ appendBinaryStrVal(
+ &(lex->inc_state->partial_token), s, end - s);
return JSON_INCOMPLETE;
}
@@ -1680,8 +1851,8 @@ json_lex_string(JsonLexContext *lex)
do { \
if (lex->incremental && !lex->inc_state->is_last_chunk) \
{ \
- appendBinaryStringInfo(&lex->inc_state->partial_token, \
- lex->token_start, end - lex->token_start); \
+ appendBinaryStrVal(&lex->inc_state->partial_token, \
+ lex->token_start, end - lex->token_start); \
return JSON_INCOMPLETE; \
} \
lex->token_terminator = s; \
@@ -1694,8 +1865,15 @@ json_lex_string(JsonLexContext *lex)
return code; \
} while (0)
- if (lex->strval != NULL)
- resetStringInfo(lex->strval);
+ if (lex->parse_strval)
+ {
+#ifdef FRONTEND
+ /* make sure initialization succeeded */
+ if (lex->strval == NULL)
+ return JSON_OUT_OF_MEMORY;
+#endif
+ resetStrVal(lex->strval);
+ }
Assert(lex->input_length > 0);
s = lex->token_start;
@@ -1732,7 +1910,7 @@ json_lex_string(JsonLexContext *lex)
else
FAIL_AT_CHAR_END(JSON_UNICODE_ESCAPE_FORMAT);
}
- if (lex->strval != NULL)
+ if (lex->parse_strval)
{
/*
* Combine surrogate pairs.
@@ -1789,19 +1967,19 @@ json_lex_string(JsonLexContext *lex)
unicode_to_utf8(ch, (unsigned char *) utf8str);
utf8len = pg_utf_mblen((unsigned char *) utf8str);
- appendBinaryStringInfo(lex->strval, utf8str, utf8len);
+ appendBinaryPQExpBuffer(lex->strval, utf8str, utf8len);
}
else if (ch <= 0x007f)
{
/* The ASCII range is the same in all encodings */
- appendStringInfoChar(lex->strval, (char) ch);
+ appendPQExpBufferChar(lex->strval, (char) ch);
}
else
FAIL_AT_CHAR_END(JSON_UNICODE_HIGH_ESCAPE);
#endif /* FRONTEND */
}
}
- else if (lex->strval != NULL)
+ else if (lex->parse_strval)
{
if (hi_surrogate != -1)
FAIL_AT_CHAR_END(JSON_UNICODE_LOW_SURROGATE);
@@ -1811,22 +1989,22 @@ json_lex_string(JsonLexContext *lex)
case '"':
case '\\':
case '/':
- appendStringInfoChar(lex->strval, *s);
+ appendStrValChar(lex->strval, *s);
break;
case 'b':
- appendStringInfoChar(lex->strval, '\b');
+ appendStrValChar(lex->strval, '\b');
break;
case 'f':
- appendStringInfoChar(lex->strval, '\f');
+ appendStrValChar(lex->strval, '\f');
break;
case 'n':
- appendStringInfoChar(lex->strval, '\n');
+ appendStrValChar(lex->strval, '\n');
break;
case 'r':
- appendStringInfoChar(lex->strval, '\r');
+ appendStrValChar(lex->strval, '\r');
break;
case 't':
- appendStringInfoChar(lex->strval, '\t');
+ appendStrValChar(lex->strval, '\t');
break;
default:
@@ -1861,7 +2039,7 @@ json_lex_string(JsonLexContext *lex)
/*
* Skip to the first byte that requires special handling, so we
- * can batch calls to appendBinaryStringInfo.
+ * can batch calls to appendBinaryStrVal.
*/
while (p < end - sizeof(Vector8) &&
!pg_lfind8('\\', (uint8 *) p, sizeof(Vector8)) &&
@@ -1885,8 +2063,8 @@ json_lex_string(JsonLexContext *lex)
}
}
- if (lex->strval != NULL)
- appendBinaryStringInfo(lex->strval, s, p - s);
+ if (lex->parse_strval)
+ appendBinaryStrVal(lex->strval, s, p - s);
/*
* s will be incremented at the top of the loop, so set it to just
@@ -1902,6 +2080,11 @@ json_lex_string(JsonLexContext *lex)
return JSON_UNICODE_LOW_SURROGATE;
}
+#ifdef FRONTEND
+ if (lex->parse_strval && PQExpBufferBroken(lex->strval))
+ return JSON_OUT_OF_MEMORY;
+#endif
+
/* Hooray, we found the end of the string! */
lex->prev_token_terminator = lex->token_terminator;
lex->token_terminator = s + 1;
@@ -2019,8 +2202,8 @@ json_lex_number(JsonLexContext *lex, const char *s,
if (lex->incremental && !lex->inc_state->is_last_chunk &&
len >= lex->input_length)
{
- appendBinaryStringInfo(&lex->inc_state->partial_token,
- lex->token_start, s - lex->token_start);
+ appendBinaryStrVal(&lex->inc_state->partial_token,
+ lex->token_start, s - lex->token_start);
if (num_err != NULL)
*num_err = error;
@@ -2096,19 +2279,25 @@ report_parse_error(JsonParseContext ctx, JsonLexContext *lex)
char *
json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
{
+ if (error == JSON_OUT_OF_MEMORY)
+ {
+ /* Short circuit. Allocating anything for this case is unhelpful. */
+ return _("out of memory");
+ }
+
if (lex->errormsg)
- resetStringInfo(lex->errormsg);
+ resetStrVal(lex->errormsg);
else
- lex->errormsg = makeStringInfo();
+ lex->errormsg = createStrVal();
/*
* A helper for error messages that should print the current token. The
* format must contain exactly one %.*s specifier.
*/
#define json_token_error(lex, format) \
- appendStringInfo((lex)->errormsg, _(format), \
- (int) ((lex)->token_terminator - (lex)->token_start), \
- (lex)->token_start);
+ appendStrVal((lex)->errormsg, _(format), \
+ (int) ((lex)->token_terminator - (lex)->token_start), \
+ (lex)->token_start);
switch (error)
{
@@ -2127,9 +2316,9 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
json_token_error(lex, "Escape sequence \"\\%.*s\" is invalid.");
break;
case JSON_ESCAPING_REQUIRED:
- appendStringInfo(lex->errormsg,
- _("Character with value 0x%02x must be escaped."),
- (unsigned char) *(lex->token_terminator));
+ appendStrVal(lex->errormsg,
+ _("Character with value 0x%02x must be escaped."),
+ (unsigned char) *(lex->token_terminator));
break;
case JSON_EXPECTED_END:
json_token_error(lex, "Expected end of input, but found \"%.*s\".");
@@ -2160,6 +2349,9 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
case JSON_INVALID_TOKEN:
json_token_error(lex, "Token \"%.*s\" is invalid.");
break;
+ case JSON_OUT_OF_MEMORY:
+ /* should have been handled above; use the error path */
+ break;
case JSON_UNICODE_CODE_POINT_ZERO:
return _("\\u0000 cannot be converted to text.");
case JSON_UNICODE_ESCAPE_FORMAT:
@@ -2191,15 +2383,23 @@ json_errdetail(JsonParseErrorType error, JsonLexContext *lex)
}
#undef json_token_error
- /*
- * We don't use a default: case, so that the compiler will warn about
- * unhandled enum values. But this needs to be here anyway to cover the
- * possibility of an incorrect input.
- */
- if (lex->errormsg->len == 0)
- appendStringInfo(lex->errormsg,
- "unexpected json parse error type: %d",
- (int) error);
+ /* Note that lex->errormsg can be NULL in FRONTEND code. */
+ if (lex->errormsg && lex->errormsg->len == 0)
+ {
+ /*
+ * We don't use a default: case, so that the compiler will warn about
+ * unhandled enum values. But this needs to be here anyway to cover
+ * the possibility of an incorrect input.
+ */
+ appendStrVal(lex->errormsg,
+ "unexpected json parse error type: %d",
+ (int) error);
+ }
+
+#ifdef FRONTEND
+ if (PQExpBufferBroken(lex->errormsg))
+ return _("out of memory while constructing error description");
+#endif
return lex->errormsg->data;
}
diff --git a/src/common/meson.build b/src/common/meson.build
index 7eb604c608..ea5f19e89e 100644
--- a/src/common/meson.build
+++ b/src/common/meson.build
@@ -125,13 +125,18 @@ common_sources_frontend_static += files(
# least cryptohash_openssl.c, hmac_openssl.c depend on it.
# controldata_utils.c depends on wait_event_types_h. That's arguably a
# layering violation, but ...
+#
+# XXX Frontend builds need libpq's pqexpbuffer.h, so adjust the include paths
+# appropriately. This seems completely broken.
pgcommon = {}
pgcommon_variants = {
'_srv': internal_lib_args + {
+ 'include_directories': include_directories('.'),
'sources': common_sources + [lwlocknames_h] + [wait_event_types_h],
'dependencies': [backend_common_code],
},
'': default_lib_args + {
+ 'include_directories': include_directories('../interfaces/libpq', '.'),
'sources': common_sources_frontend_static,
'dependencies': [frontend_common_code],
# Files in libpgcommon.a should use/export the "xxx_private" versions
@@ -140,6 +145,7 @@ pgcommon_variants = {
},
'_shlib': default_lib_args + {
'pic': true,
+ 'include_directories': include_directories('../interfaces/libpq', '.'),
'sources': common_sources_frontend_shlib,
'dependencies': [frontend_common_code],
},
@@ -157,7 +163,6 @@ foreach name, opts : pgcommon_variants
c_args = opts.get('c_args', []) + common_cflags[cflagname]
cflag_libs += static_library('libpgcommon@0@_@1@'.format(name, cflagname),
c_pch: pch_c_h,
- include_directories: include_directories('.'),
kwargs: opts + {
'sources': sources,
'c_args': c_args,
@@ -170,7 +175,6 @@ foreach name, opts : pgcommon_variants
lib = static_library('libpgcommon@0@'.format(name),
link_with: cflag_libs,
c_pch: pch_c_h,
- include_directories: include_directories('.'),
kwargs: opts + {
'dependencies': opts['dependencies'] + [ssl],
}
diff --git a/src/common/parse_manifest.c b/src/common/parse_manifest.c
index 612e120b17..0da6272336 100644
--- a/src/common/parse_manifest.c
+++ b/src/common/parse_manifest.c
@@ -139,7 +139,8 @@ json_parse_manifest_incremental_init(JsonManifestParseContext *context)
parse->state = JM_EXPECT_TOPLEVEL_START;
parse->saw_version_field = false;
- makeJsonLexContextIncremental(&(incstate->lex), PG_UTF8, true);
+ if (!makeJsonLexContextIncremental(&(incstate->lex), PG_UTF8, true))
+ context->error_cb(context, "out of memory");
incstate->sem.semstate = parse;
incstate->sem.object_start = json_manifest_object_start;
@@ -240,6 +241,8 @@ json_parse_manifest(JsonManifestParseContext *context, const char *buffer,
/* Create a JSON lexing context. */
lex = makeJsonLexContextCstringLen(NULL, buffer, size, PG_UTF8, true);
+ if (!lex)
+ json_manifest_parse_failure(context, "out of memory");
/* Set up semantic actions. */
sem.semstate = &parse;
diff --git a/src/include/common/jsonapi.h b/src/include/common/jsonapi.h
index 71a491d72d..d03a61fcd6 100644
--- a/src/include/common/jsonapi.h
+++ b/src/include/common/jsonapi.h
@@ -14,8 +14,6 @@
#ifndef JSONAPI_H
#define JSONAPI_H
-#include "lib/stringinfo.h"
-
typedef enum JsonTokenType
{
JSON_TOKEN_INVALID,
@@ -51,6 +49,7 @@ typedef enum JsonParseErrorType
JSON_EXPECTED_OBJECT_NEXT,
JSON_EXPECTED_STRING,
JSON_INVALID_TOKEN,
+ JSON_OUT_OF_MEMORY,
JSON_UNICODE_CODE_POINT_ZERO,
JSON_UNICODE_ESCAPE_FORMAT,
JSON_UNICODE_HIGH_ESCAPE,
@@ -64,6 +63,18 @@ typedef enum JsonParseErrorType
typedef struct JsonParserStack JsonParserStack;
typedef struct JsonIncrementalState JsonIncrementalState;
+/*
+ * Don't depend on the internal type header for strval; if callers need access
+ * then they can include the appropriate header themselves.
+ */
+#ifdef FRONTEND
+#define StrValType PQExpBufferData
+#else
+#define StrValType StringInfoData
+#endif
+
+typedef struct StrValType StrValType;
+
/*
* All the fields in this structure should be treated as read-only.
*
@@ -102,8 +113,9 @@ typedef struct JsonLexContext
const char *line_start; /* where that line starts within input */
JsonParserStack *pstack;
JsonIncrementalState *inc_state;
- StringInfo strval;
- StringInfo errormsg;
+ bool parse_strval;
+ StrValType *strval; /* only used if parse_strval == true */
+ StrValType *errormsg;
} JsonLexContext;
typedef JsonParseErrorType (*json_struct_action) (void *state);
diff --git a/src/test/modules/test_json_parser/Makefile b/src/test/modules/test_json_parser/Makefile
index 2dc7175b7c..f410e04cf1 100644
--- a/src/test/modules/test_json_parser/Makefile
+++ b/src/test/modules/test_json_parser/Makefile
@@ -19,6 +19,9 @@ include $(top_builddir)/src/Makefile.global
include $(top_srcdir)/contrib/contrib-global.mk
endif
+# TODO: fix this properly
+LDFLAGS_INTERNAL += -lpgcommon $(libpq_pgport)
+
all: test_json_parser_incremental$(X) test_json_parser_perf$(X)
%.o: $(top_srcdir)/$(subdir)/%.c
diff --git a/src/test/modules/test_json_parser/meson.build b/src/test/modules/test_json_parser/meson.build
index b224f3e07e..8136070233 100644
--- a/src/test/modules/test_json_parser/meson.build
+++ b/src/test/modules/test_json_parser/meson.build
@@ -13,7 +13,7 @@ endif
test_json_parser_incremental = executable('test_json_parser_incremental',
test_json_parser_incremental_sources,
- dependencies: [frontend_code],
+ dependencies: [frontend_code, libpq],
kwargs: default_bin_args + {
'install': false,
},
@@ -32,7 +32,7 @@ endif
test_json_parser_perf = executable('test_json_parser_perf',
test_json_parser_perf_sources,
- dependencies: [frontend_code],
+ dependencies: [frontend_code, libpq],
kwargs: default_bin_args + {
'install': false,
},
--
2.34.1
v23-0005-backend-add-OAUTHBEARER-SASL-mechanism.patch (application/octet-stream, 54.7K) download
From 044d8b08e99ddcac3e9680ee50c53672009bf95b Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 4 May 2021 16:21:11 -0700
Subject: [PATCH v23 5/6] backend: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) on the server side. This adds a new
auth method, oauth, to pg_hba.
Because OAuth implementations vary so wildly, and bearer token
validation is heavily dependent on the issuing party, authn/z is done by
communicating with an external validator module using callbacks.
The module is responsible for:
1. Validate the bearer token. The correct way to do this depends on the
issuer, but it generally involves either cryptographic operations to
prove that the token was issued by a trusted party, or the
presentation of the bearer token to some other party so that _it_ can
perform validation.
The command MUST maintain confidentiality of the bearer token, since
in most cases it can be used just like a password. (There are ways to
cryptographically bind tokens to client certificates, but they are
way beyond the scope of this commit message.)
If the token cannot be validated, the authorized member of the
ValidatorModuleResult struct is used to indicate failure.
Further authentication/authorization is pointless if
the bearer token wasn't issued by someone you trust.
3. Authenticate the user, authorize the user, or both:
a. To authenticate the user, use the bearer token to retrieve some
trusted identifier string for the end user. The exact process for
this is, again, issuer-dependent. The module wull return the
authenticated identity in the authn_id member.
b. To optionally authorize the user, in combination with the HBA
option trust_validator_authz=1 (see below).
The hard part is in determining whether the given token truly
authorizes the client to use the given role, which must
unfortunately be left as an exercise to the reader.
This obviously requires some care, as a poorly implemented token
validator may silently open the entire database to anyone with a
bearer token. But it may be a more portable approach, since OAuth
is designed as an authorization framework, not an authentication
framework. For example, the user's bearer token could carry an
"allow_superuser_access" claim, which would authorize pseudonymous
database access as any role. It's then up to the OAuth system
administrators to ensure that allow_superuser_access is doled out
only to the proper users.
c. It's possible that the user can be successfully authenticated but
isn't authorized to connect. In this case, the validator module may
return the authenticated ID and then fail with false authorized
member. (This can make it easier to see what's going on in the
Postgres logs.)
The oauth method supports the following HBA options (but note that two
of them are not optional, since we have no way of choosing sensible
defaults):
issuer: Required. The URL of the OAuth issuing party, which the client
must contact to receive a bearer token.
Some real-world examples as of time of writing:
- https://accounts.google.com
- https://login.microsoft.com/[tenant-id]/v2.0
scope: Required. The OAuth scope(s) required for the server to
authenticate and/or authorize the user. This is heavily
deployment-specific, but a simple example is "openid email".
map: Optional. Specify a standard PostgreSQL user map; this works
the same as with other auth methods such as peer. If a map is
not specified, the user ID returned by the token validator
must exactly match the role that's being requested (but see
trust_validator_authz, below).
trust_validator_authz:
Optional. When set to 1, this allows the token validator to
take full control of the authorization process. Standard user
mapping is skipped: if the validator command succeeds, the
client is allowed to connect under its desired role and no
further checks are done.
Several TODOs:
- port to platforms other than "modern Linux/BSD"
- implement more helpful handling of HBA misconfigurations
- use logdetail during auth failures
- allow passing the configured issuer to the oauth_validator_command, to
deal with multi-issuer setups
- ...and more.
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
---
.cirrus.tasks.yml | 15 +-
src/backend/libpq/Makefile | 1 +
src/backend/libpq/auth-oauth.c | 666 ++++++++++++++++++
src/backend/libpq/auth-sasl.c | 10 +-
src/backend/libpq/auth-scram.c | 4 +-
src/backend/libpq/auth.c | 26 +-
src/backend/libpq/hba.c | 31 +-
src/backend/libpq/meson.build | 1 +
src/backend/utils/misc/guc_tables.c | 12 +
src/common/Makefile | 2 +-
src/include/libpq/auth.h | 17 +
src/include/libpq/hba.h | 6 +-
src/include/libpq/oauth.h | 49 ++
src/include/libpq/sasl.h | 11 +
src/test/modules/Makefile | 1 +
src/test/modules/meson.build | 1 +
src/test/modules/oauth_validator/.gitignore | 4 +
src/test/modules/oauth_validator/Makefile | 22 +
.../oauth_validator/expected/validator.out | 6 +
src/test/modules/oauth_validator/meson.build | 37 +
.../modules/oauth_validator/sql/validator.sql | 1 +
.../modules/oauth_validator/t/001_server.pl | 79 +++
.../modules/oauth_validator/t/oauth_server.py | 114 +++
src/test/modules/oauth_validator/validator.c | 82 +++
src/test/perl/PostgreSQL/Test/Cluster.pm | 14 +-
src/test/perl/PostgreSQL/Test/OAuthServer.pm | 65 ++
src/tools/pgindent/typedefs.list | 3 +
27 files changed, 1241 insertions(+), 39 deletions(-)
create mode 100644 src/backend/libpq/auth-oauth.c
create mode 100644 src/include/libpq/oauth.h
create mode 100644 src/test/modules/oauth_validator/.gitignore
create mode 100644 src/test/modules/oauth_validator/Makefile
create mode 100644 src/test/modules/oauth_validator/expected/validator.out
create mode 100644 src/test/modules/oauth_validator/meson.build
create mode 100644 src/test/modules/oauth_validator/sql/validator.sql
create mode 100644 src/test/modules/oauth_validator/t/001_server.pl
create mode 100755 src/test/modules/oauth_validator/t/oauth_server.py
create mode 100644 src/test/modules/oauth_validator/validator.c
create mode 100644 src/test/perl/PostgreSQL/Test/OAuthServer.pm
diff --git a/.cirrus.tasks.yml b/.cirrus.tasks.yml
index 33646faead..95f131baa9 100644
--- a/.cirrus.tasks.yml
+++ b/.cirrus.tasks.yml
@@ -163,7 +163,7 @@ task:
chown root:postgres /tmp/cores
sysctl kern.corefile='/tmp/cores/%N.%P.core'
setup_additional_packages_script: |
- #pkg install -y ...
+ pkg install -y curl
# NB: Intentionally build without -Dllvm. The freebsd image size is already
# large enough to make VM startup slow, and even without llvm freebsd
@@ -175,6 +175,7 @@ task:
-Dcassert=true -Dinjection_points=true \
-Duuid=bsd -Dtcl_version=tcl86 -Ddtrace=auto \
-DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
+ -Doauth=curl \
-Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \
build
EOF
@@ -223,6 +224,7 @@ LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
--with-libxslt
--with-llvm
--with-lz4
+ --with-oauth=curl
--with-pam
--with-perl
--with-python
@@ -235,6 +237,7 @@ LINUX_CONFIGURE_FEATURES: &LINUX_CONFIGURE_FEATURES >-
LINUX_MESON_FEATURES: &LINUX_MESON_FEATURES >-
-Dllvm=enabled
+ -Doauth=curl
-Duuid=e2fs
@@ -310,8 +313,10 @@ task:
EOF
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install \
+ libcurl4-openssl-dev \
+ libcurl4-openssl-dev:i386 \
matrix:
- name: Linux - Debian Bullseye - Autoconf
@@ -676,8 +681,8 @@ task:
folder: $CCACHE_DIR
setup_additional_packages_script: |
- #apt-get update
- #DEBIAN_FRONTEND=noninteractive apt-get -y install ...
+ apt-get update
+ DEBIAN_FRONTEND=noninteractive apt-get -y install libcurl4-openssl-dev
###
# Test that code can be built with gcc/clang without warnings
diff --git a/src/backend/libpq/Makefile b/src/backend/libpq/Makefile
index 6d385fd6a4..98eb2a8242 100644
--- a/src/backend/libpq/Makefile
+++ b/src/backend/libpq/Makefile
@@ -15,6 +15,7 @@ include $(top_builddir)/src/Makefile.global
# be-fsstubs is here for historical reasons, probably belongs elsewhere
OBJS = \
+ auth-oauth.o \
auth-sasl.o \
auth-scram.o \
auth.o \
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
new file mode 100644
index 0000000000..024f304e4d
--- /dev/null
+++ b/src/backend/libpq/auth-oauth.c
@@ -0,0 +1,666 @@
+/*-------------------------------------------------------------------------
+ *
+ * auth-oauth.c
+ * Server-side implementation of the SASL OAUTHBEARER mechanism.
+ *
+ * See the following RFC for more details:
+ * - RFC 7628: https://tools.ietf.org/html/rfc7628
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/backend/libpq/auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include <unistd.h>
+#include <fcntl.h>
+
+#include "common/oauth-common.h"
+#include "fmgr.h"
+#include "lib/stringinfo.h"
+#include "libpq/auth.h"
+#include "libpq/hba.h"
+#include "libpq/oauth.h"
+#include "libpq/sasl.h"
+#include "storage/fd.h"
+#include "storage/ipc.h"
+#include "utils/json.h"
+
+/* GUC */
+char *OAuthValidatorLibrary = "";
+
+static void oauth_get_mechanisms(Port *port, StringInfo buf);
+static void *oauth_init(Port *port, const char *selected_mech, const char *shadow_pass);
+static int oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, const char **logdetail);
+
+static void load_validator_library(void);
+static void shutdown_validator_library(int code, Datum arg);
+
+static ValidatorModuleState *validator_module_state;
+static const OAuthValidatorCallbacks *ValidatorCallbacks;
+
+/* Mechanism declaration */
+const pg_be_sasl_mech pg_be_oauth_mech = {
+ oauth_get_mechanisms,
+ oauth_init,
+ oauth_exchange,
+
+ PG_MAX_AUTH_TOKEN_LENGTH,
+};
+
+
+typedef enum
+{
+ OAUTH_STATE_INIT = 0,
+ OAUTH_STATE_ERROR,
+ OAUTH_STATE_FINISHED,
+} oauth_state;
+
+struct oauth_ctx
+{
+ oauth_state state;
+ Port *port;
+ const char *issuer;
+ const char *scope;
+};
+
+static char *sanitize_char(char c);
+static char *parse_kvpairs_for_auth(char **input);
+static void generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen);
+static bool validate(Port *port, const char *auth);
+
+#define KVSEP 0x01
+#define AUTH_KEY "auth"
+#define BEARER_SCHEME "Bearer "
+
+static void
+oauth_get_mechanisms(Port *port, StringInfo buf)
+{
+ /* Only OAUTHBEARER is supported. */
+ appendStringInfoString(buf, OAUTHBEARER_NAME);
+ appendStringInfoChar(buf, '\0');
+}
+
+static void *
+oauth_init(Port *port, const char *selected_mech, const char *shadow_pass)
+{
+ struct oauth_ctx *ctx;
+
+ if (strcmp(selected_mech, OAUTHBEARER_NAME))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("client selected an invalid SASL authentication mechanism")));
+
+ ctx = palloc0(sizeof(*ctx));
+
+ ctx->state = OAUTH_STATE_INIT;
+ ctx->port = port;
+
+ Assert(port->hba);
+ ctx->issuer = port->hba->oauth_issuer;
+ ctx->scope = port->hba->oauth_scope;
+
+ load_validator_library();
+
+ return ctx;
+}
+
+static int
+oauth_exchange(void *opaq, const char *input, int inputlen,
+ char **output, int *outputlen, const char **logdetail)
+{
+ char *p;
+ char cbind_flag;
+ char *auth;
+
+ struct oauth_ctx *ctx = opaq;
+
+ *output = NULL;
+ *outputlen = -1;
+
+ /*
+ * If the client didn't include an "Initial Client Response" in the
+ * SASLInitialResponse message, send an empty challenge, to which the
+ * client will respond with the same data that usually comes in the
+ * Initial Client Response.
+ */
+ if (input == NULL)
+ {
+ Assert(ctx->state == OAUTH_STATE_INIT);
+
+ *output = pstrdup("");
+ *outputlen = 0;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ /*
+ * Check that the input length agrees with the string length of the input.
+ */
+ if (inputlen == 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The message is empty.")));
+ if (inputlen != strlen(input))
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message length does not match input length.")));
+
+ switch (ctx->state)
+ {
+ case OAUTH_STATE_INIT:
+ /* Handle this case below. */
+ break;
+
+ case OAUTH_STATE_ERROR:
+
+ /*
+ * Only one response is valid for the client during authentication
+ * failure: a single kvsep.
+ */
+ if (inputlen != 1 || *input != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Client did not send a kvsep response.")));
+
+ /* The (failed) handshake is now complete. */
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_FAILURE;
+
+ default:
+ elog(ERROR, "invalid OAUTHBEARER exchange state");
+ return PG_SASL_EXCHANGE_FAILURE;
+ }
+
+ /* Handle the client's initial message. */
+ p = pstrdup(input);
+
+ /*
+ * OAUTHBEARER does not currently define a channel binding (so there is no
+ * OAUTHBEARER-PLUS, and we do not accept a 'p' specifier). We accept a
+ * 'y' specifier purely for the remote chance that a future specification
+ * could define one; then future clients can still interoperate with this
+ * server implementation. 'n' is the expected case.
+ */
+ cbind_flag = *p;
+ switch (cbind_flag)
+ {
+ case 'p':
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("The server does not support channel binding for OAuth, but the client message includes channel binding data.")));
+ break;
+
+ case 'y': /* fall through */
+ case 'n':
+ p++;
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Comma expected, but found character \"%s\".",
+ sanitize_char(*p))));
+ p++;
+ break;
+
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected channel-binding flag %s.",
+ sanitize_char(cbind_flag))));
+ }
+
+ /*
+ * Forbid optional authzid (authorization identity). We don't support it.
+ */
+ if (*p == 'a')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("client uses authorization identity, but it is not supported")));
+ if (*p != ',')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Unexpected attribute %s in client-first-message.",
+ sanitize_char(*p))));
+ p++;
+
+ /* All remaining fields are separated by the RFC's kvsep (\x01). */
+ if (*p != KVSEP)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Key-value separator expected, but found character %s.",
+ sanitize_char(*p))));
+ p++;
+
+ auth = parse_kvpairs_for_auth(&p);
+ if (!auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message does not contain an auth value.")));
+
+ /* We should be at the end of our message. */
+ if (*p)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains additional data after the final terminator.")));
+
+ if (!validate(ctx->port, auth))
+ {
+ generate_error_response(ctx, output, outputlen);
+
+ ctx->state = OAUTH_STATE_ERROR;
+ return PG_SASL_EXCHANGE_CONTINUE;
+ }
+
+ ctx->state = OAUTH_STATE_FINISHED;
+ return PG_SASL_EXCHANGE_SUCCESS;
+}
+
+/*
+ * Convert an arbitrary byte to printable form. For error messages.
+ *
+ * If it's a printable ASCII character, print it as a single character.
+ * otherwise, print it in hex.
+ *
+ * The returned pointer points to a static buffer.
+ */
+static char *
+sanitize_char(char c)
+{
+ static char buf[5];
+
+ if (c >= 0x21 && c <= 0x7E)
+ snprintf(buf, sizeof(buf), "'%c'", c);
+ else
+ snprintf(buf, sizeof(buf), "0x%02x", (unsigned char) c);
+ return buf;
+}
+
+/*
+ * Performs syntactic validation of a key and value from the initial client
+ * response. (Semantic validation of interesting values must be performed
+ * later.)
+ */
+static void
+validate_kvpair(const char *key, const char *val)
+{
+ /*-----
+ * From Sec 3.1:
+ * key = 1*(ALPHA)
+ */
+ static const char *key_allowed_set =
+ "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
+
+ size_t span;
+
+ if (!key[0])
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an empty key name.")));
+
+ span = strspn(key, key_allowed_set);
+ if (key[span] != '\0')
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an invalid key name.")));
+
+ /*-----
+ * From Sec 3.1:
+ * value = *(VCHAR / SP / HTAB / CR / LF )
+ *
+ * The VCHAR (visible character) class is large; a loop is more
+ * straightforward than strspn().
+ */
+ for (; *val; ++val)
+ {
+ if (0x21 <= *val && *val <= 0x7E)
+ continue; /* VCHAR */
+
+ switch (*val)
+ {
+ case ' ':
+ case '\t':
+ case '\r':
+ case '\n':
+ continue; /* SP, HTAB, CR, LF */
+
+ default:
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an invalid value.")));
+ }
+ }
+}
+
+/*
+ * Consumes all kvpairs in an OAUTHBEARER exchange message. If the "auth" key is
+ * found, its value is returned.
+ */
+static char *
+parse_kvpairs_for_auth(char **input)
+{
+ char *pos = *input;
+ char *auth = NULL;
+
+ /*----
+ * The relevant ABNF, from Sec. 3.1:
+ *
+ * kvsep = %x01
+ * key = 1*(ALPHA)
+ * value = *(VCHAR / SP / HTAB / CR / LF )
+ * kvpair = key "=" value kvsep
+ * ;;gs2-header = See RFC 5801
+ * client-resp = (gs2-header kvsep *kvpair kvsep) / kvsep
+ *
+ * By the time we reach this code, the gs2-header and initial kvsep have
+ * already been validated. We start at the beginning of the first kvpair.
+ */
+
+ while (*pos)
+ {
+ char *end;
+ char *sep;
+ char *key;
+ char *value;
+
+ /*
+ * Find the end of this kvpair. Note that input is null-terminated by
+ * the SASL code, so the strchr() is bounded.
+ */
+ end = strchr(pos, KVSEP);
+ if (!end)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains an unterminated key/value pair.")));
+ *end = '\0';
+
+ if (pos == end)
+ {
+ /* Empty kvpair, signifying the end of the list. */
+ *input = pos + 1;
+ return auth;
+ }
+
+ /*
+ * Find the end of the key name.
+ */
+ sep = strchr(pos, '=');
+ if (!sep)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains a key without a value.")));
+ *sep = '\0';
+
+ /* Both key and value are now safely terminated. */
+ key = pos;
+ value = sep + 1;
+ validate_kvpair(key, value);
+
+ if (!strcmp(key, AUTH_KEY))
+ {
+ if (auth)
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message contains multiple auth values.")));
+
+ auth = value;
+ }
+ else
+ {
+ /*
+ * The RFC also defines the host and port keys, but they are not
+ * required for OAUTHBEARER and we do not use them. Also, per Sec.
+ * 3.1, any key/value pairs we don't recognize must be ignored.
+ */
+ }
+
+ /* Move to the next pair. */
+ pos = end + 1;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAUTHBEARER message"),
+ errdetail("Message did not contain a final terminator.")));
+
+ return NULL; /* unreachable */
+}
+
+static void
+generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
+{
+ StringInfoData buf;
+ StringInfoData issuer;
+
+ /*
+ * The admin needs to set an issuer and scope for OAuth to work. There's
+ * not really a way to hide this from the user, either, because we can't
+ * choose a "default" issuer, so be honest in the failure message.
+ *
+ * TODO: see if there's a better place to fail, earlier than this.
+ */
+ if (!ctx->issuer || !ctx->scope)
+ ereport(FATAL,
+ (errcode(ERRCODE_INTERNAL_ERROR),
+ errmsg("OAuth is not properly configured for this user"),
+ errdetail_log("The issuer and scope parameters must be set in pg_hba.conf.")));
+
+ /*------
+ * Build the .well-known URI based on our issuer.
+ * TODO: RFC 8414 defines a competing well-known URI, so we'll probably
+ * have to make this configurable too.
+ */
+ initStringInfo(&issuer);
+ appendStringInfoString(&issuer, ctx->issuer);
+ appendStringInfoString(&issuer, "/.well-known/openid-configuration");
+
+ initStringInfo(&buf);
+
+ /*
+ * TODO: note that escaping here should be belt-and-suspenders, since
+ * escapable characters aren't valid in either the issuer URI or the scope
+ * list, but the HBA doesn't enforce that yet.
+ */
+ appendStringInfoString(&buf, "{ \"status\": \"invalid_token\", ");
+
+ appendStringInfoString(&buf, "\"openid-configuration\": ");
+ escape_json(&buf, issuer.data);
+ pfree(issuer.data);
+
+ appendStringInfoString(&buf, ", \"scope\": ");
+ escape_json(&buf, ctx->scope);
+
+ appendStringInfoString(&buf, " }");
+
+ *output = buf.data;
+ *outputlen = buf.len;
+}
+
+/*-----
+ * Validates the provided Authorization header and returns the token from
+ * within it. NULL is returned on validation failure.
+ *
+ * Only Bearer tokens are accepted. The ABNF is defined in RFC 6750, Sec.
+ * 2.1:
+ *
+ * b64token = 1*( ALPHA / DIGIT /
+ * "-" / "." / "_" / "~" / "+" / "/" ) *"="
+ * credentials = "Bearer" 1*SP b64token
+ *
+ * The "credentials" construction is what we receive in our auth value.
+ *
+ * Since that spec is subordinate to HTTP (i.e. the HTTP Authorization
+ * header format; RFC 7235 Sec. 2), the "Bearer" scheme string must be
+ * compared case-insensitively. (This is not mentioned in RFC 6750, but
+ * it's pointed out in RFC 7628 Sec. 4.)
+ *
+ * Invalid formats are technically a protocol violation, but we shouldn't
+ * reflect any information about the sensitive Bearer token back to the
+ * client; log at COMMERROR instead.
+ *
+ * TODO: handle the Authorization spec, RFC 7235 Sec. 2.1.
+ */
+static const char *
+validate_token_format(const char *header)
+{
+ size_t span;
+ const char *token;
+ static const char *const b64token_allowed_set =
+ "abcdefghijklmnopqrstuvwxyz"
+ "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
+ "0123456789-._~+/";
+
+ /* If the token is empty or simply too short to be correct */
+ if (!header || strlen(header) <= 7)
+ {
+ ereport(COMMERROR,
+ (errmsg("malformed OAuth bearer token 1")));
+ return NULL;
+ }
+
+ if (pg_strncasecmp(header, BEARER_SCHEME, strlen(BEARER_SCHEME)))
+ return NULL;
+
+ /* Pull the bearer token out of the auth value. */
+ token = header + strlen(BEARER_SCHEME);
+
+ /* Swallow any additional spaces. */
+ while (*token == ' ')
+ token++;
+
+ /* Tokens must not be empty. */
+ if (!*token)
+ {
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAuth bearer token 2"),
+ errdetail("Bearer token is empty.")));
+ return NULL;
+ }
+
+ /*
+ * Make sure the token contains only allowed characters. Tokens may end
+ * with any number of '=' characters.
+ */
+ span = strspn(token, b64token_allowed_set);
+ while (token[span] == '=')
+ span++;
+
+ if (token[span] != '\0')
+ {
+ /*
+ * This error message could be more helpful by printing the
+ * problematic character(s), but that'd be a bit like printing a piece
+ * of someone's password into the logs.
+ */
+ ereport(COMMERROR,
+ (errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAuth bearer token 3"),
+ errdetail("Bearer token is not in the correct format.")));
+ return NULL;
+ }
+
+ return token;
+}
+
+static bool
+validate(Port *port, const char *auth)
+{
+ int map_status;
+ ValidatorModuleResult *ret;
+ const char *token;
+
+ /* Ensure that we have a correct token to validate */
+ if (!(token = validate_token_format(auth)))
+ return false;
+
+ /* Call the validation function from the validator module */
+ ret = ValidatorCallbacks->validate_cb(validator_module_state,
+ token, port->user_name);
+
+ if (!ret->authorized)
+ return false;
+
+ if (ret->authn_id)
+ set_authn_id(port, ret->authn_id);
+
+ if (port->hba->oauth_skip_usermap)
+ {
+ /*
+ * If the validator is our authorization authority, we're done.
+ * Authentication may or may not have been performed depending on the
+ * validator implementation; all that matters is that the validator
+ * says the user can log in with the target role.
+ */
+ return true;
+ }
+
+ /* Make sure the validator authenticated the user. */
+ if (ret->authn_id == NULL || ret->authn_id[0] == '\0')
+ {
+ /* TODO: use logdetail; reduce message duplication */
+ ereport(LOG,
+ (errmsg("OAuth bearer authentication failed for user \"%s\": validator provided no identity",
+ port->user_name)));
+ return false;
+ }
+
+ /* Finally, check the user map. */
+ map_status = check_usermap(port->hba->usermap, port->user_name,
+ MyClientConnectionInfo.authn_id, false);
+ return (map_status == STATUS_OK);
+}
+
+static void
+load_validator_library(void)
+{
+ OAuthValidatorModuleInit validator_init;
+
+ if (OAuthValidatorLibrary[0] == '\0')
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("oauth_validator_library is not set")));
+
+ validator_init = (OAuthValidatorModuleInit)
+ load_external_function(OAuthValidatorLibrary,
+ "_PG_oauth_validator_module_init", false, NULL);
+
+ if (validator_init == NULL)
+ ereport(ERROR,
+ (errmsg("%s module \"%s\" have to define the symbol %s",
+ "OAuth validator", OAuthValidatorLibrary, "_PG_oauth_validator_module_init")));
+
+ ValidatorCallbacks = (*validator_init) ();
+
+ validator_module_state = (ValidatorModuleState *) palloc0(sizeof(ValidatorModuleState));
+ if (ValidatorCallbacks->startup_cb != NULL)
+ ValidatorCallbacks->startup_cb(validator_module_state);
+
+ before_shmem_exit(shutdown_validator_library, 0);
+}
+
+static void
+shutdown_validator_library(int code, Datum arg)
+{
+ if (ValidatorCallbacks->shutdown_cb != NULL)
+ ValidatorCallbacks->shutdown_cb(validator_module_state);
+}
diff --git a/src/backend/libpq/auth-sasl.c b/src/backend/libpq/auth-sasl.c
index 08b24d90b4..4039e7fa3e 100644
--- a/src/backend/libpq/auth-sasl.c
+++ b/src/backend/libpq/auth-sasl.c
@@ -20,14 +20,6 @@
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
-/*
- * Maximum accepted size of SASL messages.
- *
- * The messages that the server or libpq generate are much smaller than this,
- * but have some headroom.
- */
-#define PG_MAX_SASL_MESSAGE_LENGTH 1024
-
/*
* Perform a SASL exchange with a libpq client, using a specific mechanism
* implementation.
@@ -103,7 +95,7 @@ CheckSASLAuth(const pg_be_sasl_mech *mech, Port *port, char *shadow_pass,
/* Get the actual SASL message */
initStringInfo(&buf);
- if (pq_getmessage(&buf, PG_MAX_SASL_MESSAGE_LENGTH))
+ if (pq_getmessage(&buf, mech->max_message_length))
{
/* EOF - pq_getmessage already logged error */
pfree(buf.data);
diff --git a/src/backend/libpq/auth-scram.c b/src/backend/libpq/auth-scram.c
index 4161959914..486a34e719 100644
--- a/src/backend/libpq/auth-scram.c
+++ b/src/backend/libpq/auth-scram.c
@@ -113,7 +113,9 @@ static int scram_exchange(void *opaq, const char *input, int inputlen,
const pg_be_sasl_mech pg_be_scram_mech = {
scram_get_mechanisms,
scram_init,
- scram_exchange
+ scram_exchange,
+
+ PG_MAX_SASL_MESSAGE_LENGTH
};
/*
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 2b607c5270..0a5a8640fc 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -29,6 +29,7 @@
#include "libpq/auth.h"
#include "libpq/crypt.h"
#include "libpq/libpq.h"
+#include "libpq/oauth.h"
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
#include "libpq/scram.h"
@@ -45,7 +46,6 @@
*/
static void auth_failed(Port *port, int status, const char *logdetail);
static char *recv_password_packet(Port *port);
-static void set_authn_id(Port *port, const char *id);
/*----------------------------------------------------------------
@@ -201,22 +201,6 @@ static int CheckRADIUSAuth(Port *port);
static int PerformRadiusTransaction(const char *server, const char *secret, const char *portstr, const char *identifier, const char *user_name, const char *passwd);
-/*
- * Maximum accepted size of GSS and SSPI authentication tokens.
- * We also use this as a limit on ordinary password packet lengths.
- *
- * Kerberos tickets are usually quite small, but the TGTs issued by Windows
- * domain controllers include an authorization field known as the Privilege
- * Attribute Certificate (PAC), which contains the user's Windows permissions
- * (group memberships etc.). The PAC is copied into all tickets obtained on
- * the basis of this TGT (even those issued by Unix realms which the Windows
- * realm trusts), and can be several kB in size. The maximum token size
- * accepted by Windows systems is determined by the MaxAuthToken Windows
- * registry setting. Microsoft recommends that it is not set higher than
- * 65535 bytes, so that seems like a reasonable limit for us as well.
- */
-#define PG_MAX_AUTH_TOKEN_LENGTH 65535
-
/*----------------------------------------------------------------
* Global authentication functions
*----------------------------------------------------------------
@@ -305,6 +289,9 @@ auth_failed(Port *port, int status, const char *logdetail)
case uaRADIUS:
errstr = gettext_noop("RADIUS authentication failed for user \"%s\"");
break;
+ case uaOAuth:
+ errstr = gettext_noop("OAuth bearer authentication failed for user \"%s\"");
+ break;
default:
errstr = gettext_noop("authentication failed for user \"%s\": invalid authentication method");
break;
@@ -340,7 +327,7 @@ auth_failed(Port *port, int status, const char *logdetail)
* lifetime of MyClientConnectionInfo, so it is safe to pass a string that is
* managed by an external library.
*/
-static void
+void
set_authn_id(Port *port, const char *id)
{
Assert(id);
@@ -627,6 +614,9 @@ ClientAuthentication(Port *port)
case uaTrust:
status = STATUS_OK;
break;
+ case uaOAuth:
+ status = CheckSASLAuth(&pg_be_oauth_mech, port, NULL, NULL);
+ break;
}
if ((status == STATUS_OK && port->hba->clientcert == clientCertFull)
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index 18271def2e..aabe0b0e68 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -114,7 +114,8 @@ static const char *const UserAuthName[] =
"ldap",
"cert",
"radius",
- "peer"
+ "peer",
+ "oauth",
};
/*
@@ -1743,6 +1744,8 @@ parse_hba_line(TokenizedAuthLine *tok_line, int elevel)
#endif
else if (strcmp(token->string, "radius") == 0)
parsedline->auth_method = uaRADIUS;
+ else if (strcmp(token->string, "oauth") == 0)
+ parsedline->auth_method = uaOAuth;
else
{
ereport(elevel,
@@ -2062,8 +2065,9 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->auth_method != uaPeer &&
hbaline->auth_method != uaGSS &&
hbaline->auth_method != uaSSPI &&
- hbaline->auth_method != uaCert)
- INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, and cert"));
+ hbaline->auth_method != uaCert &&
+ hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("map", gettext_noop("ident, peer, gssapi, sspi, cert, and oauth"));
hbaline->usermap = pstrdup(val);
}
else if (strcmp(name, "clientcert") == 0)
@@ -2446,6 +2450,27 @@ parse_hba_auth_opt(char *name, char *val, HbaLine *hbaline,
hbaline->radiusidentifiers = parsed_identifiers;
hbaline->radiusidentifiers_s = pstrdup(val);
}
+ else if (strcmp(name, "issuer") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("issuer", gettext_noop("oauth"));
+ hbaline->oauth_issuer = pstrdup(val);
+ }
+ else if (strcmp(name, "scope") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("scope", gettext_noop("oauth"));
+ hbaline->oauth_scope = pstrdup(val);
+ }
+ else if (strcmp(name, "trust_validator_authz") == 0)
+ {
+ if (hbaline->auth_method != uaOAuth)
+ INVALID_AUTH_OPTION("trust_validator_authz", gettext_noop("oauth"));
+ if (strcmp(val, "1") == 0)
+ hbaline->oauth_skip_usermap = true;
+ else
+ hbaline->oauth_skip_usermap = false;
+ }
else
{
ereport(elevel,
diff --git a/src/backend/libpq/meson.build b/src/backend/libpq/meson.build
index 7c65314512..c85527fb01 100644
--- a/src/backend/libpq/meson.build
+++ b/src/backend/libpq/meson.build
@@ -1,6 +1,7 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
backend_sources += files(
+ 'auth-oauth.c',
'auth-sasl.c',
'auth-scram.c',
'auth.c',
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index d28b0bcb40..461094f288 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -48,6 +48,7 @@
#include "jit/jit.h"
#include "libpq/auth.h"
#include "libpq/libpq.h"
+#include "libpq/oauth.h"
#include "libpq/scram.h"
#include "nodes/queryjumble.h"
#include "optimizer/cost.h"
@@ -4707,6 +4708,17 @@ struct config_string ConfigureNamesString[] =
check_synchronized_standby_slots, assign_synchronized_standby_slots, NULL
},
+ {
+ {"oauth_validator_library", PGC_SIGHUP, CONN_AUTH_AUTH,
+ gettext_noop("Sets the library that will be called to validate OAuth v2 bearer tokens."),
+ NULL,
+ GUC_SUPERUSER_ONLY | GUC_NOT_IN_SAMPLE
+ },
+ &OAuthValidatorLibrary,
+ "",
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, NULL, NULL, NULL, NULL
diff --git a/src/common/Makefile b/src/common/Makefile
index f1da2ed13d..beb9830030 100644
--- a/src/common/Makefile
+++ b/src/common/Makefile
@@ -41,7 +41,7 @@ override CPPFLAGS += -DVAL_LDFLAGS_SL="\"$(LDFLAGS_SL)\""
override CPPFLAGS += -DVAL_LIBS="\"$(LIBS)\""
override CPPFLAGS := -DFRONTEND -I. -I$(top_srcdir)/src/common -I$(libpq_srcdir) $(CPPFLAGS)
-LIBS += $(PTHREAD_LIBS)
+LIBS += $(PTHREAD_LIBS) $(libpq_pgport)
OBJS_COMMON = \
archive.o \
diff --git a/src/include/libpq/auth.h b/src/include/libpq/auth.h
index 227b41daf6..22f6ab9f1d 100644
--- a/src/include/libpq/auth.h
+++ b/src/include/libpq/auth.h
@@ -16,6 +16,22 @@
#include "libpq/libpq-be.h"
+/*
+ * Maximum accepted size of GSS and SSPI authentication tokens.
+ * We also use this as a limit on ordinary password packet lengths.
+ *
+ * Kerberos tickets are usually quite small, but the TGTs issued by Windows
+ * domain controllers include an authorization field known as the Privilege
+ * Attribute Certificate (PAC), which contains the user's Windows permissions
+ * (group memberships etc.). The PAC is copied into all tickets obtained on
+ * the basis of this TGT (even those issued by Unix realms which the Windows
+ * realm trusts), and can be several kB in size. The maximum token size
+ * accepted by Windows systems is determined by the MaxAuthToken Windows
+ * registry setting. Microsoft recommends that it is not set higher than
+ * 65535 bytes, so that seems like a reasonable limit for us as well.
+ */
+#define PG_MAX_AUTH_TOKEN_LENGTH 65535
+
extern PGDLLIMPORT char *pg_krb_server_keyfile;
extern PGDLLIMPORT bool pg_krb_caseins_users;
extern PGDLLIMPORT bool pg_gss_accept_delegation;
@@ -23,6 +39,7 @@ extern PGDLLIMPORT bool pg_gss_accept_delegation;
extern void ClientAuthentication(Port *port);
extern void sendAuthRequest(Port *port, AuthRequest areq, const char *extradata,
int extralen);
+extern void set_authn_id(Port *port, const char *id);
/* Hook for plugins to get control in ClientAuthentication() */
typedef void (*ClientAuthentication_hook_type) (Port *, int);
diff --git a/src/include/libpq/hba.h b/src/include/libpq/hba.h
index 8ea837ae82..53c999a69f 100644
--- a/src/include/libpq/hba.h
+++ b/src/include/libpq/hba.h
@@ -39,7 +39,8 @@ typedef enum UserAuth
uaCert,
uaRADIUS,
uaPeer,
-#define USER_AUTH_LAST uaPeer /* Must be last value of this enum */
+ uaOAuth,
+#define USER_AUTH_LAST uaOAuth /* Must be last value of this enum */
} UserAuth;
/*
@@ -135,6 +136,9 @@ typedef struct HbaLine
char *radiusidentifiers_s;
List *radiusports;
char *radiusports_s;
+ char *oauth_issuer;
+ char *oauth_scope;
+ bool oauth_skip_usermap;
} HbaLine;
typedef struct IdentLine
diff --git a/src/include/libpq/oauth.h b/src/include/libpq/oauth.h
new file mode 100644
index 0000000000..6f98e84cc9
--- /dev/null
+++ b/src/include/libpq/oauth.h
@@ -0,0 +1,49 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth.h
+ * Interface to libpq/auth-oauth.c
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/libpq/oauth.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_OAUTH_H
+#define PG_OAUTH_H
+
+#include "libpq/libpq-be.h"
+#include "libpq/sasl.h"
+
+extern PGDLLIMPORT char *OAuthValidatorLibrary;
+
+typedef struct ValidatorModuleState
+{
+ void *private_data;
+} ValidatorModuleState;
+
+typedef struct ValidatorModuleResult
+{
+ bool authorized;
+ char *authn_id;
+} ValidatorModuleResult;
+
+typedef void (*ValidatorStartupCB) (ValidatorModuleState *state);
+typedef void (*ValidatorShutdownCB) (ValidatorModuleState *state);
+typedef ValidatorModuleResult *(*ValidatorValidateCB) (ValidatorModuleState *state, const char *token, const char *role);
+
+typedef struct OAuthValidatorCallbacks
+{
+ ValidatorStartupCB startup_cb;
+ ValidatorShutdownCB shutdown_cb;
+ ValidatorValidateCB validate_cb;
+} OAuthValidatorCallbacks;
+
+typedef const OAuthValidatorCallbacks *(*OAuthValidatorModuleInit) (void);
+extern PGDLLEXPORT const OAuthValidatorCallbacks *_PG_oauth_validator_module_init(void);
+
+/* Implementation */
+extern const pg_be_sasl_mech pg_be_oauth_mech;
+
+#endif /* PG_OAUTH_H */
diff --git a/src/include/libpq/sasl.h b/src/include/libpq/sasl.h
index 7a1f970cca..3f2c02b8f2 100644
--- a/src/include/libpq/sasl.h
+++ b/src/include/libpq/sasl.h
@@ -26,6 +26,14 @@
#define PG_SASL_EXCHANGE_SUCCESS 1
#define PG_SASL_EXCHANGE_FAILURE 2
+/*
+ * Maximum accepted size of SASL messages.
+ *
+ * The messages that the server or libpq generate are much smaller than this,
+ * but have some headroom.
+ */
+#define PG_MAX_SASL_MESSAGE_LENGTH 1024
+
/*
* Backend SASL mechanism callbacks.
*
@@ -127,6 +135,9 @@ typedef struct pg_be_sasl_mech
const char *input, int inputlen,
char **output, int *outputlen,
const char **logdetail);
+
+ /* The maximum size allowed for client SASLResponses. */
+ int max_message_length;
} pg_be_sasl_mech;
/* Common implementation for auth.c */
diff --git a/src/test/modules/Makefile b/src/test/modules/Makefile
index 256799f520..150dc1d908 100644
--- a/src/test/modules/Makefile
+++ b/src/test/modules/Makefile
@@ -11,6 +11,7 @@ SUBDIRS = \
dummy_index_am \
dummy_seclabel \
libpq_pipeline \
+ oauth_validator \
plsample \
spgist_name_ops \
test_bloomfilter \
diff --git a/src/test/modules/meson.build b/src/test/modules/meson.build
index d8fe059d23..60efa07b42 100644
--- a/src/test/modules/meson.build
+++ b/src/test/modules/meson.build
@@ -9,6 +9,7 @@ subdir('gin')
subdir('injection_points')
subdir('ldap_password_func')
subdir('libpq_pipeline')
+subdir('oauth_validator')
subdir('plsample')
subdir('spgist_name_ops')
subdir('ssl_passphrase_callback')
diff --git a/src/test/modules/oauth_validator/.gitignore b/src/test/modules/oauth_validator/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/src/test/modules/oauth_validator/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/src/test/modules/oauth_validator/Makefile b/src/test/modules/oauth_validator/Makefile
new file mode 100644
index 0000000000..655ce75796
--- /dev/null
+++ b/src/test/modules/oauth_validator/Makefile
@@ -0,0 +1,22 @@
+export PYTHON
+export with_oauth
+
+MODULES = validator
+PGFILEDESC = "validator - test OAuth validator module"
+
+NO_INSTALLCHECK = 1
+
+TAP_TESTS = 1
+
+REGRESS = validator
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = src/test/modules/oauth_validator
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/src/test/modules/oauth_validator/expected/validator.out b/src/test/modules/oauth_validator/expected/validator.out
new file mode 100644
index 0000000000..360caa2cb3
--- /dev/null
+++ b/src/test/modules/oauth_validator/expected/validator.out
@@ -0,0 +1,6 @@
+SELECT 1;
+ ?column?
+----------
+ 1
+(1 row)
+
diff --git a/src/test/modules/oauth_validator/meson.build b/src/test/modules/oauth_validator/meson.build
new file mode 100644
index 0000000000..3db2ddea1c
--- /dev/null
+++ b/src/test/modules/oauth_validator/meson.build
@@ -0,0 +1,37 @@
+# Copyright (c) 2024, PostgreSQL Global Development Group
+
+validator_sources = files(
+ 'validator.c',
+)
+
+if host_system == 'windows'
+ validator_sources += rc_lib_gen.process(win32ver_rc, extra_args: [
+ '--NAME', 'validator',
+ '--FILEDESC', 'validator - test OAuth validator module',])
+endif
+
+validator = shared_module('validator',
+ validator_sources,
+ kwargs: pg_test_mod_args,
+)
+test_install_libs += validator
+
+tests += {
+ 'name': 'oauth_validator',
+ 'sd': meson.current_source_dir(),
+ 'bd': meson.current_build_dir(),
+ 'regress': {
+ 'sql': [
+ 'validator',
+ ],
+ },
+ 'tap': {
+ 'tests': [
+ 't/001_server.pl',
+ ],
+ 'env': {
+ 'PYTHON': python.path(),
+ 'with_oauth': oauth_library,
+ },
+ },
+}
diff --git a/src/test/modules/oauth_validator/sql/validator.sql b/src/test/modules/oauth_validator/sql/validator.sql
new file mode 100644
index 0000000000..e0ac49d1ec
--- /dev/null
+++ b/src/test/modules/oauth_validator/sql/validator.sql
@@ -0,0 +1 @@
+SELECT 1;
diff --git a/src/test/modules/oauth_validator/t/001_server.pl b/src/test/modules/oauth_validator/t/001_server.pl
new file mode 100644
index 0000000000..e3cf3ac7f2
--- /dev/null
+++ b/src/test/modules/oauth_validator/t/001_server.pl
@@ -0,0 +1,79 @@
+
+# Copyright (c) 2021-2024, PostgreSQL Global Development Group
+
+use strict;
+use warnings FATAL => 'all';
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::OAuthServer;
+use Test::More;
+
+if ($ENV{with_oauth} ne 'curl')
+{
+ plan skip_all => 'client-side OAuth not supported by this build';
+}
+
+my $node = PostgreSQL::Test::Cluster->new('primary');
+$node->init;
+$node->append_conf('postgresql.conf', "log_connections = on\n");
+$node->append_conf('postgresql.conf', "shared_preload_libraries = 'validator'\n");
+$node->append_conf('postgresql.conf', "oauth_validator_library = 'validator'\n");
+$node->start;
+
+$node->safe_psql('postgres', 'CREATE USER test;');
+$node->safe_psql('postgres', 'CREATE USER testalt;');
+
+my $webserver = PostgreSQL::Test::OAuthServer->new();
+$webserver->run();
+
+my $port = $webserver->port();
+my $issuer = "127.0.0.1:$port";
+
+unlink($node->data_dir . '/pg_hba.conf');
+$node->append_conf('pg_hba.conf', qq{
+local all test oauth issuer="$issuer" scope="openid postgres"
+local all testalt oauth issuer="$issuer/alternate" scope="openid postgres alt"
+});
+$node->reload;
+
+my ($log_start, $log_end);
+$log_start = $node->wait_for_log(qr/reloading configuration files/);
+
+my $user = "test";
+$node->connect_ok("user=$user dbname=postgres oauth_client_id=f02c6361-0635", "connect",
+ expected_stderr => qr@Visit https://example\.com/ and enter the code: postgresuser@);
+
+$log_end = $node->wait_for_log(qr/connection authorized/, $log_start);
+$node->log_check("user $user: validator receives correct parameters", $log_start,
+ log_like => [
+ qr/oauth_validator: token="9243959234", role="$user"/,
+ qr/oauth_validator: issuer="\Q$issuer\E", scope="openid postgres"/,
+ ]);
+$node->log_check("user $user: validator sets authenticated identity", $log_start,
+ log_like => [
+ qr/connection authenticated: identity="test" method=oauth/,
+ ]);
+$log_start = $log_end;
+
+# The /alternate issuer uses slightly different parameters.
+$user = "testalt";
+$node->connect_ok("user=$user dbname=postgres oauth_client_id=f02c6361-0636", "connect",
+ expected_stderr => qr@Visit https://example\.org/ and enter the code: postgresuser@);
+
+$log_end = $node->wait_for_log(qr/connection authorized/, $log_start);
+$node->log_check("user $user: validator receives correct parameters", $log_start,
+ log_like => [
+ qr/oauth_validator: token="9243959234-alt", role="$user"/,
+ qr|oauth_validator: issuer="\Q$issuer/alternate\E", scope="openid postgres alt"|,
+ ]);
+$node->log_check("user $user: validator sets authenticated identity", $log_start,
+ log_like => [
+ qr/connection authenticated: identity="testalt" method=oauth/,
+ ]);
+$log_start = $log_end;
+
+$webserver->stop();
+$node->stop;
+
+done_testing();
diff --git a/src/test/modules/oauth_validator/t/oauth_server.py b/src/test/modules/oauth_validator/t/oauth_server.py
new file mode 100755
index 0000000000..77e3883a81
--- /dev/null
+++ b/src/test/modules/oauth_validator/t/oauth_server.py
@@ -0,0 +1,114 @@
+#! /usr/bin/env python3
+
+import http.server
+import json
+import os
+import sys
+
+
+class OAuthHandler(http.server.BaseHTTPRequestHandler):
+ JsonObject = dict[str, object] # TypeAlias is not available until 3.10
+
+ def _check_issuer(self):
+ """
+ Switches the behavior of the provider depending on the issuer URI.
+ """
+ self._alt_issuer = self.path.startswith("/alternate/")
+ if self._alt_issuer:
+ self.path = self.path.removeprefix("/alternate")
+
+ def do_GET(self):
+ self._check_issuer()
+
+ if self.path == "/.well-known/openid-configuration":
+ resp = self.config()
+ else:
+ self.send_error(404, "Not Found")
+ return
+
+ self._send_json(resp)
+
+ def do_POST(self):
+ self._check_issuer()
+
+ if self.path == "/authorize":
+ resp = self.authorization()
+ elif self.path == "/token":
+ resp = self.token()
+ else:
+ self.send_error(404, "Not Found")
+ return
+
+ self._send_json(resp)
+
+ def _send_json(self, js: JsonObject) -> None:
+ """
+ Sends the provided JSON dict as an application/json response.
+ """
+
+ resp = json.dumps(js).encode("ascii")
+
+ self.send_response(200, "OK")
+ self.send_header("Content-Type", "application/json")
+ self.send_header("Content-Length", str(len(resp)))
+ self.end_headers()
+
+ self.wfile.write(resp)
+
+ def config(self) -> JsonObject:
+ port = self.server.socket.getsockname()[1]
+ issuer = f"http://localhost:{port}"
+ if self._alt_issuer:
+ issuer += "/alternate"
+
+ return {
+ "issuer": issuer,
+ "token_endpoint": issuer + "/token",
+ "device_authorization_endpoint": issuer + "/authorize",
+ "response_types_supported": ["token"],
+ "subject_types_supported": ["public"],
+ "id_token_signing_alg_values_supported": ["RS256"],
+ "grant_types_supported": ["urn:ietf:params:oauth:grant-type:device_code"],
+ }
+
+ def authorization(self) -> JsonObject:
+ uri = "https://example.com/"
+ if self._alt_issuer:
+ uri = "https://example.org/"
+
+ return {
+ "device_code": "postgres",
+ "user_code": "postgresuser",
+ "interval": 0,
+ "verification_uri": uri,
+ "expires-in": 5,
+ }
+
+ def token(self) -> JsonObject:
+ token = "9243959234"
+ if self._alt_issuer:
+ token += "-alt"
+
+ return {
+ "access_token": token,
+ "token_type": "bearer",
+ }
+
+
+def main():
+ s = http.server.HTTPServer(("127.0.0.1", 0), OAuthHandler)
+
+ # Give the parent the port number to contact (this is also the signal that
+ # we're ready to receive requests).
+ port = s.socket.getsockname()[1]
+ print(port)
+
+ stdout = sys.stdout.fileno()
+ sys.stdout.close()
+ os.close(stdout)
+
+ s.serve_forever() # we expect our parent to send a termination signal
+
+
+if __name__ == "__main__":
+ main()
diff --git a/src/test/modules/oauth_validator/validator.c b/src/test/modules/oauth_validator/validator.c
new file mode 100644
index 0000000000..09a4bf61d2
--- /dev/null
+++ b/src/test/modules/oauth_validator/validator.c
@@ -0,0 +1,82 @@
+/*-------------------------------------------------------------------------
+ *
+ * validator.c
+ * Test module for serverside OAuth token validation callbacks
+ *
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/test/modules/oauth_validator/validator.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "fmgr.h"
+#include "libpq/oauth.h"
+#include "miscadmin.h"
+#include "utils/memutils.h"
+
+PG_MODULE_MAGIC;
+
+static void validator_startup(ValidatorModuleState *state);
+static void validator_shutdown(ValidatorModuleState *state);
+static ValidatorModuleResult * validate_token(ValidatorModuleState *state,
+ const char *token,
+ const char *role);
+
+static const OAuthValidatorCallbacks validator_callbacks = {
+ .startup_cb = validator_startup,
+ .shutdown_cb = validator_shutdown,
+ .validate_cb = validate_token
+};
+
+void
+_PG_init(void)
+{
+ /* no-op */
+}
+
+const OAuthValidatorCallbacks *
+_PG_oauth_validator_module_init(void)
+{
+ return &validator_callbacks;
+}
+
+#define PRIVATE_COOKIE ((void *) 13579)
+
+static void
+validator_startup(ValidatorModuleState *state)
+{
+ state->private_data = PRIVATE_COOKIE;
+}
+
+static void
+validator_shutdown(ValidatorModuleState *state)
+{
+ /* do nothing */
+}
+
+static ValidatorModuleResult *
+validate_token(ValidatorModuleState *state, const char *token, const char *role)
+{
+ ValidatorModuleResult *res;
+
+ /* Check to make sure our private state still exists. */
+ if (state->private_data != PRIVATE_COOKIE)
+ elog(ERROR, "oauth_validator: private state cookie changed to %p",
+ state->private_data);
+
+ res = palloc(sizeof(ValidatorModuleResult));
+
+ elog(LOG, "oauth_validator: token=\"%s\", role=\"%s\"", token, role);
+ elog(LOG, "oauth_validator: issuer=\"%s\", scope=\"%s\"",
+ MyProcPort->hba->oauth_issuer,
+ MyProcPort->hba->oauth_scope);
+
+ res->authorized = true;
+ res->authn_id = pstrdup(role);
+
+ return res;
+}
diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm
index 0135c5a795..f14839f4c5 100644
--- a/src/test/perl/PostgreSQL/Test/Cluster.pm
+++ b/src/test/perl/PostgreSQL/Test/Cluster.pm
@@ -2388,6 +2388,11 @@ instead of the default.
If this regular expression is set, matches it with the output generated.
+=item expected_stderr => B<value>
+
+If this regular expression is set, matches it against the standard error
+stream; otherwise the stderr must be empty.
+
=item log_like => [ qr/required message/ ]
=item log_unlike => [ qr/prohibited message/ ]
@@ -2431,7 +2436,14 @@ sub connect_ok
like($stdout, $params{expected_stdout}, "$test_name: stdout matches");
}
- is($stderr, "", "$test_name: no stderr");
+ if (defined($params{expected_stderr}))
+ {
+ like($stderr, $params{expected_stderr}, "$test_name: stderr matches");
+ }
+ else
+ {
+ is($stderr, "", "$test_name: no stderr");
+ }
$self->log_check($test_name, $log_location, %params);
}
diff --git a/src/test/perl/PostgreSQL/Test/OAuthServer.pm b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
new file mode 100644
index 0000000000..d96733f531
--- /dev/null
+++ b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
@@ -0,0 +1,65 @@
+#!/usr/bin/perl
+
+package PostgreSQL::Test::OAuthServer;
+
+use warnings;
+use strict;
+use threads;
+use Scalar::Util;
+use Socket;
+use IO::Select;
+
+local *server_socket;
+
+sub new
+{
+ my $class = shift;
+
+ my $self = {};
+ bless($self, $class);
+
+ return $self;
+}
+
+sub port
+{
+ my $self = shift;
+
+ return $self->{'port'};
+}
+
+sub run
+{
+ my $self = shift;
+ my $port;
+
+ my $pid = open(my $read_fh, "-|", $ENV{PYTHON}, "t/oauth_server.py")
+ // die "failed to start OAuth server: $!";
+
+ read($read_fh, $port, 7) // die "failed to read port number: $!";
+ chomp $port;
+ die "server did not advertise a valid port"
+ unless Scalar::Util::looks_like_number($port);
+
+ $self->{'pid'} = $pid;
+ $self->{'port'} = $port;
+ $self->{'child'} = $read_fh;
+
+ print("# OAuth provider (PID $pid) is listening on port $port\n");
+}
+
+sub stop
+{
+ my $self = shift;
+
+ print("# Sending SIGTERM to OAuth provider PID: $self->{'pid'}\n");
+
+ kill(15, $self->{'pid'});
+ $self->{'pid'} = undef;
+
+ # Closing the popen() handle waits for the process to exit.
+ close($self->{'child'});
+ $self->{'child'} = undef;
+}
+
+1;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 8f323da558..b02ee48898 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1715,6 +1715,7 @@ NumericSortSupport
NumericSumAccum
NumericVar
OAuthStep
+OAuthValidatorCallbacks
OM_uint32
OP
OSAPerGroupState
@@ -3055,6 +3056,7 @@ VacuumRelation
VacuumStmt
ValidIOData
ValidateIndexState
+ValidatorModuleState
ValuesScan
ValuesScanState
Var
@@ -3647,6 +3649,7 @@ normal_rand_fctx
nsphash_hash
ntile_context
numeric
+oauth_state
object_access_hook_type
object_access_hook_type_str
off_t
--
2.34.1
v23-0004-libpq-add-OAUTHBEARER-SASL-mechanism.patch (application/octet-stream, 106.3K) download
From 1639f7eb9a9184ae58dcd20f8184dd01dbd820b3 Mon Sep 17 00:00:00 2001
From: Jacob Champion <pchampion@vmware.com>
Date: Tue, 13 Apr 2021 10:27:27 -0700
Subject: [PATCH v23 4/6] libpq: add OAUTHBEARER SASL mechanism
DO NOT USE THIS PROOF OF CONCEPT IN PRODUCTION.
Implement OAUTHBEARER (RFC 7628) and OAuth 2.0 Device Authorization
Grants (RFC 8628) on the client side. When speaking to a OAuth-enabled
server, it looks a bit like this:
$ psql 'host=example.org oauth_client_id=f02c6361-0635-...'
Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG
The OAuth issuer must support device authorization. No other OAuth flows
are currently implemented (but clients may provide their own flows; see
below).
The client implementation requires libcurl and its development headers.
Pass `curl` to --with-oauth/-Doauth during configuration.
Thomas Munro wrote the kqueue() implementation for oauth-curl; thanks!
= PQauthDataHook =
Clients may override two pieces of OAuth handling using the new
PQsetAuthDataHook():
- PQAUTHDATA_PROMPT_OAUTH_DEVICE: replaces the default user prompt to
standard error when using the builtin device authorization flow
- PQAUTHDATA_OAUTH_BEARER_TOKEN: replaces the entire OAuth flow with a
custom asynchronous implementation
In general, a hook implementation should examine the incoming `type` to
decide whether or not to handle a specific piece of authdata; if not, it
should delegate to the previous hook in the chain (retrievable via
PQgetAuthDataHook()). Otherwise, it should return an integer > 0 and
follow the authdata-specific instructions. Returning an integer < 0
signals an error condition and abandons the connection attempt.
== PQAUTHDATA_PROMPT_OAUTH_DEVICE ==
The hook should display the device prompt (URL + code) using whatever
method it prefers.
== PQAUTHDATA_OAUTH_BEARER_TOKEN ==
The hook should either directly return a Bearer token for the current
user/issuer/scope combination, if one is available without blocking, or
else set up an asynchronous callback to retrieve one. See the
documentation for PQoauthBearerRequest.
Several TODOs:
- don't retry forever if the server won't accept our token
- perform several sanity checks on the OAuth issuer's responses
- handle cases where the client has been set up with an issuer and
scope, but the Postgres server wants to use something different
- improve error debuggability during the OAuth handshake
- fix libcurl initialization thread-safety
- harden the libcurl flow implementation
- figure out pgsocket/int difference on Windows
- fix intermittent failure in the cleanup callback tests (race
condition?)
- support require_auth
- ...and more.
Co-authored-by: Daniel Gustafsson <daniel@yesql.se>
---
configure | 110 ++
configure.ac | 28 +
meson.build | 29 +
meson_options.txt | 4 +
src/Makefile.global.in | 1 +
src/include/common/oauth-common.h | 19 +
src/include/pg_config.h.in | 9 +
src/interfaces/libpq/Makefile | 10 +-
src/interfaces/libpq/exports.txt | 3 +
src/interfaces/libpq/fe-auth-oauth-curl.c | 1982 +++++++++++++++++++++
src/interfaces/libpq/fe-auth-oauth.c | 659 +++++++
src/interfaces/libpq/fe-auth-oauth.h | 42 +
src/interfaces/libpq/fe-auth-sasl.h | 10 +-
src/interfaces/libpq/fe-auth-scram.c | 6 +-
src/interfaces/libpq/fe-auth.c | 105 +-
src/interfaces/libpq/fe-auth.h | 9 +-
src/interfaces/libpq/fe-connect.c | 85 +-
src/interfaces/libpq/fe-misc.c | 7 +-
src/interfaces/libpq/libpq-fe.h | 75 +
src/interfaces/libpq/libpq-int.h | 14 +
src/interfaces/libpq/meson.build | 7 +
src/makefiles/meson.build | 1 +
src/tools/pgindent/typedefs.list | 10 +
23 files changed, 3199 insertions(+), 26 deletions(-)
create mode 100644 src/include/common/oauth-common.h
create mode 100644 src/interfaces/libpq/fe-auth-oauth-curl.c
create mode 100644 src/interfaces/libpq/fe-auth-oauth.c
create mode 100644 src/interfaces/libpq/fe-auth-oauth.h
diff --git a/configure b/configure
index 76f06bd8fd..864bb9b7c3 100755
--- a/configure
+++ b/configure
@@ -715,6 +715,7 @@ with_uuid
with_readline
with_systemd
with_selinux
+with_oauth
with_ldap
with_krb_srvnam
krb_srvtab
@@ -861,6 +862,7 @@ with_krb_srvnam
with_pam
with_bsd_auth
with_ldap
+with_oauth
with_bonjour
with_selinux
with_systemd
@@ -1571,6 +1573,7 @@ Optional Packages:
--with-pam build with PAM support
--with-bsd-auth build with BSD Authentication support
--with-ldap build with LDAP support
+ --with-oauth=LIB use LIB for OAuth 2.0 support (curl)
--with-bonjour build with Bonjour support
--with-selinux build with SELinux support
--with-systemd build with systemd support
@@ -8488,6 +8491,52 @@ $as_echo "$with_ldap" >&6; }
+#
+# OAuth 2.0
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with OAuth support" >&5
+$as_echo_n "checking whether to build with OAuth support... " >&6; }
+
+
+
+# Check whether --with-oauth was given.
+if test "${with_oauth+set}" = set; then :
+ withval=$with_oauth;
+ case $withval in
+ yes)
+ as_fn_error $? "argument required for --with-oauth option" "$LINENO" 5
+ ;;
+ no)
+ as_fn_error $? "argument required for --with-oauth option" "$LINENO" 5
+ ;;
+ *)
+
+ ;;
+ esac
+
+fi
+
+
+if test x"$with_oauth" = x"" ; then
+ with_oauth=no
+fi
+
+if test x"$with_oauth" = x"curl"; then
+
+$as_echo "#define USE_OAUTH 1" >>confdefs.h
+
+
+$as_echo "#define USE_OAUTH_CURL 1" >>confdefs.h
+
+elif test x"$with_oauth" != x"no"; then
+ as_fn_error $? "--with-oauth must specify curl" "$LINENO" 5
+fi
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_oauth" >&5
+$as_echo "$with_oauth" >&6; }
+
+
+
#
# Bonjour
#
@@ -13040,6 +13089,56 @@ fi
+if test "$with_oauth" = curl ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for curl_multi_init in -lcurl" >&5
+$as_echo_n "checking for curl_multi_init in -lcurl... " >&6; }
+if ${ac_cv_lib_curl_curl_multi_init+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-lcurl $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char curl_multi_init ();
+int
+main ()
+{
+return curl_multi_init ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_curl_curl_multi_init=yes
+else
+ ac_cv_lib_curl_curl_multi_init=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_curl_curl_multi_init" >&5
+$as_echo "$ac_cv_lib_curl_curl_multi_init" >&6; }
+if test "x$ac_cv_lib_curl_curl_multi_init" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBCURL 1
+_ACEOF
+
+ LIBS="-lcurl $LIBS"
+
+else
+ as_fn_error $? "library 'curl' is required for --with-oauth=curl" "$LINENO" 5
+fi
+
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for security_compute_create_name in -lselinux" >&5
@@ -14065,6 +14164,17 @@ fi
done
+fi
+
+if test "$with_oauth" = curl; then
+ ac_fn_c_check_header_mongrel "$LINENO" "curl/curl.h" "ac_cv_header_curl_curl_h" "$ac_includes_default"
+if test "x$ac_cv_header_curl_curl_h" = xyes; then :
+
+else
+ as_fn_error $? "header file <curl/curl.h> is required for OAuth" "$LINENO" 5
+fi
+
+
fi
if test "$PORTNAME" = "win32" ; then
diff --git a/configure.ac b/configure.ac
index ab2d51c21c..6aceed898c 100644
--- a/configure.ac
+++ b/configure.ac
@@ -927,6 +927,26 @@ AC_MSG_RESULT([$with_ldap])
AC_SUBST(with_ldap)
+#
+# OAuth 2.0
+#
+AC_MSG_CHECKING([whether to build with OAuth support])
+PGAC_ARG_REQ(with, oauth, [LIB], [use LIB for OAuth 2.0 support (curl)])
+if test x"$with_oauth" = x"" ; then
+ with_oauth=no
+fi
+
+if test x"$with_oauth" = x"curl"; then
+ AC_DEFINE([USE_OAUTH], 1, [Define to 1 to build with OAuth 2.0 support. (--with-oauth)])
+ AC_DEFINE([USE_OAUTH_CURL], 1, [Define to 1 to use libcurl for OAuth support.])
+elif test x"$with_oauth" != x"no"; then
+ AC_MSG_ERROR([--with-oauth must specify curl])
+fi
+
+AC_MSG_RESULT([$with_oauth])
+AC_SUBST(with_oauth)
+
+
#
# Bonjour
#
@@ -1423,6 +1443,10 @@ fi
AC_SUBST(LDAP_LIBS_FE)
AC_SUBST(LDAP_LIBS_BE)
+if test "$with_oauth" = curl ; then
+ AC_CHECK_LIB(curl, curl_multi_init, [], [AC_MSG_ERROR([library 'curl' is required for --with-oauth=curl])])
+fi
+
# for contrib/sepgsql
if test "$with_selinux" = yes; then
AC_CHECK_LIB(selinux, security_compute_create_name, [],
@@ -1614,6 +1638,10 @@ elif test "$with_uuid" = ossp ; then
[AC_MSG_ERROR([header file <ossp/uuid.h> or <uuid.h> is required for OSSP UUID])])])
fi
+if test "$with_oauth" = curl; then
+ AC_CHECK_HEADER(curl/curl.h, [], [AC_MSG_ERROR([header file <curl/curl.h> is required for OAuth])])
+fi
+
if test "$PORTNAME" = "win32" ; then
AC_CHECK_HEADERS(crtdefs.h)
fi
diff --git a/meson.build b/meson.build
index 5387bb6d5f..b01bc91c09 100644
--- a/meson.build
+++ b/meson.build
@@ -840,6 +840,33 @@ endif
+###############################################################
+# Library: oauth
+###############################################################
+
+oauth = not_found_dep
+oauth_library = 'none'
+oauthopt = get_option('oauth')
+
+if oauthopt == 'auto' and auto_features.disabled()
+ oauthopt = 'none'
+endif
+
+if oauthopt in ['auto', 'curl']
+ oauth = dependency('libcurl', required: (oauthopt == 'curl'))
+
+ if oauth.found()
+ oauth_library = 'curl'
+ cdata.set('USE_OAUTH', 1)
+ cdata.set('USE_OAUTH_CURL', 1)
+ endif
+endif
+
+if oauthopt == 'auto' and auto_features.enabled() and not oauth.found()
+ error('no OAuth implementation library found')
+endif
+
+
###############################################################
# Library: Tcl (for pltcl)
#
@@ -2974,6 +3001,7 @@ libpq_deps += [
gssapi,
ldap_r,
libintl,
+ oauth,
ssl,
]
@@ -3645,6 +3673,7 @@ if meson.version().version_compare('>=0.57')
'llvm': llvm,
'lz4': lz4,
'nls': libintl,
+ 'oauth': oauth,
'openssl': ssl,
'pam': pam,
'plperl': perl_dep,
diff --git a/meson_options.txt b/meson_options.txt
index 246cecf382..3ffe1f52c2 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -124,6 +124,10 @@ option('lz4', type: 'feature', value: 'auto',
option('nls', type: 'feature', value: 'auto',
description: 'Native language support')
+option('oauth', type : 'combo', choices : ['auto', 'none', 'curl'],
+ value: 'auto',
+ description: 'use LIB for OAuth 2.0 support (curl)')
+
option('pam', type: 'feature', value: 'auto',
description: 'PAM support')
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 83b91fe916..6f6174811d 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -193,6 +193,7 @@ with_ldap = @with_ldap@
with_libxml = @with_libxml@
with_libxslt = @with_libxslt@
with_llvm = @with_llvm@
+with_oauth = @with_oauth@
with_system_tzdata = @with_system_tzdata@
with_uuid = @with_uuid@
with_zlib = @with_zlib@
diff --git a/src/include/common/oauth-common.h b/src/include/common/oauth-common.h
new file mode 100644
index 0000000000..5ff3488bfb
--- /dev/null
+++ b/src/include/common/oauth-common.h
@@ -0,0 +1,19 @@
+/*-------------------------------------------------------------------------
+ *
+ * oauth-common.h
+ * Declarations for helper functions used for OAuth/OIDC authentication
+ *
+ * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/common/oauth-common.h
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef OAUTH_COMMON_H
+#define OAUTH_COMMON_H
+
+/* Name of SASL mechanism per IANA */
+#define OAUTHBEARER_NAME "OAUTHBEARER"
+
+#endif /* OAUTH_COMMON_H */
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index f8d3e3b6b8..1ee7f3731d 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -246,6 +246,9 @@
/* Define to 1 if you have the `crypto' library (-lcrypto). */
#undef HAVE_LIBCRYPTO
+/* Define to 1 if you have the `curl' library (-lcurl). */
+#undef HAVE_LIBCURL
+
/* Define to 1 if you have the `ldap' library (-lldap). */
#undef HAVE_LIBLDAP
@@ -726,6 +729,12 @@
/* Define to select named POSIX semaphores. */
#undef USE_NAMED_POSIX_SEMAPHORES
+/* Define to 1 to build with OAuth 2.0 support. (--with-oauth) */
+#undef USE_OAUTH
+
+/* Define to 1 to use libcurl for OAuth support. */
+#undef USE_OAUTH_CURL
+
/* Define to 1 to build with OpenSSL support. (--with-ssl=openssl) */
#undef USE_OPENSSL
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index b36a765764..0e0369cb63 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -61,6 +61,14 @@ OBJS += \
fe-secure-gssapi.o
endif
+ifneq ($(with_oauth),no)
+OBJS += fe-auth-oauth.o
+
+ifeq ($(with_oauth),curl)
+OBJS += fe-auth-oauth-curl.o
+endif
+endif
+
ifeq ($(PORTNAME), cygwin)
override shlib = cyg$(NAME)$(DLSUFFIX)
endif
@@ -79,7 +87,7 @@ endif
# that are built correctly for use in a shlib.
SHLIB_LINK_INTERNAL = -lpgcommon_shlib -lpgport_shlib
ifneq ($(PORTNAME), win32)
-SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
+SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lcurl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
else
SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi32 -lssl -lsocket -lnsl -lresolv -lintl -lm $(PTHREAD_LIBS), $(LIBS)) $(LDAP_LIBS_FE)
endif
diff --git a/src/interfaces/libpq/exports.txt b/src/interfaces/libpq/exports.txt
index 5d8213e0b5..eb8f9d65a1 100644
--- a/src/interfaces/libpq/exports.txt
+++ b/src/interfaces/libpq/exports.txt
@@ -205,3 +205,6 @@ PQcancelFinish 202
PQsocketPoll 203
PQsetChunkedRowsMode 204
PQgetCurrentTimeUSec 205
+PQsetAuthDataHook 206
+PQgetAuthDataHook 207
+PQdefaultAuthDataHook 208
diff --git a/src/interfaces/libpq/fe-auth-oauth-curl.c b/src/interfaces/libpq/fe-auth-oauth-curl.c
new file mode 100644
index 0000000000..0504f96e4e
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth-curl.c
@@ -0,0 +1,1982 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth-curl.c
+ * The libcurl implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth-curl.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include <curl/curl.h>
+#include <math.h>
+#ifdef HAVE_SYS_EPOLL_H
+#include <sys/epoll.h>
+#include <sys/timerfd.h>
+#endif
+#ifdef HAVE_SYS_EVENT_H
+#include <sys/event.h>
+#endif
+#include <unistd.h>
+
+#include "common/jsonapi.h"
+#include "fe-auth.h"
+#include "fe-auth-oauth.h"
+#include "libpq-int.h"
+#include "mb/pg_wchar.h"
+
+/*
+ * Parsed JSON Representations
+ *
+ * As a general rule, we parse and cache only the fields we're currently using.
+ * When adding new fields, ensure the corresponding free_*() function is updated
+ * too.
+ */
+
+/*
+ * The OpenID Provider configuration (alternatively named "authorization server
+ * metadata") jointly described by OpenID Connect Discovery 1.0 and RFC 8414:
+ *
+ * https://openid.net/specs/openid-connect-discovery-1_0.html
+ * https://www.rfc-editor.org/rfc/rfc8414#section-3.2
+ */
+struct provider
+{
+ char *issuer;
+ char *token_endpoint;
+ char *device_authorization_endpoint;
+ struct curl_slist *grant_types_supported;
+};
+
+static void
+free_provider(struct provider *provider)
+{
+ free(provider->issuer);
+ free(provider->token_endpoint);
+ free(provider->device_authorization_endpoint);
+ curl_slist_free_all(provider->grant_types_supported);
+}
+
+/*
+ * The Device Authorization response, described by RFC 8628:
+ *
+ * https://www.rfc-editor.org/rfc/rfc8628#section-3.2
+ */
+struct device_authz
+{
+ char *device_code;
+ char *user_code;
+ char *verification_uri;
+ char *interval_str;
+
+ /* Fields below are parsed from the corresponding string above. */
+ int interval;
+};
+
+static void
+free_device_authz(struct device_authz *authz)
+{
+ free(authz->device_code);
+ free(authz->user_code);
+ free(authz->verification_uri);
+ free(authz->interval_str);
+}
+
+/*
+ * The Token Endpoint error response, as described by RFC 6749:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-5.2
+ *
+ * Note that this response type can also be returned from the Device
+ * Authorization Endpoint.
+ */
+struct token_error
+{
+ char *error;
+ char *error_description;
+};
+
+static void
+free_token_error(struct token_error *err)
+{
+ free(err->error);
+ free(err->error_description);
+}
+
+/*
+ * The Access Token response, as described by RFC 6749:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-4.1.4
+ *
+ * During the Device Authorization flow, several temporary errors are expected
+ * as part of normal operation. To make it easy to handle these in the happy
+ * path, this contains an embedded token_error that is filled in if needed.
+ */
+struct token
+{
+ /* for successful responses */
+ char *access_token;
+ char *token_type;
+
+ /* for error responses */
+ struct token_error err;
+};
+
+static void
+free_token(struct token *tok)
+{
+ free(tok->access_token);
+ free(tok->token_type);
+ free_token_error(&tok->err);
+}
+
+/*
+ * Asynchronous State
+ */
+
+/* States for the overall async machine. */
+typedef enum
+{
+ OAUTH_STEP_INIT,
+ OAUTH_STEP_DISCOVERY,
+ OAUTH_STEP_DEVICE_AUTHORIZATION,
+ OAUTH_STEP_TOKEN_REQUEST,
+ OAUTH_STEP_WAIT_INTERVAL,
+} OAuthStep;
+
+/*
+ * The async_ctx holds onto state that needs to persist across multiple calls to
+ * pg_fe_run_oauth_flow(). Almost everything interacts with this in some way.
+ */
+struct async_ctx
+{
+ OAuthStep step; /* where are we in the flow? */
+
+#ifdef HAVE_SYS_EPOLL_H
+ int timerfd; /* a timerfd for signaling async timeouts */
+#endif
+ pgsocket mux; /* the multiplexer socket containing all
+ * descriptors tracked by cURL, plus the
+ * timerfd */
+ CURLM *curlm; /* top-level multi handle for cURL operations */
+ CURL *curl; /* the (single) easy handle for serial
+ * requests */
+
+ struct curl_slist *headers; /* common headers for all requests */
+ PQExpBufferData work_data; /* scratch buffer for general use (remember to
+ * clear out prior contents first!) */
+
+ /*------
+ * Since a single logical operation may stretch across multiple calls to
+ * our entry point, errors have three parts:
+ *
+ * - errctx: an optional static string, describing the global operation
+ * currently in progress. It'll be translated for you.
+ *
+ * - errbuf: contains the actual error message. Generally speaking, use
+ * actx_error[_str] to manipulate this. This must be filled
+ * with something useful on an error.
+ *
+ * - curl_err: an optional static error buffer used by cURL to put
+ * detailed information about failures. Unfortunately
+ * untranslatable.
+ *
+ * These pieces will be combined into a single error message looking
+ * something like the following, with errctx and/or curl_err omitted when
+ * absent:
+ *
+ * connection to server ... failed: errctx: errbuf (curl_err)
+ */
+ const char *errctx; /* not freed; must point to static allocation */
+ PQExpBufferData errbuf;
+ char curl_err[CURL_ERROR_SIZE];
+
+ /*
+ * These documents need to survive over multiple calls, and are therefore
+ * cached directly in the async_ctx.
+ */
+ struct provider provider;
+ struct device_authz authz;
+
+ bool user_prompted; /* have we already sent the authz prompt? */
+};
+
+/*
+ * Frees the async_ctx, which is stored directly on the PGconn. This is called
+ * during pqDropConnection() so that we don't leak resources even if
+ * PQconnectPoll() never calls us back.
+ *
+ * TODO: we should probably call this at the end of a successful authentication,
+ * too, to proactively free up resources.
+ */
+static void
+free_curl_async_ctx(PGconn *conn, void *ctx)
+{
+ struct async_ctx *actx = ctx;
+
+ Assert(actx); /* oauth_free() shouldn't call us otherwise */
+
+ /*
+ * TODO: in general, none of the error cases below should ever happen if
+ * we have no bugs above. But if we do hit them, surfacing those errors
+ * somehow might be the only way to have a chance to debug them. What's
+ * the best way to do that? Assertions? Spraying messages on stderr?
+ * Bubbling an error code to the top? Appending to the connection's error
+ * message only helps if the bug caused a connection failure; otherwise
+ * it'll be buried...
+ */
+
+ if (actx->curlm && actx->curl)
+ {
+ CURLMcode err = curl_multi_remove_handle(actx->curlm, actx->curl);
+
+ if (err)
+ libpq_append_conn_error(conn,
+ "cURL easy handle removal failed: %s",
+ curl_multi_strerror(err));
+ }
+
+ if (actx->curl)
+ {
+ /*
+ * curl_multi_cleanup() doesn't free any associated easy handles; we
+ * need to do that separately. We only ever have one easy handle per
+ * multi handle.
+ */
+ curl_easy_cleanup(actx->curl);
+ }
+
+ if (actx->curlm)
+ {
+ CURLMcode err = curl_multi_cleanup(actx->curlm);
+
+ if (err)
+ libpq_append_conn_error(conn,
+ "cURL multi handle cleanup failed: %s",
+ curl_multi_strerror(err));
+ }
+
+ free_provider(&actx->provider);
+ free_device_authz(&actx->authz);
+
+ curl_slist_free_all(actx->headers);
+ termPQExpBuffer(&actx->work_data);
+ termPQExpBuffer(&actx->errbuf);
+
+ if (actx->mux != PGINVALID_SOCKET)
+ close(actx->mux);
+#ifdef HAVE_SYS_EPOLL_H
+ if (actx->timerfd >= 0)
+ close(actx->timerfd);
+#endif
+
+ free(actx);
+}
+
+/*
+ * Macros for manipulating actx->errbuf. actx_error() translates and formats a
+ * string for you; actx_error_str() appends a string directly without
+ * translation.
+ */
+
+#define actx_error(ACTX, FMT, ...) \
+ appendPQExpBuffer(&(ACTX)->errbuf, libpq_gettext(FMT), ##__VA_ARGS__)
+
+#define actx_error_str(ACTX, S) \
+ appendPQExpBufferStr(&(ACTX)->errbuf, S)
+
+/*
+ * Macros for getting and setting state for the connection's two cURL handles,
+ * so you don't have to write out the error handling every time.
+ */
+
+#define CHECK_MSETOPT(ACTX, OPT, VAL, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLMcode _setopterr = curl_multi_setopt(_actx->curlm, OPT, VAL); \
+ if (_setopterr) { \
+ actx_error(_actx, "failed to set %s on OAuth connection: %s",\
+ #OPT, curl_multi_strerror(_setopterr)); \
+ FAILACTION; \
+ } \
+ } while (0)
+
+#define CHECK_SETOPT(ACTX, OPT, VAL, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLcode _setopterr = curl_easy_setopt(_actx->curl, OPT, VAL); \
+ if (_setopterr) { \
+ actx_error(_actx, "failed to set %s on OAuth connection: %s",\
+ #OPT, curl_easy_strerror(_setopterr)); \
+ FAILACTION; \
+ } \
+ } while (0)
+
+#define CHECK_GETINFO(ACTX, INFO, OUT, FAILACTION) \
+ do { \
+ struct async_ctx *_actx = (ACTX); \
+ CURLcode _getinfoerr = curl_easy_getinfo(_actx->curl, INFO, OUT); \
+ if (_getinfoerr) { \
+ actx_error(_actx, "failed to get %s from OAuth response: %s",\
+ #INFO, curl_easy_strerror(_getinfoerr)); \
+ FAILACTION; \
+ } \
+ } while (0)
+
+/*
+ * General JSON Parsing for OAuth Responses
+ */
+
+/*
+ * Represents a single name/value pair in a JSON object. This is the primary
+ * interface to parse_oauth_json().
+ *
+ * All fields are stored internally as strings or lists of strings, so clients
+ * have to explicitly parse other scalar types (though they will have gone
+ * through basic lexical validation). Storing nested objects is not currently
+ * supported, nor is parsing arrays of anything other than strings.
+ */
+struct json_field
+{
+ const char *name; /* name (key) of the member */
+
+ JsonTokenType type; /* currently supports JSON_TOKEN_STRING,
+ * JSON_TOKEN_NUMBER, and
+ * JSON_TOKEN_ARRAY_START */
+ union
+ {
+ char **scalar; /* for all scalar types */
+ struct curl_slist **array; /* for type == JSON_TOKEN_ARRAY_START */
+ };
+
+ bool required; /* REQUIRED field, or just OPTIONAL? */
+};
+
+/* Documentation macros for json_field.required. */
+#define REQUIRED true
+#define OPTIONAL false
+
+/* Parse state for parse_oauth_json(). */
+struct oauth_parse
+{
+ PQExpBuffer errbuf; /* detail message for JSON_SEM_ACTION_FAILED */
+ int nested; /* nesting level (zero is the top) */
+
+ const struct json_field *fields; /* field definition array */
+ const struct json_field *active; /* points inside the fields array */
+};
+
+#define oauth_parse_set_error(ctx, fmt, ...) \
+ appendPQExpBuffer((ctx)->errbuf, libpq_gettext(fmt), ##__VA_ARGS__)
+
+static void
+report_type_mismatch(struct oauth_parse *ctx)
+{
+ char *msgfmt;
+
+ Assert(ctx->active);
+
+ /*
+ * At the moment, the only fields we're interested in are strings,
+ * numbers, and arrays of strings.
+ */
+ switch (ctx->active->type)
+ {
+ case JSON_TOKEN_STRING:
+ msgfmt = "field \"%s\" must be a string";
+ break;
+
+ case JSON_TOKEN_NUMBER:
+ msgfmt = "field \"%s\" must be a number";
+ break;
+
+ case JSON_TOKEN_ARRAY_START:
+ msgfmt = "field \"%s\" must be an array of strings";
+ break;
+
+ default:
+ Assert(false);
+ msgfmt = "field \"%s\" has unexpected type";
+ }
+
+ oauth_parse_set_error(ctx, msgfmt, ctx->active->name);
+}
+
+static JsonParseErrorType
+oauth_json_object_start(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (ctx->active)
+ {
+ /*
+ * Currently, none of the fields we're interested in can be or contain
+ * objects, so we can reject this case outright.
+ */
+ report_type_mismatch(ctx);
+ return JSON_SEM_ACTION_FAILED;
+ }
+
+ ++ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_field_start(void *state, char *name, bool isnull)
+{
+ struct oauth_parse *ctx = state;
+
+ /* We care only about the top-level fields. */
+ if (ctx->nested == 1)
+ {
+ const struct json_field *field = ctx->fields;
+
+ /*
+ * We should never start parsing a new field while a previous one is
+ * still active.
+ *
+ * TODO: this code relies on assertions too much. We need to exit
+ * sanely on internal logic errors, to avoid turning bugs into
+ * vulnerabilities.
+ */
+ Assert(!ctx->active);
+
+ while (field->name)
+ {
+ if (strcmp(name, field->name) == 0)
+ {
+ ctx->active = field;
+ break;
+ }
+
+ ++field;
+ }
+ }
+
+ free(name);
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_end(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_start(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (!ctx->nested)
+ {
+ oauth_parse_set_error(ctx, "top-level element must be an object");
+ return JSON_SEM_ACTION_FAILED;
+ }
+
+ if (ctx->active)
+ {
+ if (ctx->active->type != JSON_TOKEN_ARRAY_START
+ /* The arrays we care about must not have arrays as values. */
+ || ctx->nested > 1)
+ {
+ report_type_mismatch(ctx);
+ return JSON_SEM_ACTION_FAILED;
+ }
+ }
+
+ ++ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_end(void *state)
+{
+ struct oauth_parse *ctx = state;
+
+ if (ctx->active)
+ {
+ /*
+ * This assumes that no target arrays can contain other arrays, which
+ * we check in the array_start callback.
+ */
+ Assert(ctx->nested == 2);
+ Assert(ctx->active->type == JSON_TOKEN_ARRAY_START);
+
+ ctx->active = NULL;
+ }
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_scalar(void *state, char *token, JsonTokenType type)
+{
+ struct oauth_parse *ctx = state;
+ JsonParseErrorType result = JSON_SUCCESS;
+
+ if (!ctx->nested)
+ {
+ oauth_parse_set_error(ctx, "top-level element must be an object");
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ if (ctx->active)
+ {
+ JsonTokenType expected;
+
+ /*
+ * Make sure this matches what the active field expects. Arrays must
+ * contain only strings with the current implementation.
+ */
+ if (ctx->active->type == JSON_TOKEN_ARRAY_START)
+ expected = JSON_TOKEN_STRING;
+ else
+ expected = ctx->active->type;
+
+ if (type != expected)
+ {
+ report_type_mismatch(ctx);
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ /*
+ * FIXME if the JSON field is duplicated, we'll leak the prior value.
+ * Error out in that case instead.
+ */
+ if (ctx->active->type != JSON_TOKEN_ARRAY_START)
+ {
+ Assert(ctx->nested == 1);
+
+ *ctx->active->scalar = token;
+ ctx->active = NULL;
+
+ return JSON_SUCCESS; /* don't free the token */
+ }
+ else /* ctx->target_array */
+ {
+ struct curl_slist *temp;
+
+ Assert(ctx->nested == 2);
+
+ temp = curl_slist_append(*ctx->active->array, token);
+ if (!temp)
+ {
+ oauth_parse_set_error(ctx, "out of memory");
+ result = JSON_SEM_ACTION_FAILED;
+ goto cleanup;
+ }
+
+ *ctx->active->array = temp;
+
+ /*
+ * Note that curl_slist_append() makes a copy of the token, so we
+ * can free it below.
+ */
+ }
+ }
+ else
+ {
+ /* otherwise we just ignore it */
+ }
+
+cleanup:
+ free(token);
+ return result;
+}
+
+/*
+ * A helper function for general JSON parsing. fields is the array of field
+ * definitions with their backing pointers. The response will be parsed from
+ * actx->curl and actx->work_data (as set up by start_request()), and any
+ * parsing errors will be placed into actx->errbuf.
+ */
+static bool
+parse_oauth_json(struct async_ctx *actx, const struct json_field *fields)
+{
+ PQExpBuffer resp = &actx->work_data;
+ char *content_type;
+ JsonLexContext lex = {0};
+ JsonSemAction sem = {0};
+ JsonParseErrorType err;
+ struct oauth_parse ctx = {0};
+ bool success = false;
+
+ /* Make sure the server thinks it's given us JSON. */
+ CHECK_GETINFO(actx, CURLINFO_CONTENT_TYPE, &content_type, return false);
+
+ if (!content_type)
+ {
+ actx_error(actx, "no content type was provided");
+ goto cleanup;
+ }
+ else if (strcasecmp(content_type, "application/json") != 0)
+ {
+ actx_error(actx, "unexpected content type \"%s\"", content_type);
+ goto cleanup;
+ }
+
+ if (strlen(resp->data) != resp->len)
+ {
+ actx_error(actx, "response contains embedded NULLs");
+ goto cleanup;
+ }
+
+ makeJsonLexContextCstringLen(&lex, resp->data, resp->len, PG_UTF8, true);
+
+ ctx.errbuf = &actx->errbuf;
+ ctx.fields = fields;
+ sem.semstate = &ctx;
+
+ sem.object_start = oauth_json_object_start;
+ sem.object_field_start = oauth_json_object_field_start;
+ sem.object_end = oauth_json_object_end;
+ sem.array_start = oauth_json_array_start;
+ sem.array_end = oauth_json_array_end;
+ sem.scalar = oauth_json_scalar;
+
+ err = pg_parse_json(&lex, &sem);
+
+ if (err != JSON_SUCCESS)
+ {
+ /*
+ * For JSON_SEM_ACTION_FAILED, we've already written the error
+ * message. Other errors come directly from pg_parse_json(), already
+ * translated.
+ */
+ if (err != JSON_SEM_ACTION_FAILED)
+ actx_error_str(actx, json_errdetail(err, &lex));
+
+ goto cleanup;
+ }
+
+ /* Check all required fields. */
+ while (fields->name)
+ {
+ if (fields->required && !*fields->scalar && !*fields->array)
+ {
+ actx_error(actx, "field \"%s\" is missing", fields->name);
+ goto cleanup;
+ }
+
+ fields++;
+ }
+
+ success = true;
+
+cleanup:
+ freeJsonLexContext(&lex);
+ return success;
+}
+
+/*
+ * JSON Parser Definitions
+ */
+
+static bool
+parse_provider(struct async_ctx *actx, struct provider *provider)
+{
+ struct json_field fields[] = {
+ {"issuer", JSON_TOKEN_STRING, {&provider->issuer}, REQUIRED},
+ {"token_endpoint", JSON_TOKEN_STRING, {&provider->token_endpoint}, REQUIRED},
+
+ /*----
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - jwks_uri
+ * - response_types_supported
+ * - subject_types_supported
+ * - id_token_signing_alg_values_supported
+ */
+
+ {"device_authorization_endpoint", JSON_TOKEN_STRING, {&provider->device_authorization_endpoint}, OPTIONAL},
+ {"grant_types_supported", JSON_TOKEN_ARRAY_START, {.array = &provider->grant_types_supported}, OPTIONAL},
+
+ {0},
+ };
+
+ return parse_oauth_json(actx, fields);
+}
+
+/*
+ * Parses the "interval" JSON number, corresponding to the number of seconds to
+ * wait between token endpoint requests.
+ *
+ * RFC 8628 is pretty silent on sanity checks for the interval. As a matter of
+ * practicality, round any fractional intervals up to the next second, and clamp
+ * the result at a minimum of one. (Zero-second intervals would result in an
+ * expensive network polling loop.)
+ *
+ * TODO: maybe clamp the upper bound too, based on the libpq timeout and/or the
+ * code expiration time?
+ */
+static int
+parse_interval(const char *interval_str)
+{
+ double parsed;
+ int cnt;
+
+ /*
+ * The JSON lexer has already validated the number, which is stricter than
+ * the %f format, so we should be good to use sscanf().
+ */
+ cnt = sscanf(interval_str, "%lf", &parsed);
+
+ if (cnt != 1)
+ {
+ /*
+ * Either the lexer screwed up or our assumption above isn't true, and
+ * either way a developer needs to take a look.
+ */
+ Assert(cnt == 1);
+ return 1; /* don't fall through in release builds */
+ }
+
+ parsed = ceil(parsed);
+
+ if (parsed < 1)
+ return 1; /* TODO this slows down the tests
+ * considerably... */
+ else if (INT_MAX <= parsed)
+ return INT_MAX;
+
+ return parsed;
+}
+
+static bool
+parse_device_authz(struct async_ctx *actx, struct device_authz *authz)
+{
+ struct json_field fields[] = {
+ {"device_code", JSON_TOKEN_STRING, {&authz->device_code}, REQUIRED},
+ {"user_code", JSON_TOKEN_STRING, {&authz->user_code}, REQUIRED},
+ {"verification_uri", JSON_TOKEN_STRING, {&authz->verification_uri}, REQUIRED},
+
+ /*
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - expires_in
+ */
+
+ {"interval", JSON_TOKEN_NUMBER, {&authz->interval_str}, OPTIONAL},
+
+ {0},
+ };
+
+ if (!parse_oauth_json(actx, fields))
+ return false;
+
+ /*
+ * Parse our numeric fields. Lexing has already completed by this time, so
+ * we at least know they're valid JSON numbers.
+ */
+ if (authz->interval_str)
+ authz->interval = parse_interval(authz->interval_str);
+ else
+ {
+ /* TODO: handle default interval of 5 seconds */
+ }
+
+ return true;
+}
+
+static bool
+parse_token_error(struct async_ctx *actx, struct token_error *err)
+{
+ bool result;
+ struct json_field fields[] = {
+ {"error", JSON_TOKEN_STRING, {&err->error}, REQUIRED},
+
+ {"error_description", JSON_TOKEN_STRING, {&err->error_description}, OPTIONAL},
+
+ {0},
+ };
+
+ result = parse_oauth_json(actx, fields);
+
+ /*
+ * Since token errors are parsed during other active error paths, only
+ * override the errctx if parsing explicitly fails.
+ */
+ if (!result)
+ actx->errctx = "failed to parse token error response";
+
+ return result;
+}
+
+static bool
+parse_access_token(struct async_ctx *actx, struct token *tok)
+{
+ struct json_field fields[] = {
+ {"access_token", JSON_TOKEN_STRING, {&tok->access_token}, REQUIRED},
+ {"token_type", JSON_TOKEN_STRING, {&tok->token_type}, REQUIRED},
+
+ /*
+ * The following fields are technically REQUIRED, but we don't use
+ * them anywhere yet:
+ *
+ * - scope (only required if different than requested -- TODO check)
+ */
+
+ {0},
+ };
+
+ return parse_oauth_json(actx, fields);
+}
+
+/*
+ * cURL Multi Setup/Callbacks
+ */
+
+/*
+ * Sets up the actx->mux, which is the altsock that PQconnectPoll clients will
+ * select() on instead of the Postgres socket during OAuth negotiation.
+ *
+ * This is just an epoll set or kqueue abstracting multiple other descriptors.
+ * A timerfd is always part of the set when using epoll; it's just disabled
+ * when we're not using it.
+ */
+static bool
+setup_multiplexer(struct async_ctx *actx)
+{
+#ifdef HAVE_SYS_EPOLL_H
+ struct epoll_event ev = {.events = EPOLLIN};
+
+ actx->mux = epoll_create1(EPOLL_CLOEXEC);
+ if (actx->mux < 0)
+ {
+ actx_error(actx, "failed to create epoll set: %m");
+ return false;
+ }
+
+ actx->timerfd = timerfd_create(CLOCK_MONOTONIC, TFD_CLOEXEC);
+ if (actx->timerfd < 0)
+ {
+ actx_error(actx, "failed to create timerfd: %m");
+ return false;
+ }
+
+ if (epoll_ctl(actx->mux, EPOLL_CTL_ADD, actx->timerfd, &ev) < 0)
+ {
+ actx_error(actx, "failed to add timerfd to epoll set: %m");
+ return false;
+ }
+
+ return true;
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ actx->mux = kqueue();
+ if (actx->mux < 0)
+ {
+ actx_error(actx, "failed to create kqueue: %m");
+ return false;
+ }
+
+ return true;
+#endif
+
+ actx_error(actx, "here's a nickel kid, get yourself a better computer");
+ return false;
+}
+
+/*
+ * Adds and removes sockets from the multiplexer set, as directed by the
+ * cURL multi handle.
+ */
+static int
+register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
+ void *socketp)
+{
+#ifdef HAVE_SYS_EPOLL_H
+ struct async_ctx *actx = ctx;
+ struct epoll_event ev = {0};
+ int res;
+ int op = EPOLL_CTL_ADD;
+
+ switch (what)
+ {
+ case CURL_POLL_IN:
+ ev.events = EPOLLIN;
+ break;
+
+ case CURL_POLL_OUT:
+ ev.events = EPOLLOUT;
+ break;
+
+ case CURL_POLL_INOUT:
+ ev.events = EPOLLIN | EPOLLOUT;
+ break;
+
+ case CURL_POLL_REMOVE:
+ op = EPOLL_CTL_DEL;
+ break;
+
+ default:
+ actx_error(actx, "unknown cURL socket operation (%d)", what);
+ return -1;
+ }
+
+ res = epoll_ctl(actx->mux, op, socket, &ev);
+ if (res < 0 && errno == EEXIST)
+ {
+ /* We already had this socket in the pollset. */
+ op = EPOLL_CTL_MOD;
+ res = epoll_ctl(actx->mux, op, socket, &ev);
+ }
+
+ if (res < 0)
+ {
+ switch (op)
+ {
+ case EPOLL_CTL_ADD:
+ actx_error(actx, "could not add to epoll set: %m");
+ break;
+
+ case EPOLL_CTL_DEL:
+ actx_error(actx, "could not delete from epoll set: %m");
+ break;
+
+ default:
+ actx_error(actx, "could not update epoll set: %m");
+ }
+
+ return -1;
+ }
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct async_ctx *actx = ctx;
+ struct kevent ev[2] = {{0}};
+ struct kevent ev_out[2];
+ struct timespec timeout = {0};
+ int nev = 0;
+ int res;
+
+ switch (what)
+ {
+ case CURL_POLL_IN:
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_OUT:
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_INOUT:
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_ADD | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ case CURL_POLL_REMOVE:
+
+ /*
+ * We don't know which of these is currently registered, perhaps
+ * both, so we try to remove both. This means we need to tolerate
+ * ENOENT below.
+ */
+ EV_SET(&ev[nev], socket, EVFILT_READ, EV_DELETE | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_DELETE | EV_RECEIPT, 0, 0, 0);
+ nev++;
+ break;
+
+ default:
+ actx_error(actx, "unknown cURL socket operation (%d)", what);
+ return -1;
+ }
+
+ res = kevent(actx->mux, ev, nev, ev_out, lengthof(ev_out), &timeout);
+ if (res < 0)
+ {
+ actx_error(actx, "could not modify kqueue: %m");
+ return -1;
+ }
+
+ /*
+ * We can't use the simple errno version of kevent, because we need to
+ * skip over ENOENT while still allowing a second change to be processed.
+ * So we need a longer-form error checking loop.
+ */
+ for (int i = 0; i < res; ++i)
+ {
+ /*
+ * EV_RECEIPT should guarantee one EV_ERROR result for every change,
+ * whether successful or not. Failed entries contain a non-zero errno
+ * in the `data` field.
+ */
+ Assert(ev_out[i].flags & EV_ERROR);
+
+ errno = ev_out[i].data;
+ if (errno && errno != ENOENT)
+ {
+ switch (what)
+ {
+ case CURL_POLL_REMOVE:
+ actx_error(actx, "could not delete from kqueue: %m");
+ break;
+ default:
+ actx_error(actx, "could not add to kqueue: %m");
+ }
+ return -1;
+ }
+ }
+#endif
+
+ return 0;
+}
+
+/*
+ * Adds or removes timeouts from the multiplexer set, as directed by the
+ * cURL multi handle. Rather than continually adding and removing the timer,
+ * we keep it in the set at all times and just disarm it when it's not
+ * needed.
+ */
+static int
+register_timer(CURLM *curlm, long timeout, void *ctx)
+{
+#if HAVE_SYS_EPOLL_H
+ struct async_ctx *actx = ctx;
+ struct itimerspec spec = {0};
+
+ if (timeout < 0)
+ {
+ /* the zero itimerspec will disarm the timer below */
+ }
+ else if (timeout == 0)
+ {
+ /*
+ * A zero timeout means cURL wants us to call back immediately. That's
+ * not technically an option for timerfd, but we can make the timeout
+ * ridiculously short.
+ *
+ * TODO: maybe just signal drive_request() to immediately call back in
+ * this case?
+ */
+ spec.it_value.tv_nsec = 1;
+ }
+ else
+ {
+ spec.it_value.tv_sec = timeout / 1000;
+ spec.it_value.tv_nsec = (timeout % 1000) * 1000000;
+ }
+
+ if (timerfd_settime(actx->timerfd, 0 /* no flags */ , &spec, NULL) < 0)
+ {
+ actx_error(actx, "setting timerfd to %ld: %m", timeout);
+ return -1;
+ }
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct async_ctx *actx = ctx;
+ struct kevent ev;
+
+ EV_SET(&ev, 1, EVFILT_TIMER, timeout < 0 ? EV_DELETE : EV_ADD,
+ 0, timeout, 0);
+ if (kevent(actx->mux, &ev, 1, NULL, 0, NULL) < 0 && errno != ENOENT)
+ {
+ actx_error(actx, "setting kqueue timer to %ld: %m", timeout);
+ return -1;
+ }
+#endif
+
+ return 0;
+}
+
+/*
+ * Initializes the two cURL handles in the async_ctx. The multi handle,
+ * actx->curlm, is what drives the asynchronous engine and tells us what to do
+ * next. The easy handle, actx->curl, encapsulates the state for a single
+ * request/response. It's added to the multi handle as needed, during
+ * start_request().
+ */
+static bool
+setup_curl_handles(struct async_ctx *actx)
+{
+ curl_version_info_data *curl_info;
+
+ /*
+ * Create our multi handle. This encapsulates the entire conversation with
+ * cURL for this connection.
+ */
+ actx->curlm = curl_multi_init();
+ if (!actx->curlm)
+ {
+ /* We don't get a lot of feedback on the failure reason. */
+ actx_error(actx, "failed to create cURL multi handle");
+ return false;
+ }
+
+ /*
+ * Extract information about the libcurl we are linked against.
+ */
+ curl_info = curl_version_info(CURLVERSION_NOW);
+
+ /*
+ * The multi handle tells us what to wait on using two callbacks. These
+ * will manipulate actx->mux as needed.
+ */
+ CHECK_MSETOPT(actx, CURLMOPT_SOCKETFUNCTION, register_socket, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_SOCKETDATA, actx, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_TIMERFUNCTION, register_timer, return false);
+ CHECK_MSETOPT(actx, CURLMOPT_TIMERDATA, actx, return false);
+
+ /*
+ * Set up an easy handle. All of our requests are made serially, so we
+ * only ever need to keep track of one.
+ */
+ actx->curl = curl_easy_init();
+ if (!actx->curl)
+ {
+ actx_error(actx, "failed to create cURL handle");
+ return false;
+ }
+
+ /*
+ * Multi-threaded applications must set CURLOPT_NOSIGNAL. This requires us
+ * to handle the possibility of SIGPIPE ourselves.
+ *
+ * TODO: handle SIGPIPE via pq_block_sigpipe(), or via a
+ * CURLOPT_SOCKOPTFUNCTION maybe...
+ */
+ CHECK_SETOPT(actx, CURLOPT_NOSIGNAL, 1L, return false);
+ if (!curl_info->ares_num)
+ {
+ /* No alternative resolver, TODO: warn about timeouts */
+ }
+
+ /* TODO investigate using conn->Pfdebug and CURLOPT_DEBUGFUNCTION here */
+ CHECK_SETOPT(actx, CURLOPT_VERBOSE, 1L, return false);
+ CHECK_SETOPT(actx, CURLOPT_ERRORBUFFER, actx->curl_err, return false);
+
+ /*
+ * Only HTTP[S] is allowed. TODO: disallow HTTP without user opt-in
+ */
+ CHECK_SETOPT(actx, CURLOPT_PROTOCOLS, CURLPROTO_HTTP | CURLPROTO_HTTPS, return false);
+
+ /*
+ * Suppress the Accept header to make our request as minimal as possible.
+ * (Ideally we would set it to "application/json" instead, but OpenID is
+ * pretty strict when it comes to provider behavior, so we have to check
+ * what comes back anyway.)
+ */
+ actx->headers = curl_slist_append(actx->headers, "Accept:"); /* TODO: check result */
+ CHECK_SETOPT(actx, CURLOPT_HTTPHEADER, actx->headers, return false);
+
+ return true;
+}
+
+/*
+ * Generic HTTP Request Handlers
+ */
+
+/*
+ * Response callback from cURL; appends the response body into actx->work_data.
+ * See start_request().
+ */
+static size_t
+append_data(char *buf, size_t size, size_t nmemb, void *userdata)
+{
+ PQExpBuffer resp = userdata;
+ size_t len = size * nmemb;
+
+ /* TODO: cap the maximum size */
+ appendBinaryPQExpBuffer(resp, buf, len);
+ /* TODO: check for broken buffer */
+
+ return len;
+}
+
+/*
+ * Begins an HTTP request on the multi handle. The caller should have set up all
+ * request-specific options on actx->curl first. The server's response body will
+ * be accumulated in actx->work_data (which will be reset, so don't store
+ * anything important there across this call).
+ *
+ * Once a request is queued, it can be driven to completion via drive_request().
+ */
+static bool
+start_request(struct async_ctx *actx)
+{
+ CURLMcode err;
+ int running;
+
+ resetPQExpBuffer(&actx->work_data);
+ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data, return false);
+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, &actx->work_data, return false);
+
+ err = curl_multi_add_handle(actx->curlm, actx->curl);
+ if (err)
+ {
+ actx_error(actx, "failed to queue HTTP request: %s",
+ curl_multi_strerror(err));
+ return false;
+ }
+
+ err = curl_multi_socket_action(actx->curlm, CURL_SOCKET_TIMEOUT, 0, &running);
+ if (err)
+ {
+ actx_error(actx, "asynchronous HTTP request failed: %s",
+ curl_multi_strerror(err));
+ return false;
+ }
+
+ /*
+ * Sanity check.
+ *
+ * TODO: even though this is nominally an asynchronous process, there are
+ * apparently operations that can synchronously fail by this point, such
+ * as connections to closed local ports. Maybe we need to let this case
+ * fall through to drive_request instead, or else perform a
+ * curl_multi_info_read immediately.
+ */
+ if (running != 1)
+ {
+ actx_error(actx, "failed to queue HTTP request");
+ return false;
+ }
+
+ return true;
+}
+
+/*
+ * Drives the multi handle towards completion. The caller should have already
+ * set up an asynchronous request via start_request().
+ */
+static PostgresPollingStatusType
+drive_request(struct async_ctx *actx)
+{
+ CURLMcode err;
+ int running;
+ CURLMsg *msg;
+ int msgs_left;
+ bool done;
+
+ err = curl_multi_socket_all(actx->curlm, &running);
+ if (err)
+ {
+ actx_error(actx, "asynchronous HTTP request failed: %s",
+ curl_multi_strerror(err));
+ return PGRES_POLLING_FAILED;
+ }
+
+ if (running)
+ {
+ /* We'll come back again. */
+ return PGRES_POLLING_READING;
+ }
+
+ done = false;
+ while ((msg = curl_multi_info_read(actx->curlm, &msgs_left)) != NULL)
+ {
+ if (msg->msg != CURLMSG_DONE)
+ {
+ /*
+ * Future cURL versions may define new message types; we don't
+ * know how to handle them, so we'll ignore them.
+ */
+ continue;
+ }
+
+ /* First check the status of the request itself. */
+ if (msg->data.result != CURLE_OK)
+ {
+ actx_error_str(actx, curl_easy_strerror(msg->data.result));
+ return PGRES_POLLING_FAILED;
+ }
+
+ /* Now remove the finished handle; we'll add it back later if needed. */
+ err = curl_multi_remove_handle(actx->curlm, msg->easy_handle);
+ if (err)
+ {
+ actx_error(actx, "cURL easy handle removal failed: %s",
+ curl_multi_strerror(err));
+ return PGRES_POLLING_FAILED;
+ }
+
+ done = true;
+ }
+
+ /* Sanity check. */
+ if (!done)
+ {
+ actx_error(actx, "no result was retrieved for the finished handle");
+ return PGRES_POLLING_FAILED;
+ }
+
+ return PGRES_POLLING_OK;
+}
+
+/*
+ * Specific HTTP Request Handlers
+ *
+ * This is finally the beginning of the actual application logic. Generally
+ * speaking, a single request consists of a start_* and a finish_* step, with
+ * drive_request() pumping the machine in between.
+ */
+
+/*
+ * Queue an OpenID Provider Configuration Request:
+ *
+ * https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderConfigurationRequest
+ * https://www.rfc-editor.org/rfc/rfc8414#section-3.1
+ *
+ * This is done first to get the endpoint URIs we need to contact and to make
+ * sure the provider provides a device authorization flow. finish_discovery()
+ * will fill in actx->provider.
+ */
+static bool
+start_discovery(struct async_ctx *actx, const char *discovery_uri)
+{
+ CHECK_SETOPT(actx, CURLOPT_HTTPGET, 1L, return false);
+ CHECK_SETOPT(actx, CURLOPT_URL, discovery_uri, return false);
+
+ return start_request(actx);
+}
+
+static bool
+finish_discovery(struct async_ctx *actx)
+{
+ long response_code;
+
+ /*----
+ * Now check the response. OIDC Discovery 1.0 is pretty strict:
+ *
+ * A successful response MUST use the 200 OK HTTP status code and
+ * return a JSON object using the application/json content type that
+ * contains a set of Claims as its members that are a subset of the
+ * Metadata values defined in Section 3.
+ *
+ * Compared to standard HTTP semantics, this makes life easy -- we don't
+ * need to worry about redirections (which would call the Issuer host
+ * validation into question), or non-authoritative responses, or any other
+ * complications.
+ */
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ if (response_code != 200)
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ actx->errctx = "failed to parse OpenID discovery document";
+ if (!parse_provider(actx, &actx->provider))
+ return false; /* error message already set */
+
+ /*
+ * Fill in any defaults for OPTIONAL/RECOMMENDED fields we care about.
+ */
+ if (!actx->provider.grant_types_supported)
+ {
+ /*
+ * Per Section 3, the default is ["authorization_code", "implicit"].
+ */
+ struct curl_slist *temp = actx->provider.grant_types_supported;
+
+ temp = curl_slist_append(temp, "authorization_code");
+ if (temp)
+ {
+ temp = curl_slist_append(temp, "implicit");
+ }
+
+ if (!temp)
+ {
+ actx_error(actx, "out of memory");
+ return false;
+ }
+
+ actx->provider.grant_types_supported = temp;
+ }
+
+ return true;
+}
+
+#define OAUTH_GRANT_TYPE_DEVICE_CODE "urn:ietf:params:oauth:grant-type:device_code"
+
+/*
+ * Ensure that the provider supports the Device Authorization flow (i.e. it
+ * accepts the device_code grant type and provides an authorization endpoint).
+ */
+static bool
+check_for_device_flow(struct async_ctx *actx)
+{
+ const struct provider *provider = &actx->provider;
+ const struct curl_slist *grant;
+ bool device_grant_found = false;
+
+ Assert(provider->issuer); /* ensured by get_discovery_document() */
+
+ /*------
+ * First, sanity checks for discovery contents that are OPTIONAL in the
+ * spec but required for our flow:
+ * - the issuer must support the device_code grant
+ * - the issuer must have actually given us a
+ * device_authorization_endpoint
+ */
+
+ grant = provider->grant_types_supported;
+ while (grant)
+ {
+ if (strcmp(grant->data, OAUTH_GRANT_TYPE_DEVICE_CODE) == 0)
+ {
+ device_grant_found = true;
+ break;
+ }
+
+ grant = grant->next;
+ }
+
+ if (!device_grant_found)
+ {
+ actx_error(actx, "issuer \"%s\" does not support device code grants",
+ provider->issuer);
+ return false;
+ }
+
+ if (!provider->device_authorization_endpoint)
+ {
+ actx_error(actx,
+ "issuer \"%s\" does not provide a device authorization endpoint",
+ provider->issuer);
+ return false;
+ }
+
+ /* TODO: check that the endpoint uses HTTPS */
+
+ return true;
+}
+
+/*
+ * Queue a Device Authorization Request:
+ *
+ * https://www.rfc-editor.org/rfc/rfc8628#section-3.1
+ *
+ * This is the second step. We ask the provider to verify the end user out of
+ * band and authorize us to act on their behalf; it will give us the required
+ * nonces for us to later poll the request status, which we'll grab in
+ * finish_device_authz().
+ */
+static bool
+start_device_authz(struct async_ctx *actx, PGconn *conn)
+{
+ const char *device_authz_uri = actx->provider.device_authorization_endpoint;
+ PQExpBuffer work_buffer = &actx->work_data;
+
+ Assert(conn->oauth_client_id); /* ensured by get_auth_token() */
+ Assert(device_authz_uri); /* ensured by check_for_device_flow() */
+
+ /* Construct our request body. TODO: url-encode */
+ resetPQExpBuffer(work_buffer);
+ appendPQExpBuffer(work_buffer, "client_id=%s", conn->oauth_client_id);
+ if (conn->oauth_scope)
+ appendPQExpBuffer(work_buffer, "&scope=%s", conn->oauth_scope);
+ /* TODO check for broken buffer */
+
+ /* Make our request. */
+ CHECK_SETOPT(actx, CURLOPT_URL, device_authz_uri, return false);
+ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data, return false);
+
+ if (conn->oauth_client_secret)
+ {
+ /*----
+ * Use HTTP Basic auth to send the password. Per RFC 6749, Sec. 2.3.1,
+ *
+ * Including the client credentials in the request-body using the
+ * two parameters is NOT RECOMMENDED and SHOULD be limited to
+ * clients unable to directly utilize the HTTP Basic authentication
+ * scheme (or other password-based HTTP authentication schemes).
+ *
+ * TODO: should we omit client_id from the body in this case?
+ */
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC, return false);
+ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id, return false);
+ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret, return false);
+ }
+ else
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE, return false);
+
+ return start_request(actx);
+}
+
+static bool
+finish_device_authz(struct async_ctx *actx)
+{
+ long response_code;
+
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ /*
+ * The device authorization endpoint uses the same error response as the
+ * token endpoint, so we have to handle 400/401 here too.
+ */
+ if (response_code != 200
+ && response_code != 400
+ /* && response_code != 401 TODO */ )
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ if (response_code != 200)
+ {
+ struct token_error err = {0};
+
+ if (!parse_token_error(actx, &err))
+ {
+ free_token_error(&err);
+ return false;
+ }
+
+ if (err.error_description)
+ appendPQExpBuffer(&actx->errbuf, "%s ", err.error_description);
+
+ appendPQExpBuffer(&actx->errbuf, "(%s)", err.error);
+
+ free_token_error(&err);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ actx->errctx = "failed to parse device authorization";
+ if (!parse_device_authz(actx, &actx->authz))
+ return false; /* error message already set */
+
+ return true;
+}
+
+/*
+ * Queue an Access Token Request:
+ *
+ * https://www.rfc-editor.org/rfc/rfc6749#section-4.1.3
+ *
+ * This is the final step. We continually poll the token endpoint to see if the
+ * user has authorized us yet. finish_token_request() will pull either the token
+ * or a (ideally temporary) error status from the provider.
+ */
+static bool
+start_token_request(struct async_ctx *actx, PGconn *conn)
+{
+ const char *token_uri = actx->provider.token_endpoint;
+ const char *device_code = actx->authz.device_code;
+ PQExpBuffer work_buffer = &actx->work_data;
+
+ Assert(conn->oauth_client_id); /* ensured by get_auth_token() */
+ Assert(token_uri); /* ensured by get_discovery_document() */
+ Assert(device_code); /* ensured by run_device_authz() */
+
+ /* Construct our request body. TODO: url-encode */
+ resetPQExpBuffer(work_buffer);
+ appendPQExpBuffer(work_buffer, "client_id=%s", conn->oauth_client_id);
+ appendPQExpBuffer(work_buffer, "&device_code=%s", device_code);
+ appendPQExpBuffer(work_buffer, "&grant_type=%s",
+ OAUTH_GRANT_TYPE_DEVICE_CODE);
+ /* TODO check for broken buffer */
+
+ /* Make our request. */
+ CHECK_SETOPT(actx, CURLOPT_URL, token_uri, return false);
+ CHECK_SETOPT(actx, CURLOPT_COPYPOSTFIELDS, work_buffer->data, return false);
+
+ if (conn->oauth_client_secret)
+ {
+ /*----
+ * Use HTTP Basic auth to send the password. Per RFC 6749, Sec. 2.3.1,
+ *
+ * Including the client credentials in the request-body using the
+ * two parameters is NOT RECOMMENDED and SHOULD be limited to
+ * clients unable to directly utilize the HTTP Basic authentication
+ * scheme (or other password-based HTTP authentication schemes).
+ *
+ * TODO: should we omit client_id from the body in this case?
+ */
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC, return false);
+ CHECK_SETOPT(actx, CURLOPT_USERNAME, conn->oauth_client_id, return false);
+ CHECK_SETOPT(actx, CURLOPT_PASSWORD, conn->oauth_client_secret, return false);
+ }
+ else
+ CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_NONE, return false);
+
+ resetPQExpBuffer(work_buffer);
+ CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data, return false);
+ CHECK_SETOPT(actx, CURLOPT_WRITEDATA, work_buffer, return false);
+
+ return start_request(actx);
+}
+
+static bool
+finish_token_request(struct async_ctx *actx, struct token *tok)
+{
+ long response_code;
+
+ CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
+
+ /*
+ * Per RFC 6749, Section 5, a successful response uses 200 OK. An error
+ * response uses either 400 Bad Request or 401 Unauthorized.
+ *
+ * TODO: there are references online to 403 appearing in the wild...
+ */
+ if (response_code != 200
+ && response_code != 400
+ /* && response_code != 401 TODO */ )
+ {
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
+ }
+
+ /*
+ * Pull the fields we care about from the document.
+ */
+ if (response_code == 200)
+ {
+ actx->errctx = "failed to parse access token response";
+ if (!parse_access_token(actx, tok))
+ return false; /* error message already set */
+ }
+ else if (!parse_token_error(actx, &tok->err))
+ return false;
+
+ return true;
+}
+
+/*
+ * The top-level, nonblocking entry point for the cURL implementation. This will
+ * be called several times to pump the async engine.
+ *
+ * The architecture is based on PQconnectPoll(). The first half drives the
+ * connection state forward as necessary, returning if we're not ready to
+ * proceed to the next step yet. The second half performs the actual transition
+ * between states.
+ *
+ * You can trace the overall OAuth flow through the second half. It's linear
+ * until we get to the end, where we flip back and forth between
+ * OAUTH_STEP_TOKEN_REQUEST and OAUTH_STEP_WAIT_INTERVAL to regularly ping the
+ * provider.
+ */
+PostgresPollingStatusType
+pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
+{
+ fe_oauth_state *state = conn->sasl_state;
+ struct async_ctx *actx;
+
+ struct token tok = {0};
+
+ /*
+ * XXX This is not safe. cURL has stringent requirements for the thread
+ * context in which you call curl_global_init(), because it's going to try
+ * initializing a bunch of other libraries (OpenSSL, Winsock...). And we
+ * probably need to consider both the TLS backend libcurl is compiled
+ * against and what the user has asked us to do via PQinit[Open]SSL.
+ *
+ * Recent versions of libcurl have improved the thread-safety situation,
+ * but you apparently can't check at compile time whether the
+ * implementation is thread-safe, and there's a chicken-and-egg problem
+ * where you can't check the thread safety until you've initialized cURL,
+ * which you can't do before you've made sure it's thread-safe...
+ *
+ * We know we've already initialized Winsock by this point, so we should
+ * be able to safely skip that bit. But we have to tell cURL to initialize
+ * everything else, because other pieces of our client executable may
+ * already be using cURL for their own purposes. If we initialize libcurl
+ * first, with only a subset of its features, we could break those other
+ * clients nondeterministically, and that would probably be a nightmare to
+ * debug.
+ */
+ curl_global_init(CURL_GLOBAL_ALL
+ & ~CURL_GLOBAL_WIN32); /* we already initialized Winsock */
+
+ if (!state->async_ctx)
+ {
+ /*
+ * Create our asynchronous state, and hook it into the upper-level
+ * OAuth state immediately, so any failures below won't leak the
+ * context allocation.
+ */
+ actx = calloc(1, sizeof(*actx));
+ if (!actx)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return PGRES_POLLING_FAILED;
+ }
+
+ actx->mux = PGINVALID_SOCKET;
+#ifdef HAVE_SYS_EPOLL_H
+ actx->timerfd = -1;
+#endif
+
+ state->async_ctx = actx;
+ state->free_async_ctx = free_curl_async_ctx;
+
+ initPQExpBuffer(&actx->work_data);
+ initPQExpBuffer(&actx->errbuf);
+
+ if (!setup_multiplexer(actx))
+ goto error_return;
+
+ if (!setup_curl_handles(actx))
+ goto error_return;
+ }
+
+ actx = state->async_ctx;
+
+ /* By default, the multiplexer is the altsock. Reassign as desired. */
+ *altsock = actx->mux;
+
+ switch (actx->step)
+ {
+ case OAUTH_STEP_INIT:
+ break;
+
+ case OAUTH_STEP_DISCOVERY:
+ case OAUTH_STEP_DEVICE_AUTHORIZATION:
+ case OAUTH_STEP_TOKEN_REQUEST:
+ {
+ PostgresPollingStatusType status;
+
+ status = drive_request(actx);
+
+ if (status == PGRES_POLLING_FAILED)
+ goto error_return;
+ else if (status != PGRES_POLLING_OK)
+ {
+ /* not done yet */
+ free_token(&tok);
+ return status;
+ }
+ }
+
+ case OAUTH_STEP_WAIT_INTERVAL:
+ /* TODO check that the timer has expired */
+ break;
+ }
+
+ switch (actx->step)
+ {
+ case OAUTH_STEP_INIT:
+ actx->errctx = "failed to fetch OpenID discovery document";
+ if (!start_discovery(actx, conn->oauth_discovery_uri))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_DISCOVERY;
+ break;
+
+ case OAUTH_STEP_DISCOVERY:
+ if (!finish_discovery(actx))
+ goto error_return;
+
+ /* TODO: check issuer */
+
+ actx->errctx = "cannot run OAuth device authorization";
+ if (!check_for_device_flow(actx))
+ goto error_return;
+
+ actx->errctx = "failed to obtain device authorization";
+ if (!start_device_authz(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_DEVICE_AUTHORIZATION;
+ break;
+
+ case OAUTH_STEP_DEVICE_AUTHORIZATION:
+ if (!finish_device_authz(actx))
+ goto error_return;
+
+ actx->errctx = "failed to obtain access token";
+ if (!start_token_request(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_TOKEN_REQUEST;
+ break;
+
+ case OAUTH_STEP_TOKEN_REQUEST:
+ {
+ const struct token_error *err;
+#ifdef HAVE_SYS_EPOLL_H
+ struct itimerspec spec = {0};
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ struct kevent ev = {0};
+#endif
+
+ if (!finish_token_request(actx, &tok))
+ goto error_return;
+
+ if (!actx->user_prompted)
+ {
+ int res;
+ PQpromptOAuthDevice prompt = {
+ .verification_uri = actx->authz.verification_uri,
+ .user_code = actx->authz.user_code,
+ /* TODO: optional fields */
+ };
+
+ /*
+ * Now that we know the token endpoint isn't broken, give
+ * the user the login instructions.
+ */
+ res = PQauthDataHook(PQAUTHDATA_PROMPT_OAUTH_DEVICE, conn,
+ &prompt);
+
+ if (!res)
+ {
+ fprintf(stderr, "Visit %s and enter the code: %s",
+ prompt.verification_uri, prompt.user_code);
+ }
+ else if (res < 0)
+ {
+ actx_error(actx, "device prompt failed");
+ goto error_return;
+ }
+
+ actx->user_prompted = true;
+ }
+
+ if (tok.access_token)
+ {
+ /* Construct our Bearer token. */
+ resetPQExpBuffer(&actx->work_data);
+ appendPQExpBuffer(&actx->work_data, "Bearer %s",
+ tok.access_token);
+
+ if (PQExpBufferDataBroken(actx->work_data))
+ {
+ actx_error(actx, "out of memory");
+ goto error_return;
+ }
+
+ state->token = strdup(actx->work_data.data);
+ break;
+ }
+
+ /*
+ * authorization_pending and slow_down are the only acceptable
+ * errors; anything else and we bail.
+ */
+ err = &tok.err;
+ if (!err->error || (strcmp(err->error, "authorization_pending")
+ && strcmp(err->error, "slow_down")))
+ {
+ /* TODO handle !err->error */
+ if (err->error_description)
+ appendPQExpBuffer(&actx->errbuf, "%s ",
+ err->error_description);
+
+ appendPQExpBuffer(&actx->errbuf, "(%s)", err->error);
+
+ goto error_return;
+ }
+
+ /*
+ * A slow_down error requires us to permanently increase our
+ * retry interval by five seconds. RFC 8628, Sec. 3.5.
+ */
+ if (strcmp(err->error, "slow_down") == 0)
+ {
+ actx->authz.interval += 5; /* TODO check for overflow? */
+ }
+
+ /*
+ * Wait for the required interval before issuing the next
+ * request.
+ */
+ Assert(actx->authz.interval > 0);
+#ifdef HAVE_SYS_EPOLL_H
+ spec.it_value.tv_sec = actx->authz.interval;
+
+ if (timerfd_settime(actx->timerfd, 0 /* no flags */ , &spec, NULL) < 0)
+ {
+ actx_error(actx, "failed to set timerfd: %m");
+ goto error_return;
+ }
+
+ *altsock = actx->timerfd;
+#endif
+#ifdef HAVE_SYS_EVENT_H
+ /* XXX: I guess this wants to be hidden in a routine */
+ EV_SET(&ev, 1, EVFILT_TIMER, EV_ADD, 0,
+ actx->authz.interval * 1000, 0);
+ if (kevent(actx->mux, &ev, 1, NULL, 0, NULL) < 0)
+ {
+ actx_error(actx, "failed to set kqueue timer: %m");
+ goto error_return;
+ }
+ /* XXX: why did we change the altsock in the epoll version? */
+#endif
+ actx->step = OAUTH_STEP_WAIT_INTERVAL;
+ break;
+ }
+
+ case OAUTH_STEP_WAIT_INTERVAL:
+ actx->errctx = "failed to obtain access token";
+ if (!start_token_request(actx, conn))
+ goto error_return;
+
+ actx->step = OAUTH_STEP_TOKEN_REQUEST;
+ break;
+ }
+
+ free_token(&tok);
+
+ /* If we've stored a token, we're done. Otherwise come back later. */
+ return state->token ? PGRES_POLLING_OK : PGRES_POLLING_READING;
+
+error_return:
+
+ /*
+ * Assemble the three parts of our error: context, body, and detail. See
+ * also the documentation for struct async_ctx.
+ */
+ if (actx->errctx)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext(actx->errctx));
+ appendPQExpBufferStr(&conn->errorMessage, ": ");
+ }
+
+ if (PQExpBufferDataBroken(actx->errbuf))
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("out of memory"));
+ else
+ appendPQExpBufferStr(&conn->errorMessage, actx->errbuf.data);
+
+ if (actx->curl_err[0])
+ {
+ size_t len;
+
+ appendPQExpBuffer(&conn->errorMessage, " (%s)", actx->curl_err);
+
+ /* Sometimes libcurl adds a newline to the error buffer. :( */
+ len = conn->errorMessage.len;
+ if (len >= 2 && conn->errorMessage.data[len - 2] == '\n')
+ {
+ conn->errorMessage.data[len - 2] = ')';
+ conn->errorMessage.data[len - 1] = '\0';
+ conn->errorMessage.len--;
+ }
+ }
+
+ appendPQExpBufferStr(&conn->errorMessage, "\n");
+
+ free_token(&tok);
+ return PGRES_POLLING_FAILED;
+}
diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
new file mode 100644
index 0000000000..66ee8ff076
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth.c
@@ -0,0 +1,659 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth.c
+ * The front-end (client) implementation of OAuth/OIDC authentication.
+ *
+ * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/interfaces/libpq/fe-auth-oauth.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres_fe.h"
+
+#include "common/base64.h"
+#include "common/hmac.h"
+#include "common/jsonapi.h"
+#include "common/oauth-common.h"
+#include "fe-auth.h"
+#include "fe-auth-oauth.h"
+#include "mb/pg_wchar.h"
+
+/* The exported OAuth callback mechanism. */
+static void *oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism);
+static SASLStatus oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen);
+static bool oauth_channel_bound(void *opaq);
+static void oauth_free(void *opaq);
+
+const pg_fe_sasl_mech pg_oauth_mech = {
+ oauth_init,
+ oauth_exchange,
+ oauth_channel_bound,
+ oauth_free,
+};
+
+static void *
+oauth_init(PGconn *conn, const char *password,
+ const char *sasl_mechanism)
+{
+ fe_oauth_state *state;
+
+ /*
+ * We only support one SASL mechanism here; anything else is programmer
+ * error.
+ */
+ Assert(sasl_mechanism != NULL);
+ Assert(strcmp(sasl_mechanism, OAUTHBEARER_NAME) == 0);
+
+ state = calloc(1, sizeof(*state));
+ if (!state)
+ return NULL;
+
+ state->state = FE_OAUTH_INIT;
+ state->conn = conn;
+
+ return state;
+}
+
+#define kvsep "\x01"
+
+static char *
+client_initial_response(PGconn *conn, const char *token)
+{
+ static const char *const resp_format = "n,," kvsep "auth=%s" kvsep kvsep;
+
+ PQExpBufferData buf;
+ char *response = NULL;
+
+ if (!token)
+ {
+ /*
+ * Either programmer error, or something went badly wrong during the
+ * asynchronous fetch.
+ *
+ * TODO: users shouldn't see this; what action should they take if
+ * they do?
+ */
+ libpq_append_conn_error(conn, "no OAuth token was set for the connection");
+ return NULL;
+ }
+
+ initPQExpBuffer(&buf);
+ appendPQExpBuffer(&buf, resp_format, token);
+
+ if (!PQExpBufferDataBroken(buf))
+ response = strdup(buf.data);
+
+ termPQExpBuffer(&buf);
+ return response;
+}
+
+#define ERROR_STATUS_FIELD "status"
+#define ERROR_SCOPE_FIELD "scope"
+#define ERROR_OPENID_CONFIGURATION_FIELD "openid-configuration"
+
+struct json_ctx
+{
+ char *errmsg; /* any non-NULL value stops all processing */
+ PQExpBufferData errbuf; /* backing memory for errmsg */
+ int nested; /* nesting level (zero is the top) */
+
+ const char *target_field_name; /* points to a static allocation */
+ char **target_field; /* see below */
+
+ /* target_field, if set, points to one of the following: */
+ char *status;
+ char *scope;
+ char *discovery_uri;
+};
+
+#define oauth_json_has_error(ctx) \
+ (PQExpBufferDataBroken((ctx)->errbuf) || (ctx)->errmsg)
+
+#define oauth_json_set_error(ctx, ...) \
+ do { \
+ appendPQExpBuffer(&(ctx)->errbuf, __VA_ARGS__); \
+ (ctx)->errmsg = (ctx)->errbuf.data; \
+ } while (0)
+
+static JsonParseErrorType
+oauth_json_object_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ ++ctx->nested;
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_end(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ --ctx->nested;
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_object_field_start(void *state, char *name, bool isnull)
+{
+ struct json_ctx *ctx = state;
+
+ if (ctx->nested == 1)
+ {
+ if (strcmp(name, ERROR_STATUS_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_STATUS_FIELD;
+ ctx->target_field = &ctx->status;
+ }
+ else if (strcmp(name, ERROR_SCOPE_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_SCOPE_FIELD;
+ ctx->target_field = &ctx->scope;
+ }
+ else if (strcmp(name, ERROR_OPENID_CONFIGURATION_FIELD) == 0)
+ {
+ ctx->target_field_name = ERROR_OPENID_CONFIGURATION_FIELD;
+ ctx->target_field = &ctx->discovery_uri;
+ }
+ }
+
+ free(name);
+ return JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_array_start(void *state)
+{
+ struct json_ctx *ctx = state;
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static JsonParseErrorType
+oauth_json_scalar(void *state, char *token, JsonTokenType type)
+{
+ struct json_ctx *ctx = state;
+
+ if (!ctx->nested)
+ {
+ ctx->errmsg = libpq_gettext("top-level element must be an object");
+ }
+ else if (ctx->target_field)
+ {
+ Assert(ctx->nested == 1);
+
+ if (type == JSON_TOKEN_STRING)
+ {
+ *ctx->target_field = token;
+
+ ctx->target_field = NULL;
+ ctx->target_field_name = NULL;
+
+ return JSON_SUCCESS; /* don't free the token we're using */
+ }
+
+ oauth_json_set_error(ctx,
+ libpq_gettext("field \"%s\" must be a string"),
+ ctx->target_field_name);
+ }
+
+ free(token);
+ return oauth_json_has_error(ctx) ? JSON_SEM_ACTION_FAILED : JSON_SUCCESS;
+}
+
+static bool
+handle_oauth_sasl_error(PGconn *conn, char *msg, int msglen)
+{
+ JsonLexContext lex = {0};
+ JsonSemAction sem = {0};
+ JsonParseErrorType err;
+ struct json_ctx ctx = {0};
+ char *errmsg = NULL;
+
+ /* Sanity check. */
+ if (strlen(msg) != msglen)
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server's error message contained an embedded NULL, and was discarded"));
+ return false;
+ }
+
+ makeJsonLexContextCstringLen(&lex, msg, msglen, PG_UTF8, true);
+
+ initPQExpBuffer(&ctx.errbuf);
+ sem.semstate = &ctx;
+
+ sem.object_start = oauth_json_object_start;
+ sem.object_end = oauth_json_object_end;
+ sem.object_field_start = oauth_json_object_field_start;
+ sem.array_start = oauth_json_array_start;
+ sem.scalar = oauth_json_scalar;
+
+ err = pg_parse_json(&lex, &sem);
+
+ if (err == JSON_SEM_ACTION_FAILED)
+ {
+ if (PQExpBufferDataBroken(ctx.errbuf))
+ errmsg = libpq_gettext("out of memory");
+ else if (ctx.errmsg)
+ errmsg = ctx.errmsg;
+ else
+ {
+ /*
+ * Developer error: one of the action callbacks didn't call
+ * oauth_json_set_error() before erroring out.
+ */
+ Assert(oauth_json_has_error(&ctx));
+ errmsg = "<unexpected empty error>";
+ }
+ }
+ else if (err != JSON_SUCCESS)
+ errmsg = json_errdetail(err, &lex);
+
+ if (errmsg)
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("failed to parse server's error response: %s"),
+ errmsg);
+
+ /* Don't need the error buffer or the JSON lexer anymore. */
+ termPQExpBuffer(&ctx.errbuf);
+ freeJsonLexContext(&lex);
+
+ if (errmsg)
+ return false;
+
+ /* TODO: what if these override what the user already specified? */
+ if (ctx.discovery_uri)
+ {
+ if (conn->oauth_discovery_uri)
+ free(conn->oauth_discovery_uri);
+
+ conn->oauth_discovery_uri = ctx.discovery_uri;
+ }
+
+ if (ctx.scope)
+ {
+ if (conn->oauth_scope)
+ free(conn->oauth_scope);
+
+ conn->oauth_scope = ctx.scope;
+ }
+ /* TODO: missing error scope should clear any existing connection scope */
+
+ if (!ctx.status)
+ {
+ appendPQExpBuffer(&conn->errorMessage,
+ libpq_gettext("server sent error response without a status"));
+ return false;
+ }
+
+ if (strcmp(ctx.status, "invalid_token") == 0)
+ {
+ /*
+ * invalid_token is the only error code we'll automatically retry for,
+ * but only if we have enough information to do so.
+ */
+ if (conn->oauth_discovery_uri)
+ conn->oauth_want_retry = true;
+ }
+ /* TODO: include status in hard failure message */
+
+ return true;
+}
+
+static void
+free_request(PGconn *conn, void *vreq)
+{
+ PQoauthBearerRequest *request = vreq;
+
+ if (request->cleanup)
+ request->cleanup(conn, request);
+
+ free(request);
+}
+
+static PostgresPollingStatusType
+run_user_oauth_flow(PGconn *conn, pgsocket *altsock)
+{
+ fe_oauth_state *state = conn->sasl_state;
+ PQoauthBearerRequest *request = state->async_ctx;
+ PostgresPollingStatusType status;
+
+ if (!request->async)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow provided neither a token nor an async callback");
+ return PGRES_POLLING_FAILED;
+ }
+
+ status = request->async(conn, request, altsock);
+ if (status == PGRES_POLLING_FAILED)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow failed");
+ return status;
+ }
+ else if (status == PGRES_POLLING_OK)
+ {
+ /*
+ * We already have a token, so copy it into the state. (We can't hold
+ * onto the original string, since it may not be safe for us to free()
+ * it.)
+ */
+ PQExpBufferData token;
+
+ if (!request->token)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow did not provide a token");
+ return PGRES_POLLING_FAILED;
+ }
+
+ initPQExpBuffer(&token);
+ appendPQExpBuffer(&token, "Bearer %s", request->token);
+
+ if (PQExpBufferDataBroken(token))
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return PGRES_POLLING_FAILED;
+ }
+
+ state->token = token.data;
+ return PGRES_POLLING_OK;
+ }
+
+ /* TODO: what if no altsock was set? */
+ return status;
+}
+
+static bool
+setup_token_request(PGconn *conn, fe_oauth_state *state)
+{
+ int res;
+ PQoauthBearerRequest request = {
+ .openid_configuration = conn->oauth_discovery_uri,
+ .scope = conn->oauth_scope,
+ };
+
+ Assert(request.openid_configuration);
+
+ /* The client may have overridden the OAuth flow. */
+ res = PQauthDataHook(PQAUTHDATA_OAUTH_BEARER_TOKEN, conn, &request);
+ if (res > 0)
+ {
+ PQoauthBearerRequest *request_copy;
+
+ if (request.token)
+ {
+ /*
+ * We already have a token, so copy it into the state. (We can't
+ * hold onto the original string, since it may not be safe for us
+ * to free() it.)
+ */
+ PQExpBufferData token;
+
+ initPQExpBuffer(&token);
+ appendPQExpBuffer(&token, "Bearer %s", request.token);
+
+ if (PQExpBufferDataBroken(token))
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ goto fail;
+ }
+
+ state->token = token.data;
+
+ /* short-circuit */
+ if (request.cleanup)
+ request.cleanup(conn, &request);
+ return true;
+ }
+
+ request_copy = malloc(sizeof(*request_copy));
+ if (!request_copy)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ goto fail;
+ }
+
+ memcpy(request_copy, &request, sizeof(request));
+
+ conn->async_auth = run_user_oauth_flow;
+ state->async_ctx = request_copy;
+ state->free_async_ctx = free_request;
+ }
+ else if (res < 0)
+ {
+ libpq_append_conn_error(conn, "user-defined OAuth flow failed");
+ goto fail;
+ }
+ else
+ {
+ /* Use our built-in OAuth flow. */
+ conn->async_auth = pg_fe_run_oauth_flow;
+ }
+
+ return true;
+
+fail:
+ if (request.cleanup)
+ request.cleanup(conn, &request);
+ return false;
+}
+
+static bool
+derive_discovery_uri(PGconn *conn)
+{
+ PQExpBufferData discovery_buf;
+
+ if (conn->oauth_discovery_uri || !conn->oauth_issuer)
+ {
+ /*
+ * Either we already have one, or we aren't able to derive one
+ * ourselves. The latter case is not an error condition; we'll just
+ * ask the server to provide one for us.
+ */
+ return true;
+ }
+
+ initPQExpBuffer(&discovery_buf);
+
+ Assert(!conn->oauth_discovery_uri);
+ Assert(conn->oauth_issuer);
+
+ /*
+ * If we don't yet have a discovery URI, but the user gave us an explicit
+ * issuer, use the .well-known discovery URI for that issuer.
+ */
+ appendPQExpBufferStr(&discovery_buf, conn->oauth_issuer);
+ appendPQExpBufferStr(&discovery_buf, "/.well-known/openid-configuration");
+
+ if (PQExpBufferDataBroken(discovery_buf))
+ goto cleanup;
+
+ conn->oauth_discovery_uri = strdup(discovery_buf.data);
+
+cleanup:
+ termPQExpBuffer(&discovery_buf);
+ return (conn->oauth_discovery_uri != NULL);
+}
+
+static SASLStatus
+oauth_exchange(void *opaq, bool final,
+ char *input, int inputlen,
+ char **output, int *outputlen)
+{
+ fe_oauth_state *state = opaq;
+ PGconn *conn = state->conn;
+
+ *output = NULL;
+ *outputlen = 0;
+
+ switch (state->state)
+ {
+ case FE_OAUTH_INIT:
+ Assert(inputlen == -1);
+
+ if (!derive_discovery_uri(conn))
+ return SASL_FAILED;
+
+ if (conn->oauth_discovery_uri)
+ {
+ if (!conn->oauth_client_id)
+ {
+ /* We can't talk to a server without a client identifier. */
+ libpq_append_conn_error(conn, "no oauth_client_id is set for the connection");
+ return SASL_FAILED;
+ }
+
+ /*
+ * Decide whether we're using a user-provided OAuth flow, or
+ * the one we have built in.
+ */
+ if (!setup_token_request(conn, state))
+ return SASL_FAILED;
+
+ if (state->token)
+ {
+ /*
+ * A really smart user implementation may have already
+ * given us the token (e.g. if there was an unexpired copy
+ * already cached). In that case, we can just fall
+ * through.
+ */
+ }
+ else
+ {
+ /*
+ * Otherwise, we have to hand the connection over to our
+ * OAuth implementation. This involves a number of HTTP
+ * connections and timed waits, so we escape the
+ * synchronous auth processing and tell PQconnectPoll to
+ * transfer control to our async implementation.
+ */
+ Assert(conn->async_auth); /* should have been set
+ * already */
+ state->state = FE_OAUTH_REQUESTING_TOKEN;
+ return SASL_ASYNC;
+ }
+ }
+ else
+ {
+ /*
+ * If we don't have a discovery URI to be able to request a
+ * token, we ask the server for one explicitly with an empty
+ * token. This doesn't require any asynchronous work.
+ */
+ state->token = strdup("");
+ if (!state->token)
+ {
+ libpq_append_conn_error(conn, "out of memory");
+ return SASL_FAILED;
+ }
+ }
+
+ /* fall through */
+
+ case FE_OAUTH_REQUESTING_TOKEN:
+ /* We should still be in the initial response phase. */
+ Assert(inputlen == -1);
+
+ *output = client_initial_response(conn, state->token);
+ if (!*output)
+ return SASL_FAILED;
+
+ *outputlen = strlen(*output);
+ state->state = FE_OAUTH_BEARER_SENT;
+
+ return SASL_CONTINUE;
+
+ case FE_OAUTH_BEARER_SENT:
+ if (final)
+ {
+ /* TODO: ensure there is no message content here. */
+ return SASL_COMPLETE;
+ }
+
+ /*
+ * Error message sent by the server.
+ */
+ if (!handle_oauth_sasl_error(conn, input, inputlen))
+ return SASL_FAILED;
+
+ /*
+ * Respond with the required dummy message (RFC 7628, sec. 3.2.3).
+ */
+ *output = strdup(kvsep);
+ *outputlen = strlen(*output); /* == 1 */
+
+ state->state = FE_OAUTH_SERVER_ERROR;
+ return SASL_CONTINUE;
+
+ case FE_OAUTH_SERVER_ERROR:
+
+ /*
+ * After an error, the server should send an error response to
+ * fail the SASL handshake, which is handled in higher layers.
+ *
+ * If we get here, the server either sent *another* challenge
+ * which isn't defined in the RFC, or completed the handshake
+ * successfully after telling us it was going to fail. Neither is
+ * acceptable.
+ */
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("server sent additional OAuth data after error\n"));
+ return SASL_FAILED;
+
+ default:
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("invalid OAuth exchange state\n"));
+ break;
+ }
+
+ return SASL_FAILED;
+}
+
+static bool
+oauth_channel_bound(void *opaq)
+{
+ /* This mechanism does not support channel binding. */
+ return false;
+}
+
+static void
+oauth_free(void *opaq)
+{
+ fe_oauth_state *state = opaq;
+
+ free(state->token);
+ if (state->async_ctx)
+ state->free_async_ctx(state->conn, state->async_ctx);
+
+ free(state);
+}
diff --git a/src/interfaces/libpq/fe-auth-oauth.h b/src/interfaces/libpq/fe-auth-oauth.h
new file mode 100644
index 0000000000..8d4ea45aa8
--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth.h
@@ -0,0 +1,42 @@
+/*-------------------------------------------------------------------------
+ *
+ * fe-auth-oauth.h
+ *
+ * Definitions for OAuth authentication implementations
+ *
+ * Portions Copyright (c) 2023, PostgreSQL Global Development Group
+ *
+ * src/interfaces/libpq/fe-auth-oauth.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef FE_AUTH_OAUTH_H
+#define FE_AUTH_OAUTH_H
+
+#include "libpq-fe.h"
+#include "libpq-int.h"
+
+
+typedef enum
+{
+ FE_OAUTH_INIT,
+ FE_OAUTH_REQUESTING_TOKEN,
+ FE_OAUTH_BEARER_SENT,
+ FE_OAUTH_SERVER_ERROR,
+} fe_oauth_state_enum;
+
+typedef struct
+{
+ fe_oauth_state_enum state;
+
+ PGconn *conn;
+ char *token;
+
+ void *async_ctx;
+ void (*free_async_ctx) (PGconn *conn, void *ctx);
+} fe_oauth_state;
+
+extern PostgresPollingStatusType pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock);
+
+#endif /* FE_AUTH_OAUTH_H */
diff --git a/src/interfaces/libpq/fe-auth-sasl.h b/src/interfaces/libpq/fe-auth-sasl.h
index 4eecf53a15..83bdab1f40 100644
--- a/src/interfaces/libpq/fe-auth-sasl.h
+++ b/src/interfaces/libpq/fe-auth-sasl.h
@@ -30,6 +30,7 @@ typedef enum
SASL_COMPLETE = 0,
SASL_FAILED,
SASL_CONTINUE,
+ SASL_ASYNC,
} SASLStatus;
/*
@@ -77,6 +78,8 @@ typedef struct pg_fe_sasl_mech
*
* state: The opaque mechanism state returned by init()
*
+ * final: true if the server has sent a final exchange outcome
+ *
* input: The challenge data sent by the server, or NULL when
* generating a client-first initial response (that is, when
* the server expects the client to send a message to start
@@ -101,12 +104,17 @@ typedef struct pg_fe_sasl_mech
*
* SASL_CONTINUE: The output buffer is filled with a client response.
* Additional server challenge is expected
+ * SASL_ASYNC: Some asynchronous processing external to the
+ * connection needs to be done before a response can be
+ * generated. The mechanism is responsible for setting up
+ * conn->async_auth appropriately before returning.
* SASL_COMPLETE: The SASL exchange has completed successfully.
* SASL_FAILED: The exchange has failed and the connection should be
* dropped.
*--------
*/
- SASLStatus (*exchange) (void *state, char *input, int inputlen,
+ SASLStatus (*exchange) (void *state, bool final,
+ char *input, int inputlen,
char **output, int *outputlen);
/*--------
diff --git a/src/interfaces/libpq/fe-auth-scram.c b/src/interfaces/libpq/fe-auth-scram.c
index 0bb820e0d9..da168eb2f5 100644
--- a/src/interfaces/libpq/fe-auth-scram.c
+++ b/src/interfaces/libpq/fe-auth-scram.c
@@ -24,7 +24,8 @@
/* The exported SCRAM callback mechanism. */
static void *scram_init(PGconn *conn, const char *password,
const char *sasl_mechanism);
-static SASLStatus scram_exchange(void *opaq, char *input, int inputlen,
+static SASLStatus scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen);
static bool scram_channel_bound(void *opaq);
static void scram_free(void *opaq);
@@ -202,7 +203,8 @@ scram_free(void *opaq)
* Exchange a SCRAM message with backend.
*/
static SASLStatus
-scram_exchange(void *opaq, char *input, int inputlen,
+scram_exchange(void *opaq, bool final,
+ char *input, int inputlen,
char **output, int *outputlen)
{
fe_scram_state *state = (fe_scram_state *) opaq;
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index 3b25d8afda..d02424e11b 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -39,6 +39,7 @@
#endif
#include "common/md5.h"
+#include "common/oauth-common.h"
#include "common/scram-common.h"
#include "fe-auth.h"
#include "fe-auth-sasl.h"
@@ -419,7 +420,7 @@ pg_SSPI_startup(PGconn *conn, int use_negotiate, int payloadlen)
* Initialize SASL authentication exchange.
*/
static int
-pg_SASL_init(PGconn *conn, int payloadlen)
+pg_SASL_init(PGconn *conn, int payloadlen, bool *async)
{
char *initialresponse = NULL;
int initialresponselen;
@@ -437,7 +438,7 @@ pg_SASL_init(PGconn *conn, int payloadlen)
goto error;
}
- if (conn->sasl_state)
+ if (conn->sasl_state && !conn->async_auth)
{
libpq_append_conn_error(conn, "duplicate SASL authentication request");
goto error;
@@ -524,6 +525,15 @@ pg_SASL_init(PGconn *conn, int payloadlen)
conn->sasl = &pg_scram_mech;
conn->password_needed = true;
}
+#ifdef USE_OAUTH
+ else if (strcmp(mechanism_buf.data, OAUTHBEARER_NAME) == 0 &&
+ !selected_mechanism)
+ {
+ selected_mechanism = OAUTHBEARER_NAME;
+ conn->sasl = &pg_oauth_mech;
+ conn->password_needed = false;
+ }
+#endif
}
if (!selected_mechanism)
@@ -563,26 +573,48 @@ pg_SASL_init(PGconn *conn, int payloadlen)
Assert(conn->sasl);
- /*
- * Initialize the SASL state information with all the information gathered
- * during the initial exchange.
- *
- * Note: Only tls-unique is supported for the moment.
- */
- conn->sasl_state = conn->sasl->init(conn,
- password,
- selected_mechanism);
if (!conn->sasl_state)
- goto oom_error;
+ {
+ /*
+ * Initialize the SASL state information with all the information
+ * gathered during the initial exchange.
+ *
+ * Note: Only tls-unique is supported for the moment.
+ */
+ conn->sasl_state = conn->sasl->init(conn,
+ password,
+ selected_mechanism);
+ if (!conn->sasl_state)
+ goto oom_error;
+ }
+ else
+ {
+ /*
+ * This is only possible if we're returning from an async loop.
+ * Disconnect it now.
+ */
+ Assert(conn->async_auth);
+ conn->async_auth = NULL;
+ }
/* Get the mechanism-specific Initial Client Response, if any */
- status = conn->sasl->exchange(conn->sasl_state,
+ status = conn->sasl->exchange(conn->sasl_state, false,
NULL, -1,
&initialresponse, &initialresponselen);
if (status == SASL_FAILED)
goto error;
+ if (status == SASL_ASYNC)
+ {
+ /*
+ * The mechanism should have set up the necessary callbacks; all we
+ * need to do is signal the caller.
+ */
+ *async = true;
+ return STATUS_OK;
+ }
+
/*
* Build a SASLInitialResponse message, and send it.
*/
@@ -625,7 +657,7 @@ oom_error:
* the protocol.
*/
static int
-pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
+pg_SASL_continue(PGconn *conn, int payloadlen, bool final, bool *async)
{
char *output;
int outputlen;
@@ -650,11 +682,21 @@ pg_SASL_continue(PGconn *conn, int payloadlen, bool final)
/* For safety and convenience, ensure the buffer is NULL-terminated. */
challenge[payloadlen] = '\0';
- status = conn->sasl->exchange(conn->sasl_state,
+ status = conn->sasl->exchange(conn->sasl_state, final,
challenge, payloadlen,
&output, &outputlen);
free(challenge); /* don't need the input anymore */
+ if (status == SASL_ASYNC)
+ {
+ /*
+ * The mechanism should have set up the necessary callbacks; all we
+ * need to do is signal the caller.
+ */
+ *async = true;
+ return STATUS_OK;
+ }
+
if (final && status == SASL_CONTINUE)
{
if (outputlen != 0)
@@ -956,12 +998,18 @@ check_expected_areq(AuthRequest areq, PGconn *conn)
* it. We are responsible for reading any remaining extra data, specific
* to the authentication method. 'payloadlen' is the remaining length in
* the message.
+ *
+ * If *async is set to true on return, the client doesn't yet have enough
+ * information to respond, and the caller must temporarily switch to
+ * conn->async_auth() to continue driving the exchange.
*/
int
-pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
+pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn, bool *async)
{
int oldmsglen;
+ *async = false;
+
if (!check_expected_areq(areq, conn))
return STATUS_ERROR;
@@ -1119,7 +1167,7 @@ pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
* The request contains the name (as assigned by IANA) of the
* authentication mechanism.
*/
- if (pg_SASL_init(conn, payloadlen) != STATUS_OK)
+ if (pg_SASL_init(conn, payloadlen, async) != STATUS_OK)
{
/* pg_SASL_init already set the error message */
return STATUS_ERROR;
@@ -1136,7 +1184,8 @@ pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn)
}
oldmsglen = conn->errorMessage.len;
if (pg_SASL_continue(conn, payloadlen,
- (areq == AUTH_REQ_SASL_FIN)) != STATUS_OK)
+ (areq == AUTH_REQ_SASL_FIN),
+ async) != STATUS_OK)
{
/* Use this message if pg_SASL_continue didn't supply one */
if (conn->errorMessage.len == oldmsglen)
@@ -1452,3 +1501,23 @@ PQchangePassword(PGconn *conn, const char *user, const char *passwd)
}
}
}
+
+PQauthDataHook_type PQauthDataHook = PQdefaultAuthDataHook;
+
+PQauthDataHook_type
+PQgetAuthDataHook(void)
+{
+ return PQauthDataHook;
+}
+
+void
+PQsetAuthDataHook(PQauthDataHook_type hook)
+{
+ PQauthDataHook = hook ? hook : PQdefaultAuthDataHook;
+}
+
+int
+PQdefaultAuthDataHook(PGAuthData type, PGconn *conn, void *data)
+{
+ return 0; /* handle nothing */
+}
diff --git a/src/interfaces/libpq/fe-auth.h b/src/interfaces/libpq/fe-auth.h
index a18c508688..3e2bc1333f 100644
--- a/src/interfaces/libpq/fe-auth.h
+++ b/src/interfaces/libpq/fe-auth.h
@@ -18,8 +18,12 @@
#include "libpq-int.h"
+extern PQauthDataHook_type PQauthDataHook;
+
+
/* Prototypes for functions in fe-auth.c */
-extern int pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn);
+extern int pg_fe_sendauth(AuthRequest areq, int payloadlen, PGconn *conn,
+ bool *async);
extern char *pg_fe_getusername(uid_t user_id, PQExpBuffer errorMessage);
extern char *pg_fe_getauthname(PQExpBuffer errorMessage);
@@ -29,4 +33,7 @@ extern char *pg_fe_scram_build_secret(const char *password,
int iterations,
const char **errstr);
+/* Mechanisms in fe-auth-oauth.c */
+extern const pg_fe_sasl_mech pg_oauth_mech;
+
#endif /* FE_AUTH_H */
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 071b1b34aa..44f38f1836 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -364,6 +364,23 @@ static const internalPQconninfoOption PQconninfoOptions[] = {
"Load-Balance-Hosts", "", 8, /* sizeof("disable") = 8 */
offsetof(struct pg_conn, load_balance_hosts)},
+ /* OAuth v2 */
+ {"oauth_issuer", NULL, NULL, NULL,
+ "OAuth-Issuer", "", 40,
+ offsetof(struct pg_conn, oauth_issuer)},
+
+ {"oauth_client_id", NULL, NULL, NULL,
+ "OAuth-Client-ID", "", 40,
+ offsetof(struct pg_conn, oauth_client_id)},
+
+ {"oauth_client_secret", NULL, NULL, NULL,
+ "OAuth-Client-Secret", "", 40,
+ offsetof(struct pg_conn, oauth_client_secret)},
+
+ {"oauth_scope", NULL, NULL, NULL,
+ "OAuth-Scope", "", 15,
+ offsetof(struct pg_conn, oauth_scope)},
+
/* Terminating entry --- MUST BE LAST */
{NULL, NULL, NULL, NULL,
NULL, NULL, 0}
@@ -627,6 +644,7 @@ pqDropServerData(PGconn *conn)
conn->write_failed = false;
free(conn->write_err_msg);
conn->write_err_msg = NULL;
+ /* conn->oauth_want_retry = false; TODO */
/*
* Cancel connections need to retain their be_pid and be_key across
@@ -2644,6 +2662,7 @@ PQconnectPoll(PGconn *conn)
case CONNECTION_NEEDED:
case CONNECTION_GSS_STARTUP:
case CONNECTION_CHECK_TARGET:
+ case CONNECTION_AUTHENTICATING:
break;
default:
@@ -3680,6 +3699,7 @@ keep_going: /* We will come back to here until there is
int avail;
AuthRequest areq;
int res;
+ bool async;
/*
* Scan the message from current point (note that if we find
@@ -3835,6 +3855,16 @@ keep_going: /* We will come back to here until there is
/* Check to see if we should mention pgpassfile */
pgpassfileWarning(conn);
+#ifdef USE_OAUTH
+ if (conn->sasl == &pg_oauth_mech
+ && conn->oauth_want_retry)
+ {
+ /* TODO: only allow retry once */
+ need_new_connection = true;
+ goto keep_going;
+ }
+#endif
+
CONNECTION_FAILED();
}
else if (beresp == PqMsg_NegotiateProtocolVersion)
@@ -3868,7 +3898,17 @@ keep_going: /* We will come back to here until there is
* Note that conn->pghost must be non-NULL if we are going to
* avoid the Kerberos code doing a hostname look-up.
*/
- res = pg_fe_sendauth(areq, msgLength, conn);
+ res = pg_fe_sendauth(areq, msgLength, conn, &async);
+
+ if (async && (res == STATUS_OK))
+ {
+ /*
+ * We'll come back later once we're ready to respond.
+ * Don't consume the request yet.
+ */
+ conn->status = CONNECTION_AUTHENTICATING;
+ goto keep_going;
+ }
/* OK, we have processed the message; mark data consumed */
conn->inStart = conn->inCursor;
@@ -3901,6 +3941,41 @@ keep_going: /* We will come back to here until there is
goto keep_going;
}
+ case CONNECTION_AUTHENTICATING:
+ {
+ PostgresPollingStatusType status;
+ pgsocket altsock = PGINVALID_SOCKET;
+
+ if (!conn->async_auth)
+ {
+ /* programmer error; should not happen */
+ libpq_append_conn_error(conn, "async authentication has no handler");
+ goto error_return;
+ }
+
+ status = conn->async_auth(conn, &altsock);
+
+ if (status == PGRES_POLLING_FAILED)
+ goto error_return;
+
+ if (status == PGRES_POLLING_OK)
+ {
+ /*
+ * Reenter the authentication exchange with the server. We
+ * didn't consume the message that started external
+ * authentication, so it'll be reprocessed as if we just
+ * received it.
+ */
+ conn->status = CONNECTION_AWAITING_RESPONSE;
+ conn->altsock = PGINVALID_SOCKET; /* TODO: what frees
+ * this? */
+ goto keep_going;
+ }
+
+ conn->altsock = altsock;
+ return status;
+ }
+
case CONNECTION_AUTH_OK:
{
/*
@@ -4582,6 +4657,7 @@ pqMakeEmptyPGconn(void)
conn->verbosity = PQERRORS_DEFAULT;
conn->show_context = PQSHOW_CONTEXT_ERRORS;
conn->sock = PGINVALID_SOCKET;
+ conn->altsock = PGINVALID_SOCKET;
conn->Pfdebug = NULL;
/*
@@ -4699,6 +4775,11 @@ freePGconn(PGconn *conn)
free(conn->rowBuf);
free(conn->target_session_attrs);
free(conn->load_balance_hosts);
+ free(conn->oauth_issuer);
+ free(conn->oauth_discovery_uri);
+ free(conn->oauth_client_id);
+ free(conn->oauth_client_secret);
+ free(conn->oauth_scope);
termPQExpBuffer(&conn->errorMessage);
termPQExpBuffer(&conn->workBuffer);
@@ -7181,6 +7262,8 @@ PQsocket(const PGconn *conn)
{
if (!conn)
return -1;
+ if (conn->altsock != PGINVALID_SOCKET)
+ return conn->altsock;
return (conn->sock != PGINVALID_SOCKET) ? conn->sock : -1;
}
diff --git a/src/interfaces/libpq/fe-misc.c b/src/interfaces/libpq/fe-misc.c
index f235bfbb41..aa1fee38c8 100644
--- a/src/interfaces/libpq/fe-misc.c
+++ b/src/interfaces/libpq/fe-misc.c
@@ -1041,10 +1041,13 @@ static int
pqSocketCheck(PGconn *conn, int forRead, int forWrite, pg_usec_time_t end_time)
{
int result;
+ pgsocket sock;
if (!conn)
return -1;
- if (conn->sock == PGINVALID_SOCKET)
+
+ sock = (conn->altsock != PGINVALID_SOCKET) ? conn->altsock : conn->sock;
+ if (sock == PGINVALID_SOCKET)
{
libpq_append_conn_error(conn, "invalid socket");
return -1;
@@ -1061,7 +1064,7 @@ pqSocketCheck(PGconn *conn, int forRead, int forWrite, pg_usec_time_t end_time)
/* We will retry as long as we get EINTR */
do
- result = PQsocketPoll(conn->sock, forRead, forWrite, end_time);
+ result = PQsocketPoll(sock, forRead, forWrite, end_time);
while (result < 0 && SOCK_ERRNO == EINTR);
if (result < 0)
diff --git a/src/interfaces/libpq/libpq-fe.h b/src/interfaces/libpq/libpq-fe.h
index 87a6f3df07..25f216afcf 100644
--- a/src/interfaces/libpq/libpq-fe.h
+++ b/src/interfaces/libpq/libpq-fe.h
@@ -38,6 +38,8 @@ extern "C"
#define LIBPQ_HAS_TRACE_FLAGS 1
/* Indicates that PQsslAttribute(NULL, "library") is useful */
#define LIBPQ_HAS_SSL_LIBRARY_DETECTION 1
+/* Indicates presence of the PQAUTHDATA_PROMPT_OAUTH_DEVICE authdata hook */
+#define LIBPQ_HAS_PROMPT_OAUTH_DEVICE 1
/*
* Option flags for PQcopyResult
@@ -82,6 +84,8 @@ typedef enum
CONNECTION_CHECK_STANDBY, /* Checking if server is in standby mode. */
CONNECTION_ALLOCATED, /* Waiting for connection attempt to be
* started. */
+ CONNECTION_AUTHENTICATING, /* Authentication is in progress with some
+ * external system. */
} ConnStatusType;
typedef enum
@@ -163,6 +167,13 @@ typedef enum
PQ_PIPELINE_ABORTED
} PGpipelineStatus;
+typedef enum
+{
+ PQAUTHDATA_PROMPT_OAUTH_DEVICE, /* user must visit a device-authorization
+ * URL */
+ PQAUTHDATA_OAUTH_BEARER_TOKEN, /* server requests an OAuth Bearer token */
+} PGAuthData;
+
/* PGconn encapsulates a connection to the backend.
* The contents of this struct are not supposed to be known to applications.
*/
@@ -695,10 +706,74 @@ extern int PQenv2encoding(void);
/* === in fe-auth.c === */
+typedef struct _PQpromptOAuthDevice
+{
+ const char *verification_uri; /* verification URI to visit */
+ const char *user_code; /* user code to enter */
+} PQpromptOAuthDevice;
+
+typedef struct _PQoauthBearerRequest
+{
+ /* Hook inputs (constant across all calls) */
+ const char *const openid_configuration; /* OIDC discovery URI */
+ const char *const scope; /* required scope(s), or NULL */
+
+ /* Hook outputs */
+
+ /*---------
+ * Callback implementing a custom asynchronous OAuth flow.
+ *
+ * The callback may return
+ * - PGRES_POLLING_READING/WRITING, to indicate that a file descriptor
+ * has been stored in *altsock and libpq should wait until it is
+ * readable or writable before calling back;
+ * - PGRES_POLLING_OK, to indicate that the flow is complete and
+ * request->token has been set; or
+ * - PGRES_POLLING_FAILED, to indicate that token retrieval has failed.
+ *
+ * This callback is optional. If the token can be obtained without
+ * blocking during the original call to the PQAUTHDATA_OAUTH_BEARER_TOKEN
+ * hook, it may be returned directly, but one of request->async or
+ * request->token must be set by the hook.
+ */
+ PostgresPollingStatusType (*async) (PGconn *conn,
+ struct _PQoauthBearerRequest *request,
+ int *altsock);
+
+ /*
+ * Callback to clean up custom allocations. A hook implementation may use
+ * this to free request->token and any resources in request->user.
+ *
+ * This is technically optional, but highly recommended, because there is
+ * no other indication as to when it is safe to free the token.
+ */
+ void (*cleanup) (PGconn *conn, struct _PQoauthBearerRequest *request);
+
+ /*
+ * The hook should set this to the Bearer token contents for the
+ * connection, once the flow is completed. The token contents must remain
+ * available to libpq until the hook's cleanup callback is called.
+ */
+ char *token;
+
+ /*
+ * Hook-defined data. libpq will not modify this pointer across calls to
+ * the async callback, so it can be used to keep track of
+ * application-specific state. Resources allocated here should be freed by
+ * the cleanup callback.
+ */
+ void *user;
+} PQoauthBearerRequest;
+
extern char *PQencryptPassword(const char *passwd, const char *user);
extern char *PQencryptPasswordConn(PGconn *conn, const char *passwd, const char *user, const char *algorithm);
extern PGresult *PQchangePassword(PGconn *conn, const char *user, const char *passwd);
+typedef int (*PQauthDataHook_type) (PGAuthData type, PGconn *conn, void *data);
+extern void PQsetAuthDataHook(PQauthDataHook_type hook);
+extern PQauthDataHook_type PQgetAuthDataHook(void);
+extern int PQdefaultAuthDataHook(PGAuthData type, PGconn *conn, void *data);
+
/* === in encnames.c === */
extern int pg_char_to_encoding(const char *name);
diff --git a/src/interfaces/libpq/libpq-int.h b/src/interfaces/libpq/libpq-int.h
index f36d76bf3f..c9d9213cf3 100644
--- a/src/interfaces/libpq/libpq-int.h
+++ b/src/interfaces/libpq/libpq-int.h
@@ -357,6 +357,8 @@ typedef struct pg_conn_host
* found in password file. */
} pg_conn_host;
+typedef PostgresPollingStatusType (*AsyncAuthFunc) (PGconn *conn, pgsocket *altsock);
+
/*
* PGconn stores all the state data associated with a single connection
* to a backend.
@@ -420,6 +422,15 @@ struct pg_conn
* cancel request, instead of being a normal
* connection that's used for queries */
+ /* OAuth v2 */
+ char *oauth_issuer; /* token issuer URL */
+ char *oauth_discovery_uri; /* URI of the issuer's discovery
+ * document */
+ char *oauth_client_id; /* client identifier */
+ char *oauth_client_secret; /* client secret */
+ char *oauth_scope; /* access token scope */
+ bool oauth_want_retry; /* should we retry on failure? */
+
/* Optional file to write trace info to */
FILE *Pfdebug;
int traceFlags;
@@ -491,6 +502,9 @@ struct pg_conn
bool client_finished_auth; /* have we finished our half of the
* authentication exchange? */
+ AsyncAuthFunc async_auth; /* callback for external async authentication */
+ pgsocket altsock; /* alternative socket for client to poll */
+
/* Transient state needed while establishing connection */
PGTargetServerType target_server_type; /* desired session properties */
diff --git a/src/interfaces/libpq/meson.build b/src/interfaces/libpq/meson.build
index ed2a4048d1..0181e5cc03 100644
--- a/src/interfaces/libpq/meson.build
+++ b/src/interfaces/libpq/meson.build
@@ -40,6 +40,13 @@ if gssapi.found()
)
endif
+if oauth.found()
+ libpq_sources += files('fe-auth-oauth.c')
+ if oauth_library == 'curl'
+ libpq_sources += files('fe-auth-oauth-curl.c')
+ endif
+endif
+
export_file = custom_target('libpq.exports',
kwargs: gen_export_kwargs,
)
diff --git a/src/makefiles/meson.build b/src/makefiles/meson.build
index 5618050b30..830da57994 100644
--- a/src/makefiles/meson.build
+++ b/src/makefiles/meson.build
@@ -233,6 +233,7 @@ pgxs_deps = {
'llvm': llvm,
'lz4': lz4,
'nls': libintl,
+ 'oauth': oauth,
'pam': pam,
'perl': perl_dep,
'python': python3_dep,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index e6c1caf649..8f323da558 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -153,6 +153,7 @@ ArrayMetaState
ArraySubWorkspace
ArrayToken
ArrayType
+AsyncAuthFunc
AsyncQueueControl
AsyncQueueEntry
AsyncRequest
@@ -366,6 +367,8 @@ CState
CTECycleClause
CTEMaterialize
CTESearchClause
+CURL
+CURLM
CV
CachedExpression
CachedPlan
@@ -1711,6 +1714,7 @@ NumericDigit
NumericSortSupport
NumericSumAccum
NumericVar
+OAuthStep
OM_uint32
OP
OSAPerGroupState
@@ -1775,6 +1779,7 @@ PFN
PGAlignedBlock
PGAlignedXLogBlock
PGAsyncStatusType
+PGAuthData
PGCALL2
PGChecksummablePage
PGContextVisibility
@@ -1935,11 +1940,14 @@ PQArgBlock
PQEnvironmentOption
PQExpBuffer
PQExpBufferData
+PQauthDataHook_type
PQcommMethods
PQconninfoOption
PQnoticeProcessor
PQnoticeReceiver
+PQoauthBearerRequest
PQprintOpt
+PQpromptOAuthDevice
PQsslKeyPassHook_OpenSSL_type
PREDICATELOCK
PREDICATELOCKTAG
@@ -3438,6 +3446,8 @@ explain_get_index_name_hook_type
f_smgr
fasthash_state
fd_set
+fe_oauth_state
+fe_oauth_state_enum
fe_scram_state
fe_scram_state_enum
fetch_range_request
--
2.34.1
v23-0006-Review-comments.patch (application/octet-stream, 32.6K) download
From 84c6893325491fc860bcb903741e894b60478f9a Mon Sep 17 00:00:00 2001
From: Daniel Gustafsson <daniel@yesql.se>
Date: Thu, 28 Mar 2024 21:59:02 +0100
Subject: [PATCH v23 6/6] Review comments
Fixes and tidy-ups following a review of v21, a few items
are (listed in no specific order):
* Implement a version check for libcurl in autoconf, the
equivalent check for Meson is still a TODO.
* Address a few TODOs in the code
* libpq JSON support memory management fixups [ed: these have been moved
to an earlier commit]
---
config/programs.m4 | 21 ++
configure | 34 +++
configure.ac | 1 +
src/backend/libpq/auth-oauth.c | 28 +-
src/include/common/oauth-common.h | 2 +-
src/interfaces/libpq/Makefile | 4 +-
src/interfaces/libpq/fe-auth-oauth-curl.c | 275 +++++++++++--------
src/interfaces/libpq/fe-auth-oauth.c | 9 +-
src/interfaces/libpq/fe-auth-oauth.h | 2 +-
src/test/modules/oauth_validator/validator.c | 2 +-
src/test/perl/PostgreSQL/Test/OAuthServer.pm | 10 +-
src/tools/pgindent/typedefs.list | 1 +
12 files changed, 256 insertions(+), 133 deletions(-)
diff --git a/config/programs.m4 b/config/programs.m4
index 490ec9fe9f..157da7eec5 100644
--- a/config/programs.m4
+++ b/config/programs.m4
@@ -142,7 +142,28 @@ if test "$pgac_cv_ldap_safe" != yes; then
*** also uses LDAP will crash on exit.])
fi])
+# PGAC_CHECK_LIBCURL
+# ------------------
+# Check for libcurl 8.4.0 or higher since earlier versions can be compiled
+# with a codepatch containing exit(), and PostgreSQL does not allow any lib
+# linked to libpq which can call exit.
+# PGAC_CHECK_LIBCURL
+AC_DEFUN([PGAC_CHECK_LIBCURL],
+[AC_CACHE_CHECK([for compatible libcurl], [pgac_cv_check_libcurl],
+[AC_COMPILE_IFELSE([AC_LANG_PROGRAM(
+[#include <curl/curlver.h>
+#if LIBCURL_VERSION_MAJOR <= 8 && LIBCURL_VERSION_MINOR < 4
+choke me
+#endif], [])],
+[pgac_cv_check_libcurl=yes],
+[pgac_cv_check_libcurl=no])])
+
+if test "$pgac_cv_check_libcurl" != yes; then
+ AC_MSG_ERROR([
+*** The installed version of libcurl is too old to use with PostgreSQL.
+*** libcurl version 8.4.0 or later is required.])
+fi])
# PGAC_CHECK_READLINE
# -------------------
diff --git a/configure b/configure
index 864bb9b7c3..1a785006b9 100755
--- a/configure
+++ b/configure
@@ -13137,6 +13137,40 @@ else
as_fn_error $? "library 'curl' is required for --with-oauth=curl" "$LINENO" 5
fi
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for compatible libcurl" >&5
+$as_echo_n "checking for compatible libcurl... " >&6; }
+if ${pgac_cv_check_libcurl+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+#include <curl/curlver.h>
+#if LIBCURL_VERSION_MAJOR <= 8 && LIBCURL_VERSION_MINOR < 4
+choke me
+#endif
+int
+main ()
+{
+
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_compile "$LINENO"; then :
+ pgac_cv_check_libcurl=yes
+else
+ pgac_cv_check_libcurl=no
+fi
+rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $pgac_cv_check_libcurl" >&5
+$as_echo "$pgac_cv_check_libcurl" >&6; }
+
+if test "$pgac_cv_check_libcurl" != yes; then
+ as_fn_error $? "
+*** The installed version of libcurl is too old to use with PostgreSQL.
+*** libcurl version 8.4.0 or later is required." "$LINENO" 5
+fi
fi
# for contrib/sepgsql
diff --git a/configure.ac b/configure.ac
index 6aceed898c..cdc6bea660 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1445,6 +1445,7 @@ AC_SUBST(LDAP_LIBS_BE)
if test "$with_oauth" = curl ; then
AC_CHECK_LIB(curl, curl_multi_init, [], [AC_MSG_ERROR([library 'curl' is required for --with-oauth=curl])])
+ PGAC_CHECK_LIBCURL
fi
# for contrib/sepgsql
diff --git a/src/backend/libpq/auth-oauth.c b/src/backend/libpq/auth-oauth.c
index 024f304e4d..ec1418c3fc 100644
--- a/src/backend/libpq/auth-oauth.c
+++ b/src/backend/libpq/auth-oauth.c
@@ -476,9 +476,9 @@ generate_error_response(struct oauth_ctx *ctx, char **output, int *outputlen)
initStringInfo(&buf);
/*
- * TODO: note that escaping here should be belt-and-suspenders, since
- * escapable characters aren't valid in either the issuer URI or the scope
- * list, but the HBA doesn't enforce that yet.
+ * Escaping the string here is belt-and-suspenders defensive programming
+ * since escapable characters aren't valid in either the issuer URI or the
+ * scope list, but the HBA doesn't enforce that yet.
*/
appendStringInfoString(&buf, "{ \"status\": \"invalid_token\", ");
@@ -533,7 +533,9 @@ validate_token_format(const char *header)
if (!header || strlen(header) <= 7)
{
ereport(COMMERROR,
- (errmsg("malformed OAuth bearer token 1")));
+ errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAuth bearer token"),
+ errdetail_log("Bearer token is less than 8 bytes."));
return NULL;
}
@@ -551,9 +553,9 @@ validate_token_format(const char *header)
if (!*token)
{
ereport(COMMERROR,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("malformed OAuth bearer token 2"),
- errdetail("Bearer token is empty.")));
+ errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAuth bearer token"),
+ errdetail_log("Bearer token is empty."));
return NULL;
}
@@ -573,9 +575,9 @@ validate_token_format(const char *header)
* of someone's password into the logs.
*/
ereport(COMMERROR,
- (errcode(ERRCODE_PROTOCOL_VIOLATION),
- errmsg("malformed OAuth bearer token 3"),
- errdetail("Bearer token is not in the correct format.")));
+ errcode(ERRCODE_PROTOCOL_VIOLATION),
+ errmsg("malformed OAuth bearer token"),
+ errdetail_log("Bearer token is not in the correct format."));
return NULL;
}
@@ -617,10 +619,10 @@ validate(Port *port, const char *auth)
/* Make sure the validator authenticated the user. */
if (ret->authn_id == NULL || ret->authn_id[0] == '\0')
{
- /* TODO: use logdetail; reduce message duplication */
ereport(LOG,
- (errmsg("OAuth bearer authentication failed for user \"%s\": validator provided no identity",
- port->user_name)));
+ errmsg("OAuth bearer authentication failed for user \"%s\"",
+ port->user_name),
+ errdetail_log("Validator provided no identity"));
return false;
}
diff --git a/src/include/common/oauth-common.h b/src/include/common/oauth-common.h
index 5ff3488bfb..8fe5626778 100644
--- a/src/include/common/oauth-common.h
+++ b/src/include/common/oauth-common.h
@@ -3,7 +3,7 @@
* oauth-common.h
* Declarations for helper functions used for OAuth/OIDC authentication
*
- * Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* src/include/common/oauth-common.h
diff --git a/src/interfaces/libpq/Makefile b/src/interfaces/libpq/Makefile
index 0e0369cb63..9a290782f2 100644
--- a/src/interfaces/libpq/Makefile
+++ b/src/interfaces/libpq/Makefile
@@ -116,6 +116,8 @@ backend_src = $(top_srcdir)/src/backend
# which seems to insert references to that even in pure C code. Excluding
# __tsan_func_exit is necessary when using ThreadSanitizer data race detector
# which use this function for instrumentation of function exit.
+# libcurl registers an exit handler in the memory debugging code when running
+# with LeakSanitizer.
# Skip the test when profiling, as gcc may insert exit() calls for that.
# Also skip the test on platforms where libpq infrastructure may be provided
# by statically-linked libraries, as we can't expect them to honor this
@@ -123,7 +125,7 @@ backend_src = $(top_srcdir)/src/backend
libpq-refs-stamp: $(shlib)
ifneq ($(enable_coverage), yes)
ifeq (,$(filter solaris,$(PORTNAME)))
- @if nm -A -u $< 2>/dev/null | grep -v -e __cxa_atexit -e __tsan_func_exit | grep exit; then \
+ @if nm -A -u $< 2>/dev/null | grep -v -e __cxa_atexit -e __tsan_func_exit -e _atexit | grep exit; then \
echo 'libpq must not be calling any function which invokes exit'; exit 1; \
fi
endif
diff --git a/src/interfaces/libpq/fe-auth-oauth-curl.c b/src/interfaces/libpq/fe-auth-oauth-curl.c
index 0504f96e4e..9dd8454cac 100644
--- a/src/interfaces/libpq/fe-auth-oauth-curl.c
+++ b/src/interfaces/libpq/fe-auth-oauth-curl.c
@@ -3,7 +3,7 @@
* fe-auth-oauth-curl.c
* The libcurl implementation of OAuth/OIDC authentication.
*
- * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* IDENTIFICATION
@@ -31,6 +31,8 @@
#include "libpq-int.h"
#include "mb/pg_wchar.h"
+#define MAX_OAUTH_RESPONSE_SIZE (1024 * 1024)
+
/*
* Parsed JSON Representations
*
@@ -143,7 +145,7 @@ free_token(struct token *tok)
/* States for the overall async machine. */
typedef enum
{
- OAUTH_STEP_INIT,
+ OAUTH_STEP_INIT = 0,
OAUTH_STEP_DISCOVERY,
OAUTH_STEP_DEVICE_AUTHORIZATION,
OAUTH_STEP_TOKEN_REQUEST,
@@ -151,8 +153,9 @@ typedef enum
} OAuthStep;
/*
- * The async_ctx holds onto state that needs to persist across multiple calls to
- * pg_fe_run_oauth_flow(). Almost everything interacts with this in some way.
+ * The async_ctx holds onto state that needs to persist across multiple calls
+ * to pg_fe_run_oauth_flow(). Almost everything interacts with this in some
+ * way.
*/
struct async_ctx
{
@@ -162,9 +165,10 @@ struct async_ctx
int timerfd; /* a timerfd for signaling async timeouts */
#endif
pgsocket mux; /* the multiplexer socket containing all
- * descriptors tracked by cURL, plus the
+ * descriptors tracked by libcurl, plus the
* timerfd */
- CURLM *curlm; /* top-level multi handle for cURL operations */
+ CURLM *curlm; /* top-level multi handle for libcurl
+ * operations */
CURL *curl; /* the (single) easy handle for serial
* requests */
@@ -183,7 +187,7 @@ struct async_ctx
* actx_error[_str] to manipulate this. This must be filled
* with something useful on an error.
*
- * - curl_err: an optional static error buffer used by cURL to put
+ * - curl_err: an optional static error buffer used by libcurl to put
* detailed information about failures. Unfortunately
* untranslatable.
*
@@ -195,7 +199,7 @@ struct async_ctx
*/
const char *errctx; /* not freed; must point to static allocation */
PQExpBufferData errbuf;
- char curl_err[CURL_ERROR_SIZE];
+ PQExpBufferData curl_err;
/*
* These documents need to survive over multiple calls, and are therefore
@@ -205,6 +209,8 @@ struct async_ctx
struct device_authz authz;
bool user_prompted; /* have we already sent the authz prompt? */
+
+ int running;
};
/*
@@ -238,7 +244,7 @@ free_curl_async_ctx(PGconn *conn, void *ctx)
if (err)
libpq_append_conn_error(conn,
- "cURL easy handle removal failed: %s",
+ "libcurl easy handle removal failed: %s",
curl_multi_strerror(err));
}
@@ -258,7 +264,7 @@ free_curl_async_ctx(PGconn *conn, void *ctx)
if (err)
libpq_append_conn_error(conn,
- "cURL multi handle cleanup failed: %s",
+ "libcurl multi handle cleanup failed: %s",
curl_multi_strerror(err));
}
@@ -292,8 +298,8 @@ free_curl_async_ctx(PGconn *conn, void *ctx)
appendPQExpBufferStr(&(ACTX)->errbuf, S)
/*
- * Macros for getting and setting state for the connection's two cURL handles,
- * so you don't have to write out the error handling every time.
+ * Macros for getting and setting state for the connection's two libcurl
+ * handles, so you don't have to write out the error handling every time.
*/
#define CHECK_MSETOPT(ACTX, OPT, VAL, FAILACTION) \
@@ -622,19 +628,28 @@ parse_oauth_json(struct async_ctx *actx, const struct json_field *fields)
actx_error(actx, "no content type was provided");
goto cleanup;
}
- else if (strcasecmp(content_type, "application/json") != 0)
+
+ /*
+ * We only check the media-type and not the parameters, so we need to
+ * perform a length limited comparison and not compare the whole string.
+ */
+ if (pg_strncasecmp(content_type, "application/json", strlen("application/json")) != 0)
{
- actx_error(actx, "unexpected content type \"%s\"", content_type);
- goto cleanup;
+ actx_error(actx, "unexpected content type: \"%s\"", content_type);
+ return false;
}
if (strlen(resp->data) != resp->len)
{
actx_error(actx, "response contains embedded NULLs");
- goto cleanup;
+ return false;
}
- makeJsonLexContextCstringLen(&lex, resp->data, resp->len, PG_UTF8, true);
+ if (!makeJsonLexContextCstringLen(&lex, resp->data, resp->len, PG_UTF8, true))
+ {
+ actx_error(actx, "out of memory");
+ return false;
+ }
ctx.errbuf = &actx->errbuf;
ctx.fields = fields;
@@ -787,7 +802,11 @@ parse_device_authz(struct async_ctx *actx, struct device_authz *authz)
authz->interval = parse_interval(authz->interval_str);
else
{
- /* TODO: handle default interval of 5 seconds */
+ /*
+ * RFC 8628 specify 5 seconds as the default value if the server
+ * doesn't provide an interval.
+ */
+ authz->interval = 5;
}
return true;
@@ -838,7 +857,7 @@ parse_access_token(struct async_ctx *actx, struct token *tok)
}
/*
- * cURL Multi Setup/Callbacks
+ * libcurl Multi Setup/Callbacks
*/
/*
@@ -894,7 +913,7 @@ setup_multiplexer(struct async_ctx *actx)
/*
* Adds and removes sockets from the multiplexer set, as directed by the
- * cURL multi handle.
+ * libcurl multi handle.
*/
static int
register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
@@ -925,7 +944,7 @@ register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
break;
default:
- actx_error(actx, "unknown cURL socket operation (%d)", what);
+ actx_error(actx, "unknown libcurl socket operation: %d", what);
return -1;
}
@@ -997,7 +1016,7 @@ register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
break;
default:
- actx_error(actx, "unknown cURL socket operation (%d)", what);
+ actx_error(actx, "unknown libcurl socket operation: %d", what);
return -1;
}
@@ -1018,7 +1037,7 @@ register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
/*
* EV_RECEIPT should guarantee one EV_ERROR result for every change,
* whether successful or not. Failed entries contain a non-zero errno
- * in the `data` field.
+ * in the data field.
*/
Assert(ev_out[i].flags & EV_ERROR);
@@ -1043,9 +1062,8 @@ register_socket(CURL *curl, curl_socket_t socket, int what, void *ctx,
/*
* Adds or removes timeouts from the multiplexer set, as directed by the
- * cURL multi handle. Rather than continually adding and removing the timer,
- * we keep it in the set at all times and just disarm it when it's not
- * needed.
+ * libcurl multi handle. Rather than continually adding and removing the timer,
+ * we keep it in the set at all times and just disarm it when it's not needed.
*/
static int
register_timer(CURLM *curlm, long timeout, void *ctx)
@@ -1061,9 +1079,9 @@ register_timer(CURLM *curlm, long timeout, void *ctx)
else if (timeout == 0)
{
/*
- * A zero timeout means cURL wants us to call back immediately. That's
- * not technically an option for timerfd, but we can make the timeout
- * ridiculously short.
+ * A zero timeout means libcurl wants us to call back immediately.
+ * That's not technically an option for timerfd, but we can make the
+ * timeout ridiculously short.
*
* TODO: maybe just signal drive_request() to immediately call back in
* this case?
@@ -1098,8 +1116,21 @@ register_timer(CURLM *curlm, long timeout, void *ctx)
return 0;
}
+static int
+debug_callback(CURL *handle, curl_infotype *type, char *data, size_t size,
+ void *clientp)
+{
+ struct async_ctx *actx = (struct async_ctx *) clientp;
+
+ /* For now we only store TEXT debug information, extending is a TODO */
+ if (type == CURLINFO_TEXT)
+ appendBinaryPQExpBuffer(&actx->curl_err, data, size);
+
+ return 0;
+}
+
/*
- * Initializes the two cURL handles in the async_ctx. The multi handle,
+ * Initializes the two libcurl handles in the async_ctx. The multi handle,
* actx->curlm, is what drives the asynchronous engine and tells us what to do
* next. The easy handle, actx->curl, encapsulates the state for a single
* request/response. It's added to the multi handle as needed, during
@@ -1108,17 +1139,17 @@ register_timer(CURLM *curlm, long timeout, void *ctx)
static bool
setup_curl_handles(struct async_ctx *actx)
{
- curl_version_info_data *curl_info;
+ curl_version_info_data *curl_info;
/*
* Create our multi handle. This encapsulates the entire conversation with
- * cURL for this connection.
+ * libcurl for this connection.
*/
actx->curlm = curl_multi_init();
if (!actx->curlm)
{
/* We don't get a lot of feedback on the failure reason. */
- actx_error(actx, "failed to create cURL multi handle");
+ actx_error(actx, "failed to create libcurl multi handle");
return false;
}
@@ -1143,7 +1174,7 @@ setup_curl_handles(struct async_ctx *actx)
actx->curl = curl_easy_init();
if (!actx->curl)
{
- actx_error(actx, "failed to create cURL handle");
+ actx_error(actx, "failed to create libcurl handle");
return false;
}
@@ -1160,9 +1191,14 @@ setup_curl_handles(struct async_ctx *actx)
/* No alternative resolver, TODO: warn about timeouts */
}
- /* TODO investigate using conn->Pfdebug and CURLOPT_DEBUGFUNCTION here */
+ /*
+ * Set a callback for retrieving error information from libcurl, the
+ * function only takes effect when CURLOPT_VERBOSE has been set so make
+ * sure the order is kept.
+ */
+ CHECK_SETOPT(actx, CURLOPT_DEBUGDATA, actx, return false);
+ CHECK_SETOPT(actx, CURLOPT_DEBUGFUNCTION, debug_callback, return false);
CHECK_SETOPT(actx, CURLOPT_VERBOSE, 1L, return false);
- CHECK_SETOPT(actx, CURLOPT_ERRORBUFFER, actx->curl_err, return false);
/*
* Only HTTP[S] is allowed. TODO: disallow HTTP without user opt-in
@@ -1175,7 +1211,12 @@ setup_curl_handles(struct async_ctx *actx)
* pretty strict when it comes to provider behavior, so we have to check
* what comes back anyway.)
*/
- actx->headers = curl_slist_append(actx->headers, "Accept:"); /* TODO: check result */
+ actx->headers = curl_slist_append(actx->headers, "Accept:");
+ if (actx->headers == NULL)
+ {
+ actx_error(actx, "out of memory");
+ return false;
+ }
CHECK_SETOPT(actx, CURLOPT_HTTPHEADER, actx->headers, return false);
return true;
@@ -1186,8 +1227,10 @@ setup_curl_handles(struct async_ctx *actx)
*/
/*
- * Response callback from cURL; appends the response body into actx->work_data.
- * See start_request().
+ * Response callback from libcurl which appends the response body into
+ * actx->work_data (see start_request()). The maximum size of the data is
+ * defined by CURL_MAX_WRITE_SIZE which by default is 16kb (and can only be
+ * changed by recompiling libcurl).
*/
static size_t
append_data(char *buf, size_t size, size_t nmemb, void *userdata)
@@ -1195,9 +1238,19 @@ append_data(char *buf, size_t size, size_t nmemb, void *userdata)
PQExpBuffer resp = userdata;
size_t len = size * nmemb;
- /* TODO: cap the maximum size */
+ /* In case we receive data over the threshold, abort the transfer */
+ if ((resp->len + len) > MAX_OAUTH_RESPONSE_SIZE)
+ return 0;
+
+ /* The data passed from libcurl is not null-terminated */
appendBinaryPQExpBuffer(resp, buf, len);
- /* TODO: check for broken buffer */
+
+ /*
+ * Signal an error in order to abort the transfer in case we ran out of
+ * memory in accepting the data.
+ */
+ if (PQExpBufferBroken(resp))
+ return 0;
return len;
}
@@ -1214,7 +1267,6 @@ static bool
start_request(struct async_ctx *actx)
{
CURLMcode err;
- int running;
resetPQExpBuffer(&actx->work_data);
CHECK_SETOPT(actx, CURLOPT_WRITEFUNCTION, append_data, return false);
@@ -1228,7 +1280,7 @@ start_request(struct async_ctx *actx)
return false;
}
- err = curl_multi_socket_action(actx->curlm, CURL_SOCKET_TIMEOUT, 0, &running);
+ err = curl_multi_socket_action(actx->curlm, CURL_SOCKET_TIMEOUT, 0, &actx->running);
if (err)
{
actx_error(actx, "asynchronous HTTP request failed: %s",
@@ -1237,19 +1289,11 @@ start_request(struct async_ctx *actx)
}
/*
- * Sanity check.
- *
- * TODO: even though this is nominally an asynchronous process, there are
- * apparently operations that can synchronously fail by this point, such
- * as connections to closed local ports. Maybe we need to let this case
- * fall through to drive_request instead, or else perform a
- * curl_multi_info_read immediately.
+ * Even though this is nominally an asynchronous process, there are some
+ * operations that can synchronously fail by this point like connections
+ * to closed local ports. Fall through and leave the sanity check for the
+ * next state consuming actx.
*/
- if (running != 1)
- {
- actx_error(actx, "failed to queue HTTP request");
- return false;
- }
return true;
}
@@ -1262,12 +1306,18 @@ static PostgresPollingStatusType
drive_request(struct async_ctx *actx)
{
CURLMcode err;
- int running;
CURLMsg *msg;
int msgs_left;
bool done;
- err = curl_multi_socket_all(actx->curlm, &running);
+ /* Sanity check the previous operation */
+ if (actx->running != 1)
+ {
+ actx_error(actx, "failed to queue HTTP request");
+ return false;
+ }
+
+ err = curl_multi_socket_all(actx->curlm, &actx->running);
if (err)
{
actx_error(actx, "asynchronous HTTP request failed: %s",
@@ -1275,7 +1325,7 @@ drive_request(struct async_ctx *actx)
return PGRES_POLLING_FAILED;
}
- if (running)
+ if (actx->running)
{
/* We'll come back again. */
return PGRES_POLLING_READING;
@@ -1287,7 +1337,7 @@ drive_request(struct async_ctx *actx)
if (msg->msg != CURLMSG_DONE)
{
/*
- * Future cURL versions may define new message types; we don't
+ * Future libcurl versions may define new message types; we don't
* know how to handle them, so we'll ignore them.
*/
continue;
@@ -1304,7 +1354,7 @@ drive_request(struct async_ctx *actx)
err = curl_multi_remove_handle(actx->curlm, msg->easy_handle);
if (err)
{
- actx_error(actx, "cURL easy handle removal failed: %s",
+ actx_error(actx, "libcurl easy handle removal failed: %s",
curl_multi_strerror(err));
return PGRES_POLLING_FAILED;
}
@@ -1489,7 +1539,12 @@ start_device_authz(struct async_ctx *actx, PGconn *conn)
appendPQExpBuffer(work_buffer, "client_id=%s", conn->oauth_client_id);
if (conn->oauth_scope)
appendPQExpBuffer(work_buffer, "&scope=%s", conn->oauth_scope);
- /* TODO check for broken buffer */
+
+ if (PQExpBufferBroken(work_buffer))
+ {
+ actx_error(actx, "out of memory");
+ return false;
+ }
/* Make our request. */
CHECK_SETOPT(actx, CURLOPT_URL, device_authz_uri, return false);
@@ -1631,37 +1686,40 @@ finish_token_request(struct async_ctx *actx, struct token *tok)
CHECK_GETINFO(actx, CURLINFO_RESPONSE_CODE, &response_code, return false);
/*
- * Per RFC 6749, Section 5, a successful response uses 200 OK. An error
- * response uses either 400 Bad Request or 401 Unauthorized.
- *
- * TODO: there are references online to 403 appearing in the wild...
+ * Per RFC 6749, Section 5, a successful response uses 200 OK.
*/
- if (response_code != 200
- && response_code != 400
- /* && response_code != 401 TODO */ )
+ if (response_code == 200)
{
- actx_error(actx, "unexpected response code %ld", response_code);
- return false;
+ actx->errctx = "failed to parse access token response";
+ if (!parse_access_token(actx, tok))
+ return false; /* error message already set */
+
+ return true;
}
/*
- * Pull the fields we care about from the document.
+ * An error response uses either 400 Bad Request or 401 Unauthorized.
+ * There are references online to implementations using 403 for error
+ * return which would violate the specification. For now we stick to the
+ * specification but we might have to revisit this.
*/
- if (response_code == 200)
+ if (response_code == 400 || response_code == 401)
{
- actx->errctx = "failed to parse access token response";
- if (!parse_access_token(actx, tok))
- return false; /* error message already set */
+ if (!parse_token_error(actx, &tok->err))
+ return false;
+
+ return true;
}
- else if (!parse_token_error(actx, &tok->err))
- return false;
- return true;
+ /* Any other response codes are considered invalid */
+ actx_error(actx, "unexpected response code %ld", response_code);
+ return false;
}
+
/*
- * The top-level, nonblocking entry point for the cURL implementation. This will
- * be called several times to pump the async engine.
+ * The top-level, nonblocking entry point for the libcurl implementation. This
+ * will be called several times to pump the async engine.
*
* The architecture is based on PQconnectPoll(). The first half drives the
* connection state forward as necessary, returning if we're not ready to
@@ -1682,7 +1740,7 @@ pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
struct token tok = {0};
/*
- * XXX This is not safe. cURL has stringent requirements for the thread
+ * XXX This is not safe. libcurl has stringent requirements for the thread
* context in which you call curl_global_init(), because it's going to try
* initializing a bunch of other libraries (OpenSSL, Winsock...). And we
* probably need to consider both the TLS backend libcurl is compiled
@@ -1691,16 +1749,16 @@ pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
* Recent versions of libcurl have improved the thread-safety situation,
* but you apparently can't check at compile time whether the
* implementation is thread-safe, and there's a chicken-and-egg problem
- * where you can't check the thread safety until you've initialized cURL,
- * which you can't do before you've made sure it's thread-safe...
+ * where you can't check the thread safety until you've initialized
+ * libcurl, which you can't do before you've made sure it's thread-safe...
*
* We know we've already initialized Winsock by this point, so we should
- * be able to safely skip that bit. But we have to tell cURL to initialize
- * everything else, because other pieces of our client executable may
- * already be using cURL for their own purposes. If we initialize libcurl
- * first, with only a subset of its features, we could break those other
- * clients nondeterministically, and that would probably be a nightmare to
- * debug.
+ * be able to safely skip that bit. But we have to tell libcurl to
+ * initialize everything else, because other pieces of our client
+ * executable may already be using libcurl for their own purposes. If we
+ * initialize libcurl first, with only a subset of its features, we could
+ * break those other clients nondeterministically, and that would probably
+ * be a nightmare to debug.
*/
curl_global_init(CURL_GLOBAL_ALL
& ~CURL_GLOBAL_WIN32); /* we already initialized Winsock */
@@ -1729,6 +1787,7 @@ pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
initPQExpBuffer(&actx->work_data);
initPQExpBuffer(&actx->errbuf);
+ initPQExpBuffer(&actx->curl_err);
if (!setup_multiplexer(actx))
goto error_return;
@@ -1873,16 +1932,20 @@ pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
* errors; anything else and we bail.
*/
err = &tok.err;
- if (!err->error || (strcmp(err->error, "authorization_pending")
- && strcmp(err->error, "slow_down")))
+ if (!err->error)
+ {
+ actx_error(actx, "unknown error");
+ goto error_return;
+ }
+
+ if (strcmp(err->error, "authorization_pending") != 0 &&
+ strcmp(err->error, "slow_down") != 0)
{
- /* TODO handle !err->error */
if (err->error_description)
appendPQExpBuffer(&actx->errbuf, "%s ",
err->error_description);
appendPQExpBuffer(&actx->errbuf, "(%s)", err->error);
-
goto error_return;
}
@@ -1892,7 +1955,14 @@ pg_fe_run_oauth_flow(PGconn *conn, pgsocket *altsock)
*/
if (strcmp(err->error, "slow_down") == 0)
{
- actx->authz.interval += 5; /* TODO check for overflow? */
+ int prev_interval = actx->authz.interval;
+
+ actx->authz.interval += 5;
+ if (actx->authz.interval < prev_interval)
+ {
+ actx_error(actx, "slow_down interval overflow");
+ goto error_return;
+ }
}
/*
@@ -1959,21 +2029,8 @@ error_return:
else
appendPQExpBufferStr(&conn->errorMessage, actx->errbuf.data);
- if (actx->curl_err[0])
- {
- size_t len;
-
- appendPQExpBuffer(&conn->errorMessage, " (%s)", actx->curl_err);
-
- /* Sometimes libcurl adds a newline to the error buffer. :( */
- len = conn->errorMessage.len;
- if (len >= 2 && conn->errorMessage.data[len - 2] == '\n')
- {
- conn->errorMessage.data[len - 2] = ')';
- conn->errorMessage.data[len - 1] = '\0';
- conn->errorMessage.len--;
- }
- }
+ if (actx->curl_err.len > 0)
+ appendPQExpBuffer(&conn->errorMessage, " (%s)", actx->curl_err.data);
appendPQExpBufferStr(&conn->errorMessage, "\n");
diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
index 66ee8ff076..61de9ac451 100644
--- a/src/interfaces/libpq/fe-auth-oauth.c
+++ b/src/interfaces/libpq/fe-auth-oauth.c
@@ -3,7 +3,7 @@
* fe-auth-oauth.c
* The front-end (client) implementation of OAuth/OIDC authentication.
*
- * Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
* IDENTIFICATION
@@ -247,7 +247,12 @@ handle_oauth_sasl_error(PGconn *conn, char *msg, int msglen)
return false;
}
- makeJsonLexContextCstringLen(&lex, msg, msglen, PG_UTF8, true);
+ if (!makeJsonLexContextCstringLen(&lex, msg, msglen, PG_UTF8, true))
+ {
+ appendPQExpBufferStr(&conn->errorMessage,
+ libpq_gettext("out of memory"));
+ return false;
+ }
initPQExpBuffer(&ctx.errbuf);
sem.semstate = &ctx;
diff --git a/src/interfaces/libpq/fe-auth-oauth.h b/src/interfaces/libpq/fe-auth-oauth.h
index 8d4ea45aa8..6e5e946364 100644
--- a/src/interfaces/libpq/fe-auth-oauth.h
+++ b/src/interfaces/libpq/fe-auth-oauth.h
@@ -4,7 +4,7 @@
*
* Definitions for OAuth authentication implementations
*
- * Portions Copyright (c) 2023, PostgreSQL Global Development Group
+ * Portions Copyright (c) 2024, PostgreSQL Global Development Group
*
* src/interfaces/libpq/fe-auth-oauth.h
*
diff --git a/src/test/modules/oauth_validator/validator.c b/src/test/modules/oauth_validator/validator.c
index 09a4bf61d2..7b4dc9c494 100644
--- a/src/test/modules/oauth_validator/validator.c
+++ b/src/test/modules/oauth_validator/validator.c
@@ -66,7 +66,7 @@ validate_token(ValidatorModuleState *state, const char *token, const char *role)
/* Check to make sure our private state still exists. */
if (state->private_data != PRIVATE_COOKIE)
elog(ERROR, "oauth_validator: private state cookie changed to %p",
- state->private_data);
+ state->private_data);
res = palloc(sizeof(ValidatorModuleResult));
diff --git a/src/test/perl/PostgreSQL/Test/OAuthServer.pm b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
index d96733f531..9e18186f23 100644
--- a/src/test/perl/PostgreSQL/Test/OAuthServer.pm
+++ b/src/test/perl/PostgreSQL/Test/OAuthServer.pm
@@ -4,10 +4,10 @@ package PostgreSQL::Test::OAuthServer;
use warnings;
use strict;
-use threads;
use Scalar::Util;
use Socket;
use IO::Select;
+use Test::More;
local *server_socket;
@@ -34,9 +34,9 @@ sub run
my $port;
my $pid = open(my $read_fh, "-|", $ENV{PYTHON}, "t/oauth_server.py")
- // die "failed to start OAuth server: $!";
+ or die "failed to start OAuth server: $!";
- read($read_fh, $port, 7) // die "failed to read port number: $!";
+ read($read_fh, $port, 7) or die "failed to read port number: $!";
chomp $port;
die "server did not advertise a valid port"
unless Scalar::Util::looks_like_number($port);
@@ -45,14 +45,14 @@ sub run
$self->{'port'} = $port;
$self->{'child'} = $read_fh;
- print("# OAuth provider (PID $pid) is listening on port $port\n");
+ diag("OAuth provider (PID $pid) is listening on port $port\n");
}
sub stop
{
my $self = shift;
- print("# Sending SIGTERM to OAuth provider PID: $self->{'pid'}\n");
+ diag("Sending SIGTERM to OAuth provider PID: $self->{'pid'}\n");
kill(15, $self->{'pid'});
$self->{'pid'} = undef;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index b02ee48898..7e9b1f564a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -3057,6 +3057,7 @@ VacuumStmt
ValidIOData
ValidateIndexState
ValidatorModuleState
+ValidatorModuleResult
ValuesScan
ValuesScanState
Var
--
2.34.1
v23-0001-Revert-ECPG-s-use-of-pnstrdup.patch (application/octet-stream, 1.8K) download
From aa553b7700376d5eba055b21fa15dd355e1a3939 Mon Sep 17 00:00:00 2001
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Tue, 2 Jul 2024 12:26:04 -0700
Subject: [PATCH v23 1/6] Revert ECPG's use of pnstrdup()
Commit 0b9466fce added a dependency on fe_memutils' pnstrdup() inside
informix.c. This 1) makes it hard to remove fe_memutils from
libpgcommon_shlib, and 2) adds an exit() path where it perhaps should
not be. (See the !str check after the call to pnstrdup, which looks like
it should not be possible.)
Revert that part of the patch for now, pending further discussion on the
thread.
---
src/interfaces/ecpg/compatlib/informix.c | 23 +++++++++++++++++++++--
1 file changed, 21 insertions(+), 2 deletions(-)
diff --git a/src/interfaces/ecpg/compatlib/informix.c b/src/interfaces/ecpg/compatlib/informix.c
index 8ea89e640a..65a0b2e46c 100644
--- a/src/interfaces/ecpg/compatlib/informix.c
+++ b/src/interfaces/ecpg/compatlib/informix.c
@@ -175,6 +175,25 @@ deccopy(decimal *src, decimal *target)
memcpy(target, src, sizeof(decimal));
}
+static char *
+ecpg_strndup(const char *str, size_t len)
+{
+ size_t real_len = strlen(str);
+ int use_len = (int) ((real_len > len) ? len : real_len);
+
+ char *new = malloc(use_len + 1);
+
+ if (new)
+ {
+ memcpy(new, str, use_len);
+ new[use_len] = '\0';
+ }
+ else
+ errno = ENOMEM;
+
+ return new;
+}
+
int
deccvasc(const char *cp, int len, decimal *np)
{
@@ -186,8 +205,8 @@ deccvasc(const char *cp, int len, decimal *np)
if (risnull(CSTRINGTYPE, cp))
return 0;
- str = pnstrdup(cp, len); /* decimal_in always converts the complete
- * string */
+ str = ecpg_strndup(cp, len); /* decimal_in always converts the complete
+ * string */
if (!str)
ret = ECPG_INFORMIX_NUM_UNDERFLOW;
else
--
2.34.1
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-07-29 12:01 Peter Eisentraut <peter@eisentraut.org>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
1 sibling, 1 reply; 424+ messages in thread
From: Peter Eisentraut @ 2024-07-29 12:01 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: pgsql-hackers; Daniel Gustafsson <daniel@yesql.se>
I have some comments about the first three patches, that deal with
memory management.
v24-0001-Revert-ECPG-s-use-of-pnstrdup.patch
This looks right.
I suppose another approach would be to put a full replacement for
strndup() into src/port/. But since there is currently only one user,
and most other users should be using pnstrdup(), the presented approach
seems ok.
We should take the check for exit() calls from libpq and expand it to
cover the other libraries as well. Maybe there are other problems like
this?
v24-0002-Remove-fe_memutils-from-libpgcommon_shlib.patch
I don't quite understand how this problem can arise. The description says
"""
libpq appears to have no need for this, and the exit() references cause
our libpq-refs-stamp test to fail if the linker doesn't strip out the
unused code.
"""
But under what circumstances does "the linker doesn't strip out" happen?
If this happens accidentally, then we should have seen some buildfarm
failures or something?
Also, one could look further and notice that restricted_token.c and
sprompt.c both a) are not needed by libpq and b) can trigger exit()
calls. Then it's not clear why those are not affected.
v24-0003-common-jsonapi-support-libpq-as-a-client.patch
I'm reminded of thread [0]. I think there is quite a bit of confusion
about the pqexpbuffer vs. stringinfo APIs, and they are probably used
incorrectly quite a bit. There are now also programs that use both of
them! This patch now introduces another layer on top of them. I fear,
at the end, nobody is going to understand any of this anymore. Also,
changing all the programs to link in libpq for pqexpbuffer seems like
the opposite direction from what was suggested in [0].
I think we need to do some deeper thinking here about how we want the
memory management on the client side to work. Maybe we could just use
one API but have some flags or callbacks to control the out-of-memory
behavior.
[0]:
https://www.postgresql.org/message-id/flat/16d0beac-a141-e5d3-60e9-323da75f49bf%40eisentraut.org
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-07-29 20:51 Daniel Gustafsson <daniel@yesql.se>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
1 sibling, 1 reply; 424+ messages in thread
From: Daniel Gustafsson @ 2024-07-29 20:51 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: pgsql-hackers; Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>; Andrew Dunstan <andrew@dunslane.net>; Peter Eisentraut <peter@eisentraut.org>
Thanks for working on this patchset, I'm looking over 0004 and 0005 but came
across a thing I wanted to bring up one thing sooner than waiting for the
review. In parse_device_authz we have this:
{"user_code", JSON_TOKEN_STRING, {&authz->user_code}, REQUIRED},
{"verification_uri", JSON_TOKEN_STRING, {&authz->verification_uri}, REQUIRED},
/*
* The following fields are technically REQUIRED, but we don't use
* them anywhere yet:
*
* - expires_in
*/
{"interval", JSON_TOKEN_NUMBER, {&authz->interval_str}, OPTIONAL},
Together with a colleage we found the Azure provider use "verification_url"
rather than xxx_uri. Another discrepancy is that it uses a string for the
interval (ie: "interval":"5"). One can of course argue that Azure is wrong and
should feel bad, but I fear that virtually all (major) providers will have
differences like this, so we will have to deal with it in an extensible fashion
(compile time, not runtime configurable).
I was toying with making the name json_field name member an array, to allow
variations. That won't help with the fieldtype differences though, so another
train of thought was to have some form of REQUIRED_XOR where fields can tied
together. What do you think about something along these lines?
Another thing, shouldn't we really parse and interpret *all* REQUIRED fields
even if we don't use them to ensure that the JSON is wellformed? If the JSON
we get is malformed in any way it seems like the safe/conservative option to
error out.
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-07-29 22:30 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Peter Eisentraut <peter@eisentraut.org>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2024-07-29 22:30 UTC (permalink / raw)
To: Peter Eisentraut <peter@eisentraut.org>; +Cc: pgsql-hackers; Daniel Gustafsson <daniel@yesql.se>
On Mon, Jul 29, 2024 at 5:02 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> We should take the check for exit() calls from libpq and expand it to
> cover the other libraries as well. Maybe there are other problems like
> this?
Seems reasonable, yeah.
> But under what circumstances does "the linker doesn't strip out" happen?
> If this happens accidentally, then we should have seen some buildfarm
> failures or something?
On my machine, for example, I see differences with optimization
levels. Say you inadvertently call pfree() in a _shlib build, as I did
multiple times upthread. By itself, that shouldn't actually be a
problem (it eventually redirects to free()), so it should be legal to
call pfree(), and with -O2 the build succeeds. But with -Og, the
exit() check trips, and when I disassemble I see that pg_malloc() et
all have infected the shared object. After all, we did tell the linker
to put that object file in, and we don't ask it to garbage-collect
sections.
> Also, one could look further and notice that restricted_token.c and
> sprompt.c both a) are not needed by libpq and b) can trigger exit()
> calls. Then it's not clear why those are not affected.
I think it's easier for the linker to omit whole object files rather
than partial ones. If libpq doesn't use any of those APIs there's not
really a reason to trip over it.
(Maybe the _shlib variants should just contain the minimum objects
required to compile.)
> I'm reminded of thread [0]. I think there is quite a bit of confusion
> about the pqexpbuffer vs. stringinfo APIs, and they are probably used
> incorrectly quite a bit. There are now also programs that use both of
> them! This patch now introduces another layer on top of them. I fear,
> at the end, nobody is going to understand any of this anymore.
"anymore"? :)
In all seriousness -- I agree that this isn't sustainable. At the
moment the worst pain (the new layer) is isolated to jsonapi.c, which
seems like an okay place to try something new, since there aren't that
many clients. But to be honest I'm not excited about deciding the Best
Way Forward based on a sample size of JSON.
> Also,
> changing all the programs to link in libpq for pqexpbuffer seems like
> the opposite direction from what was suggested in [0].
(I don't really want to keep that new libpq dependency. We'd just have
to decide where PQExpBuffer is going to go if we're not okay with it.)
> I think we need to do some deeper thinking here about how we want the
> memory management on the client side to work. Maybe we could just use
> one API but have some flags or callbacks to control the out-of-memory
> behavior.
Any src/common code that needs to handle both in-band and out-of-band
failure modes will still have to decide whether it's going to 1)
duplicate code paths or 2) just act as if in-band failures can always
happen. I think that's probably essential complexity; an ideal API
might make it nicer to deal with but it can't abstract it away.
Thanks!
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-07-29 23:15 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Daniel Gustafsson <daniel@yesql.se>
0 siblings, 0 replies; 424+ messages in thread
From: Jacob Champion @ 2024-07-29 23:15 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: pgsql-hackers; Shlok Kyal <shlok.kyal.oss@gmail.com>; mahendrakar s <mahendrakarforpg@gmail.com>; Andrey Chudnovsky <achudnovskij@gmail.com>; Thomas Munro <thomas.munro@gmail.com>; hlinnaka@iki.fi <hlinnaka@iki.fi>; michael@paquier.xyz <michael@paquier.xyz>; smilingsamay@gmail.com <smilingsamay@gmail.com>; Stephen Frost <sfrost@snowman.net>; Andrew Dunstan <andrew@dunslane.net>; Peter Eisentraut <peter@eisentraut.org>
On Mon, Jul 29, 2024 at 1:51 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> Together with a colleage we found the Azure provider use "verification_url"
> rather than xxx_uri.
Yeah, I think that's originally a Google-ism. (As far as I can tell
they helped author the spec for this and then didn't follow it. :/ ) I
didn't recall Azure having used it back when I was testing against it,
though, so that's good to know.
> Another discrepancy is that it uses a string for the
> interval (ie: "interval":"5").
Oh, that's a new one. I don't remember needing to hack around that
either; maybe iddawc handled it silently?
> One can of course argue that Azure is wrong and
> should feel bad, but I fear that virtually all (major) providers will have
> differences like this, so we will have to deal with it in an extensible fashion
> (compile time, not runtime configurable).
Such is life... verification_url we will just have to deal with by
default, I think, since Google does/did it too. Not sure about
interval -- but do we want to make our distribution maintainers deal
with a compile-time setting for libpq, just to support various OAuth
flavors? To me it seems like we should just hold our noses and support
known (large) departures in the core.
> I was toying with making the name json_field name member an array, to allow
> variations. That won't help with the fieldtype differences though, so another
> train of thought was to have some form of REQUIRED_XOR where fields can tied
> together. What do you think about something along these lines?
If I designed it right, just adding alternative spellings directly to
the fields list should work. (The "required" check is by struct
member, not name, so both spellings can point to the same
destination.) The alternative typing on the other hand might require
something like a new sentinel "type" that will accept both... I hadn't
expected that.
> Another thing, shouldn't we really parse and interpret *all* REQUIRED fields
> even if we don't use them to ensure that the JSON is wellformed? If the JSON
> we get is malformed in any way it seems like the safe/conservative option to
> error out.
Good, I was hoping to have a conversation about that. I am fine with
either option in principle. In practice I expect to add code to use
`expires_in` (so that we can pass it to custom OAuth hook
implementations) and `scope` (to check if the server has changed it on
us).
That leaves the provider... Forcing the provider itself to implement
unused stuff in order to interoperate seems like it could backfire on
us, especially since IETF standardized an alternate .well-known URI
[1] that changes some of these REQUIRED things into OPTIONAL. (One way
for us to interpret this: those fields may be required for OpenID, but
your OAuth provider might not be an OpenID provider, and our code
doesn't require OpenID.) I think we should probably tread lightly in
that particular case. Thoughts on that?
Thanks!
--Jacob
[1] https://www.rfc-editor.org/rfc/rfc8414.html
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-08-02 17:13 Peter Eisentraut <peter@eisentraut.org>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 1 reply; 424+ messages in thread
From: Peter Eisentraut @ 2024-08-02 17:13 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: pgsql-hackers; Daniel Gustafsson <daniel@yesql.se>
On 30.07.24 00:30, Jacob Champion wrote:
>> But under what circumstances does "the linker doesn't strip out" happen?
>> If this happens accidentally, then we should have seen some buildfarm
>> failures or something?
> On my machine, for example, I see differences with optimization
> levels. Say you inadvertently call pfree() in a _shlib build, as I did
> multiple times upthread. By itself, that shouldn't actually be a
> problem (it eventually redirects to free()), so it should be legal to
> call pfree(), and with -O2 the build succeeds. But with -Og, the
> exit() check trips, and when I disassemble I see that pg_malloc() et
> all have infected the shared object. After all, we did tell the linker
> to put that object file in, and we don't ask it to garbage-collect
> sections.
I'm tempted to say, this is working as intended.
libpgcommon is built as a static library. So we can put all the object
files in the library, and its users only use the object files they
really need. So this garbage collection you allude to actually does
happen, on an object-file level.
You shouldn't use pfree() interchangeably with free(), even if that is
not enforced because it's the same thing underneath. First, it just
makes sense to keep the alloc and free pairs matched up. And second, on
Windows there is some additional restriction (vague knowledge) that the
allocate and free functions must be in the same library, so mixing them
freely might not even work.
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-08-02 17:51 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Peter Eisentraut <peter@eisentraut.org>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2024-08-02 17:51 UTC (permalink / raw)
To: Peter Eisentraut <peter@eisentraut.org>; +Cc: pgsql-hackers; Daniel Gustafsson <daniel@yesql.se>
On Fri, Aug 2, 2024 at 10:13 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> You shouldn't use pfree() interchangeably with free(), even if that is
> not enforced because it's the same thing underneath. First, it just
> makes sense to keep the alloc and free pairs matched up. And second, on
> Windows there is some additional restriction (vague knowledge) that the
> allocate and free functions must be in the same library, so mixing them
> freely might not even work.
Ah, I forgot about the CRT problems on Windows. So my statement of
"the linker might not garbage collect" is pretty much irrelevant.
But it sounds like we agree that we shouldn't be using fe_memutils at
all in shlib builds. (If you can't use palloc -- it calls exit -- then
you can't use pfree either.) Is 0002 still worth pursuing, once I've
correctly wordsmithed the commit? Or did I misunderstand your point?
Thanks!
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-08-02 18:48 Peter Eisentraut <peter@eisentraut.org>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 1 reply; 424+ messages in thread
From: Peter Eisentraut @ 2024-08-02 18:48 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: pgsql-hackers; Daniel Gustafsson <daniel@yesql.se>
On 02.08.24 19:51, Jacob Champion wrote:
> But it sounds like we agree that we shouldn't be using fe_memutils at
> all in shlib builds. (If you can't use palloc -- it calls exit -- then
> you can't use pfree either.) Is 0002 still worth pursuing, once I've
> correctly wordsmithed the commit? Or did I misunderstand your point?
Yes, I think with an adjusted comment and commit message, the actual
change makes sense.
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-08-07 07:34 Peter Eisentraut <peter@eisentraut.org>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 1 reply; 424+ messages in thread
From: Peter Eisentraut @ 2024-08-07 07:34 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: pgsql-hackers; Daniel Gustafsson <daniel@yesql.se>
On 05.08.24 19:53, Jacob Champion wrote:
> On Fri, Aug 2, 2024 at 11:48 AM Peter Eisentraut <peter@eisentraut.org> wrote:
>> Yes, I think with an adjusted comment and commit message, the actual
>> change makes sense.
>
> Done in v25.
>
> ...along with a bunch of other stuff:
I have committed 0001, and I plan to backpatch it once the release
freeze lifts.
I'll work on 0002 next.
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-08-12 06:37 Peter Eisentraut <peter@eisentraut.org>
parent: Peter Eisentraut <peter@eisentraut.org>
0 siblings, 1 reply; 424+ messages in thread
From: Peter Eisentraut @ 2024-08-12 06:37 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: pgsql-hackers; Daniel Gustafsson <daniel@yesql.se>
On 07.08.24 09:34, Peter Eisentraut wrote:
> On 05.08.24 19:53, Jacob Champion wrote:
>> On Fri, Aug 2, 2024 at 11:48 AM Peter Eisentraut
>> <peter@eisentraut.org> wrote:
>>> Yes, I think with an adjusted comment and commit message, the actual
>>> change makes sense.
>>
>> Done in v25.
>>
>> ...along with a bunch of other stuff:
>
> I have committed 0001, and I plan to backpatch it once the release
> freeze lifts.
>
> I'll work on 0002 next.
I have committed 0002 now.
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-08-26 08:18 Peter Eisentraut <peter@eisentraut.org>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 1 reply; 424+ messages in thread
From: Peter Eisentraut @ 2024-08-26 08:18 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: pgsql-hackers; Daniel Gustafsson <daniel@yesql.se>
On 13.08.24 23:11, Jacob Champion wrote:
> On Sun, Aug 11, 2024 at 11:37 PM Peter Eisentraut <peter@eisentraut.org> wrote:
>> I have committed 0002 now.
>
> Thanks Peter! Rebased over both in v26.
I have looked again at the jsonapi memory management patch (v26-0001).
As previously mentioned, I think adding a third or fourth (depending
on how you count) memory management API is maybe something we should
avoid. Also, the weird layering where src/common/ now (sometimes)
depends on libpq seems not great.
I'm thinking, maybe we leave the use of StringInfo at the source code
level, but #define the symbols to use PQExpBuffer. Something like
#ifdef JSONAPI_USE_PQEXPBUFFER
#define StringInfo PQExpBuffer
#define appendStringInfo appendPQExpBuffer
#define appendBinaryStringInfo appendBinaryPQExpBuffer
#define palloc malloc
//etc.
#endif
(simplified, the argument lists might differ)
Or, if people find that too scary, something like
#ifdef JSONAPI_USE_PQEXPBUFFER
#define jsonapi_StringInfo PQExpBuffer
#define jsonapi_appendStringInfo appendPQExpBuffer
#define jsonapi_appendBinaryStringInfo appendBinaryPQExpBuffer
#define jsonapi_palloc malloc
//etc.
#else
#define jsonapi_StringInfo StringInfo
#define jsonapi_appendStringInfo appendStringInfo
#define jsonapi_appendBinaryStringInfo appendBinaryStringInfo
#define jsonapi_palloc palloc
//etc.
#endif
That way, it's at least more easy to follow the source code because
you see a mostly-familiar API.
Also, we should make this PQExpBuffer-using mode only used by libpq,
not by frontend programs. So libpq takes its own copy of jsonapi.c
and compiles it using JSONAPI_USE_PQEXPBUFFER. That will make the
libpq build descriptions a bit more complicated, but everyone who is
not libpq doesn't need to change.
Once you get past all the function renaming, the logic changes in
jsonapi.c all look pretty reasonable. Refactoring like
allocate_incremental_state() makes sense.
You could add pg_nodiscard attributes to
makeJsonLexContextCstringLen() and makeJsonLexContextIncremental() so
that callers who are using the libpq mode are forced to check for
errors. Or maybe there is a clever way to avoid even that: Create a
fixed JsonLexContext like
static const JsonLexContext failed_oom;
and on OOM you return that one from makeJsonLexContext*(). And then
in pg_parse_json(), when you get handed that context, you return
JSON_OUT_OF_MEMORY immediately.
Other than that detail and the need to use freeJsonLexContext(), it
looks like this new mode doesn't impose any additional burden on
callers, since during parsing they need to check for errors anyway,
and this just adds one more error type for out of memory. That's a good
outcome.
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-08-26 23:23 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Peter Eisentraut <peter@eisentraut.org>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2024-08-26 23:23 UTC (permalink / raw)
To: Peter Eisentraut <peter@eisentraut.org>; +Cc: pgsql-hackers; Daniel Gustafsson <daniel@yesql.se>
On Mon, Aug 26, 2024 at 1:18 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> Or, if people find that too scary, something like
>
> #ifdef JSONAPI_USE_PQEXPBUFFER
>
> #define jsonapi_StringInfo PQExpBuffer
> [...]
>
> That way, it's at least more easy to follow the source code because
> you see a mostly-familiar API.
I was having trouble reasoning about the palloc-that-isn't-palloc code
during the first few drafts, so I will try a round with the jsonapi_
prefix.
> Also, we should make this PQExpBuffer-using mode only used by libpq,
> not by frontend programs. So libpq takes its own copy of jsonapi.c
> and compiles it using JSONAPI_USE_PQEXPBUFFER. That will make the
> libpq build descriptions a bit more complicated, but everyone who is
> not libpq doesn't need to change.
Sounds reasonable. It complicates the test coverage situation a little
bit, but I think my current patch was maybe insufficient there anyway,
since the coverage for the backend flavor silently dropped...
> Or maybe there is a clever way to avoid even that: Create a
> fixed JsonLexContext like
>
> static const JsonLexContext failed_oom;
>
> and on OOM you return that one from makeJsonLexContext*(). And then
> in pg_parse_json(), when you get handed that context, you return
> JSON_OUT_OF_MEMORY immediately.
I like this idea.
Thanks!
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-08-30 09:49 Peter Eisentraut <peter@eisentraut.org>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 1 reply; 424+ messages in thread
From: Peter Eisentraut @ 2024-08-30 09:49 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: pgsql-hackers; Daniel Gustafsson <daniel@yesql.se>
On 28.08.24 18:31, Jacob Champion wrote:
> On Mon, Aug 26, 2024 at 4:23 PM Jacob Champion
> <jacob.champion@enterprisedb.com> wrote:
>> I was having trouble reasoning about the palloc-that-isn't-palloc code
>> during the first few drafts, so I will try a round with the jsonapi_
>> prefix.
>
> v27 takes a stab at that. I have kept the ALLOC/FREE naming to match
> the strategy in other src/common source files.
This looks pretty good to me. Maybe on the naming side, this seems like
a gratuitous divergence:
+#define jsonapi_createStringInfo makeStringInfo
> The name of the variable JSONAPI_USE_PQEXPBUFFER leads to sections of
> code that look like this:
>
> +#ifdef JSONAPI_USE_PQEXPBUFFER
> + if (!new_prediction || !new_fnames || !new_fnull)
> + return false;
> +#endif
>
> To me it wouldn't be immediately obvious why "using PQExpBuffer" has
> anything to do with this code; the key idea is that we expect any
> allocations to be able to fail. Maybe a name like JSONAPI_ALLOW_OOM or
> JSONAPI_SHLIB_ALLOCATIONS or...?
Seems ok to me as is. I think the purpose of JSONAPI_USE_PQEXPBUFFER is
adequately explained by this comment
+/*
+ * By default, we will use palloc/pfree along with StringInfo. In libpq,
+ * use malloc and PQExpBuffer, and return JSON_OUT_OF_MEMORY on
out-of-memory.
+ */
+#ifdef JSONAPI_USE_PQEXPBUFFER
For some of the other proposed names, I'd be afraid that someone might
think you are free to mix and match APIs, OOM behavior, and compilation
options.
Some comments on src/include/common/jsonapi.h:
-#include "lib/stringinfo.h"
I suspect this will fail headerscheck? Probably needs an exception
added there.
+#ifdef JSONAPI_USE_PQEXPBUFFER
+#define StrValType PQExpBufferData
+#else
+#define StrValType StringInfoData
+#endif
Maybe use jsonapi_StrValType here.
+typedef struct StrValType StrValType;
I don't think that is needed. It would just duplicate typedefs that
already exist elsewhere, depending on what StrValType is set to.
+ bool parse_strval;
+ StrValType *strval; /* only used if
parse_strval == true */
The parse_strval field could use a better explanation.
I actually don't understand the need for this field. AFAICT, this is
just used to record whether strval is valid. But in the cases where
it's not valid, why do we need to record that? Couldn't you just return
failed_oom in those cases?
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-09-04 09:28 Peter Eisentraut <peter@eisentraut.org>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 1 reply; 424+ messages in thread
From: Peter Eisentraut @ 2024-09-04 09:28 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: pgsql-hackers; Daniel Gustafsson <daniel@yesql.se>
On 03.09.24 22:56, Jacob Champion wrote:
>> The parse_strval field could use a better explanation.
>>
>> I actually don't understand the need for this field. AFAICT, this is
>> just used to record whether strval is valid.
> No, it's meant to track the value of the need_escapes argument to the
> constructor. I've renamed it and moved the assignment to hopefully
> make that a little more obvious. WDYT?
Yes, this is clearer.
This patch (v28-0001) looks good to me now.
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-09-11 07:37 Peter Eisentraut <peter@eisentraut.org>
parent: Peter Eisentraut <peter@eisentraut.org>
0 siblings, 2 replies; 424+ messages in thread
From: Peter Eisentraut @ 2024-09-11 07:37 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: pgsql-hackers; Daniel Gustafsson <daniel@yesql.se>
On 04.09.24 11:28, Peter Eisentraut wrote:
> On 03.09.24 22:56, Jacob Champion wrote:
>>> The parse_strval field could use a better explanation.
>>>
>>> I actually don't understand the need for this field. AFAICT, this is
>>> just used to record whether strval is valid.
>> No, it's meant to track the value of the need_escapes argument to the
>> constructor. I've renamed it and moved the assignment to hopefully
>> make that a little more obvious. WDYT?
>
> Yes, this is clearer.
>
> This patch (v28-0001) looks good to me now.
This has been committed.
About the subsequent patches:
Is there any sense in dealing with the libpq and backend patches
separately in sequence, or is this split just for ease of handling?
(I suppose the 0004 "review comments" patch should be folded into the
respective other patches?)
What could be the next steps to keep this moving along, other than stare
at the remaining patches until we're content with them? ;-)
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-09-11 22:54 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Daniel Gustafsson <daniel@yesql.se>
0 siblings, 3 replies; 424+ messages in thread
From: Jacob Champion @ 2024-09-11 22:54 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
(Thanks for the commit, Peter!)
On Wed, Sep 11, 2024 at 6:44 AM Daniel Gustafsson <daniel@yesql.se> wrote:
>
> > On 11 Sep 2024, at 09:37, Peter Eisentraut <peter@eisentraut.org> wrote:
>
> > Is there any sense in dealing with the libpq and backend patches separately in sequence, or is this split just for ease of handling?
>
> I think it's just make reviewing a bit easier. At this point I think they can
> be merged together, it's mostly out of historic reasons IIUC since the patchset
> earlier on supported more than one library.
I can definitely do that (and yeah, it was to make the review slightly
less daunting). The server side could potentially be committed
independently, if you want to parallelize a bit, but it'd have to be
torn back out if the libpq stuff didn't land in time.
> > (I suppose the 0004 "review comments" patch should be folded into the respective other patches?)
Yes. I'm using that patch as a holding area while I write tests for
the hunks, and then moving them backwards.
> I added a warning to autconf in case --with-oauth is used without --with-python
> since this combination will error out in running the tests. Might be
> superfluous but I had an embarrassingly long headscratcher myself as to why the
> tests kept failing =)
Whoops, sorry. I guess we should just skip them if Python isn't there?
> CURL_IGNORE_DEPRECATION(x;) broke pgindent, it needs to keep the semicolon on
> the outside like CURL_IGNORE_DEPRECATION(x);. This doesn't really work well
> with how the macro is defined, not sure how we should handle that best (the
> attached makes the style as per how pgindent want's it with the semicolon
> returned).
Ugh... maybe a case for a pre_indent rule in pgindent?
> The oauth_validator test module need to load Makefile.global before exporting
> the symbols from there.
Hm. Why was that passing the CI, though...?
> There is a first stab at documenting the validator module API, more to come (it
> doesn't compile right now).
>
> It contains a pgindent and pgperltidy run to keep things as close to in final
> sync as we can to catch things like the curl deprecation macro mentioned above
> early.
Thanks!
> > What could be the next steps to keep this moving along, other than stare at the remaining patches until we're content with them? ;-)
>
> I'm in the "stare at things" stage now to try and get this into the tree =)
Yeah, and I still owe you all an updated roadmap.
While I fix up the tests, I've also been picking away at the JSON
encoding problem that was mentioned in [1]; the recent SASLprep fix
was fallout from that, since I'm planning to pull in pieces of its
UTF-8 validation. I will eventually want to fuzz the heck out of this.
> To further pick away at this huge patch I propose to merge the SASL message
> length hunk which can be extracted separately. The attached .txt (to keep the
> CFBot from poking at it) contains a diff which can be committed ahead of the
> rest of this patch to make it a tad smaller and to keep the history of that
> change a bit clearer.
LGTM!
--
Peter asked me if there were plans to provide a "standard" validator
module, say as part of contrib. The tricky thing is that Bearer
validation is issuer-specific, and many providers give you an opaque
token that you're not supposed to introspect at all.
We could use token introspection (RFC 7662) for online verification,
but last I looked at it, no one had actually implemented those
endpoints. For offline verification, I think the best we could do
would be to provide a generic JWT Profile (RFC 9068) validator, but
again I don't know if anyone is actually providing those token formats
in practice. I'm inclined to push that out into the future.
Thanks,
--Jacob
[1] https://www.postgresql.org/message-id/ZjxQnOD1OoCkEeMN%40paquier.xyz
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-09-16 19:13 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
2 siblings, 0 replies; 424+ messages in thread
From: Jacob Champion @ 2024-09-16 19:13 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
On Wed, Sep 11, 2024 at 3:54 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Yeah, and I still owe you all an updated roadmap.
Okay, here goes. New reviewers: start here!
== What is This? ==
OAuth 2.0 is a way for a trusted third party (a "provider") to tell a
server whether a client on the other end of the line is allowed to do
something. This patchset adds OAuth support to libpq with libcurl,
provides a server-side API so that extension modules can add support
for specific OAuth providers, and extends our SASL support to carry
the OAuth access tokens over the OAUTHBEARER mechanism.
Most OAuth clients use a web browser to perform the third-party
handshake. (These are your "Okta logins", "sign in with XXX", etc.)
But there are plenty of people who use psql without a local browser,
and invoking a browser safely across all supported platforms is
actually surprisingly fraught. So this patchset implements something
called device authorization, where the client will display a link and
a code, and then you can log in on whatever device is convenient for
you. Once you've told your provider that you trust libpq to connect to
Postgres on your behalf, it'll give libpq an access token, and libpq
will forward that on to the server.
== How This Fits, or: The Sales Pitch ==
The most popular third-party auth methods we have today are probably
the Kerberos family (AD/GSS/SSPI) and LDAP. If you're not already in
an MS ecosystem, it's unlikely that you're using the former. And users
of the latter are, in my experience, more-or-less resigned to its use,
in spite of LDAP's architectural security problems and the fact that
you have to run weird synchronization scripts to tell Postgres what
certain users are allowed to do.
OAuth provides a decently mature and widely-deployed third option. You
don't have to be running the infrastructure yourself, as long as you
have a provider you trust. If you are running your own infrastructure
(or if your provider is configurable), the tokens being passed around
can carry org-specific user privileges, so that Postgres can figure
out who's allowed to do what without out-of-band synchronization
scripts. And those access tokens are a straight upgrade over
passwords: even if they're somehow stolen, they are time-limited, they
are optionally revocable, and they can be scoped to specific actions.
== Extension Points ==
This patchset provides several points of customization:
Server-side validation is farmed out entirely to an extension, which
we do not provide. (Each OAuth provider is free to come up with its
own proprietary method of verifying its access tokens, and so far the
big players have absolutely not standardized.) Depending on the
provider, the extension may need to contact an external server to see
what the token has been authorized to do, or it may be able to do that
offline using signing keys and an agreed-upon token format.
The client driver using libpq may replace the device authorization
prompt (which by default is done on standard error), for example to
move it into an existing GUI, display a scannable QR code instead of a
link, and so on.
The driver may also replace the entire OAuth flow. For example, a
client that already interacts with browsers may be able to use one of
the more standard web-based methods to get an access token. And
clients attached to a service rather than an end user could use a more
straightforward server-to-server flow, with pre-established
credentials.
== Architecture ==
The client needs to speak HTTP, which is implemented entirely with
libcurl. Originally, I used another OAuth library for rapid
prototyping, but the quality just wasn't there and I ported the
implementation. An internal abstraction layer remains in the libpq
code, so if a better client library comes along, switching to it
shouldn't be too painful.
The client-side hooks all go through a single extension point, so that
we don't continually add entry points in the API for each new piece of
authentication data that a driver may be able to provide. If we wanted
to, we could potentially move the existing SSL passphrase hook into
that, or even handle password retries within libpq itself, but I don't
see any burning reason to do that now.
I wanted to make sure that OAuth could be dropped into existing
deployments without driver changes. (Drivers will probably *want* to
look at the extension hooks for better UX, but they shouldn't
necessarily *have* to.) That has driven several parts of the design.
Drivers using the async APIs should continue to work without blocking,
even during the long HTTP handshakes. So the new client code is
structured as a typical event-driven state machine (similar to
PQconnectPoll). The protocol machine hands off control to the OAuth
machine during authentication, without really needing to know how it
works, because the OAuth machine replaces the PQsocket with a
general-purpose multiplexer that handles all of the HTTP sockets and
events. Once that's completed, the OAuth machine hands control right
back and we return to the Postgres protocol on the wire.
This decision led to a major compromise: Windows client support is
nonexistent. Multiplexer handles exist in Windows (for example with
WSAEventSelect, IIUC), but last I checked they were completely
incompatible with Winsock select(), which means existing async-aware
drivers would fail. We could compromise by providing synchronous-only
support, or by cobbling together a socketpair plus thread pool (or
IOCP?), or simply by saying that existing Windows clients need a new
API other than PQsocket() to be able to work properly. None of those
approaches have been attempted yet, though.
== Areas of Concern ==
Here are the iffy things that a committer is signing up for:
The client implementation is roughly 3k lines, requiring domain
knowledge of Curl, HTTP, JSON, and OAuth, the specifications of which
are spread across several separate standards bodies. (And some big
providers ignore those anyway.)
The OAUTHBEARER mechanism is extensible, but not in the same way as
HTTP. So sometimes, it looks like people design new OAuth features
that rely heavily on HTTP and forget to "port" them over to SASL. That
may be a point of future frustration.
C is not really anyone's preferred language for implementing an
extensible authn/z protocol running on top of HTTP, and constant
vigilance is going to be required to maintain safety. What's more, we
don't really "trust" the endpoints we're talking to in the same way
that we normally trust our servers. It's a fairly hostile environment
for maintainers.
Along the same lines, our JSON implementation assumes some level of
trust in the JSON data -- which is true for the backend, and can be
assumed for a DBA running our utilities, but is absolutely not the
case for a libpq client downloading data from Some Server on the
Internet. I've been working to fuzz the implementation and there are a
few known problems registered in the CF already.
Curl is not a lightweight dependency by any means. Typically, libcurl
is configured with a wide variety of nice options, a tiny subset of
which we're actually going to use, but all that code (and its
transitive dependencies!) is going to arrive in our process anyway.
That might not be a lot of fun if you're not using OAuth.
It's possible that the application embedding libpq is also a direct
client of libcurl. We need to make sure we're not stomping on their
toes at any point.
== TODOs/Known Issues ==
The client does not deal with verification failure well at the moment;
it just keeps retrying with a new OAuth handshake.
Some people are not going to be okay with just contacting any web
server that Postgres tells them to. There's a more paranoid mode
sketched out that lets the connection string specify the trusted
issuer, but it's not complete.
The new code still needs to play well with orthogonal connection
options, like connect_timeout and require_auth.
The server does not deal well with multi-issuer setups yet. And you
only get one oauth_validator_library...
Harden, harden, harden. There are still a handful of inline TODOs
around double-checking certain pieces of the response before
continuing with the handshake. Servers should not be able to run our
recursive descent parser out of stack. And my JSON code is using
assertions too liberally, which will turn bugs into DoS vectors. I've
been working to fit a fuzzer into more and more places, and I'm hoping
to eventually drive it directly from the socket.
Documentation still needs to be filled in. (Thanks Daniel for your work here!)
== Future Features ==
There is no support for token caching (refresh or otherwise). Each new
connection needs a new approval, and the only way to change that for
v1 is to replace the entire flow. I think that's eventually going to
annoy someone. The question is, where do you persist it? Does that
need to be another extensibility point?
We already have pretty good support for client certificates, and it'd
be great if we could bind our tokens to those. That way, even if you
somehow steal the tokens, you can't do anything with them without the
private key! But the state of proof-of-possession in OAuth is an
absolute mess, involving at least three competing standards (Token
Binding, mTLS, DPoP). I don't know what's going to win.
--
Hope this helps! Next I'll be working to fold the patches together, as
discussed upthread.
Thanks,
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-09-27 17:58 Antonin Houska <ah@cybertec.at>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
2 siblings, 1 reply; 424+ messages in thread
From: Antonin Houska @ 2024-09-27 17:58 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: Daniel Gustafsson <daniel@yesql.se>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> Peter asked me if there were plans to provide a "standard" validator
> module, say as part of contrib. The tricky thing is that Bearer
> validation is issuer-specific, and many providers give you an opaque
> token that you're not supposed to introspect at all.
>
> We could use token introspection (RFC 7662) for online verification,
> but last I looked at it, no one had actually implemented those
> endpoints. For offline verification, I think the best we could do
> would be to provide a generic JWT Profile (RFC 9068) validator, but
> again I don't know if anyone is actually providing those token formats
> in practice. I'm inclined to push that out into the future.
Have you considered sending the token for validation to the server, like this
curl -X GET "https://www.googleapis.com/oauth2/v3/userinfo"; -H "Authorization: Bearer $TOKEN"
and getting the userid (e.g. email address) from the response, as described in
[1]? ISTM that this is what pgadmin4 does - in paricular, see the
get_user_profile() function in web/pgadmin/authenticate/oauth2.py.
[1] https://www.oauth.com/oauth2-servers/signing-in-with-google/verifying-the-user-info/
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-09-27 20:45 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2024-09-27 20:45 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Daniel Gustafsson <daniel@yesql.se>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
On Fri, Sep 27, 2024 at 10:58 AM Antonin Houska <ah@cybertec.at> wrote:
> Have you considered sending the token for validation to the server, like this
>
> curl -X GET "https://www.googleapis.com/oauth2/v3/userinfo"; -H "Authorization: Bearer $TOKEN"
In short, no, but I'm glad you asked. I think it's going to be a
common request, and I need to get better at explaining why it's not
safe, so we can document it clearly. Or else someone can point out
that I'm misunderstanding, which honestly would make all this much
easier and less complicated. I would love to be able to do it that
way.
We cannot, for the same reason libpq must send the server an access
token instead of an ID token. The /userinfo endpoint tells you who the
end user is, but it doesn't tell you whether the Bearer is actually
allowed to access the database. That difference is critical: it's
entirely possible for an end user to be authorized to access the
database, *and yet* the Bearer token may not actually carry that
authorization on their behalf. (In fact, the user may have actively
refused to give the Bearer that permission.) That's why people are so
pedantic about saying that OAuth is an authorization framework and not
an authentication framework.
To illustrate, think about all the third-party web services out there
that ask you to Sign In with Google. They ask Google for permission to
access your personal ID, and Google asks you if you're okay with that,
and you either allow or deny it. Now imagine that I ran one of those
services, and I decided to become evil. I could take my legitimately
acquired Bearer token -- which should only give me permission to query
your Google ID -- and send it to a Postgres database you're authorized
to access.
The server is supposed to introspect it, say, "hey, this token doesn't
give the bearer access to the database at all," and shut everything
down. For extra credit, the server could notice that the client ID
tied to the access token isn't even one that it recognizes! But if all
the server does is ask Google, "what's the email address associated
with this token's end user?", then it's about to make some very bad
decisions. The email address it gets back doesn't belong to Jacob the
Evil Bearer; it belongs to you.
Now, the token introspection endpoint I mentioned upthread should give
us the required information (scopes, etc.). But Google doesn't
implement that one. In fact they don't seem to have implemented custom
scopes at all in the years since I started work on this feature, which
makes me think that people are probably not going to be able to safely
log into Postgres using Google tokens. Hopefully there's some feature
buried somewhere that I haven't seen.
Let me know if that makes sense. (And again: I'd love to be proven
wrong. It would improve the reach of the feature considerably if I
am.)
Thanks,
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-09-30 13:38 Antonin Houska <ah@cybertec.at>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 2 replies; 424+ messages in thread
From: Antonin Houska @ 2024-09-30 13:38 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: Daniel Gustafsson <daniel@yesql.se>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> On Fri, Sep 27, 2024 at 10:58 AM Antonin Houska <ah@cybertec.at> wrote:
> > Have you considered sending the token for validation to the server, like this
> >
> > curl -X GET "https://www.googleapis.com/oauth2/v3/userinfo"; -H "Authorization: Bearer $TOKEN"
>
> In short, no, but I'm glad you asked. I think it's going to be a
> common request, and I need to get better at explaining why it's not
> safe, so we can document it clearly. Or else someone can point out
> that I'm misunderstanding, which honestly would make all this much
> easier and less complicated. I would love to be able to do it that
> way.
>
> We cannot, for the same reason libpq must send the server an access
> token instead of an ID token. The /userinfo endpoint tells you who the
> end user is, but it doesn't tell you whether the Bearer is actually
> allowed to access the database. That difference is critical: it's
> entirely possible for an end user to be authorized to access the
> database, *and yet* the Bearer token may not actually carry that
> authorization on their behalf. (In fact, the user may have actively
> refused to give the Bearer that permission.)
> That's why people are so pedantic about saying that OAuth is an
> authorization framework and not an authentication framework.
This statement alone sounds as if you missed *authentication*, but you seem to
admit above that the /userinfo endpoint provides it ("tells you who the end
user is"). I agree that it does. My understanding is that this endpoint, as
well as the concept of "claims" and "scopes", is introduced by OpenID, which
is an *authentication* framework, although it's built on top of OAuth.
Regarding *authorization*, I agree that the bearer token may not contain
enough information to determine whether the owner of the token is allowed to
access the database. However, I consider database a special kind of
"application", which can handle authorization on its own. In this case, the
authorization can be controlled by (not) assigning the user the LOGIN
attribute, as well as by (not) granting it privileges on particular database
objects. In short, I think that *authentication* is all we need.
> To illustrate, think about all the third-party web services out there
> that ask you to Sign In with Google. They ask Google for permission to
> access your personal ID, and Google asks you if you're okay with that,
> and you either allow or deny it. Now imagine that I ran one of those
> services, and I decided to become evil. I could take my legitimately
> acquired Bearer token -- which should only give me permission to query
> your Google ID -- and send it to a Postgres database you're authorized
> to access.
>
> The server is supposed to introspect it, say, "hey, this token doesn't
> give the bearer access to the database at all," and shut everything
> down. For extra credit, the server could notice that the client ID
> tied to the access token isn't even one that it recognizes! But if all
> the server does is ask Google, "what's the email address associated
> with this token's end user?", then it's about to make some very bad
> decisions. The email address it gets back doesn't belong to Jacob the
> Evil Bearer; it belongs to you.
Are you sure you can legitimately acquire the bearer token containing my email
address? I think the email address returned by the /userinfo endpoint is one
of the standard claims [1]. Thus by returning the particular value of "email"
from the endpoint the identity provider asserts that the token owner does have
this address. (And that, if "email_verified" claim is "true", it spent some
effort to verify that the email address is controlled by that user.)
> Now, the token introspection endpoint I mentioned upthread
Can you please point me to the particular message?
> should give us the required information (scopes, etc.). But Google doesn't
> implement that one. In fact they don't seem to have implemented custom
> scopes at all in the years since I started work on this feature, which makes
> me think that people are probably not going to be able to safely log into
> Postgres using Google tokens. Hopefully there's some feature buried
> somewhere that I haven't seen.
>
> Let me know if that makes sense. (And again: I'd love to be proven
> wrong. It would improve the reach of the feature considerably if I
> am.)
Another question, assuming the token verification is resolved somehow:
wouldn't it be sufficient for the initial implementation if the client could
pass the bearer token to libpq in the connection string?
Obviously, one use case is than an application / web server which needs the
token to authenticate the user could eventually pass the token to the database
server. Thus, if users could authenticate to the database using their
individual ids, it would no longer be necessary to store a separate userid /
password for the application in a configuration file.
Also, if libpq accepted the bearer token via the connection string, it would
be possible to implement the authorization as a separate front-end application
(e.g. pg_oauth_login) rather than adding more complexity to libpq itself.
(I'm learning this stuff on-the-fly, so there might be something naive in my
comments.)
[1] https://openid.net/specs/openid-connect-core-1_0.html#StandardClaims
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-09-30 16:54 Antonin Houska <ah@cybertec.at>
parent: Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 424+ messages in thread
From: Antonin Houska @ 2024-09-30 16:54 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: Daniel Gustafsson <daniel@yesql.se>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
Antonin Houska <ah@cybertec.at> wrote:
> Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> > Now, the token introspection endpoint I mentioned upthread
>
> Can you please point me to the particular message?
Please ignore this dumb question. You probably referred to the email I was
responding to.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-10-01 00:05 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2024-10-01 00:05 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Daniel Gustafsson <daniel@yesql.se>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
On Mon, Sep 30, 2024 at 6:38 AM Antonin Houska <ah@cybertec.at> wrote:
>
> Jacob Champion <jacob.champion@enterprisedb.com> wrote:
>
> > On Fri, Sep 27, 2024 at 10:58 AM Antonin Houska <ah@cybertec.at> wrote:
> > That's why people are so pedantic about saying that OAuth is an
> > authorization framework and not an authentication framework.
>
> This statement alone sounds as if you missed *authentication*, but you seem to
> admit above that the /userinfo endpoint provides it ("tells you who the end
> user is"). I agree that it does. My understanding is that this endpoint, as
> well as the concept of "claims" and "scopes", is introduced by OpenID, which
> is an *authentication* framework, although it's built on top of OAuth.
OpenID is an authentication framework, but it's generally focused on a
type of client known as a Relying Party. In the architecture of this
patchset, the Relying Party would be libpq, which has the option of
retrieving authentication claims from the provider. Unfortunately for
us, libpq has no use for those claims. It's not trying to authenticate
the user for its own purposes.
The Postgres server, on the other hand, is not a Relying Party. (It's
an OAuth resource server, in this architecture.) It's not performing
any of the OIDC flows, it's not talking to the end user and the
provider at the same time, and it is very restricted in its ability to
influence the client exchange via the SASL mechanism.
> Regarding *authorization*, I agree that the bearer token may not contain
> enough information to determine whether the owner of the token is allowed to
> access the database. However, I consider database a special kind of
> "application", which can handle authorization on its own. In this case, the
> authorization can be controlled by (not) assigning the user the LOGIN
> attribute, as well as by (not) granting it privileges on particular database
> objects. In short, I think that *authentication* is all we need.
Authorizing the *end user's* access to the database using scopes is
optional. Authorizing the *bearer's* ability to connect on behalf of
the end user, however, is mandatory. Hopefully the below clarifies.
(I agree that most people probably want to use authentication, so that
the database can then make decisions based on HBA settings. OIDC is a
fine way to do that.)
> Are you sure you can legitimately acquire the bearer token containing my email
> address?
Yes. In general that's how OpenID-based "Sign in with <Service>"
works. All those third-party services are running around with tokens
that identify you, but unless they've asked for more abilities and
you've granted them the associated scopes, identifying you is all they
can do.
> I think the email address returned by the /userinfo endpoint is one
> of the standard claims [1]. Thus by returning the particular value of "email"
> from the endpoint the identity provider asserts that the token owner does have
> this address.
We agree that /userinfo gives authentication claims for the end user.
It's just insufficient for our use case.
For example, there are enterprise applications out there that will ask
for read access to your Google Calendar. If you're willing to grant
that, then you probably won't mind if those applications also know
your email address, but you probably do mind if they're suddenly able
to access your production databases just because you gave them your
email.
Put another way: if you log into Postgres using OAuth, and your
provider doesn't show you a big message saying "this application is
about to access *your* prod database using *your* identity; do you
want to allow that?", then your DBA has deployed a really dangerous
configuration. That's a critical protection feature you get from your
OAuth provider. Otherwise, what's stopping somebody else from setting
up their own malicious service to farm access tokens? All they'd have
to do is ask for your email.
> Another question, assuming the token verification is resolved somehow:
> wouldn't it be sufficient for the initial implementation if the client could
> pass the bearer token to libpq in the connection string?
It was discussed wayyy upthread:
https://postgr.es/m/CAAWbhmhmBe9v3aCffz5j8Sg4HMWWkB5FvTDCSZ_Vh8E1fX91Gw%40mail.gmail.com
Basically, at that point the entire implementation becomes an exercise
for the reader. I want to avoid that if possible. I'm not adamantly
opposed to it, but I think the client-side hook implementation is
going to be better for the use cases that have been discussed so far.
> Also, if libpq accepted the bearer token via the connection string, it would
> be possible to implement the authorization as a separate front-end application
> (e.g. pg_oauth_login) rather than adding more complexity to libpq itself.
The application would still need to parse the server error response.
There was (a small) consensus at the time [1] that parsing error
messages for that purpose would be really unpleasant; hence the hook
architecture.
> (I'm learning this stuff on-the-fly, so there might be something naive in my
> comments.)
No worries! Please keep the questions coming; this OAuth architecture
is unintuitive, and I need to be able to defend it.
Thanks,
--Jacob
[1] https://postgr.es/m/CACrwV54_euYe%2Bv7bcLrxnje-JuM%3DKRX5azOcmmrXJ5qrffVZfg%40mail.gmail.com
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-10-08 10:46 Antonin Houska <ah@cybertec.at>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 2 replies; 424+ messages in thread
From: Antonin Houska @ 2024-10-08 10:46 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: Daniel Gustafsson <daniel@yesql.se>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> On Mon, Sep 30, 2024 at 6:38 AM Antonin Houska <ah@cybertec.at> wrote:
> >
> > Are you sure you can legitimately acquire the bearer token containing my email
> > address?
>
> Yes. In general that's how OpenID-based "Sign in with <Service>"
> works. All those third-party services are running around with tokens
> that identify you, but unless they've asked for more abilities and
> you've granted them the associated scopes, identifying you is all they
> can do.
>
> > I think the email address returned by the /userinfo endpoint is one
> > of the standard claims [1]. Thus by returning the particular value of "email"
> > from the endpoint the identity provider asserts that the token owner does have
> > this address.
>
> We agree that /userinfo gives authentication claims for the end user.
> It's just insufficient for our use case.
>
> For example, there are enterprise applications out there that will ask
> for read access to your Google Calendar. If you're willing to grant
> that, then you probably won't mind if those applications also know
> your email address, but you probably do mind if they're suddenly able
> to access your production databases just because you gave them your
> email.
>
> Put another way: if you log into Postgres using OAuth, and your
> provider doesn't show you a big message saying "this application is
> about to access *your* prod database using *your* identity; do you
> want to allow that?", then your DBA has deployed a really dangerous
> configuration. That's a critical protection feature you get from your
> OAuth provider. Otherwise, what's stopping somebody else from setting
> up their own malicious service to farm access tokens? All they'd have
> to do is ask for your email.
Perhaps I understand now. I use getmail [2] to retrieve email messages from my
Google account. What made me confused is that the getmail application,
although installed on my workstation (and thus the bearer token it eventually
gets contains my email address), it's "someone else" (in particular the
"Relying Party") from the perspective of the OpenID protocol. And the same
applies to "psql" in the context of your patch.
Thus, in addition to the email, we'd need special claims which authorize the
RPs to access the database and only the database. Does this sound correct?
> > (I'm learning this stuff on-the-fly, so there might be something naive in my
> > comments.)
>
> No worries! Please keep the questions coming; this OAuth architecture
> is unintuitive, and I need to be able to defend it.
I'd like to play with the code a bit and provide some review before or during
the next CF. That will probably generate some more questions.
>
> [1] https://postgr.es/m/CACrwV54_euYe%2Bv7bcLrxnje-JuM%3DKRX5azOcmmrXJ5qrffVZfg%40mail.gmail.com
[2] https://github.com/getmail6/getmail6/
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-10-08 15:19 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Antonin Houska <ah@cybertec.at>
1 sibling, 0 replies; 424+ messages in thread
From: Jacob Champion @ 2024-10-08 15:19 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Daniel Gustafsson <daniel@yesql.se>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
On Tue, Oct 8, 2024 at 3:46 AM Antonin Houska <ah@cybertec.at> wrote:
> Perhaps I understand now. I use getmail [2] to retrieve email messages from my
> Google account. What made me confused is that the getmail application,
> although installed on my workstation (and thus the bearer token it eventually
> gets contains my email address), it's "someone else" (in particular the
> "Relying Party") from the perspective of the OpenID protocol. And the same
> applies to "psql" in the context of your patch.
>
> Thus, in addition to the email, we'd need special claims which authorize the
> RPs to access the database and only the database. Does this sound correct?
Yes. (One nitpick: the "special claims" in this case are not OpenID
claims at all, but OAuth scopes. The HBA will be configured with the
list of scopes that the server requires, and it requests those from
the client during the SASL handshake.)
> I'd like to play with the code a bit and provide some review before or during
> the next CF. That will probably generate some more questions.
Thanks very much for the review!
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-10-15 18:00 Alexander Lakhin <exclusion@gmail.com>
parent: Peter Eisentraut <peter@eisentraut.org>
1 sibling, 1 reply; 424+ messages in thread
From: Alexander Lakhin @ 2024-10-15 18:00 UTC (permalink / raw)
To: Peter Eisentraut <peter@eisentraut.org>; Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: pgsql-hackers; Daniel Gustafsson <daniel@yesql.se>
Hello Peter,
11.09.2024 10:37, Peter Eisentraut wrote:
>
> This has been committed.
>
I've discovered that starting from 0785d1b8b,
make check -C src/bin/pg_combinebackup
fails under Valgrind, with the following diagnostics:
2024-10-15 14:29:52.883 UTC [3338981] 002_compare_backups.pl STATEMENT: UPLOAD_MANIFEST
==00:00:00:20.028 3338981== Conditional jump or move depends on uninitialised value(s)
==00:00:00:20.028 3338981== at 0xA3E68F: json_lex (jsonapi.c:1496)
==00:00:00:20.028 3338981== by 0xA3ED13: json_lex (jsonapi.c:1666)
==00:00:00:20.028 3338981== by 0xA3D5AF: pg_parse_json_incremental (jsonapi.c:822)
==00:00:00:20.028 3338981== by 0xA40ECF: json_parse_manifest_incremental_chunk (parse_manifest.c:194)
==00:00:00:20.028 3338981== by 0x31656B: FinalizeIncrementalManifest (basebackup_incremental.c:237)
==00:00:00:20.028 3338981== by 0x73B4A4: UploadManifest (walsender.c:709)
==00:00:00:20.028 3338981== by 0x73DF4A: exec_replication_command (walsender.c:2185)
==00:00:00:20.028 3338981== by 0x7C58C3: PostgresMain (postgres.c:4762)
==00:00:00:20.028 3338981== by 0x7BBDA7: BackendMain (backend_startup.c:107)
==00:00:00:20.028 3338981== by 0x6CF60F: postmaster_child_launch (launch_backend.c:274)
==00:00:00:20.028 3338981== by 0x6D546F: BackendStartup (postmaster.c:3415)
==00:00:00:20.028 3338981== by 0x6D2B21: ServerLoop (postmaster.c:1648)
==00:00:00:20.028 3338981==
(Initializing
dummy_lex.inc_state = NULL;
before
partial_result = json_lex(&dummy_lex);
makes these TAP tests pass for me.)
Best regards,
Alexander
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-10-15 18:10 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Alexander Lakhin <exclusion@gmail.com>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2024-10-15 18:10 UTC (permalink / raw)
To: Alexander Lakhin <exclusion@gmail.com>; +Cc: Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers; Daniel Gustafsson <daniel@yesql.se>
On Tue, Oct 15, 2024 at 11:00 AM Alexander Lakhin <exclusion@gmail.com> wrote:
> I've discovered that starting from 0785d1b8b,
> make check -C src/bin/pg_combinebackup
> fails under Valgrind, with the following diagnostics:
Yep, sorry for that (and thanks for the report!). It's currently
tracked over at [1], but I should have mentioned it here. The patch I
used is attached, renamed to not stress out the cfbot.
--Jacob
[1] https://www.postgresql.org/message-id/CAOYmi%2BkiiM83%3DH6YQ77NSSCtkGAzAnZfC0vZS%3DaLM9QZx%3DRn_A%40...
Attachments:
v4-0002-jsonapi-fully-initialize-dummy-lexer.patch.txt (text/plain, 998B) download
From d3e639ba2bacf64fc0d2eb3aa9364a87030335f5 Mon Sep 17 00:00:00 2001
From: Jacob Champion <jacob.champion@enterprisedb.com>
Date: Mon, 7 Oct 2024 14:41:31 -0700
Subject: [PATCH v4 2/2] jsonapi: fully initialize dummy lexer
Valgrind reports that checks on lex->inc_state are undefined for the
"dummy lexer" used for incremental parsing, since it's only partially
initialized on the stack. This was introduced in 0785d1b8b2.
Zero-initialize the whole struct.
---
src/common/jsonapi.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/common/jsonapi.c b/src/common/jsonapi.c
index df6e633b5e..0e2a82ad7a 100644
--- a/src/common/jsonapi.c
+++ b/src/common/jsonapi.c
@@ -1622,7 +1622,7 @@ json_lex(JsonLexContext *lex)
jsonapi_StrValType *ptok = &(lex->inc_state->partial_token);
size_t added = 0;
bool tok_done = false;
- JsonLexContext dummy_lex;
+ JsonLexContext dummy_lex = {0};
JsonParseErrorType partial_result;
if (ptok->data[0] == '"')
--
2.34.1
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-10-17 06:28 Peter Eisentraut <peter@eisentraut.org>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 0 replies; 424+ messages in thread
From: Peter Eisentraut @ 2024-10-17 06:28 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; Alexander Lakhin <exclusion@gmail.com>; +Cc: pgsql-hackers; Daniel Gustafsson <daniel@yesql.se>
On 15.10.24 20:10, Jacob Champion wrote:
> On Tue, Oct 15, 2024 at 11:00 AM Alexander Lakhin <exclusion@gmail.com> wrote:
>> I've discovered that starting from 0785d1b8b,
>> make check -C src/bin/pg_combinebackup
>> fails under Valgrind, with the following diagnostics:
>
> Yep, sorry for that (and thanks for the report!). It's currently
> tracked over at [1], but I should have mentioned it here. The patch I
> used is attached, renamed to not stress out the cfbot.
I have committed this fix.
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-10-18 05:51 Antonin Houska <ah@cybertec.at>
parent: Antonin Houska <ah@cybertec.at>
1 sibling, 1 reply; 424+ messages in thread
From: Antonin Houska @ 2024-10-18 05:51 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: Daniel Gustafsson <daniel@yesql.se>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
Antonin Houska <ah@cybertec.at> wrote:
> I'd like to play with the code a bit and provide some review before or during
> the next CF. That will probably generate some more questions.
This is the 1st round, based on reading the code. I'll continue paying
attention to the project and possibly post some more comments in the future.
* Information on the new method should be added to pg_hba.conf.sample.method.
* Is it important that fe_oauth_state.token also contains the "Bearer"
keyword? I'd expect only the actual token value here. The keyword can be
added to the authentication message w/o storing it.
The same applies to the 'token' structure in fe-auth-oauth-curl.c.
* Does PQdefaultAuthDataHook() have to be declared extern and exported via
libpq/exports.txt ? Even if the user was interested in it, he can use
PQgetAuthDataHook() to get the pointer (unless he already installed his
custom hook).
* I wonder if the hooks (PQauthDataHook) can be implemented in a separate
diff. Couldn't the first version of the feature be commitable without these
hooks?
* Instead of allocating an instance of PQoauthBearerRequest, assigning it to
fe_oauth_state.async_ctx, and eventually having to all its cleanup()
function, wouldn't it be simpler to embed PQoauthBearerRequest as a member
in fe_oauth_state ?
* oauth_validator_library is defined as PGC_SIGHUP - is that intentional?
And regardless, the library appears to be loaded by every backend during
authentication. Why isn't it loaded by postmaster like libraries listed in
shared_preload_libraries? fork() would then ensure that the backends do have
the library in their address space.
* pg_fe_run_oauth_flow()
When first time here
case OAUTH_STEP_TOKEN_REQUEST:
if (!handle_token_response(actx, &state->token))
goto error_return;
the user hasn't been prompted yet so ISTM that the first token request must
always fail. It seems more logical if the prompt is set to the user before
sending the token request to the server. (Although the user probably won't
be that fast to make the first request succeed, so consider this just a
hint.)
* As long as I understand, the following comment would make sense:
inline patch 1: src/interfaces/libpq/fe-auth-oauth.c (+1/-0 lines) download inline-1.patch
diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
index f943a31cc08..97259fb5654 100644
--- a/src/interfaces/libpq/fe-auth-oauth.c
+++ b/src/interfaces/libpq/fe-auth-oauth.c
@@ -518,6 +518,7 @@ oauth_exchange(void *opaq, bool final,
switch (state->state)
{
case FE_OAUTH_INIT:
+ /* Initial Client Response */
Assert(inputlen == -1);
if (!derive_discovery_uri(conn))
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-10-23 22:40 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Antonin Houska <ah@cybertec.at>
0 siblings, 2 replies; 424+ messages in thread
From: Jacob Champion @ 2024-10-23 22:40 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Daniel Gustafsson <daniel@yesql.se>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
On Thu, Oct 17, 2024 at 10:51 PM Antonin Houska <ah@cybertec.at> wrote:
> This is the 1st round, based on reading the code. I'll continue paying
> attention to the project and possibly post some more comments in the future.
Thanks again for the reviews!
> * Information on the new method should be added to pg_hba.conf.sample.method.
Whoops, this will be fixed in v34.
> * Is it important that fe_oauth_state.token also contains the "Bearer"
> keyword? I'd expect only the actual token value here. The keyword can be
> added to the authentication message w/o storing it.
>
> The same applies to the 'token' structure in fe-auth-oauth-curl.c.
Excellent question; I've waffled a bit on that myself. I think you're
probably right, but here's some background on why I originally made
that decision.
RFC 7628 defines not only OAUTHBEARER but also a generic template for
future OAuth-based SASL methods, and as part of that, the definition
of the "auth" key is incredibly vague:
auth (REQUIRED): The payload that would be in the HTTP
Authorization header if this OAuth exchange was being carried
out over HTTP.
I was worried that forcing a specific format would prevent future
extensibility, if say the Bearer scheme were updated to add additional
auth-params. I was also wondering if maybe a future specification
would allow OAUTHBEARER to carry a different scheme altogether, such
as DPoP [1].
However:
- auth-param support for Bearer was considered at the draft stage and
explicitly removed, with the old drafts stating "If additional
parameters are needed in the future, a different scheme would need to
be defined."
- I think the intent of RFC 7628 is that a new SASL mechanism will be
named for each new scheme (even if the new scheme shares all of the
bones of the old one). So DPoP tokens wouldn't piggyback on
OAUTHBEARER, and instead something like an OAUTHDPOP mech would need
to be defined.
So: the additional complexity in the current API is probably a YAGNI
violation, and I should just hardcode the Bearer format as you
suggest. Any future OAuth SASL mechanisms we support will have to go
through a different PQAUTHDATA type, e.g. PQAUTHDATA_OAUTH_DPOP_TOKEN.
And I'll need to make sure that I'm not improperly coupling the
concepts elsewhere in the API.
> * Does PQdefaultAuthDataHook() have to be declared extern and exported via
> libpq/exports.txt ? Even if the user was interested in it, he can use
> PQgetAuthDataHook() to get the pointer (unless he already installed his
> custom hook).
I guess I don't have a strongly held opinion, but is there a good
reason not to? Exposing it means that a client application may answer
questions like "is the current hook set to the default?" and so on.
IME, hook-chain maintenance is not a lot of fun in general, and having
more visibility can be nice for third-party developers.
> * I wonder if the hooks (PQauthDataHook) can be implemented in a separate
> diff. Couldn't the first version of the feature be commitable without these
> hooks?
I am more than happy to split things up as needed! But in the end, I
think this is a question that can only be answered by the first brave
committer to take a bite. :)
(The original patchset didn't have these hooks; they were added as a
compromise, to prevent the builtin implementation from having to be
all things for all people.)
> * Instead of allocating an instance of PQoauthBearerRequest, assigning it to
> fe_oauth_state.async_ctx, and eventually having to all its cleanup()
> function, wouldn't it be simpler to embed PQoauthBearerRequest as a member
> in fe_oauth_state ?
Hmm, that would maybe be simpler. But you'd still have to call
cleanup() and set the async_ctx, right? The primary gain would be in
reducing the number of malloc calls.
> * oauth_validator_library is defined as PGC_SIGHUP - is that intentional?
Yes, I think it's going to be important to let DBAs migrate their
authentication modules without a full restart. That probably deserves
more explicit testing, now that you mention it. Is there a specific
concern that you have with that?
> And regardless, the library appears to be loaded by every backend during
> authentication. Why isn't it loaded by postmaster like libraries listed in
> shared_preload_libraries? fork() would then ensure that the backends do have
> the library in their address space.
It _can_ be, if you want -- there's nothing that I know of preventing
the validator from also being preloaded with its own _PG_init(), is
there? But I don't think it's a good idea to force that, for the same
reason we want to allow SIGHUP.
> * pg_fe_run_oauth_flow()
>
> When first time here
> case OAUTH_STEP_TOKEN_REQUEST:
> if (!handle_token_response(actx, &state->token))
> goto error_return;
>
> the user hasn't been prompted yet so ISTM that the first token request must
> always fail. It seems more logical if the prompt is set to the user before
> sending the token request to the server. (Although the user probably won't
> be that fast to make the first request succeed, so consider this just a
> hint.)
That's also intentional -- if the first token response fails for a
reason _other_ than "we're waiting for the user", then we want to
immediately fail hard instead of making them dig out their phone and
go on a two-minute trip, because they're going to come back and find
that it was all for nothing.
There's a comment immediately below the part you quoted that mentions
this briefly; maybe I should move it up a bit?
> * As long as I understand, the following comment would make sense:
>
> diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
> index f943a31cc08..97259fb5654 100644
> --- a/src/interfaces/libpq/fe-auth-oauth.c
> +++ b/src/interfaces/libpq/fe-auth-oauth.c
> @@ -518,6 +518,7 @@ oauth_exchange(void *opaq, bool final,
> switch (state->state)
> {
> case FE_OAUTH_INIT:
> + /* Initial Client Response */
> Assert(inputlen == -1);
>
> if (!derive_discovery_uri(conn))
There are multiple "initial client response" cases, though. What
questions are you hoping to clarify with the comment? Maybe we can
find a more direct answer.
> Or, doesn't the FE_OAUTH_INIT branch of the switch statement actually fit
> better into oauth_init()?
oauth_init() is the mechanism initialization for the SASL framework
itself, which is shared with SCRAM. In the current architecture, the
init callback doesn't take the initial client response into
consideration at all.
Generating the client response is up to the exchange callback -- and
even if we moved the SASL_ASYNC processing elsewhere, I don't think we
can get rid of its added complexity. Something has to signal upwards
that it's time to transfer control to an async engine. And we can't
make the asynchronicity a static attribute of the mechanism itself,
because we can skip the flow if something gives us a cached token.
> * Finally, the user documentation is almost missing. I say that just for the
> sake of completeness, you obviously know it. (On the other hand, I think
> that the lack of user information might discourage some people from running
> the code and testing it.)
Yeah, the catch-22 of writing huge features... By the way, if anyone's
reading along and dissuaded by the lack of docs, please say so!
(Daniel has been helping me out so much with the docs; thanks again,
Daniel.)
--Jacob
[1] https://datatracker.ietf.org/doc/html/rfc9449
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-10-24 22:28 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Daniel Gustafsson <daniel@yesql.se>
1 sibling, 0 replies; 424+ messages in thread
From: Jacob Champion @ 2024-10-24 22:28 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers; Antonin Houska <ah@cybertec.at>
On Fri, Oct 18, 2024 at 4:38 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> In validate() it seems to me we should clear out ret->authn_id on failure to
> pair belts with suspenders. Fixed by calling explicit_bzero on it in the error
> path.
The new hunk says:
> cleanup:
> /*
> * Clear and free the validation result from the validator module once
> * we're done with it to avoid accidental re-use.
> */
> if (ret->authn_id != NULL)
> {
> explicit_bzero(ret->authn_id, strlen(ret->authn_id));
> pfree(ret->authn_id);
> }
> pfree(ret);
But I'm not clear on what's being protected against. Which code would
reuse this result?
Thanks,
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-10-28 13:24 Daniel Gustafsson <daniel@yesql.se>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
1 sibling, 1 reply; 424+ messages in thread
From: Daniel Gustafsson @ 2024-10-28 13:24 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: Antonin Houska <ah@cybertec.at>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
> On 25 Oct 2024, at 20:22, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> I have combed almost all of Daniel's feedback backwards into the main
> patch (just the new bzero code remains, with the open question
> upthread),
Re-reading I can't see a vector there, I guess I am just scarred from what
seemed to be harmless leaks in auth codepaths and treat every bit as
potentially important. Feel free to drop from the patchset for now.
> Next up is, hopefully, url-encoding. I hadn't realized what an
> absolute mess that would be [1].
Everything and anything involving urls is a hot mess =/
Looking more at the patchset I think we need to apply conditional compilation
of the backend for oauth like how we do with other opt-in schemes in configure
and meson. The attached .txt has a diff for making --with-oauth a requirement
for compiling support into backend libpq.
--
Daniel Gustafsson
Attachments:
backend_with_oauth.txt (text/plain, 5.4K) download
diff --git a/configure b/configure
index 39fe5a0542..b40cd836f1 100755
--- a/configure
+++ b/configure
@@ -8439,9 +8439,6 @@ fi
if test x"$with_oauth" = x"curl"; then
-$as_echo "#define USE_OAUTH 1" >>confdefs.h
-
-
$as_echo "#define USE_OAUTH_CURL 1" >>confdefs.h
# OAuth requires python for testing
diff --git a/configure.ac b/configure.ac
index f7e1400b6e..82217e652e 100644
--- a/configure.ac
+++ b/configure.ac
@@ -927,8 +927,7 @@ if test x"$with_oauth" = x"" ; then
fi
if test x"$with_oauth" = x"curl"; then
- AC_DEFINE([USE_OAUTH], 1, [Define to 1 to build with OAuth 2.0 support. (--with-oauth)])
- AC_DEFINE([USE_OAUTH_CURL], 1, [Define to 1 to use libcurl for OAuth support.])
+ AC_DEFINE([USE_OAUTH_CURL], 1, [Define to 1 to build with OAuth 2.0 support. (--with-oauth=curl)])
# OAuth requires python for testing
if test "$with_python" != yes; then
AC_MSG_WARN([*** OAuth support tests requires --with-python to run])
diff --git a/src/backend/libpq/Makefile b/src/backend/libpq/Makefile
index 98eb2a8242..ba502b0442 100644
--- a/src/backend/libpq/Makefile
+++ b/src/backend/libpq/Makefile
@@ -15,7 +15,6 @@ include $(top_builddir)/src/Makefile.global
# be-fsstubs is here for historical reasons, probably belongs elsewhere
OBJS = \
- auth-oauth.o \
auth-sasl.o \
auth-scram.o \
auth.o \
@@ -30,6 +29,10 @@ OBJS = \
pqmq.o \
pqsignal.o
+ifneq ($(with_oauth),)
+OBJS += auth-oauth.o
+endif
+
ifeq ($(with_ssl),openssl)
OBJS += be-secure-openssl.o
endif
diff --git a/src/backend/libpq/auth.c b/src/backend/libpq/auth.c
index 0cf3e31c9f..57638f922e 100644
--- a/src/backend/libpq/auth.c
+++ b/src/backend/libpq/auth.c
@@ -29,7 +29,6 @@
#include "libpq/auth.h"
#include "libpq/crypt.h"
#include "libpq/libpq.h"
-#include "libpq/oauth.h"
#include "libpq/pqformat.h"
#include "libpq/sasl.h"
#include "libpq/scram.h"
@@ -201,6 +200,14 @@ static int CheckRADIUSAuth(Port *port);
static int PerformRadiusTransaction(const char *server, const char *secret, const char *portstr, const char *identifier, const char *user_name, const char *passwd);
+/*----------------------------------------------------------------
+ * OAuth Authentication
+ *----------------------------------------------------------------
+ */
+#ifdef USE_OAUTH
+#include "libpq/oauth.h"
+#endif
+
/*----------------------------------------------------------------
* Global authentication functions
*----------------------------------------------------------------
@@ -614,8 +621,13 @@ ClientAuthentication(Port *port)
case uaTrust:
status = STATUS_OK;
break;
+
case uaOAuth:
+#ifdef USE_OAUTH
status = CheckSASLAuth(&pg_be_oauth_mech, port, NULL, NULL);
+#else
+ Assert(false);
+#endif
break;
}
diff --git a/src/backend/libpq/hba.c b/src/backend/libpq/hba.c
index c623b8463d..719c7a881e 100644
--- a/src/backend/libpq/hba.c
+++ b/src/backend/libpq/hba.c
@@ -1749,7 +1749,11 @@ parse_hba_line(TokenizedAuthLine *tok_line, int elevel)
else if (strcmp(token->string, "radius") == 0)
parsedline->auth_method = uaRADIUS;
else if (strcmp(token->string, "oauth") == 0)
+#ifdef USE_OAUTH
parsedline->auth_method = uaOAuth;
+#else
+ unsupauth = "oauth";
+#endif
else
{
ereport(elevel,
diff --git a/src/backend/libpq/meson.build b/src/backend/libpq/meson.build
index c85527fb01..1c76dd80cc 100644
--- a/src/backend/libpq/meson.build
+++ b/src/backend/libpq/meson.build
@@ -1,7 +1,6 @@
# Copyright (c) 2022-2024, PostgreSQL Global Development Group
backend_sources += files(
- 'auth-oauth.c',
'auth-sasl.c',
'auth-scram.c',
'auth.c',
@@ -17,6 +16,10 @@ backend_sources += files(
'pqsignal.c',
)
+if oauth.found()
+ backend_sources += files('auth-oauth.c')
+endif
+
if ssl.found()
backend_sources += files('be-secure-openssl.c')
endif
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 2f4e3a8f63..a60df328bb 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -703,10 +703,7 @@
/* Define to select named POSIX semaphores. */
#undef USE_NAMED_POSIX_SEMAPHORES
-/* Define to 1 to build with OAuth 2.0 support. (--with-oauth) */
-#undef USE_OAUTH
-
-/* Define to 1 to use libcurl for OAuth support. */
+/* Define to 1 to build with OAuth 2.0 support. (--with-oauth=curl) */
#undef USE_OAUTH_CURL
/* Define to 1 to build with OpenSSL support. (--with-ssl=openssl) */
diff --git a/src/include/pg_config_manual.h b/src/include/pg_config_manual.h
index e49eb13e43..9bd655e306 100644
--- a/src/include/pg_config_manual.h
+++ b/src/include/pg_config_manual.h
@@ -172,6 +172,14 @@
#define USE_SSL
#endif
+/*
+ * USE_OAUTH code should be compiled only when compiling with OAuth support
+ * enabled.
+ */
+#ifdef USE_OAUTH_CURL
+#define USE_OAUTH
+#endif
+
/*
* This is the default directory in which AF_UNIX socket files are
* placed. Caution: changing this risks breaking your existing client
diff --git a/src/test/modules/oauth_validator/t/001_server.pl b/src/test/modules/oauth_validator/t/001_server.pl
index 3f06f5be76..46684a56e0 100644
--- a/src/test/modules/oauth_validator/t/001_server.pl
+++ b/src/test/modules/oauth_validator/t/001_server.pl
@@ -19,7 +19,7 @@ if (!$ENV{PG_TEST_EXTRA} || $ENV{PG_TEST_EXTRA} !~ /\boauth\b/)
if ($ENV{with_oauth} ne 'curl')
{
- plan skip_all => 'client-side OAuth not supported by this build';
+ plan skip_all => 'OAuth not supported by this build';
}
if ($ENV{with_python} ne 'yes')
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-10-28 16:09 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Daniel Gustafsson <daniel@yesql.se>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2024-10-28 16:09 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: Antonin Houska <ah@cybertec.at>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
On Mon, Oct 28, 2024 at 6:24 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> > On 25 Oct 2024, at 20:22, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
>
> > I have combed almost all of Daniel's feedback backwards into the main
> > patch (just the new bzero code remains, with the open question
> > upthread),
>
> Re-reading I can't see a vector there, I guess I am just scarred from what
> seemed to be harmless leaks in auth codepaths and treat every bit as
> potentially important. Feel free to drop from the patchset for now.
Okay. For authn_id specifically, which isn't secret and doesn't have
any power unless it's somehow copied into the ClientConnectionInfo,
I'm not sure that the bzero() gives us much. But I do see value in
clearing out, say, the Bearer token once we're finished with it.
Also in this validate() code path, I'm taking a look at the added
memory management with the pfree():
1. Should we add any more ceremony to the returned struct, to try to
ensure that the ABI matches? Or is it good enough to declare that
modules need to be compiled against a specific server version?
2. Should we split off a separate memory context to contain
allocations made by the validator?
> Looking more at the patchset I think we need to apply conditional compilation
> of the backend for oauth like how we do with other opt-in schemes in configure
> and meson. The attached .txt has a diff for making --with-oauth a requirement
> for compiling support into backend libpq.
Do we get the flexibility we need with that approach? With other
opt-in schemes, the backend and the frontend both need some sort of
third-party dependency, but that's not true for OAuth. I could see
some people wanting to support an offline token validator on the
server side but not wanting to build the HTTP dependency into their
clients.
I was considering going in the opposite direction: With the client
hooks, a user could plug in their own implementation without ever
having to touch the built-in flow, and I'm wondering if --with-oauth
should really just be --with-builtin-oauth or similar. Then if the
server sends OAUTHBEARER, the client only complains if it doesn't have
a flow available to use, rather than checking USE_OAUTH. This kind of
ties into the other big open question of "what do we do about users
that don't want the additional overhead of something they're not
using?"
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-10-29 10:52 Daniel Gustafsson <daniel@yesql.se>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 1 reply; 424+ messages in thread
From: Daniel Gustafsson @ 2024-10-29 10:52 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: Antonin Houska <ah@cybertec.at>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
> On 28 Oct 2024, at 17:09, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> On Mon, Oct 28, 2024 at 6:24 AM Daniel Gustafsson <daniel@yesql.se> wrote:
>> Looking more at the patchset I think we need to apply conditional compilation
>> of the backend for oauth like how we do with other opt-in schemes in configure
>> and meson. The attached .txt has a diff for making --with-oauth a requirement
>> for compiling support into backend libpq.
>
> Do we get the flexibility we need with that approach? With other
> opt-in schemes, the backend and the frontend both need some sort of
> third-party dependency, but that's not true for OAuth. I could see
> some people wanting to support an offline token validator on the
> server side but not wanting to build the HTTP dependency into their
> clients.
Currently we don't support any conditional compilation which only affects
backend or frontend, all --without-XXX flags turn it off for both. Maybe this
is something which should change but I'm not sure that property should be
altered as part of a patch rather than discussed on its own merit.
> I was considering going in the opposite direction: With the client
> hooks, a user could plug in their own implementation without ever
> having to touch the built-in flow, and I'm wondering if --with-oauth
> should really just be --with-builtin-oauth or similar. Then if the
> server sends OAUTHBEARER, the client only complains if it doesn't have
> a flow available to use, rather than checking USE_OAUTH. This kind of
> ties into the other big open question of "what do we do about users
> that don't want the additional overhead of something they're not
> using?"
We already know that GSS cause measurable performance impact on connections
even when compiled but not in use [0], so I think we should be careful about
piling on more.
--
Daniel Gustafsson
[0] 20240610181212.auytluwmbfl7lb5n@awork3.anarazel.de
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-10-29 16:40 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Daniel Gustafsson <daniel@yesql.se>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2024-10-29 16:40 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: Antonin Houska <ah@cybertec.at>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
On Tue, Oct 29, 2024 at 3:52 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> Currently we don't support any conditional compilation which only affects
> backend or frontend, all --without-XXX flags turn it off for both.
I don't think that's strictly true; see --with-pam which affects only
server-side code, since the hard part is in the server. Similarly,
--with-oauth currently affects only client-side code.
But in any case, that confusion is why I'm proposing a change to the
option name. I chose --with-oauth way before the architecture
solidified, and it doesn't reflect reality anymore. OAuth support on
the server side doesn't require Curl, and likely never will. So if you
want to support that on a Windows server, it's going to be strange if
we also force you to build the client with a libcurl dependency that
we won't even make use of on that platform.
> We already know that GSS cause measurable performance impact on connections
> even when compiled but not in use [0], so I think we should be careful about
> piling on more.
I agree, but if the server asks for OAUTHBEARER, that's the end of it.
Either the client supports OAuth and initiates a token flow, or it
doesn't and the connection fails. That's very different from the
client-initiated transport negotiation.
On the other hand, if we're concerned about the link-time overhead
(time and/or RAM) of the new dependency, I think that's going to need
something different from a build-time switch. My guess is that
maintainers are only going to want to ship one libpq.
Thanks,
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-10-29 17:41 Daniel Gustafsson <daniel@yesql.se>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 1 reply; 424+ messages in thread
From: Daniel Gustafsson @ 2024-10-29 17:41 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: Antonin Houska <ah@cybertec.at>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
> On 29 Oct 2024, at 17:40, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
>
> On Tue, Oct 29, 2024 at 3:52 AM Daniel Gustafsson <daniel@yesql.se> wrote:
>> Currently we don't support any conditional compilation which only affects
>> backend or frontend, all --without-XXX flags turn it off for both.
>
> I don't think that's strictly true; see --with-pam which affects only
> server-side code, since the hard part is in the server. Similarly,
> --with-oauth currently affects only client-side code.
Fair, maybe it's an unwarranted concern. Question is though, if we added PAM
today would we have done the same?
> But in any case, that confusion is why I'm proposing a change to the
> option name.
+1
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-10-29 20:39 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Daniel Gustafsson <daniel@yesql.se>
0 siblings, 0 replies; 424+ messages in thread
From: Jacob Champion @ 2024-10-29 20:39 UTC (permalink / raw)
To: Daniel Gustafsson <daniel@yesql.se>; +Cc: Antonin Houska <ah@cybertec.at>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
On Tue, Oct 29, 2024 at 10:41 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> Question is though, if we added PAM
> today would we have done the same?
I assume so; the client can't tell PAM apart from LDAP or any other
plaintext method. (In the same vein, the server can't tell if the
client uses libcurl to grab a token, or something entirely different.)
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-10-31 11:05 Antonin Houska <ah@cybertec.at>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
1 sibling, 1 reply; 424+ messages in thread
From: Antonin Houska @ 2024-10-31 11:05 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: Daniel Gustafsson <daniel@yesql.se>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> On Thu, Oct 17, 2024 at 10:51 PM Antonin Houska <ah@cybertec.at> wrote:
> > * oauth_validator_library is defined as PGC_SIGHUP - is that intentional?
>
> Yes, I think it's going to be important to let DBAs migrate their
> authentication modules without a full restart. That probably deserves
> more explicit testing, now that you mention it. Is there a specific
> concern that you have with that?
No concern. I was just trying to imagine when the module needs to be changed.
> > And regardless, the library appears to be loaded by every backend during
> > authentication. Why isn't it loaded by postmaster like libraries listed in
> > shared_preload_libraries? fork() would then ensure that the backends do have
> > the library in their address space.
>
> It _can_ be, if you want -- there's nothing that I know of preventing
> the validator from also being preloaded with its own _PG_init(), is
> there? But I don't think it's a good idea to force that, for the same
> reason we want to allow SIGHUP.
Loading the library by postmaster does not prevent the backends from reloading
it on SIGHUP later. I was simply concerned about performance. (I proposed
loading the library at another stage of backend initialization rather than
adding _PG_init() to it.)
> > * pg_fe_run_oauth_flow()
> >
> > When first time here
> > case OAUTH_STEP_TOKEN_REQUEST:
> > if (!handle_token_response(actx, &state->token))
> > goto error_return;
> >
> > the user hasn't been prompted yet so ISTM that the first token request must
> > always fail. It seems more logical if the prompt is set to the user before
> > sending the token request to the server. (Although the user probably won't
> > be that fast to make the first request succeed, so consider this just a
> > hint.)
>
> That's also intentional -- if the first token response fails for a
> reason _other_ than "we're waiting for the user", then we want to
> immediately fail hard instead of making them dig out their phone and
> go on a two-minute trip, because they're going to come back and find
> that it was all for nothing.
>
> There's a comment immediately below the part you quoted that mentions
> this briefly; maybe I should move it up a bit?
That's fine, I understand now.
> > * As long as I understand, the following comment would make sense:
> >
> > diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
> > index f943a31cc08..97259fb5654 100644
> > --- a/src/interfaces/libpq/fe-auth-oauth.c
> > +++ b/src/interfaces/libpq/fe-auth-oauth.c
> > @@ -518,6 +518,7 @@ oauth_exchange(void *opaq, bool final,
> > switch (state->state)
> > {
> > case FE_OAUTH_INIT:
> > + /* Initial Client Response */
> > Assert(inputlen == -1);
> >
> > if (!derive_discovery_uri(conn))
>
> There are multiple "initial client response" cases, though. What
> questions are you hoping to clarify with the comment? Maybe we can
> find a more direct answer.
Easiness of reading is the only "question" here :-) It's might not always be
obvious why a variable should have some particular value. In general, the
Assert() statements are almost always preceded with a comment in the PG
source.
> > Or, doesn't the FE_OAUTH_INIT branch of the switch statement actually fit
> > better into oauth_init()?
>
> oauth_init() is the mechanism initialization for the SASL framework
> itself, which is shared with SCRAM. In the current architecture, the
> init callback doesn't take the initial client response into
> consideration at all.
Sure. The FE_OAUTH_INIT branch in oauth_exchange() (FE) also does not generate
the initial client response.
Based on reading the SCRAM implementation, I concluded that the init()
callback can do authentication method specific things, but unlike exchange()
it does not generate any output.
> Generating the client response is up to the exchange callback -- and
> even if we moved the SASL_ASYNC processing elsewhere, I don't think we
> can get rid of its added complexity. Something has to signal upwards
> that it's time to transfer control to an async engine. And we can't
> make the asynchronicity a static attribute of the mechanism itself,
> because we can skip the flow if something gives us a cached token.
I didn't want to skip the flow. I thought that the init() callback could be
made responsible for getting the token, but forgot that it still needs some
way to signal to the caller that the async flow is needed.
Anyway, are you sure that pg_SASL_continue() can also receive the SASL_ASYNC
value from oauth_exchange()? My understanding is that pg_SASL_init() receives
it if there is no token, but after that, oauth_exchange() is not called util
the token is available, and thus it should not return SASL_ASYNC anymore.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-10-31 16:24 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Antonin Houska <ah@cybertec.at>
0 siblings, 1 reply; 424+ messages in thread
From: Jacob Champion @ 2024-10-31 16:24 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Daniel Gustafsson <daniel@yesql.se>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
On Thu, Oct 31, 2024 at 4:05 AM Antonin Houska <ah@cybertec.at> wrote:
> > > And regardless, the library appears to be loaded by every backend during
> > > authentication. Why isn't it loaded by postmaster like libraries listed in
> > > shared_preload_libraries? fork() would then ensure that the backends do have
> > > the library in their address space.
> >
> > It _can_ be, if you want -- there's nothing that I know of preventing
> > the validator from also being preloaded with its own _PG_init(), is
> > there? But I don't think it's a good idea to force that, for the same
> > reason we want to allow SIGHUP.
>
> Loading the library by postmaster does not prevent the backends from reloading
> it on SIGHUP later. I was simply concerned about performance. (I proposed
> loading the library at another stage of backend initialization rather than
> adding _PG_init() to it.)
Okay. I think this is going to be one of the slower authentication
methods by necessity: the builtin flow in libpq requires a human in
the loop, and an online validator is going to be making several HTTP
calls from the backend. So if it turns out later that we need to
optimize the backend logic, I'd prefer to have a case study in hand;
otherwise I think we're likely to optimize the wrong things.
> Easiness of reading is the only "question" here :-) It's might not always be
> obvious why a variable should have some particular value. In general, the
> Assert() statements are almost always preceded with a comment in the PG
> source.
Oh, an assertion label! I can absolutely add one; I originally thought
you were proposing a label for the case itself.
> > > Or, doesn't the FE_OAUTH_INIT branch of the switch statement actually fit
> > > better into oauth_init()?
> >
> > oauth_init() is the mechanism initialization for the SASL framework
> > itself, which is shared with SCRAM. In the current architecture, the
> > init callback doesn't take the initial client response into
> > consideration at all.
>
> Sure. The FE_OAUTH_INIT branch in oauth_exchange() (FE) also does not generate
> the initial client response.
It might, if it ends up falling through to FE_OAUTH_REQUESTING_TOKEN.
There are two paths that can do that: the case where we have no
discovery URI, and the case where a custom user flow returns a token
synchronously (it was probably cached).
> Anyway, are you sure that pg_SASL_continue() can also receive the SASL_ASYNC
> value from oauth_exchange()? My understanding is that pg_SASL_init() receives
> it if there is no token, but after that, oauth_exchange() is not called util
> the token is available, and thus it should not return SASL_ASYNC anymore.
Correct -- the only way for the current implementation of the
OAUTHBEARER mechanism to return SASL_ASYNC is during the very first
call. That's not an assumption I want to put into the higher levels,
though; I think Michael will be unhappy with me if I introduce
additional SASL coupling after the decoupling work that's been done
over the last few releases. :D
Thanks again,
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-11-04 05:00 jian he <jian.universality@gmail.com>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 2 replies; 424+ messages in thread
From: jian he @ 2024-11-04 05:00 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: Antonin Houska <ah@cybertec.at>; Daniel Gustafsson <daniel@yesql.se>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
Hi there.
zero knowledge of Oath, just reading through the v35-0001.
forgive me if my comments are naive.
+static int
+parse_interval(struct async_ctx *actx, const char *interval_str)
+{
+ double parsed;
+ int cnt;
+
+ /*
+ * The JSON lexer has already validated the number, which is stricter than
+ * the %f format, so we should be good to use sscanf().
+ */
+ cnt = sscanf(interval_str, "%lf", &parsed);
+
+ if (cnt != 1)
+ {
+ /*
+ * Either the lexer screwed up or our assumption above isn't true, and
+ * either way a developer needs to take a look.
+ */
+ Assert(cnt == 1);
+ return 1; /* don't fall through in release builds */
+ }
+
+ parsed = ceil(parsed);
+
+ if (parsed < 1)
+ return actx->debugging ? 0 : 1;
+
+ else if (INT_MAX <= parsed)
+ return INT_MAX;
+
+ return parsed;
+}
The above Assert looks very wrong to me.
we can also use PG_INT32_MAX, instead of INT_MAX
(generally i think PG_INT32_MAX looks more intuitive to me)
+/*
+ * The Device Authorization response, described by RFC 8628:
+ *
+ * https://www.rfc-editor.org/rfc/rfc8628#section-3.2
+ */
+struct device_authz
+{
+ char *device_code;
+ char *user_code;
+ char *verification_uri;
+ char *interval_str;
+
+ /* Fields below are parsed from the corresponding string above. */
+ int interval;
+};
click through the link https://www.rfc-editor.org/rfc/rfc8628#section-3.2
it says
"
expires_in
REQUIRED. The lifetime in seconds of the "device_code" and
"user_code".
interval
OPTIONAL. The minimum amount of time in seconds that the client
SHOULD wait between polling requests to the token endpoint. If no
value is provided, clients MUST use 5 as the default.
"
these two fields seem to differ from struct device_authz.
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-11-04 09:11 Daniel Gustafsson <daniel@yesql.se>
parent: jian he <jian.universality@gmail.com>
1 sibling, 0 replies; 424+ messages in thread
From: Daniel Gustafsson @ 2024-11-04 09:11 UTC (permalink / raw)
To: jian he <jian.universality@gmail.com>; +Cc: Jacob Champion <jacob.champion@enterprisedb.com>; Antonin Houska <ah@cybertec.at>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
> On 4 Nov 2024, at 06:00, jian he <jian.universality@gmail.com> wrote:
> + if (cnt != 1)
> + {
> + /*
> + * Either the lexer screwed up or our assumption above isn't true, and
> + * either way a developer needs to take a look.
> + */
> + Assert(cnt == 1);
> + return 1; /* don't fall through in release builds */
> + }
> The above Assert looks very wrong to me.
I think the point is to fail hard in development builds to ensure whatever
caused the disconnect between the json lexer and sscanf parsing is looked at.
It should probably be changed to Assert(false); which is the common pattern for
erroring out like this.
--
Daniel Gustafsson
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-11-05 23:07 Jacob Champion <jacob.champion@enterprisedb.com>
parent: jian he <jian.universality@gmail.com>
1 sibling, 0 replies; 424+ messages in thread
From: Jacob Champion @ 2024-11-05 23:07 UTC (permalink / raw)
To: jian he <jian.universality@gmail.com>; +Cc: Antonin Houska <ah@cybertec.at>; Daniel Gustafsson <daniel@yesql.se>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
On Sun, Nov 3, 2024 at 9:00 PM jian he <jian.universality@gmail.com> wrote:
> The above Assert looks very wrong to me.
I can switch to Assert(false) if that's preferred, but it makes part
of the libc assert() report useless. (I wish we had more fluent ways
to say "this shouldn't happen, but if it does, we still need to get
out safely.")
> we can also use PG_INT32_MAX, instead of INT_MAX
> (generally i think PG_INT32_MAX looks more intuitive to me)
That's a fixed-width max; we want the maximum for the `int` type here.
> expires_in
> REQUIRED. The lifetime in seconds of the "device_code" and
> "user_code".
> interval
> OPTIONAL. The minimum amount of time in seconds that the client
> SHOULD wait between polling requests to the token endpoint. If no
> value is provided, clients MUST use 5 as the default.
> "
> these two fields seem to differ from struct device_authz.
Yeah, Daniel and I had talked about being stricter about REQUIRED
fields that are not currently used. There's a comment making note of
this in parse_device_authz(). The v1 code will need to make expires_in
REQUIRED, so that future developers can develop features that depend
on it without worrying about breaking
currently-working-but-noncompliant deployments. (And if there are any
noncompliant deployments out there now, we need to know about them so
we can have that explicit discussion.)
Thanks,
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-11-05 23:39 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
1 sibling, 0 replies; 424+ messages in thread
From: Jacob Champion @ 2024-11-05 23:39 UTC (permalink / raw)
To: Antonin Houska <ah@cybertec.at>; +Cc: Daniel Gustafsson <daniel@yesql.se>; Peter Eisentraut <peter@eisentraut.org>; pgsql-hackers
On Tue, Nov 5, 2024 at 3:33 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Done in v36, attached.
Forgot to draw attention to this part:
> +# XXX libcurl must link after libgssapi_krb5 on FreeBSD to avoid segfaults
> +# during gss_acquire_cred(). This is possibly related to Curl's Heimdal
> +# dependency on that platform?
Best I can tell, libpq for FreeBSD has a dependency diamond for GSS
symbols: libpq links against MIT krb5, libcurl links against Heimdal,
libpq links against libcurl. Link order becomes critical to avoid
nasty segfaults, but I have not dug deeply into the root cause.
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-11-08 09:21 Peter Eisentraut <peter@eisentraut.org>
parent: Jacob Champion <jacob.champion@enterprisedb.com>
1 sibling, 1 reply; 424+ messages in thread
From: Peter Eisentraut @ 2024-11-08 09:21 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: Daniel Gustafsson <daniel@yesql.se>; Antonin Houska <ah@cybertec.at>; pgsql-hackers
On 06.11.24 00:33, Jacob Champion wrote:
> Done in v36, attached.
Assorted review comments from me:
Everything in the commit message between
= Debug Mode =
and
Several TODOs:
should be moved to the documentation. In some cases, it already is,
but it doesn't always have the same level of detail.
(You could point from the commit message to .sgml files if you want to
highlight usage instructions, but I don't think this is generally
necessary.)
* config/programs.m4
Can we do the libcurl detection using pkg-config only? Seems simpler,
and maintains consistency to meson.
* doc/src/sgml/client-auth.sgml
In the list of terms (this could be a <variablelist>), state how these
terms map to a PostgreSQL installation. You already explain what the
client and the resource server are, but not who the resource owner is
and what the authorization server is. It would also be good to be
explicit and upfront that the authorization server is a third-party
component that needs to be obtained separately.
trust_validator_authz: Personally, I'm not a fan of the "authz" and
"authn" abbreviations. I know this is security jargon. But are
regular users going to understand this? Can we just spell it out?
* doc/src/sgml/config.sgml
Also here maybe state that these OAuth libraries have to be obtained
separately.
* doc/src/sgml/installation.sgml
I find the way the installation options are structured a bit odd. I
would have expected --with-libcurl and -Dlibcurl (or --with-curl and
-Dcurl). These build options usually just say, use this library. We
don't spell out what, for example, libldap is used for, we just use it
and enable all the features that require it.
* doc/src/sgml/libpq.sgml
Maybe oauth_issuer should be oauth_issuer_url? Otherwise one might
expect to just write "google" here or something. Or there might be
other ways to contact an issuer in the future? Just a thought.
* doc/src/sgml/oauth-validators.sgml
This chapter says "libpq" several times, but I think this is a server
side plugin, so libpq does not participate. Check please.
* src/backend/libpq/auth-oauth.c
I'm confused by the use of PG_MAX_AUTH_TOKEN_LENGTH in the
pg_be_oauth_mech definition. What does that mean?
+#define KVSEP 0x01
+#define AUTH_KEY "auth"
+#define BEARER_SCHEME "Bearer "
Add comments to these.
Also, add comments to all functions defined here that don't have one
yet.
* src/backend/utils/misc/guc_tables.c
Why is oauth_validator_library GUC_NOT_IN_SAMPLE?
Also, shouldn't this be an hba option instead? What if you want to
use different validators for different connections?
* src/interfaces/libpq/fe-auth-oauth-curl.c
The CURL_IGNORE_DEPRECATION thing needs clarification. Is that in
progress?
+#define MAX_OAUTH_RESPONSE_SIZE (1024 * 1024)
Add a comment about why this value.
+ union
+ {
+ char **scalar; /* for all scalar types */
+ struct curl_slist **array; /* for type == JSON_TOKEN_ARRAY_START */
+ };
This is an anonymous union, which requires C11. Strangely, I cannot
get clang to warn about this with -Wc11-extensions. Probably better
to fix anyway. (The trailing supported MSVC versions don't support
C11 yet.)
* src/interfaces/libpq/fe-auth.h
+extern const pg_fe_sasl_mech pg_oauth_mech;
Should this rather be in fe-auth-oauth.h?
* src/interfaces/libpq/libpq-fe.h
The naming scheme of types and functions in this file is clearly
obscure and has grown randomly over time. But at least my intuition
is that the preferred way is
types start with PG
function start with PQ
and the next letter is usually lower case. (PQconnectdb, PQhost,
PGconn, PQresult)
Maybe check your additions against that.
* src/interfaces/libpq/pqexpbuffer.c
* src/interfaces/libpq/pqexpbuffer.h
Let's try to do this without opening up additional APIs here.
This is only used once, in append_urlencoded(), and there are other
ways to communicate errors, for example returning a bool.
* src/test/modules/oauth_validator/
Everything in this directory needs more comments, at least on a file
level.
Add a README in this directory. Also update the README in the upper
directory.
* src/test/modules/oauth_validator/t/001_server.pl
On Cirrus CI Windows task, this test reports SKIP. Can't tell why,
because the log is not kept. I suppose you expect this to work on
Windows (but see my comment below), so it would be good to get this
test running.
* src/test/modules/oauth_validator/t/002_client.pl
+my $issuer = "https://127.0.0.1:54321";;
Use PostgreSQL::Test::Cluster::get_free_port() instead of hardcoding
port numbers.
Or is this a real port? I don't see it used anywhere else.
+ diag "running '" . join("' '", @cmd) . "'";
This should be "note" instead. Otherwise it garbles the output.
* src/test/perl/PostgreSQL/Test/OAuthServer.pm
Add some comments to this file, what it's for.
Is this meant to work on Windows? Just thinking, things like
kill(15, $self->{'pid'});
pgperlcritic complains:
src/test/perl/PostgreSQL/Test/OAuthServer.pm: Return value of flagged
function ignored - read at line 39, column 2.
* src/tools/pgindent/typedefs.list
We don't need to typedef every locally used enum or similar into a
full typedef. I suggest the following might be unnecessary:
AsyncAuthFunc
OAuthStep
fe_oauth_state_enum
^ permalink raw reply [nested|flat] 424+ messages in thread
* Re: [PoC] Federated Authn/z with OAUTHBEARER
@ 2024-11-12 21:47 Jacob Champion <jacob.champion@enterprisedb.com>
parent: Peter Eisentraut <peter@eisentraut.org>
0 siblings, 2 replies; 424+ messages in thread
From: Jacob Champion @ 2024-11-12 21:47 UTC (permalink / raw)
To: Peter Eisentraut <peter@eisentraut.org>; +Cc: Daniel Gustafsson <daniel@yesql.se>; Antonin Houska <ah@cybertec.at>; pgsql-hackers
On Fri, Nov 8, 2024 at 1:21 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> Assorted review comments from me:
Thank you! I will cherry-pick some responses here and plan to address
the rest in a future patchset.
> trust_validator_authz: Personally, I'm not a fan of the "authz" and
> "authn" abbreviations. I know this is security jargon. But are
> regular users going to understand this? Can we just spell it out?
Yes. That name's a holdover from the very first draft, actually.
Is "trust_validator_authorization" a great name in the first place?
The key concept is that user mapping is being delegated to the OAuth
system itself, so you'd better make sure that the validator has been
built to do that. (Anyone have any suggestions?)
> I find the way the installation options are structured a bit odd. I
> would have expected --with-libcurl and -Dlibcurl (or --with-curl and
> -Dcurl). These build options usually just say, use this library.
It's patterned directly off of -Dssl/--with-ssl (which I liberally
borrowed from) because the builtin client implementation used to have
multiple options for the library in use. I can change it if needed,
but I thought it'd be helpful for future devs if I didn't undo the
generalization.
> Maybe oauth_issuer should be oauth_issuer_url? Otherwise one might
> expect to just write "google" here or something. Or there might be
> other ways to contact an issuer in the future? Just a thought.
More specifically this is an "issuer identifier", as defined by the
OAuth/OpenID discovery specs. It's a subset of a URL, and I want to
make sure users know how to differentiate between an "issuer" they
trust and the "discovery URI" that's in use for that issuer. They may
want to set one or the other -- a discovery URI is associated with
exactly one issuer, but unfortunately an issuer may have multiple
discovery URIs, which I'm actively working on. (There is also some
relation to the multiple-issuers problem mentioned below.)
> I'm confused by the use of PG_MAX_AUTH_TOKEN_LENGTH in the
> pg_be_oauth_mech definition. What does that mean?
Just that Bearer tokens can be pretty long, so we don't want to limit
them to 1k like SCRAM does. 64k is probably overkill, but I've seen
anecdotal reports of tens of KBs and it seemed reasonable to match
what we're doing for GSS tokens.
> Also, shouldn't [oauth_validator_library] be an hba option instead? What if you want to
> use different validators for different connections?
Yes. This is again the multiple-issuers problem; I will split that off
into its own email since this one's getting long. It has security
implications.
> The CURL_IGNORE_DEPRECATION thing needs clarification. Is that in
> progress?
Thanks for the nudge, I've started a thread:
https://curl.se/mail/lib-2024-11/0028.html
> This is an anonymous union, which requires C11. Strangely, I cannot
> get clang to warn about this with -Wc11-extensions. Probably better
> to fix anyway. (The trailing supported MSVC versions don't support
> C11 yet.)
Oh, that's not going to be fun.
> This is only used once, in append_urlencoded(), and there are other
> ways to communicate errors, for example returning a bool.
I'd rather not introduce two parallel error indicators for the caller
to have to check for that particular part. But I can change over to
using the (identical!) termPQExpBuffer. I felt like the other API
signaled the intent a little better, though.
> On Cirrus CI Windows task, this test reports SKIP. Can't tell why,
> because the log is not kept. I suppose you expect this to work on
> Windows (but see my comment below)
No, builtin client support does not exist on Windows. If/when it's
added, the 001_server tests will need to be ported.
> +my $issuer = "https://127.0.0.1:54321";;
>
> Use PostgreSQL::Test::Cluster::get_free_port() instead of hardcoding
> port numbers.
>
> Or is this a real port? I don't see it used anywhere else.
It's not real; 002_client.pl doesn't start an authorization server at
all. I can make that more explicit.
> src/test/perl/PostgreSQL/Test/OAuthServer.pm: Return value of flagged
> function ignored - read at line 39, column 2.
So perlcritic recognizes "or" but not the "//" operator... Lovely.
Thanks!
--Jacob
^ permalink raw reply [nested|flat] 424+ messages in thread
278 further messages in this thread omitted from this page (render size limit).
Use the per-message pages, or the t.mbox.gz byte stream, for the complete thread.
end of thread, other threads:[~2025-08-08 16:23 UTC | newest]
Thread overview: 424+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-06-08 16:37 [PoC] Federated Authn/z with OAUTHBEARER Jacob Champion <pchampion@vmware.com>
2021-06-18 04:07 ` Michael Paquier <michael@paquier.xyz>
2021-06-22 23:26 ` Jacob Champion <pchampion@vmware.com>
2021-06-23 06:10 ` Michael Paquier <michael@paquier.xyz>
2021-06-18 08:31 ` Heikki Linnakangas <hlinnaka@iki.fi>
2021-06-22 23:22 ` Jacob Champion <pchampion@vmware.com>
2021-08-25 18:41 ` Jacob Champion <pchampion@vmware.com>
2021-08-25 22:25 ` Zhihong Yu <zyu@yugabyte.com>
2021-08-25 23:24 ` Zhihong Yu <zyu@yugabyte.com>
2021-08-26 16:13 ` Jacob Champion <pchampion@vmware.com>
2021-08-26 16:20 ` Zhihong Yu <zyu@yugabyte.com>
2021-08-27 02:32 ` Michael Paquier <michael@paquier.xyz>
2021-08-31 20:48 ` Jacob Champion <pchampion@vmware.com>
2022-03-04 19:13 ` Jacob Champion <pchampion@vmware.com>
2022-03-22 21:48 ` samay sharma <smilingsamay@gmail.com>
2022-03-22 22:44 ` Jacob Champion <pchampion@vmware.com>
2022-03-26 00:00 ` Jacob Champion <pchampion@vmware.com>
2022-09-23 22:39 ` Jacob Champion <jchampion@timescale.com>
2022-09-27 01:39 ` Andrey Chudnovsky <achudnovskij@gmail.com>
2022-09-27 21:45 ` Jacob Champion <jchampion@timescale.com>
2022-09-30 14:47 ` Andrey Chudnovsky <achudnovskij@gmail.com>
2022-09-30 20:45 ` Jacob Champion <jchampion@timescale.com>
2022-10-03 18:04 ` Andrey Chudnovsky <achudnovskij@gmail.com>
2022-11-23 09:58 ` mahendrakar s <mahendrakarforpg@gmail.com>
2022-11-23 20:05 ` Jacob Champion <jchampion@timescale.com>
2022-11-24 03:45 ` Andrey Chudnovsky <achudnovskij@gmail.com>
2022-11-24 08:20 ` mahendrakar s <mahendrakarforpg@gmail.com>
2022-11-29 21:19 ` Jacob Champion <jchampion@timescale.com>
2022-12-06 00:15 ` Andrey Chudnovsky <achudnovskij@gmail.com>
2022-12-07 19:06 ` Jacob Champion <jchampion@timescale.com>
2022-12-07 23:22 ` Andrey Chudnovsky <achudnovskij@gmail.com>
2022-12-08 04:25 ` Andrey Chudnovsky <achudnovskij@gmail.com>
2022-12-09 00:41 ` Jacob Champion <jchampion@timescale.com>
2022-12-13 05:06 ` Andrey Chudnovsky <achudnovskij@gmail.com>
2022-12-16 23:18 ` Jacob Champion <jchampion@timescale.com>
2023-01-12 19:08 ` mahendrakar s <mahendrakarforpg@gmail.com>
2023-01-15 20:03 ` Andrey Chudnovsky <achudnovskij@gmail.com>
2023-01-17 22:43 ` Jacob Champion <jchampion@timescale.com>
2023-01-18 01:53 ` Andrey Chudnovsky <achudnovskij@gmail.com>
2023-01-25 04:46 ` mahendrakar s <mahendrakarforpg@gmail.com>
2023-02-20 22:35 ` Stephen Frost <sfrost@snowman.net>
2023-02-21 22:24 ` Jacob Champion <jchampion@timescale.com>
2023-02-22 07:00 ` Andrey Chudnovsky <achudnovskij@gmail.com>
2023-02-23 18:47 ` Stephen Frost <sfrost@snowman.net>
2023-02-23 23:04 ` Andrey Chudnovsky <achudnovskij@gmail.com>
2023-02-27 20:31 ` Stephen Frost <sfrost@snowman.net>
2022-11-29 21:12 ` Jacob Champion <jchampion@timescale.com>
2023-04-27 17:35 ` Jacob Champion <jchampion@timescale.com>
2023-05-19 22:01 ` Jacob Champion <jchampion@timescale.com>
2023-05-23 11:22 ` Daniele Varrazzo <daniele.varrazzo@gmail.com>
2023-05-23 15:56 ` Jacob Champion <jchampion@timescale.com>
2023-06-29 16:28 ` Jacob Champion <jchampion@timescale.com>
2023-07-01 04:28 ` Thomas Munro <thomas.munro@gmail.com>
2023-07-05 21:00 ` Jacob Champion <jchampion@timescale.com>
2023-07-05 22:07 ` Thomas Munro <thomas.munro@gmail.com>
2023-07-06 16:56 ` Jacob Champion <jchampion@timescale.com>
2023-07-06 20:47 ` Thomas Munro <thomas.munro@gmail.com>
2023-07-07 18:48 ` Jacob Champion <jchampion@timescale.com>
2023-07-07 21:16 ` Andrey Chudnovsky <achudnovskij@gmail.com>
2023-07-10 23:21 ` Jacob Champion <jchampion@timescale.com>
2023-07-08 01:00 ` Thomas Munro <thomas.munro@gmail.com>
2023-07-10 23:50 ` Jacob Champion <jchampion@timescale.com>
2023-07-11 17:50 ` Jacob Champion <jchampion@timescale.com>
2023-07-12 01:37 ` Thomas Munro <thomas.munro@gmail.com>
2023-07-13 04:51 ` Andrey Chudnovsky <achudnovskij@gmail.com>
2023-07-17 23:55 ` Jacob Champion <jchampion@timescale.com>
2023-07-18 23:03 ` Thomas Munro <thomas.munro@gmail.com>
2023-07-26 16:43 ` Jacob Champion <jchampion@timescale.com>
2023-08-30 22:57 ` Jacob Champion <jchampion@timescale.com>
2023-09-06 22:11 ` Jacob Champion <jchampion@timescale.com>
2023-11-03 12:28 ` Shlok Kyal <shlok.kyal.oss@gmail.com>
2023-11-03 23:48 ` Jacob Champion <champion.p@gmail.com>
2023-11-08 19:00 ` Jacob Champion <champion.p@gmail.com>
2023-11-10 01:42 ` Andrey Chudnovsky <achudnovskij@gmail.com>
2023-11-15 20:20 ` Jacob Champion <champion.p@gmail.com>
2023-12-05 09:43 ` Daniel Gustafsson <daniel@yesql.se>
2024-01-09 18:48 ` Jacob Champion <champion.p@gmail.com>
2024-02-21 01:00 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-02-22 14:08 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-02-24 01:01 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-02-27 19:33 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-02-28 14:05 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-02-28 17:40 ` Daniel Gustafsson <daniel@yesql.se>
2024-03-01 00:04 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-03-01 01:08 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-03-01 17:46 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-03-11 22:51 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-03-22 18:21 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-03-28 22:34 ` Daniel Gustafsson <daniel@yesql.se>
2024-04-01 22:07 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-07-03 17:02 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-07-10 00:05 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-07-29 12:01 ` Peter Eisentraut <peter@eisentraut.org>
2024-07-29 22:30 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-08-02 17:13 ` Peter Eisentraut <peter@eisentraut.org>
2024-08-02 17:51 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-08-02 18:48 ` Peter Eisentraut <peter@eisentraut.org>
2024-08-05 17:53 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-08-07 07:34 ` Peter Eisentraut <peter@eisentraut.org>
2024-08-12 06:37 ` Peter Eisentraut <peter@eisentraut.org>
2024-08-13 21:11 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-08-26 08:18 ` Peter Eisentraut <peter@eisentraut.org>
2024-08-26 23:23 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-08-28 16:31 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-08-30 09:49 ` Peter Eisentraut <peter@eisentraut.org>
2024-09-03 20:56 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-09-04 09:28 ` Peter Eisentraut <peter@eisentraut.org>
2024-09-11 07:37 ` Peter Eisentraut <peter@eisentraut.org>
2024-09-11 13:44 ` Daniel Gustafsson <daniel@yesql.se>
2024-09-11 22:54 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-09-16 19:13 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-09-27 17:58 ` Antonin Houska <ah@cybertec.at>
2024-09-27 20:45 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-09-30 13:38 ` Antonin Houska <ah@cybertec.at>
2024-09-30 16:54 ` Antonin Houska <ah@cybertec.at>
2024-10-01 00:05 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-10-08 10:46 ` Antonin Houska <ah@cybertec.at>
2024-10-08 15:19 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-10-18 05:51 ` Antonin Houska <ah@cybertec.at>
2024-10-23 22:40 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-10-25 18:22 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-10-28 13:24 ` Daniel Gustafsson <daniel@yesql.se>
2024-10-28 16:09 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-10-29 10:52 ` Daniel Gustafsson <daniel@yesql.se>
2024-10-29 16:40 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-10-29 17:41 ` Daniel Gustafsson <daniel@yesql.se>
2024-10-29 20:39 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-10-29 20:34 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-11-05 23:33 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-11-05 23:39 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-11-08 09:21 ` Peter Eisentraut <peter@eisentraut.org>
2024-11-12 21:47 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-11-14 17:45 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-11-26 13:51 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-11-27 17:27 ` Daniel Gustafsson <daniel@yesql.se>
2024-12-05 18:29 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-12-12 00:18 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-12-15 22:18 ` Daniel Gustafsson <daniel@yesql.se>
2024-12-17 00:51 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-12-20 01:00 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-12-20 22:20 ` Daniel Gustafsson <daniel@yesql.se>
2025-01-07 22:20 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-01-08 09:07 ` Kashif Zeeshan <kashi.zeeshan@gmail.com>
2025-01-02 10:11 ` Peter Eisentraut <peter@eisentraut.org>
2025-01-07 22:24 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-01-08 17:13 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-01-08 19:37 ` Peter Eisentraut <peter@eisentraut.org>
2025-01-08 20:29 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-01-09 16:17 ` Peter Eisentraut <peter@eisentraut.org>
2025-01-09 19:18 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-01-10 13:27 ` Peter Eisentraut <peter@eisentraut.org>
2025-01-13 23:21 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-01-14 01:00 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-01-17 19:02 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-01-20 06:19 ` Kashif Zeeshan <kashi.zeeshan@gmail.com>
2025-01-20 22:10 ` Daniel Gustafsson <daniel@yesql.se>
2025-01-21 00:40 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-01-21 00:43 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-01-21 09:29 ` Daniel Gustafsson <daniel@yesql.se>
2025-01-21 16:46 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-01-27 22:49 ` Daniel Gustafsson <daniel@yesql.se>
2025-01-28 00:59 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-01-31 16:23 ` Daniel Gustafsson <daniel@yesql.se>
2025-02-06 22:02 ` Daniel Gustafsson <daniel@yesql.se>
2025-02-07 05:48 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-02-07 20:12 ` Daniel Gustafsson <daniel@yesql.se>
2025-02-08 01:56 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-02-12 14:55 ` Peter Eisentraut <peter@eisentraut.org>
2025-02-12 14:59 ` Peter Eisentraut <peter@eisentraut.org>
2025-02-13 21:23 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-02-13 23:01 ` Daniel Gustafsson <daniel@yesql.se>
2025-02-13 22:56 ` Daniel Gustafsson <daniel@yesql.se>
2025-02-15 01:14 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-02-17 12:03 ` Daniel Gustafsson <daniel@yesql.se>
2025-02-17 18:15 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-02-17 23:51 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-02-19 14:13 ` Daniel Gustafsson <daniel@yesql.se>
2025-02-19 19:54 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-02-20 16:29 ` Daniel Gustafsson <daniel@yesql.se>
2025-02-20 17:28 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-02-21 17:18 ` Andres Freund <andres@anarazel.de>
2025-02-21 21:18 ` Daniel Gustafsson <daniel@yesql.se>
2025-02-24 17:39 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-02-24 20:30 ` Andres Freund <andres@anarazel.de>
2025-02-24 22:02 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-02-25 17:22 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-02-28 13:43 ` Thomas Munro <thomas.munro@gmail.com>
2025-02-28 17:37 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-02-28 21:57 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-03-01 00:36 ` Thomas Munro <thomas.munro@gmail.com>
2025-03-04 00:07 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-03-04 04:10 ` Thomas Munro <thomas.munro@gmail.com>
2025-03-04 17:08 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-03-04 22:37 ` Thomas Munro <thomas.munro@gmail.com>
2025-03-04 22:44 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-03-06 20:57 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-03-07 00:35 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-03-07 05:12 ` Thomas Munro <thomas.munro@gmail.com>
2025-03-07 17:30 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-03-07 21:51 ` Thomas Munro <thomas.munro@gmail.com>
2025-03-07 23:02 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-03-17 11:36 ` Nazir Bilal Yavuz <byavuz81@gmail.com>
2025-03-17 15:08 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-03-19 04:09 ` Thomas Munro <thomas.munro@gmail.com>
2025-04-03 18:02 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-03 19:50 ` Daniel Gustafsson <daniel@yesql.se>
2025-03-19 04:17 ` Tom Lane <tgl@sss.pgh.pa.us>
2025-03-19 04:34 ` Thomas Munro <thomas.munro@gmail.com>
2025-03-19 13:46 ` Andres Freund <andres@anarazel.de>
2025-03-19 04:57 ` Tom Lane <tgl@sss.pgh.pa.us>
2025-03-19 13:31 ` Bruce Momjian <bruce@momjian.us>
2025-03-19 13:38 ` Daniel Gustafsson <daniel@yesql.se>
2025-03-19 20:11 ` Thomas Munro <thomas.munro@gmail.com>
2025-03-19 21:03 ` Tom Lane <tgl@sss.pgh.pa.us>
2025-03-19 22:02 ` Thomas Munro <thomas.munro@gmail.com>
2025-03-19 22:14 ` Thomas Munro <thomas.munro@gmail.com>
2025-03-19 22:19 ` Tom Lane <tgl@sss.pgh.pa.us>
2025-03-19 22:28 ` Thomas Munro <thomas.munro@gmail.com>
2025-03-19 23:11 ` Bruce Momjian <bruce@momjian.us>
2025-03-19 23:59 ` Bruce Momjian <bruce@momjian.us>
2025-03-20 20:33 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-03-20 20:50 ` Bruce Momjian <bruce@momjian.us>
2025-03-20 21:08 ` Tom Lane <tgl@sss.pgh.pa.us>
2025-03-20 23:26 ` Andres Freund <andres@anarazel.de>
2025-03-21 12:40 ` Andrew Dunstan <andrew@dunslane.net>
2025-03-21 18:02 ` Daniel Gustafsson <daniel@yesql.se>
2025-03-26 19:09 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-03-31 14:06 ` Christoph Berg <myon@debian.org>
2025-04-01 22:40 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-05 00:27 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-06 20:48 ` Christoph Berg <myon@debian.org>
2025-04-07 13:59 ` Peter Eisentraut <peter@eisentraut.org>
2025-04-07 14:43 ` Andres Freund <andres@anarazel.de>
2025-04-07 16:38 ` Peter Eisentraut <peter@eisentraut.org>
2025-04-07 16:52 ` Andres Freund <andres@anarazel.de>
2025-04-07 16:41 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-07 17:05 ` Andres Freund <andres@anarazel.de>
2025-04-07 17:32 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-07 18:47 ` Christoph Berg <myon@debian.org>
2025-04-07 21:49 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-07 21:58 ` Christoph Berg <myon@debian.org>
2025-04-07 22:26 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-08 02:10 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-08 10:34 ` Daniel Gustafsson <daniel@yesql.se>
2025-04-08 14:32 ` Bruce Momjian <bruce@momjian.us>
2025-04-08 15:04 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-08 15:13 ` Daniel Gustafsson <daniel@yesql.se>
2025-04-08 15:13 ` Bruce Momjian <bruce@momjian.us>
2025-04-08 15:20 ` Andres Freund <andres@anarazel.de>
2025-04-08 15:22 ` Bruce Momjian <bruce@momjian.us>
2025-04-08 16:01 ` Wolfgang Walther <walther@technowledgy.de>
2025-04-08 16:14 ` Bruce Momjian <bruce@momjian.us>
2025-04-08 16:17 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-08 16:33 ` Bruce Momjian <bruce@momjian.us>
2025-04-08 16:42 ` Daniel Gustafsson <daniel@yesql.se>
2025-04-08 16:48 ` Bruce Momjian <bruce@momjian.us>
2025-04-08 16:43 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-08 16:49 ` Bruce Momjian <bruce@momjian.us>
2025-04-08 17:00 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-08 17:03 ` Bruce Momjian <bruce@momjian.us>
2025-04-08 16:35 ` Wolfgang Walther <walther@technowledgy.de>
2025-04-08 16:14 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-08 16:32 ` Wolfgang Walther <walther@technowledgy.de>
2025-04-08 16:50 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-08 16:57 ` Wolfgang Walther <walther@technowledgy.de>
2025-04-08 17:02 ` Bruce Momjian <bruce@momjian.us>
2025-04-08 18:11 ` Andres Freund <andres@anarazel.de>
2025-04-08 18:25 ` Bruce Momjian <bruce@momjian.us>
2025-04-08 19:22 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-08 17:04 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-08 17:10 ` Wolfgang Walther <walther@technowledgy.de>
2025-04-08 17:13 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-08 17:15 ` Bruce Momjian <bruce@momjian.us>
2025-04-08 17:19 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-08 17:44 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-11 17:32 ` Peter Eisentraut <peter@eisentraut.org>
2025-04-08 17:36 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-08 19:07 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-08 21:32 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-09 02:57 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-09 08:14 ` Christoph Berg <myon@debian.org>
2025-04-09 12:50 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-09 17:44 ` Christoph Berg <myon@debian.org>
2025-04-09 23:08 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-09 23:42 ` Jelte Fennema-Nio <postgres@jeltef.nl>
2025-04-11 00:12 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-11 16:21 ` Wolfgang Walther <walther@technowledgy.de>
2025-04-14 16:12 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-14 17:09 ` Wolfgang Walther <walther@technowledgy.de>
2025-04-14 18:27 ` Wolfgang Walther <walther@technowledgy.de>
2025-04-14 19:14 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-14 19:46 ` Wolfgang Walther <walther@technowledgy.de>
2025-04-14 20:17 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-15 00:13 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-20 17:12 ` Ivan Kush <ivan.kush@tantorlabs.com>
2025-04-21 16:57 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-22 21:29 ` Ivan Kush <ivan.kush@tantorlabs.com>
2025-05-01 20:41 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-14 16:42 ` Andres Freund <andres@anarazel.de>
2025-04-15 12:31 ` Peter Eisentraut <peter@eisentraut.org>
2025-04-15 15:34 ` Christoph Berg <myon@debian.org>
2025-04-15 18:02 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-15 17:53 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-15 18:57 ` Christoph Berg <myon@debian.org>
2025-04-15 19:44 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-15 21:38 ` Jelte Fennema-Nio <postgres@jeltef.nl>
2025-04-18 00:47 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-18 17:01 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-19 12:03 ` Christoph Berg <myon@debian.org>
2025-04-21 18:49 ` Devrim Gündüz <devrim@gunduz.org>
2025-04-21 23:19 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-22 10:02 ` Daniel Gustafsson <daniel@yesql.se>
2025-04-22 23:41 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-23 15:39 ` Christoph Berg <myon@debian.org>
2025-04-23 16:07 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-23 16:38 ` Christoph Berg <myon@debian.org>
2025-04-23 17:46 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-23 20:12 ` Christoph Berg <myon@debian.org>
2025-04-24 17:02 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-25 09:02 ` Christoph Berg <myon@debian.org>
2025-04-25 20:31 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-29 00:10 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-30 12:55 ` Daniel Gustafsson <daniel@yesql.se>
2025-04-30 17:59 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-30 18:09 ` Daniel Gustafsson <daniel@yesql.se>
2025-05-01 17:38 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-05-01 19:24 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-05-01 20:31 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-05-02 15:11 ` Nathan Bossart <nathandbossart@gmail.com>
2025-05-02 15:46 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-05-02 15:59 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-05-02 17:05 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-05-02 17:31 ` Nathan Bossart <nathandbossart@gmail.com>
2025-05-02 17:42 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-06-30 16:58 ` Andres Freund <andres@anarazel.de>
2025-06-30 17:01 ` Daniel Gustafsson <daniel@yesql.se>
2025-06-30 18:33 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-06-30 22:52 ` Daniel Gustafsson <daniel@yesql.se>
2025-06-30 23:42 ` Andres Freund <andres@anarazel.de>
2025-07-18 17:26 ` Andres Freund <andres@anarazel.de>
2025-07-18 22:29 ` Daniel Gustafsson <daniel@yesql.se>
2025-07-18 23:31 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-08-05 18:54 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-08-08 16:23 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-05-02 17:35 ` Tom Lane <tgl@sss.pgh.pa.us>
2025-05-02 17:42 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-05-02 17:59 ` Nathan Bossart <nathandbossart@gmail.com>
2025-05-02 18:25 ` Tom Lane <tgl@sss.pgh.pa.us>
2025-05-02 18:46 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-05-02 18:52 ` Tom Lane <tgl@sss.pgh.pa.us>
2025-05-02 18:56 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-05-02 19:05 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-05-02 19:05 ` Tom Lane <tgl@sss.pgh.pa.us>
2025-05-04 12:58 ` Wolfgang Walther <walther@technowledgy.de>
2025-05-06 17:49 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-05-03 14:54 ` Christoph Berg <myon@debian.org>
2025-05-05 12:37 ` Devrim Gündüz <devrim@gunduz.org>
2025-04-15 19:21 ` Robert Haas <robertmhaas@gmail.com>
2025-04-15 19:45 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-04-09 08:38 ` Jelte Fennema-Nio <postgres@jeltef.nl>
2025-04-07 14:21 ` Andres Freund <andres@anarazel.de>
2025-06-12 19:58 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-06-20 10:08 ` Ivan Kush <ivan.kush@tantorlabs.com>
2025-06-23 15:32 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-07-02 12:45 ` Ivan Kush <ivan.kush@tantorlabs.com>
2025-07-02 14:46 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-07-09 17:36 ` Tom Lane <tgl@sss.pgh.pa.us>
2025-07-09 17:46 ` Andres Freund <andres@anarazel.de>
2025-07-09 17:59 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-07-09 18:13 ` Tom Lane <tgl@sss.pgh.pa.us>
2025-07-09 18:39 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-07-09 19:07 ` Tom Lane <tgl@sss.pgh.pa.us>
2025-07-09 19:28 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-07-09 19:42 ` Tom Lane <tgl@sss.pgh.pa.us>
2025-07-09 23:54 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-07-10 14:41 ` ivan.kush@tantorlabs.com
2025-07-11 17:33 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-02-28 23:38 ` Thomas Munro <thomas.munro@gmail.com>
2025-02-20 20:07 ` Tom Lane <tgl@sss.pgh.pa.us>
2025-02-20 20:21 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-02-20 20:37 ` Daniel Gustafsson <daniel@yesql.se>
2025-02-23 16:49 ` Tom Lane <tgl@sss.pgh.pa.us>
2025-02-23 17:08 ` Daniel Gustafsson <daniel@yesql.se>
2025-02-23 23:45 ` Daniel Gustafsson <daniel@yesql.se>
2025-02-24 09:00 ` Daniel Gustafsson <daniel@yesql.se>
2025-02-24 17:43 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-02-24 17:39 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-02-24 14:41 ` Andrew Dunstan <andrew@dunslane.net>
2025-02-24 14:48 ` Daniel Gustafsson <daniel@yesql.se>
2025-02-24 15:10 ` Andres Freund <andres@anarazel.de>
2025-01-09 20:40 ` Daniel Gustafsson <daniel@yesql.se>
2025-01-09 22:35 ` Jacob Champion <jacob.champion@enterprisedb.com>
2025-01-09 22:44 ` Daniel Gustafsson <daniel@yesql.se>
2024-11-19 11:05 ` Peter Eisentraut <peter@eisentraut.org>
2024-11-21 18:51 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-11-27 16:51 ` Daniel Gustafsson <daniel@yesql.se>
2024-10-31 11:05 ` Antonin Houska <ah@cybertec.at>
2024-10-31 16:24 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-11-04 05:00 ` jian he <jian.universality@gmail.com>
2024-11-04 09:11 ` Daniel Gustafsson <daniel@yesql.se>
2024-11-05 23:07 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-10-10 23:08 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-10-16 17:21 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-10-18 11:38 ` Daniel Gustafsson <daniel@yesql.se>
2024-10-23 15:46 ` Daniel Gustafsson <daniel@yesql.se>
2024-10-23 16:46 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-10-24 22:28 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-10-15 18:00 ` Alexander Lakhin <exclusion@gmail.com>
2024-10-15 18:10 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-10-17 06:28 ` Peter Eisentraut <peter@eisentraut.org>
2024-07-29 20:51 ` Daniel Gustafsson <daniel@yesql.se>
2024-07-29 23:15 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-02-28 21:50 ` Andrew Dunstan <andrew@dunslane.net>
2024-02-28 21:52 ` Daniel Gustafsson <daniel@yesql.se>
2024-02-29 14:49 ` Jacob Champion <jacob.champion@enterprisedb.com>
2024-02-29 21:08 ` Daniel Gustafsson <daniel@yesql.se>
2024-03-01 00:11 ` Jacob Champion <jacob.champion@enterprisedb.com>
2022-09-20 05:03 ` mahendrakar s <mahendrakarforpg@gmail.com>
2022-09-20 23:19 ` Jacob Champion <jchampion@timescale.com>
2022-09-21 16:03 ` Jacob Champion <jchampion@timescale.com>
2022-09-21 22:10 ` Andrey Chudnovskiy <Andrey.Chudnovskiy@microsoft.com>
2022-09-21 22:31 ` Jacob Champion <jchampion@timescale.com>
2022-09-22 04:55 ` Andrey Chudnovsky <achudnovskij@gmail.com>
2022-09-22 21:53 ` Jacob Champion <jchampion@timescale.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox