public inbox for [email protected]
help / color / mirror / Atom feedFrom: Greg Sabino Mullane <[email protected]>
To: [email protected] <[email protected]>
Subject: POC: Carefully exposing information without authentication
Date: Thu, 29 May 2025 10:32:57 -0400
Message-ID: <CAKAnmm+T-CEDLmRezWfH-7ZEsFfD_kU2KY1TgB288K+wprB_4Q@mail.gmail.com> (raw)
Proposal: Allow a carefully curated selection of information to be shown
without authentication.
A common task for an HA system or a load balancer is to quickly determine
which of your Postgres clusters is the primary, and which are the replicas.
The canonical way to do this is to log in to each server with a valid
username and password, and then run pg_is_in_recovery().
That's a lot of work to determine if a server is a replica or not, and it
struck me that this true/false information about a running cluster is not
super-sensitive information. In other words, would it really be wrong if
there was a way to advertise that information without having to log in? I
toyed with the idea of Postgres maintaining some sort of signal file, but
then I realized that we already have a process, listening on a known port,
that has that information available to us.
Thus, this POC (proof of concept), which lets the postmaster scan for
incoming requests and quickly handle them *before* doing forking and
authenticating. We scan for a simple trigger string, and immediately return
the information to the client.
It also occured to me that since we are going to need to provide a
non-Postgres-protocol special trigger string, and we might as well do
something like "GET /info" to allow existing programs to treat Postgres as
a mini http server. To that end, we end
up with something like this:
$ psql -p 5432 -tc 'select pg_is_in_recovery()'
t
$ curl http://localhost:5432/foobar
curl: (52) Empty reply from server
$ curl http://localhost:5432/info
RECOVERY: 1
To accomplish this, we have boolean GUC flags (defaulting to false) that
control which information is exposed. For the example above, the
expose_recovery boolean has been set to true. If any of these GUCs are
true, we take a slight detour right after we accept() but before we
actually fork. We use recv with the MSG_PEEK flag to take a quick scan of
the incoming data, and use strncmp to see if it matches. If it does, we
send() some information and move on without forking. If it doesn't, we
simply move on as if we were never there, and proceed to the next step of
forking a new backend to start the authentication process.
There are three pieces of information that can be exposed with this patch.
There may be more in the future, but these are all simple, global, and not
(IMO) security leaks. The GUCs are expose_recovery, expose_sysid, and
expose_version. Each one adds a line to the output in a KEY: VALUE format
for the GET /info endpoint. The raw value is output for the direct
endpoints:
* GET /replica
* GET /sysid
* GET /version
The expose_recovery GUC uses RecoveryInProgress() to return a 1 or a 0.
This is returned by the GET /replica endpoint.
The expose_sysid GUC returns GetSystemIdentifier(). Since this can be
thought of as a fingerprint for the server, it's a nice way for external
programs to determine if the cluster is the same one it saw last time, or
for leader/replica matching.
The purpose of the expose_version GUC is to output PG_VERSION_NUM. This
will allow external tools - particularly security scanners - to know the
exact version of Postgres that is running. While some may consider this
privileged information, tools are already taking advantage of our debug
loophole to make an educated guess about the version. See my old post about
this:
https://www.endpointdev.com/blog/2010/05/finding-postgresql-version-without/
Note that this guess by security scanners is sometimes wrong, or only able
to cover a range of versions. Thus, we should give them the correct answer,
rather than providing a dubious one via some trickery.
Here's some example output with all three enabled:
$ psql -c 'alter system set expose_recovery=on'
$ psql -c 'alter system set expose_version=on'
$ psql -c 'alter system set expose_sysid=on'
$ psql -c 'select pg_reload_conf()'
$ curl http://localhost:5432/version
180000
$ curl http://localhost:5432/info
RECOVERY: 1
SYSID: 7504513530771111839
VERSION: 180000
But wait! We can do more. For the recovery, we don't even need a string
that spells out "RECOVERY:", we only need to know if the server is in
recovery or not. In short, a boolean. Patroni does this in its API with a
call of HEAD /replica. It returns a different HTTP code if the server is a
replica (200) or not a replica (503). We can do the same thing! What's
more, we can do it in a way that will allow existing calls to simply point
to the postgres server instead of a Patroni process, and get the same
result back, but faster. Here's an example of what that looks like:
## Calling Patroni
$ curl -s -w "%{http_code}" -o /dev/null -I http://localhost:8008/replica
200
## Calling Postgres directly
$ curl -s -w "%{http_code}" -o /dev/null -I http://localhost:5432/replica
200
Here's a simple Python program showing how easy it is to grab this
information:
import socket
try:
with socket.create_connection(('localhost', 5432), timeout=1) as s:
s.sendall(b'GET /sysid')
print(s.recv(200).split(b'\r\n\r\n',1)[1].decode())
except Exception as e:
print(f"Error: {e}")
That's the basic idea: proof of concept patch is attached. Additional
things to do:
* handling socket quirks (esp. Win32)
* docs (once details are hashed out)
* moving things around (everything is in one function right now for reading
ease)
Cheers,
Greg
--
Crunchy Data - https://www.crunchydata.com
Enterprise Postgres Software Products & Tech Support
Attachments:
[application/octet-stream] 0001-Allow-specific-information-to-be-output-directly-by-Postgres.patch (11.3K, 3-0001-Allow-specific-information-to-be-output-directly-by-Postgres.patch)
download | inline diff:
From fd58612f74a308aed3f0d3caf79250c0bfce2068 Mon Sep 17 00:00:00 2001
From: Greg Sabino Mullane <[email protected]>
Date: Tue, 27 May 2025 07:30:07 -0400
Subject: [PATCH] Allow specific information to be output directly by Postgres.
---
src/backend/postmaster/postmaster.c | 196 +++++++++++++++++-
src/backend/utils/misc/guc_tables.c | 27 +++
src/backend/utils/misc/postgresql.conf.sample | 8 +
src/include/postmaster/postmaster.h | 4 +
src/test/postmaster/t/004_exposed.pl | 105 ++++++++++
5 files changed, 339 insertions(+), 1 deletion(-)
create mode 100644 src/test/postmaster/t/004_exposed.pl
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
index 490f7ce..5818f63 100644
--- a/src/backend/postmaster/postmaster.c
+++ b/src/backend/postmaster/postmaster.c
@@ -92,6 +92,7 @@
#include "access/xlog.h"
#include "access/xlog_internal.h"
#include "access/xlogrecovery.h"
+#include "access/xlogutils.h"
#include "common/file_perm.h"
#include "common/pg_prng.h"
#include "lib/ilist.h"
@@ -197,6 +198,9 @@ btmask_contains(BackendTypeMask mask, BackendType t)
}
+int total_expose = 0;
+
+
BackgroundWorker *MyBgworkerEntry = NULL;
/* The socket number we are listening for connections on */
@@ -247,6 +251,10 @@ char *bonjour_name;
bool restart_after_crash = true;
bool remove_temp_files_after_crash = true;
+bool expose_recovery = false;
+bool expose_sysid = false;
+bool expose_version = false;
+
/*
* When terminating child processes after fatal errors, like a crash of a
* child process, we normally send SIGQUIT -- and most other comments in this
@@ -453,6 +461,8 @@ static void StartSysLogger(void);
static void StartAutovacuumWorker(void);
static bool StartBackgroundWorker(RegisteredBgWorker *rw);
static void InitPostmasterDeathWatchHandle(void);
+static bool ExposeInformation(int fd);
+
#ifdef WIN32
#define WNOHANG 0 /* ignored, so any integer value will do */
@@ -1699,7 +1709,13 @@ ServerLoop(void)
ClientSocket s;
if (AcceptConnection(events[i].fd, &s) == STATUS_OK)
- BackendStartup(&s);
+ {
+ if ((expose_recovery || expose_sysid || expose_version)
+ && ExposeInformation(s.sock))
+ total_expose++;
+ else
+ BackendStartup(&s);
+ }
/* We no longer need the open socket in this process */
if (s.sock != PGINVALID_SOCKET)
@@ -4616,3 +4632,181 @@ InitPostmasterDeathWatchHandle(void)
GetLastError())));
#endif /* WIN32 */
}
+
+
+static
+bool
+ExposeInformation(int fd)
+{
+
+/*
+ * ExposeInformation
+ *
+ *
+ * Handle early socket probe before full backend startup.
+ * Responds to small set of predefined endpoints (e.g. GET /info)
+ *
+ * Requires at least one "expose_" GUC to be true.
+ *
+ * Returns true if endpoint is recognized.
+ * Caller is responsible for closing the socket.
+ */
+
+#define EXPOSE_MIN_QUERY 9 /* Shortest possible line: "Get /info" */
+#define EXPOSE_MAX_QUERY 16 /* Longest possible GET line */
+
+/* What information is being returned */
+ typedef enum
+ {
+ EXPOSE_NOTHING,
+ EXPOSE_HEAD_REPLICA,
+ EXPOSE_GET_ALL,
+ EXPOSE_GET_REPLICA,
+ EXPOSE_GET_SYSID,
+ EXPOSE_GET_VERSION,
+ } ReturnType;
+
+ typedef struct
+ {
+ const char *endpoint;
+ const bool *require;
+ ReturnType type;
+ } endpoint_action;
+
+ static endpoint_action endpoint_actions[] =
+ {
+ {
+ "HEAD /replica", &expose_recovery, EXPOSE_HEAD_REPLICA
+ },
+ {
+ "GET /replica", &expose_recovery, EXPOSE_GET_REPLICA
+ },
+ {
+ "GET /sysid", &expose_sysid, EXPOSE_GET_SYSID
+ },
+ {
+ "GET /version", &expose_version, EXPOSE_GET_VERSION
+ },
+ {
+ "GET /info", NULL, EXPOSE_GET_ALL
+ }
+ };
+
+ ssize_t n;
+ char buf[EXPOSE_MAX_QUERY + 1];
+ int type;
+
+ Assert(expose_recovery || expose_sysid || expose_version);
+
+ do
+ {
+ n = recv(fd, buf, EXPOSE_MAX_QUERY, MSG_PEEK);
+ } while (n < 0 && errno == EINTR);
+
+ /*
+ * Leave as soon as possible if not chance we are interested. We also
+ * simply return false for n == -1
+ */
+ if (n < EXPOSE_MIN_QUERY)
+ return false;
+
+ buf[n] = '\0';
+
+ type = EXPOSE_NOTHING;
+ for (int i = 0; i < lengthof(endpoint_actions); i++)
+ {
+ if (
+ strncmp(buf, endpoint_actions[i].endpoint, strlen(endpoint_actions[i].endpoint)) == 0
+ &&
+ (endpoint_actions[i].require == NULL
+ ||
+ *(endpoint_actions[i].require)
+ ))
+ {
+ type = endpoint_actions[i].type;
+ break;
+ }
+ }
+
+ if (type == EXPOSE_NOTHING)
+ return false;
+
+ {
+ static const char http_version[] = "HTTP/1.1";
+ static const char http_type[] = "Content-Type: text/plain";
+ static const char *http_conn = "Connection: close";
+ static const char http_len[] = "Content-Length";
+
+ StringInfoData msg;
+
+ if (type == EXPOSE_HEAD_REPLICA)
+ {
+ /*
+ * Caller only cares about the HTTP response code, so no content
+ * needed
+ */
+
+ initStringInfoExt(&msg, 64);
+
+ appendStringInfo(&msg,
+ "%s %s\r\n"
+ "%s\r\n"
+ "%s\r\n\r\n",
+ http_version,
+ (RecoveryInProgress() ? "200 OK" : "503 Service Unavailable"),
+ http_type,
+ http_conn
+ );
+ }
+ else
+ {
+ StringInfoData content;
+
+ initStringInfoExt(&content, 64);
+
+ if (expose_recovery && (type == EXPOSE_GET_ALL || type == EXPOSE_GET_REPLICA))
+ appendStringInfo(&content, "%s%d\r\n",
+ type == EXPOSE_GET_ALL ? "RECOVERY: " : "",
+ RecoveryInProgress() ? 1 : 0);
+ if (expose_sysid && (type == EXPOSE_GET_ALL || type == EXPOSE_GET_SYSID))
+ appendStringInfo(&content, "%s%lu\r\n",
+ type == EXPOSE_GET_ALL ? "SYSID: " : "",
+ GetSystemIdentifier());
+ if (expose_version && (type == EXPOSE_GET_ALL || type == EXPOSE_GET_VERSION))
+ appendStringInfo(&content, "%s%d\r\n",
+ type == EXPOSE_GET_ALL ? "VERSION: " : "",
+ PG_VERSION_NUM);
+
+ initStringInfoExt(&msg, 256);
+
+ appendStringInfo(&msg,
+ "%s 200 OK\r\n"
+ "%s\r\n"
+ "%s: %d\r\n"
+ "%s\r\n\r\n"
+ "%s",
+ http_version,
+ http_type,
+ http_len, content.len,
+ http_conn,
+ content.data
+ );
+
+ pfree(content.data);
+ }
+
+ do
+ {
+ n = send(fd, msg.data, msg.len, 0);
+ } while (n < 0 && errno == EINTR);
+
+ pfree(msg.data);
+
+ if (n < 0)
+ elog(DEBUG1, "could not send to client: %m");
+
+ return true;
+
+ }
+
+}
diff --git a/src/backend/utils/misc/guc_tables.c b/src/backend/utils/misc/guc_tables.c
index 2f8cbd8..38bea71 100644
--- a/src/backend/utils/misc/guc_tables.c
+++ b/src/backend/utils/misc/guc_tables.c
@@ -1701,6 +1701,33 @@ struct config_bool ConfigureNamesBool[] =
true,
NULL, NULL, NULL
},
+ {
+ {"expose_recovery", PGC_SIGHUP, CLIENT_CONN_STATEMENT,
+ gettext_noop("Exposes if the server is in recovery without a login."),
+ NULL
+ },
+ &expose_recovery,
+ false,
+ NULL, NULL, NULL
+ },
+ {
+ {"expose_sysid", PGC_SIGHUP, CLIENT_CONN_STATEMENT,
+ gettext_noop("Exposes the system identifier without a login."),
+ NULL
+ },
+ &expose_sysid,
+ false,
+ NULL, NULL, NULL
+ },
+ {
+ {"expose_version", PGC_SIGHUP, CLIENT_CONN_STATEMENT,
+ gettext_noop("Exposes the version without a login."),
+ NULL
+ },
+ &expose_version,
+ false,
+ NULL, NULL, NULL
+ },
{
{"array_nulls", PGC_USERSET, COMPAT_OPTIONS_PREVIOUS,
gettext_noop("Enables input of NULL elements in arrays."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 87ce76b..a16d372 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -91,6 +91,14 @@
# disconnection while running queries;
# 0 for never
+
+# - Expose information -
+
+#expose_recovery = off
+#expose_sysid = off
+#expose_version = off
+
+
# - Authentication -
#authentication_timeout = 1min # 1s-600s
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
index 92497cd..15662ce 100644
--- a/src/include/postmaster/postmaster.h
+++ b/src/include/postmaster/postmaster.h
@@ -70,6 +70,10 @@ extern PGDLLIMPORT bool restart_after_crash;
extern PGDLLIMPORT bool remove_temp_files_after_crash;
extern PGDLLIMPORT bool send_abort_for_crash;
extern PGDLLIMPORT bool send_abort_for_kill;
+extern PGDLLIMPORT bool expose_recovery;
+extern PGDLLIMPORT bool expose_sysid;
+extern PGDLLIMPORT bool expose_version;
+
#ifdef WIN32
extern PGDLLIMPORT HANDLE PostmasterHandle;
diff --git a/src/test/postmaster/t/004_exposed.pl b/src/test/postmaster/t/004_exposed.pl
new file mode 100644
index 0000000..97cc6b1
--- /dev/null
+++ b/src/test/postmaster/t/004_exposed.pl
@@ -0,0 +1,105 @@
+
+# Copyright (c) 2025, PostgreSQL Global Development Group
+
+# Test pre-fork HTTP endpoint information
+
+use strict;
+use warnings FATAL => 'all';
+use PostgreSQL::Test::Cluster;
+use Test::More;
+
+my $node = PostgreSQL::Test::Cluster->new('primary');
+$node->init();
+$node->start;
+
+SKIP:
+{
+ skip "this test requires working raw_connect()"
+ unless $node->raw_connect_works();
+
+ my ($endpoint, $sock, $reply);
+
+ $endpoint = 'GET /info';
+ $sock = $node->raw_connect();
+ $sock->send($endpoint);
+ $sock->recv($reply, 200);
+ is($reply, '', "nothing is returned by $endpoint");
+ $sock->close();
+
+ $endpoint = 'HEAD /replica';
+ $sock = $node->raw_connect();
+ $sock->send($endpoint);
+ $sock->recv($reply, 200);
+ is($reply, '', "nothing is returned by $endpoint");
+
+ $endpoint = 'GET /replica';
+ $sock = $node->raw_connect();
+ $sock->send($endpoint);
+ $sock->recv($reply, 200);
+ is($reply, '', "nothing is returned by $endpoint");
+ $sock->close();
+
+ $node->append_conf('postgresql.conf', "expose_recovery = on");
+ $node->reload();
+
+ $sock = $node->raw_connect();
+ $sock->send($endpoint);
+ $sock->recv($reply, 200);
+ like($reply, qr[\r\n0\b], "recovery information is returned by $endpoint");
+ $sock->close();
+
+ $endpoint = 'GET /sysid';
+ $sock = $node->raw_connect();
+ $sock->send($endpoint);
+ $sock->recv($reply, 200);
+ is($reply, '', "nothing is returned by $endpoint");
+ $sock->close();
+
+ $node->append_conf('postgresql.conf', "expose_sysid = on");
+ $node->reload();
+
+ $sock = $node->raw_connect();
+ $sock->send($endpoint);
+ $sock->recv($reply, 200);
+ like($reply, qr[\r\n\d{10}], "system identifier is returned by $endpoint");
+ $sock->close();
+
+ $endpoint = 'GET /version';
+ $sock = $node->raw_connect();
+ $sock->send($endpoint);
+ $sock->recv($reply, 200);
+ is($reply, '', "nothing is returned by $endpoint");
+ $sock->close();
+
+ $node->append_conf('postgresql.conf', "expose_version = on");
+ $node->reload();
+
+ $sock = $node->raw_connect();
+ $sock->send($endpoint);
+ $sock->recv($reply, 200);
+ like($reply, qr[\r\n\d{6}\b], "version number is returned by $endpoint");
+ $sock->close();
+
+
+ $endpoint = 'GET /info';
+ $sock = $node->raw_connect();
+ $sock->send($endpoint);
+ $sock->recv($reply, 200);
+ like($reply, qr[RECOVERY: 0\b], "recovery is returned by $endpoint");
+ like($reply, qr[SYSID: \d{10}], "sysid is returned by $endpoint");
+ like($reply, qr[VERSION: \d{6}\b], "version number is returned by $endpoint");
+ $sock->close();
+
+ $endpoint = 'HEAD /replica';
+ $sock = $node->raw_connect();
+ $sock->send($endpoint);
+ $sock->recv($reply, 200);
+ like($reply, qr[503 Service Unavailable], "503 returned by $endpoint");
+ $sock->close();
+
+ ## Not working yet:
+ ## $node->backup('testbackup1');
+
+}
+
+done_testing();
--
2.30.2
view thread (8+ messages) latest in thread
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected]
Subject: Re: POC: Carefully exposing information without authentication
In-Reply-To: <CAKAnmm+T-CEDLmRezWfH-7ZEsFfD_kU2KY1TgB288K+wprB_4Q@mail.gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox