Subject: Rotate SSL certificates on reload (SIGHUP) without restart

From: Bob Ross <[email-masked]>
Date: Thu, 23 Oct 2025 07:02:59 +0200

Hello,

Please consider adding support for rotating SSL certificates on reloading
pgpool2 (i.e., sending SIGHUP to the pgpool parent), so that certificate
rotations do not require a full service restart. PostgreSQL can pick up new
certificates on reload/SIGHUP; pgpool currently requires a restart, which
causes connection disruptions.

*Current behavior:*

   - Replace certificate/key files used by pgpool (e.g., server.crt,
   server.key, related CA chain).
   - Run systemctl reload pgpool2 (send SIGHUP to the pgpool parent).
   - Observations: Existing and new client connections continue to present
   the old certificate. Only systemctl restart pgpool2 applies the new certs
   (causing connection interruptions).

*Expected behavior:*

   - After systemctl reload pgpool2 / SIGHUP, pgpool should re-read
   SSL-related configuration (server cert, private key, chain/CA, CRL if
   configured) and use them for new client connections, without requiring a
   full restart.
   - Existing connections can continue with the old context; only new
   handshakes should use the updated materials.
   - If reload fails, log a clear error and keep using the previous context
   to avoid breaking clients.
   - Consider parity with PostgreSQL’s SIGHUP behavior for certificate
   reloads where feasible.

Regards,
Bob Ross

---

From: Tatsuo Ishii <[email-masked]>
Date: Fri, 24 Oct 2025 13:44:47 +0900 (JST)

>    (causing connection interruptions).

Yes, that's the current behavior as described in the docs.

>    configured) and use them for new client connections, without requiring a

Doable but needs major surgery to the SSL subsystem
(src/utils/pool_ssl.c) as it assumes that SSL configurations are never
changed until restarting.

>    handshakes should use the updated materials.

Probably doable.

>    - Consider parity with PostgreSQL’s SIGHUP behavior for certificate

Not sure if it's doable. Needs more research on current code.

BTW, PostgreSQL behaves interestingly.

# "server.key" is the correct ssl_key_file.

test=# show ssl_key_file;
 ssl_key_file 
--------------
 server.key
(1 row)

test=# \q
t-ishii$ psql -p 11002 -h localhost test
psql (18.0)
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, compression: off, ALPN: postgresql)
Type "help" for help.

# Change ssl_key_file to "server.key1" which does not exists.
# and reload

t-ishii$ pg_ctl -D data0 reload
server signaled

# keep on using SSL connection

t-ishii$ psql -p 11002 -h localhost test
psql (18.0)
SSL connection (protocol: TLSv1.3, cipher: TLS_AES_256_GCM_SHA384, compression: off, ALPN: postgresql)
Type "help" for help.

# It seems PostgreSQL keep on using th previous ssl_key_file value,
# but it shows the new ssl_key_file value.

test=# show ssl_key_file;
 ssl_key_file 
--------------
 server.key1
(1 row)

Best regards,
--
Tatsuo Ishii
SRA OSS K.K.
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp

---

From: Bob Ross <[email-masked]>
Date: Mon, 16 Mar 2026 10:20:38 +0100

Hi Tatsuo,

Have there been any further considerations regarding changes to the pgPool
codebase to support SSL certificate rotation on reload?

As DigiCert has announced last year (
https://www.digicert.com/blog/tls-certificate-lifetimes-will-officially-reduce-to-47-days),
TLS/SSL certificate lifetimes will be reduced progressively in the coming
years, with the industry moving toward much shorter validity periods. This
makes the current requirement to fully restart the service for certificate
renewal increasingly impractical.

Please let us know whether this enhancement is being considered, or if
there are any plans or timelines for addressing it.

Regards,
Bob

---

From: Tatsuo Ishii <[email-masked]>
Date: Thu, 19 Mar 2026 19:22:25 +0900 (JST)

Hi Bob,

> there are any plans or timelines for addressing it.

I just have too many things to do for now (fixing bugs and evaluating
proposed patches), and I cannot estimate timelines for this. Plus, I
am not super familiar with this are (SSL). If you could provide
patches for this, it would greatly help me.

Best regards,

---

From: Bob Ross <[email-masked]>
Date: Wed, 25 Mar 2026 12:04:03 +0100

Hi Tatsuo,

Please find the attached patch that implements this feature. This patch
allows Pgpool-II to pick up rotated TLS certificates upon receiving a
SIGHUP without a restart, aligning its behavior with PG 12+.
As this is my first time contributing to the Pgpool-II project, please bear
with me if I missed any specific formatting or submission conventions. I am
happy to make any necessary adjustments to the code.

Thanks,
Bob

---

From: Tatsuo Ishii <[email-masked]>
Date: Thu, 26 Mar 2026 17:07:16 +0900 (JST)

Hi Bob,

> happy to make any necessary adjustments to the code.

Thank you for the patch! I will look into the patch.

Regards,

---

From: Tatsuo Ishii <[email-masked]>
Date: Thu, 26 Mar 2026 19:19:02 +0900 (JST)

Hi Bob,

> Thank you for the patch! I will look into the patch.

I skimmed through the patch. You changed main.c and child.c to call
SSL_ServerSide_init() so that they reload new SSL parameters. However,
in Pgpool-II there are two more places which could use SSL connection
to PostgreSQL server: health check and streaming replication check. I
suspect they need to call SSL_ServerSide_init() while reloading the
config file. What do you think?

Regards,

---

From: Bob Ross <[email-masked]>
Date: Fri, 27 Mar 2026 19:16:12 +0100

Hi Tatsuo,

I believe no additional changes are needed for healthcheck and streaming
replication check.

My patch focuses on the frontend-accept path, where SSL_ServerSide_init()
maintains a process-wide cached SSL_frontend_context reused across incoming
TLS handshakes, and therefore must be explicitly refreshed on reload.

In contrast, the health check (src/main/health_check.c) and the SR check
worker (src/streaming_replication/pool_worker_child.c) establish outgoing
connections to PostgreSQL backends via pool_ssl_negotiate_clientserver().
This invokes init_ssl_ctx() for each new connection, creating a fresh
SSL_CTX and reading SSL-related settings (ssl_cert, ssl_key, ssl_ca_cert,
etc.) directly from pool_config at connection time. There is no long-lived
SSL context to refresh in these processes.

Given that the SSL configuration parameters are now marked as CFGCXT_RELOAD
in src/config/pool_config_variables.c, a SIGHUP causes both processes to
reload pool_config via their respective reload_config() functions. As a
result, subsequent outgoing backend connections will naturally pick up the
updated certificate settings.

Please let me know what you think.

Best regards,
Bob

---

From: Tatsuo Ishii <[email-masked]>
Date: Tue, 31 Mar 2026 18:48:32 +0900 (JST)

> updated certificate settings.

Thank you for the explanation. Yes, health check and SR check worker
establish connection to backend every time they need. I should have
remembered that.

I am going to write a regression test for this feature unless you are
willing to work on it.

Regards,

---

From: Bob Ross <[email-masked]>
Date: Tue, 31 Mar 2026 11:53:55 +0200

Hi Tatsuo,

Thanks for double-checking! Please feel free to go ahead and write the
regression tests if you're up for it. I'd really appreciate the help.

Regards,
Bob

---

From: Tatsuo Ishii <[email-masked]>
Date: Wed, 01 Apr 2026 18:05:42 +0900 (JST)

Hi Bob,

> regression tests if you're up for it. I'd really appreciate the help.

I have written the first version of the regression test. This test
performs:

1. Set bad value (fixed string "bad_value") to a config param and
   restart pgpool so that SSL connection does not establish between
   client and pgpool.

2. Set good value to the config and reload pgpool so that SSL
   connection establishes.

The test is run against:
ssl_cert
ssl_ciphers
ssl_crl_file
ssl_ecdh_curve
ssl_key

It does not test ssl_ca_cert and ssl_ca_cert_dir because the test is
based on 023.ssl_connection which does not check cert auth. Should we
test cert auth as well?

Also this does not test followings:

- ssl_dh_params_file
If bad value is set to the parameter, it falls back to a builtin
value. So it is not possible to set a bad value to the parameter.
Do you have an idea to test this?

- ssl_passphrase_command
Our cert does not require pass passphrase.

- ssl_prefer_server_ciphers
This only affects server side (backend) ciphers. The test only tests
SSL connection between client and pgpool.

Attached is the v1 patch including your patch (I have remove "-----"
from your commit message. Otherwise the commit message cuts in the
middle)  and the test script.
What do you think?

Regards,

---

From: Bob Ross <[email-masked]>
Date: Thu, 2 Apr 2026 21:57:11 +0200

Hi Tatsuo,

Thanks for putting together the regression tests.

Thoughts on your questions:
- CA Certificates - Yes, adding a cert auth test is highly recommended. We
could test this by generating two different dummy CA certificates. Start
pgpool trusting CA #1, swap the config to CA #2, reload and verify if
client connection correctly gets rejected.
- DH parameters - perhaps we can test this by providing a non-existent file
path and then use grep to check pgpool.log for specific warning message
(per pool_ssl.c it’s “DH: could not load DH parameters”) when pgpool tries
to load the file.

Regards,
Bob

---

From: Bob Ross <[email-masked]>
Date: Mon, 13 Apr 2026 09:28:40 +0200

Hi Tatsuo,

Please let me know if you need any assistance with updating your test
cases. I am be happy to help.

Thanks,
Bob

On Thu, Apr 2, 2026 at 9:57 PM Bob Ross <[email-masked]> wrote:

> - DH parameters - perhaps we can test this by providing a non-existent
> file path and then use grep to check pgpool.log for specific warning
> message (per pool_ssl.c it’s “DH: could not load DH parameters”) when
> pgpool tries to load the file.
>
> Regards,
> Bob
>
>

---

From: Tatsuo Ishii <[email-masked]>
Date: Tue, 14 Apr 2026 17:01:51 +0900 (JST)

Hi Bob,

> cases. I am be happy to help.

Sorry for late. I was busy with personal affairs and some other
projects.

>> client connection correctly gets rejected.

If you could extend the test file I posted so that it performs a cert
auth test, that would be helpful.

>> pgpool tries to load the file.

I think it will not work.

===================================================================
static bool
initialize_dh(SSL_CTX *context)
{
	DH		   *dh = NULL;

	SSL_CTX_set_options(context, SSL_OP_SINGLE_DH_USE);

	if (pool_config->ssl_dh_params_file[0])
		dh = load_dh_file(pool_config->ssl_dh_params_file);
	if (!dh)
		dh = load_dh_buffer(FILE_DH2048, sizeof(FILE_DH2048));
	if (!dh)
	{
		ereport(WARNING,
				(errmsg("DH: could not load DH parameters")));
		return false;
	}
:
:
===================================================================

The ereport message is printed when the built-in DH parameter file is
broken. But as long as the source file is fine, it would never happen.

Maybe we should fix the code above so that it emits ereport when it
fails to load the DH parameter file specified by ssl_dh_params_file?

Regards,

---

From: Bob Ross <[email-masked]>
Date: Wed, 15 Apr 2026 08:36:06 +0200

Hi Tatsuo,

Please see attached v2. It adds regression coverage for SSL cert reload
with client certificate authentication; there are no functional code
changes.

Best regards,
Bob

---

From: Tatsuo Ishii <[email-masked]>
Date: Thu, 16 Apr 2026 19:31:38 +0900 (JST)

Hi Bob,

Thank you for the patch!

Unfortunately after applying the patch, the test failed. From
src/test/regression/log/042.ssl_reload:

===== ssl_ca_cert swap (client cert auth reload) =====
waiting for server to start....1462289 2026-04-16 19:25:49.490 JST LOG:  redirecting log output to logging collector process
1462289 2026-04-16 19:25:49.490 JST HINT:  Future log output will appear in directory "log".
 done
server started
CA cert swap: CA1-signed client cert rejected before reload – unexpected.

From pgpool.log:

2026-04-16 19:26:10.150: child pid 1462331: DETAIL:  Protocol Major: 1234 Minor: 5679 database:  user: 
2026-04-16 19:26:10.150: child pid 1462331: DEBUG:  selecting backend connection
2026-04-16 19:26:10.150: child pid 1462331: DETAIL:  SSLRequest from client
2026-04-16 19:26:10.150: child pid 1462331: DEBUG:  pool_write: to frontend: kind:S po:0
2026-04-16 19:26:10.150: child pid 1462331: DEBUG:  pool_flush_it: flush size: 1
2026-04-16 19:26:10.167: child pid 1462331: LOG:  pool_ssl: "SSL_accept": "tlsv1 alert unknown ca"
2026-04-16 19:26:10.167: child pid 1462331: DEBUG:  unable to read data from frontend
2026-04-16 19:26:10.167: child pid 1462331: DETAIL:  socket read failed with error "Connection reset by peer"

Please let me know if you need more info.

Regards,

---

From: Bob Ross <[email-masked]>
Date: Thu, 16 Apr 2026 14:25:17 +0200

Hi Tatsuo,

Thank you for the report and the log, that was very helpful.

The bug was in the test, not in the core patch. Test 4 in 042.ssl_reload
set PGSSLROOTCERT="$CADIR/ca1.crt" on both psql invocations. The intent was
to use ca1.crt for server-side client-certificate verification, but
PGSSLROOTCERT actually controls client-side server certificate
verification, and ca1.crt is a freshly generated CA that never signed
pgpool's server.crt. Because libpq upgrades sslmode=require to verify-ca
behavior when a root CA file is provided, psql immediately sent unknown_ca
back to pgpool and aborted the handshake before server-side client-cert
verification ever ran.

Fix: both lines now use PGSSLROOTCERT="etc/$SSL_CRT". Since server.crt is
self-signed it acts as its own root, so psql accepts pgpool's server cert,
the handshake completes, and the test exercises what it was always meant to
test: whether pgpool enforces the new ssl_ca_cert after reload.

No changes to the core patch.

Please find v3 attached. Please let me know if it fixes the test issue.

Best regards,
Bob

On Thu, Apr 16, 2026 at 12:31 PM Tatsuo Ishii <[email-masked]> wrote:

> waiting for server to start....1462289 2026-04-16 19:25:49.490 JST LOG:
> redirecting log output to logging collector process
> >> >> - CA Certificates - Yes, adding a cert auth test is highly
> recommended.
> >> We

---

From: Tatsuo Ishii <[email-masked]>
Date: Fri, 17 Apr 2026 07:37:33 +0900 (JST)

Hi Bob,

Thank you for the new patch. This time it failed with different pgpool.log. 
src/test/regression/log/042.ssl_reload was same.

2026-04-17 07:30:46.893: child pid 1475041: DEBUG:  attempting to negotiate a secure connection
2026-04-17 07:30:46.893: child pid 1475041: DETAIL:  sending client->server SSL request
2026-04-17 07:30:46.893: child pid 1475041: DEBUG:  pool_flush_it: flush size: 8
2026-04-17 07:30:46.894: child pid 1475041: DEBUG:  pool_read: read 1 bytes from backend 0
2026-04-17 07:30:46.894: child pid 1475041: DEBUG:  attempting to negotiate a secure connection
2026-04-17 07:30:46.894: child pid 1475041: DETAIL:  client->server SSL response: S
2026-04-17 07:30:46.899: child pid 1475041: LOG:  pool_ssl: "SSL_connect": "certificate verify failed"
2026-04-17 07:30:46.899: child pid 1475041: DEBUG:  pool_flush_it: flush size: 58
2026-04-17 07:30:46.899: child pid 1475041: DEBUG:  pool_read: read 360 bytes from backend 0
2026-04-17 07:30:46.900: child pid 1475041: ERROR:  backend authentication failed
2026-04-17 07:30:46.900: child pid 1475041: DETAIL:  backend response with kind '' when expecting 'R'
2026-04-17 07:30:46.900: child pid 1475041: DEBUG:  pool_write: to frontend: kind:E po:0
2026-04-17 07:30:46.900: child pid 1475041: DEBUG:  pool_write: to frontend: length:4 po:1
2026-04-17 07:30:46.900: child pid 1475041: DEBUG:  pool_write: to frontend: length:115 po:5
2026-04-17 07:30:46.900: child pid 1475041: DEBUG:  pool_flush_it: flush size: 120

---

From: Bob Ross <[email-masked]>
Date: Fri, 17 Apr 2026 11:50:48 +0200

Hi Tatsuo,

I've fixed the test failure. The issue was that the original test used the
static self-signed server.crt as the CA bundle. When ssl_ca_cert is set,
pgpool verifies the backend certificate against it, and the self-signed
cert failed that check. The fix generates a dedicated server CA in the test
and issues a backend cert signed by it, so pgpool can always verify the
backend while ca1/ca2 are independently swapped to test client-cert trust
rotation.

The test now passes:

*CA cert swap: CA1-signed client cert accepted before reload – ok.CA cert
swap: CA1-signed client cert rejected after reload to CA2 – ok.testing
042.ssl_reload...ok.out of 1 ok:1 failed:0 timeout:0*

pgpool.log showing client cert accepted before reload:

*2026-04-17 09:23:31.449: child pid 81378: DEBUG:  got the SSL
certificate2026-04-17 09:23:31.449: child pid 81378: DETAIL:  Protocol
Major: 3 Minor: 0 database: test user: ssltest2026-04-17 09:23:31.450:
child pid 81378: DETAIL:  client->server SSL response: S2026-04-17
09:23:31.455: child pid 81378: DETAIL:  auth kind:0*

SSL certificate reload completed:

*2026-04-17 09:23:31.471: main pid 81347: LOG:  reload SSL certificates.*

pgpool.log showing client cert rejected after reload:

*2026-04-17 09:23:32.485: psql pid 81385: DETAIL:  SSLRequest from
client2026-04-17 09:23:32.493: psql pid 81385: LOG:  pool_ssl:
"SSL_accept": "certificate verify failed"2026-04-17 09:23:32.493: psql pid
81385: ERROR:  failed while reading startup packet*

No changes to the core patch.
The updated v4 is attached.

Regards,
Bob

On Fri, Apr 17, 2026 at 12:37 AM Tatsuo Ishii <[email-masked]> wrote:

> 2026-04-17 07:30:46.900: child pid 1475041: DETAIL:  backend response with
> kind ' ' when expecting 'R'
> >> 2026-04-16 19:26:10.150: child pid 1462331: DETAIL:  Protocol Major:
> 1234
> >> Minor: 5679 database:  user:
> >> 2026-04-16 19:26:10.167: child pid 1462331: LOG:  pool_ssl:
> "SSL_accept":
> >> "tlsv1 alert unknown ca"

---

From: Tatsuo Ishii <[email-masked]>
Date: Fri, 17 Apr 2026 20:06:56 +0900 (JST)

Hi Bob,

> backend while ca1/ca2 are independently swapped to test client-cert trust

That makes sense.

> The updated v4 is attached.

Now the test succeeded here.  Many thanks. I will push the patch
tomorrow.

Regards,

---

From: Tatsuo Ishii <[email-masked]>
Date: Sat, 18 Apr 2026 19:08:14 +0900 (JST)

Hi Bob,

> Now the test succeeded here.  Many thanks. I will push the patch

I have pushed this to master branch along with documentation changes.
https://git.postgresql.org/gitweb/?p=pgpool2.git;a=commit;h=931a37e7f63fe514e989d427e1b0c2b7f4344c0d

Thank you!