Subject: Patroni 4.1.0 causes issues because of systemd notify

From: "Hauke Bruno Wollentin" <[email-masked]>
Date: Wed, 22 Oct 2025 16:16:42 +0000

Hi all,

while spinning up new Patroni cluster with version 4.1.0 (which has been 
added to the Debian repo 2 days ago), my clusters won’t bootstrap 
properly anymore.

At first, logs looks clear but anyways, all systemd operations like 
start/stop/restart will run into a timeout, causing the service to stop 
which then will break the cluster at all.

It seems like the new systemd notify feature is causing this, since 
using Patroni 4.0.7 with `Type=simple` in the systemd unit file works 
pretty fine.

I’m on Debian 12 and psql 17.

This is the patroni log after bootstrapping. Everything works fine until 
the 30s timeout of the systemd unit kicks in:

Oct 22 17:43:09 hetzner-lab01 patroni[13066]: Success. You can now start 
the database server using:
Oct 22 17:43:09 hetzner-lab01 patroni[13066]:     
/usr/lib/postgresql/17/bin/pg_ctl -D /var/lib/postgresql/patroni/data -l 
logfile start
Oct 22 17:43:09 hetzner-lab01 patroni[13059]: 2025-10-22 17:43:09,768 
INFO: postmaster pid=13083
Oct 22 17:43:09 hetzner-lab01 patroni[13084]: localhost:5432 - no 
response
Oct 22 17:43:09 hetzner-lab01 patroni[13083]: 2025-10-22 17:43:09.780 
CEST [13083] LOG:  starting PostgreSQL 17.6 (Debian 17.6-2.pgdg12+1) on 
x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14+deb12u1) 12.2.0, 
64-bit
Oct 22 17:43:09 hetzner-lab01 patroni[13083]: 2025-10-22 17:43:09.780 
CEST [13083] LOG:  listening on IPv4 address "0.0.0.0", port 5432
Oct 22 17:43:09 hetzner-lab01 patroni[13083]: 2025-10-22 17:43:09.780 
CEST [13083] LOG:  listening on IPv6 address "::", port 5432
Oct 22 17:43:09 hetzner-lab01 patroni[13083]: 2025-10-22 17:43:09.782 
CEST [13083] LOG:  listening on Unix socket 
"/var/run/postgresql/.s.PGSQL.5432”
Oct 22 17:43:09 hetzner-lab01 patroni[13087]: 2025-10-22 17:43:09.789 
CEST [13087] LOG:  database system was shut down at 2025-10-22 17:43:09 
CEST
Oct 22 17:43:09 hetzner-lab01 patroni[13083]: 2025-10-22 17:43:09.798 
CEST [13083] LOG:  database system is ready to accept connections
Oct 22 17:43:09 hetzner-lab01 systemd[1]: patroni.service: Got 
notification message from PID 13083, but reception only permitted for 
main PID 13059
Oct 22 17:43:10 hetzner-lab01 patroni[13091]: localhost:5432 - accepting 
connections
Oct 22 17:43:10 hetzner-lab01 patroni[13093]: localhost:5432 - accepting 
connections
Oct 22 17:43:10 hetzner-lab01 patroni[13059]: 2025-10-22 17:43:10,814 
INFO: establishing a new patroni heartbeat connection to postgres
Oct 22 17:43:10 hetzner-lab01 patroni[13059]: 2025-10-22 17:43:10,918 
INFO: running post_bootstrap
Oct 22 17:43:11 hetzner-lab01 patroni[13059]: 2025-10-22 17:43:11,225 
INFO: initialized a new cluster
Oct 22 17:43:11 hetzner-lab01 patroni[13059]: 2025-10-22 17:43:11,373 
INFO: no action. I am (hetzner-lab01), the leader with the lock
Oct 22 17:43:21 hetzner-lab01 patroni[13059]: 2025-10-22 17:43:21,276 
INFO: no action. I am (hetzner-lab01), the leader with the lock
Oct 22 17:43:31 hetzner-lab01 patroni[13059]: 2025-10-22 17:43:31,368 
INFO: no action. I am (hetzner-lab01), the leader with the lock
Oct 22 17:43:37 hetzner-lab01 systemd[1]: patroni.service: start 
operation timed out. Terminating.
Oct 22 17:43:37 hetzner-lab01 patroni[13083]: 2025-10-22 17:43:37.987 
CEST [13083] LOG:  received fast shutdown request
Oct 22 17:43:37 hetzner-lab01 patroni[13083]: 2025-10-22 17:43:37.989 
CEST [13083] LOG:  aborting any active transactions
Oct 22 17:43:37 hetzner-lab01 patroni[13095]: 2025-10-22 17:43:37.990 
CEST [13095] FATAL:  terminating connection due to administrator command
Oct 22 17:43:37 hetzner-lab01 systemd[1]: patroni.service: Got 
notification message from PID 13083, but reception only permitted for 
main PID 13059
Oct 22 17:43:37 hetzner-lab01 patroni[13083]: 2025-10-22 17:43:37.992 
CEST [13083] LOG:  background worker "logical replication launcher" (PID 
13090) exited with exit code 1
Oct 22 17:43:37 hetzner-lab01 patroni[13085]: 2025-10-22 17:43:37.994 
CEST [13085] LOG:  shutting down
Oct 22 17:43:37 hetzner-lab01 patroni[13085]: 2025-10-22 17:43:37.996 
CEST [13085] LOG:  checkpoint starting: shutdown immediate
Oct 22 17:43:38 hetzner-lab01 patroni[13085]: 2025-10-22 17:43:38.014 
CEST [13085] LOG:  checkpoint complete: wrote 17 buffers (0.1%); 0 WAL 
file(s) added, 0 removed, 0 recycled; write=0.005 s, sync=0.004 s, 
total=0.020 s; sync files=13, longest=0.002 s, average=0.001 s; 
distance=36 kB, estimate=36 kB; lsn=0/152D6F0, redo lsn=0/152D6F0
Oct 22 17:43:38 hetzner-lab01 patroni[13083]: 2025-10-22 17:43:38.020 
CEST [13083] LOG:  database system is shut down
Oct 22 17:43:39 hetzner-lab01 systemd[1]: patroni.service: Failed with 
result ‘timeout’.
Oct 22 17:43:39 hetzner-lab01 systemd[1]: Failed to start 
patroni.service - Runners to orchestrate a high-availability PostgreSQL.

Any recommendations how to fix/workaround this?

cheers,
Hauke

—
🥷 Hauke Bruno Wollentin

---

From: Richard Huxton <[email-masked]>
Date: Thu, 06 Nov 2025 20:34:06 +0000

On 2025-10-22 17:16, Hauke Bruno Wollentin wrote:
> Hi all,
> 
> while spinning up new Patroni cluster with version 4.1.0 (which has 
> been added to the Debian repo 2 days ago), my clusters won’t bootstrap 
> properly anymore.
> 

I think you may well be right - it seems that patroni is sending the 
notify from a forked child process and that isn't what systemd is 
expecting

> Any recommendations how to fix/workaround this?

https://www.freedesktop.org/software/systemd/man/latest/systemd.service.html#NotifyAccess=

I don't have a test patroni installation handy right now, but I think if 
you set

     NotifyAccess=all

in the service config (which you should be able to do by creating a file 
with just that line called `/etc/systemd/system/patroni/override.conf`)

It defaults to "main" IIRC which is the most restricted, "exec" might 
work, but "all" (the least restricted) seems the most likely to work 
first time.

Hopefully that will do the trick. If so, maybe try "exec" then and let 
the list know which works and the package can be updated.

-- 
   Richard Huxton

---

From: Aaron Pavely <[email-masked]>
Date: Thu, 6 Nov 2025 22:06:31 -0600

On Sun, Nov 2, 2025 at 2:27 PM Hauke Bruno Wollentin <[email-masked]> wrote:

> added to the Debian repo 2 days ago), my clusters won’t bootstrap properly
> anymore.
>
> It seems like the new systemd notify feature is causing this, since using
> Patroni 4.0.7 with `Type=simple` in the systemd unit file works pretty fine.
>

Just a quick question: Has the python3-systemd package been installed as
well?

There are multiple issues from the upstream Patroni GitHub repository
reporting similar problems since the release of 4.1.0, and the
corresponding solution is installing that python3-systemd package. It is
not (yet) listed as a patroni package dependency, so it may have been
missed given the new functionality in 4.1.0.

Ref:
https://github.com/patroni/patroni/issues/3480
https://github.com/patroni/patroni/issues/3482

-- Aaron Pavely

---

From: "Hauke Bruno Wollentin" <[email-masked]>
Date: Fri, 07 Nov 2025 06:27:21 +0000

Hi Aaron,

I can confirm that installing python3-systemd will resolve this issue, 
the package doesn't come as a dependency yet.

have a nice one,
Hauke

------ Original Message ------