From: mble (@mble) <noreply+mble@github.com>
To: pgbouncer/pgbouncer <noreply+pgbouncer-pgbouncer@github.com>
Subject: [pgbouncer/pgbouncer] issue #1711: Users removed from auth_file are never freed
Date: Wed, 18 Nov 2026 12:47:47 +0000
Message-ID: <gh-pgbouncer-pgbouncer-1711@github.com> (raw)
Hi folks,
## Summary
When a user is removed from `auth_file` and PgBouncer reloads it, the `PgGlobalUser` object is not freed. `disable_users()` only clears the password, and nothing returns the object to `user_cache`. So every distinct user name that has ever appeared in `auth_file` stays in memory until the process restarts.
This behaviour is old (`disable_users()` was added in f50cd52, 2007). Before 1.26 each kept user cost about 3.5 KiB. #1310 (1.26.0) raised `MAX_PASSWORD` from 2048 to 65531. `PgCredentials.pass
wd[MAX_PASSWORD]` is embedded in `PgGlobalUser`, and `slab_alloc()` zeroes every new object, so each kept user now costs about 100 KiB of resident memory. The leak is now about 30× faster.
#1677 reports the same root cause (a `PgGlobalUser` is never freed, about 64 KB each since 1.26.0) for users that are looked up through `auth_query`.
Some setups create and drop short-lived roles often: per-session or per-job credentials, roles with a TTL, CI or preview environments. In these setups an `auth_file` that only ever has a handful of users makes RSS grow without limit.
We saw this in production. The pgbouncer process runs in a container with a 32 MiB memory limit, and `auth_file` holds 4–5 users at any one time. A workload creates and drops about 250 short-lived roles per day. On 1.26.1 the kernel OOM-killed pgbouncer about once a day (anon-rss about 26.9 MiB at each kill). On 1.25 the same workload grows about 1 MiB/day, and the process normally restarts before that matters.
## Reproduction
The script below starts PgBouncer with an `auth_file` of 4 fixed users plus 1 temporary user. Then it repeats these steps: replace the file with a new temporary user name, send `SIGHUP`, and record RSS. `control` mode uses the same temporary user name every time.
The file is replaced with `mv` (new inode) because `auth_loaded()` compares `st_mtime` with one-second granularity. A fast in-place rewrite of the same size is not detected as a change.
<details>
<summary>Repro script</summary>
```bash
#!/usr/bin/env bash
# usage: repro.sh <pgbouncer-binary> <churn|control> [iterations]
set -euo pipefail
BIN=$1; MODE=$2; N=${3:-1000}
D=$(mktemp -d); trap 'kill $(cat $D/pgb.pid) 2>/dev/null || true; rm -rf $D' EXIT
cat > $D/pgb.ini <<INI
[databases]
db = host=127.0.0.1 port=5432
[pgbouncer]
listen_addr = 127.0.0.1
listen_port = 16432
unix_socket_dir =
auth_type = scram-sha-256
auth_file = $D/users.txt
pidfile = $D/pgb.pid
logfile = $D/pgb.log
INI
write_auth() {
{ for u in app1 app2 app3 app4; do echo "\"$u\" \"SCRAM-SHA-256\$4096:c2FsdA==\$a2V5:a2V5\""; done
[ -n "${1:-}" ] && echo "\"$1\" \"SCRAM-SHA-256\$4096:c2FsdA==\$a2V5:a2V5\""; } > $D/users.txt.new
mv $D/users.txt.new $D/users.txt
}
write_auth tmp_0
"$BIN" -d "$D/pgb.ini"; sleep 0.5
PID=$(cat $D/pgb.pid)
rss() { ps -o rss= -p "$PID" | tr -d ' '; }
echo "iter rss_kib"; echo "0 $(rss)"
for i in $(seq 1 "$N"); do
if [ "$MODE" = churn ]; then write_auth "tmp_$i"; else write_auth "tmp_0"; fi
kill -HUP "$PID"; sleep 0.05
(( i % (N/5) == 0 )) && echo "$i $(rss)"
done
```
No Postgres server is needed.
</details>
### Results
Debian bookworm (aarch64), built from tags `pgbouncer_1_25_0` and `pgbouncer_1_26_1` with `./configure --without-cares --without-pam --without-systemd`, 1000 iterations. RSS in KiB:
| Iteration | 1.25.0 churn | 1.25.0 control | 1.26.1 churn | 1.26.1 control |
|---:|---:|---:|---:|---:|
| 0 | 3464 | 3460 | 9628 | 9612 |
| 200 | 4308 | 3460 | 32168 | 9628 |
| 400 | 5260 | 3460 | 57924 | 9628 |
| 600 | 5164 | 3460 | 57924 | 9628 |
| 800 | 7068 | 3460 | 109432 | 9628 |
| 1000 | 7068 | 3460 | 109356 | 9628 |
- 1.26.1: about +100 MiB per 1000 distinct removed users (about 100 KiB each). It grows in steps because `user_cache` grows in fragments.
- 1.25.0: about +3.5 MiB per 1000 (about 3.5 KiB each).
- Control: flat in both versions, so the growth comes from distinct user names, not from reloads.
Short term, freeing users that no longer have references is probably the right direction. Longer-term, dynamically allocating the password or storing it out of line would allow each user to cost only what is required.