pg.ddx.io pgsql-bugs@postgresql.org mailing list archive
help / color / mirror / Atom feedBUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
17+ messages / 5 participants
[nested] [flat]
Commits from this thread:
2026-10-30 1c9efb57ec 5aed88ff9f 94d625f5d9 df9e6d7c7b 72216a2454 libpq: handle GSS credentials more sanely.
* BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
@ 2026-02-16 20:44 PG Bug reporting form <noreply@postgresql.org>
2026-02-17 17:01 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 17+ messages in thread
From: PG Bug reporting form @ 2026-02-16 20:44 UTC (permalink / raw)
To: pgsql-bugs@lists.postgresql.org; +Cc: mcarter@twosigma.com
The following bug has been logged on the website:
Bug reference: 19411
Logged by: Matt Carter
Email address: mcarter@twosigma.com
PostgreSQL version: 16.7
Operating system: Debian GNU/Linux 12 with kernel 6.12.51-ts1-amd64
Description:
This report is for the libpq client library for PostgreSQL.
I noticed the issue because my Apache Airflow task supervisor processes were
experiencing memory growth of ~600 KB/min. When running hundreds of tasks,
my 32 GB machine OOM-crashed every few hours. I traced the problem to when
I upgraded libpq from version 13 to 16. Varying Python version, and
PostgreSQL DB version didn't make a difference. When I updated to psycopg3,
the leak rate dropped dramatically.
Confirmed leaking:
libpq 16.0.4
libpq 16.0.7
Confirmed NOT leaking:
libpq 13.0.11
libpq 13.11
My environment for the below reproducer is:
Client libpq: 16.0.4 or 16.0.7 (runtime version)
Python: 3.10.16 or 3.11.11
psycopg2: 2.9.10
SQLAlchemy: 1.4.46
PostgreSQL Server: 13.11 OR 16.7 (both affected equally)
Platform: Linux, K8s containers
Reliable reproducer:
```
#!/usr/bin/env python3
"""
Reproduces ~800 KB/min memory leak in libpq 16.x
MUST monitor with 'ps aux', NOT Python's resource module!
"""
import datetime, os, subprocess
from time import sleep
from sqlalchemy import Column, Integer, types, create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import scoped_session, sessionmaker
from sqlalchemy.pool import NullPool
# Verify libpq version
import psycopg2
print(f"psycopg2 compiled against libpq: {psycopg2.__libpq_version__}")
# Check ACTUAL runtime libpq (critical!)
so_path = os.path.dirname(psycopg2.__file__) +
"/_psycopg.cpython-311-x86_64-linux-gnu.so"
result = subprocess.run(["ldd", so_path], capture_output=True, text=True)
for line in result.stdout.split('\n'):
if 'libpq.so' in line:
print(f"ACTUAL runtime libpq:
{line.split('=>')[1].split('(')[0].strip()}")
print(f"\nMonitor this PID: {os.getpid()}")
print("Run in another terminal: watch -n 10 'ps -p <PID> -o
pid,rss,vsz,cmd'")
print()
# Setup SQLAlchemy with NullPool (creates/destroys connections)
Base = declarative_base(metadata=None)
class TestTable(Base):
__tablename__ = "test_leak_table"
id = Column(Integer, primary_key=True)
updated_at = Column(types.DateTime(timezone=True))
obj = TestTable()
engine = create_engine(
"postgresql://your-server/your-database",
poolclass=NullPool, # Critical: no pooling, create/destroy each time
encoding="utf-8"
)
Session = scoped_session(sessionmaker(
autocommit=False,
autoflush=False,
bind=engine,
expire_on_commit=False,
))
session = Session()
session.add(obj)
# Monitor memory using ps aux (CRITICAL - internal monitoring doesn't show
leak!)
def get_rss_kb():
"""Get RSS from ps aux - DO NOT use resource.getrusage()!"""
pid = os.getpid()
result = subprocess.run(["ps", "aux"], capture_output=True, text=True)
for line in result.stdout.split('\n'):
if str(pid) in line and 'python' in line:
return int(line.split()[5]) # RSS is 6th column
return None
print("Starting 10-minute test...")
print(f"{'Time(s)':<8} | {'Iteration':<10} | {'RSS(KB)':<10} |
{'Growth(KB)':<10}")
print("-" * 55)
start_time = datetime.datetime.now()
baseline_rss = None
for i in range(1, 601): # 600 iterations = 10 minutes
# Update and commit (creates/destroys connection with NullPool)
obj.updated_at = datetime.datetime.now(datetime.timezone.utc)
session.commit()
# Report every 30 iterations
if i % 30 == 0:
rss = get_rss_kb()
if rss:
if i == 30:
baseline_rss = rss
growth = 0
else:
growth = rss - baseline_rss
elapsed = (datetime.datetime.now() - start_time).total_seconds()
print(f"{int(elapsed):<8} | {i:<10} | {rss:<10} | {growth:<10}",
flush=True)
sleep(1)
# Final results
final_rss = get_rss_kb()
total_elapsed = (datetime.datetime.now() - start_time).total_seconds()
growth_total = final_rss - baseline_rss
leak_rate = growth_total / (total_elapsed / 60)
print()
print("=" * 60)
print(f"Baseline RSS (t=30s): {baseline_rss} KB")
print(f"Final RSS (t=600s): {final_rss} KB")
print(f"Total Growth: {growth_total} KB")
print(f"Leak Rate: {leak_rate:.1f} KB/min")
print("=" * 60)
```
The DB set-up for this reproducer is simple:
```
CREATE TABLE test_leak_table (
id SERIAL PRIMARY KEY,
updated_at TIMESTAMP WITH TIME ZONE
);
INSERT INTO test_leak_table (updated_at) VALUES (NOW());
```
When I run this with libpq 13.x, I get:
Baseline: ~41,000 KB
Final: ~41,200 KB
Growth: ~200 KB (stabilizes)
Leak Rate: ~14 KB/min
But when I run it with libpq 16.x, I reproduce the leak behavior:
Baseline: ~42,000 KB
Final: ~52,000 KB
Growth: ~10,000 KB (continuous linear growth)
Leak Rate: ~800 KB/min
Notes:
1. PostgreSQL server version does NOT matter
PG 13.11: ~797 KB/min average
PG 16.7: ~802 KB/min average
Difference: <1% (within measurement error)
2. Leak is linear and continuous
Grows ~530 KB every 30 seconds
No stabilization
No sawtooth pattern
Suggests unbounded memory accumulation
3. Leak is in C Library Memory
The leak is:
- Visible in `ps aux` RSS (operating system view)
- NOT visible in Python's resource.getrusage() (application view)
- NOT any Python object (I checked with the Python VM's `gc.get_objects()`)
This indicates the leak is in libpq's C-level allocations, not Python heap.
4. The leak requires connection churn. In particular, it needs:
- NullPool (no connection pooling)
- Repeated connection create/destroy
- UPDATE + COMMIT operations each iteration
Note that the only psycopg2 version I tested with was 2.9.10, as it is the
only version available in my test environment.
Also, I only tested with SQLAlchemy version 1.4.46, as v2 would require a
rewrite due to an incompatible API.
Workarounds:
Use libpq 13.x (43x improvement: 14-22 KB/min vs 800 KB/min).
Use connection pooling to reduce connection churn.
Use psycopg3.
Recycle processes frequently.
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
2026-02-16 20:44 BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed PG Bug reporting form <noreply@postgresql.org>
@ 2026-02-17 17:01 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 17:54 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
0 siblings, 1 reply; 17+ messages in thread
From: Tom Lane @ 2026-02-17 17:01 UTC (permalink / raw)
To: mcarter@twosigma.com; +Cc: pgsql-bugs@lists.postgresql.org
PG Bug reporting form <noreply@postgresql.org> writes:
> This report is for the libpq client library for PostgreSQL.
> I noticed the issue because my Apache Airflow task supervisor processes were
> experiencing memory growth of ~600 KB/min. When running hundreds of tasks,
> my 32 GB machine OOM-crashed every few hours. I traced the problem to when
> I upgraded libpq from version 13 to 16. Varying Python version, and
> PostgreSQL DB version didn't make a difference. When I updated to psycopg3,
> the leak rate dropped dramatically.
I think there are way too many moving parts here, and too few
configuration details, to allow assigning blame confidently.
I tried making a simple C test program that just connected, issued
BEGIN/UPDATE/COMMIT, and disconnected in a tight loop. I see zero
leakage with that. So either the leak isn't actually in libpq, or
there's some critical environmental factor you didn't mention.
Plausible such factors include connection type and authentication
method. (For example, years ago libpq did leak memory while using
GSSAPI encryption.) I tried both regular and SSL connections but
didn't really push hard on that, since I'd just be guessing blindly
about what your setup is.
regards, tom lane
^ permalink raw reply [nested|flat] 17+ messages in thread
* RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
2026-02-16 20:44 BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed PG Bug reporting form <noreply@postgresql.org>
2026-02-17 17:01 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
@ 2026-02-17 17:54 ` Matt Carter <Matt.Carter@twosigma.com>
2026-02-17 18:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 17+ messages in thread
From: Matt Carter @ 2026-02-17 17:54 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
Tom Lane <tgl@sss.pgh.pa.us> wrote:
> I think there are way too many moving parts here, and too few configuration details,
> to allow assigning blame confidently.
> I tried making a simple C test program that just connected, issued
> BEGIN/UPDATE/COMMIT, and disconnected in a tight loop. I see zero leakage with
> that. So either the leak isn't actually in libpq, or there's some critical environmental
> factor you didn't mention.
>
> Plausible such factors include connection type and authentication method. (For
> example, years ago libpq did leak memory while using GSSAPI encryption.) I tried
> both regular and SSL connections but didn't really push hard on that, since I'd just
> be guessing blindly about what your setup is.
Tom,
Thank you for taking the time to test this and for the feedback. Your C test showing no leak suggests the issue is specific to how psycopg2 uses libpq, not libpq itself. I apologize for not including enough environmental details. I used Kerberos/GSSAPI with SSL (TLS 1.2 connections). My connection string was: "postgresql://hostname/database" (no password, Kerberos auth).
Your mention of "years ago libpq did leak memory while using GSSAPI encryption" is interesting because we ARE using GSSAPI/Kerberos authentication.
Given your C test showed no leak, this appears to be specific to GSSAPI authentication, or perhaps specific to how psycopg2 uses libpq with GSSAPI.
I can test with non-GSSAPI authentication to try to isolate that variable. I can also create a pure psycopg2 reproducer (without SQLAlchemy). I can also test whether disabling GSSAPI encryption (but keeping GSSAPI auth) changes the behavior. Would testing with GSSAPI authentication help narrow this down? I can also report this to the psycopg2 project if you think it's their issue.
Thanks again for your help,
Matt
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
2026-02-16 20:44 BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed PG Bug reporting form <noreply@postgresql.org>
2026-02-17 17:01 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 17:54 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
@ 2026-02-17 18:47 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 19:45 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Andres Freund <andres@anarazel.de>
2026-02-17 20:54 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
0 siblings, 2 replies; 17+ messages in thread
From: Tom Lane @ 2026-02-17 18:47 UTC (permalink / raw)
To: Matt Carter <Matt.Carter@twosigma.com>; +Cc: pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
Matt Carter <Matt.Carter@twosigma.com> writes:
> Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> I think there are way too many moving parts here, and too few configuration details,
>> to allow assigning blame confidently.
> Thank you for taking the time to test this and for the feedback. Your C test showing no leak suggests the issue is specific to how psycopg2 uses libpq, not libpq itself. I apologize for not including enough environmental details. I used Kerberos/GSSAPI with SSL (TLS 1.2 connections). My connection string was: "postgresql://hostname/database" (no password, Kerberos auth).
> Your mention of "years ago libpq did leak memory while using GSSAPI encryption" is interesting because we ARE using GSSAPI/Kerberos authentication.
Interesting. I wondered about GSSAPI, but spinning up such an
environment is more work than I wanted to do on speculation.
> I can test with non-GSSAPI authentication to try to isolate that variable. I can also create a pure psycopg2 reproducer (without SQLAlchemy). I can also test whether disabling GSSAPI encryption (but keeping GSSAPI auth) changes the behavior. Would testing with GSSAPI authentication help narrow this down? I can also report this to the psycopg2 project if you think it's their issue.
Please try varying the connection type and encryption. I do suspect
this may be psycopg2's fault, but we lack enough data to pin blame
as yet.
regards, tom lane
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
2026-02-16 20:44 BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed PG Bug reporting form <noreply@postgresql.org>
2026-02-17 17:01 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 17:54 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-02-17 18:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
@ 2026-02-17 19:45 ` Andres Freund <andres@anarazel.de>
2026-10-29 18:40 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
1 sibling, 1 reply; 17+ messages in thread
From: Andres Freund @ 2026-02-17 19:45 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Matt Carter <Matt.Carter@twosigma.com>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
Hi,
On 2026-02-17 13:47:49 -0500, Tom Lane wrote:
> Matt Carter <Matt.Carter@twosigma.com> writes:
> > Thank you for taking the time to test this and for the feedback. Your C test showing no leak suggests the issue is specific to how psycopg2 uses libpq, not libpq itself. I apologize for not including enough environmental details. I used Kerberos/GSSAPI with SSL (TLS 1.2 connections). My connection string was: "postgresql://hostname/database" (no password, Kerberos auth).
> > Your mention of "years ago libpq did leak memory while using GSSAPI encryption" is interesting because we ARE using GSSAPI/Kerberos authentication.
>
> Interesting. I wondered about GSSAPI, but spinning up such an
> environment is more work than I wanted to do on speculation.
Heh, understandable...
> > I can test with non-GSSAPI authentication to try to isolate that variable. I can also create a pure psycopg2 reproducer (without SQLAlchemy). I can also test whether disabling GSSAPI encryption (but keeping GSSAPI auth) changes the behavior. Would testing with GSSAPI authentication help narrow this down? I can also report this to the psycopg2 project if you think it's their issue.
>
> Please try varying the connection type and encryption. I do suspect
> this may be psycopg2's fault, but we lack enough data to pin blame
> as yet.
Matt, you could try analyzing the memory usage with heaptrack, it tends to be
pretty good at finding them even in uninstrumented builds, as long as enough
debug symbols for a backtrace are available. Often enough it'll pinpoint
where the leak is coming from quite easily (but note that it will report some
constant-sized leaks that are "intentional").
Greetings,
Andres
^ permalink raw reply [nested|flat] 17+ messages in thread
* RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
2026-02-16 20:44 BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed PG Bug reporting form <noreply@postgresql.org>
2026-02-17 17:01 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 17:54 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-02-17 18:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 19:45 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Andres Freund <andres@anarazel.de>
@ 2026-10-29 18:40 ` Matt Carter <Matt.Carter@twosigma.com>
2026-10-29 20:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 17+ messages in thread
From: Matt Carter @ 2026-10-29 18:40 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
> Tom Lane wrote:
>> Interesting. I wondered about GSSAPI, but spinning up such an
>> environment is more work than I wanted to do on speculation.
Your suspicion was right, and I'm sorry for pointing at psycopg2 earlier. The leak is in libpq, and it needs GSSAPI *authentication*. Every connection to my server authenticates via "gss" in pg_hba.conf. My gssencmode=disable runs only turned off GSS encryption, so none of my tests ever removed it.
pg_GSS_continue() runs once per GSS round trip, and Kerberos with mutual authentication takes two. On every call it does
/* Only try to acquire credentials if GSS delegation isn't disabled. */
if (!pg_GSS_have_cred_cache(&conn->gcred))
conn->gcred = GSS_C_NO_CREDENTIAL;
pg_GSS_have_cred_cache() calls gss_acquire_cred() and stores the result in *cred_out. It doesn't release the credential that is already there. pqDropConnection() releases only the last one, so the credential from the first round trip is orphaned. With MIT krb5 that costs about 17.8 kB per connection, mostly 4144-byte keytab buffers allocated under gss_acquire_cred().
This is gdb on libpq 16.7 with gssencmode=disable, breaking on the GSSAPI entry points. Every connection looks like this:
PQconnectdb
gss_acquire_cred from pg_GSS_have_cred_cache
gss_init_sec_context cred=...b580 from pg_GSS_continue
gss_acquire_cred from pg_GSS_have_cred_cache
gss_init_sec_context cred=...85c0 from pg_GSS_continue
PQfinish
gss_release_cred cred=...85c0 from pqDropConnection
...b580 is never released.
The other callers already guard against this. For example, fe-secure-gssapi.c calls pg_GSS_have_cred_cache() only when conn->gcred == GSS_C_NO_CREDENTIAL. The patch below does the same in pg_GSS_continue(). It's against 18.3 and applies to 16.7 with only a line offset:
inline patch 1: src/interfaces/libpq/fe-auth.c (+7/-2 lines) download inline-1.patch
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -101,8 +101,13 @@
if (conn->Pfdebug)
pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
- /* Only try to acquire credentials if GSS delegation isn't disabled. */
- if (!pg_GSS_have_cred_cache(&conn->gcred))
+ /*
+ * Acquire credentials on the first call only. Each GSS round trip comes
+ * through here, and acquiring again would leak the credential already in
+ * conn->gcred.
+ */
+ if (conn->gcred == GSS_C_NO_CREDENTIAL &&
+ !pg_GSS_have_cred_cache(&conn->gcred))
conn->gcred = GSS_C_NO_CREDENTIAL;
if (conn->gssdelegation && conn->gssdelegation[0] == '1')
I built libpq from the 16.7 and 18.3 release tarballs, with MIT krb5 1.20.1 and OpenSSL 1.1, both with and without the patch. I then ran the reproducer below against a server using gss auth + SSL. These are the bytes per connection still allocated after warm-up, over 450 connections:
unpatched patched
16.7 17823 123
18.3 17844 98
With gssencmode=disable, the patched builds measure 31 and 74 bytes/connection. Connections still authenticate normally. With the patch, gdb shows one gss_acquire_cred() per connection, used for both gss_init_sec_context() calls and released in pqDropConnection(). Over 3,000 connections (least-squares slope after warm-up), the patched builds still grow by 25-31 bytes per connection. libpq 13.11 on the same server grows by 18 bytes per connection, so that remainder doesn't look new in v16.
The unpatched leak also shows up in libpq 17.6 as bundled in psycopg-binary 3.2.13, with MIT krb5 1.15.1: 17985 bytes/connection. libpq 13.11 and 15 don't have it, which fits this code arriving with GSS credential delegation in v16. I haven't tested master.
Andres: thanks for the heaptrack pointer. An allocation tracker with backtraces put 95% of the retained memory under gss_acquire_cred() called from libpq, which is what led here.
--- gss_leak_repro.c ---
/*
* Build: cc -o gss_leak_repro gss_leak_repro.c -I$(pg_config --includedir) \
* -L$(pg_config --libdir) -lpq
* Run: kinit; ./gss_leak_repro "host=<server with gss in pg_hba.conf> dbname=postgres" 1000
* Needs glibc >= 2.33 for mallinfo2().
*/
#include <malloc.h>
#include <stdio.h>
#include <stdlib.h>
#include <libpq-fe.h>
static size_t
heap_in_use(void)
{
struct mallinfo2 mi = mallinfo2();
return mi.uordblks + mi.hblkhd;
}
int
main(int argc, char **argv)
{
const char *conninfo = argc > 1 ? argv[1] : "dbname=postgres";
int n = argc > 2 ? atoi(argv[2]) : 1000;
int warmup = n / 10;
size_t base = 0;
printf("libpq %d\n", PQlibVersion());
for (int i = 1; i <= n; i++)
{
PGconn *conn = PQconnectdb(conninfo);
if (PQstatus(conn) != CONNECTION_OK)
{
fprintf(stderr, "%s", PQerrorMessage(conn));
return 1;
}
if (i == 1)
printf("GSSAPI auth used: %d\n", PQconnectionUsedGSSAPI(conn));
PQfinish(conn);
if (i == warmup)
base = heap_in_use();
}
printf("%.1f bytes per connection still allocated (%d connections after warm-up)\n",
(double) (heap_in_use() - base) / (n - warmup), n - warmup);
return 0;
}
--- end ---
Matt
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
2026-02-16 20:44 BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed PG Bug reporting form <noreply@postgresql.org>
2026-02-17 17:01 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 17:54 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-02-17 18:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 19:45 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Andres Freund <andres@anarazel.de>
2026-10-29 18:40 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
@ 2026-10-29 20:47 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-10-29 21:52 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
0 siblings, 1 reply; 17+ messages in thread
From: Tom Lane @ 2026-10-29 20:47 UTC (permalink / raw)
To: Matt Carter <Matt.Carter@twosigma.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
Matt Carter <Matt.Carter@twosigma.com> writes:
> Your suspicion was right, and I'm sorry for pointing at psycopg2 earlier. The leak is in libpq, and it needs GSSAPI *authentication*. Every connection to my server authenticates via "gss" in pg_hba.conf. My gssencmode=disable runs only turned off GSS encryption, so none of my tests ever removed it.
> pg_GSS_continue() runs once per GSS round trip, and Kerberos with mutual authentication takes two. On every call it does
> /* Only try to acquire credentials if GSS delegation isn't disabled. */
> if (!pg_GSS_have_cred_cache(&conn->gcred))
> conn->gcred = GSS_C_NO_CREDENTIAL;
> pg_GSS_have_cred_cache() calls gss_acquire_cred() and stores the result in *cred_out. It doesn't release the credential that is already there.
Ooops. It looks like this isn't the only place that makes that
mistake: select_next_encryption_method seems at risk too, or at least
if it's not, that's because of caller logic not anything it's doing
itself. I think that's partially because pg_GSS_have_cred_cache is
poorly named: it sounds like a status test not an action, and its API
is just bizarre. I propose changing its name altogether, as attached.
regards, tom lane
Attachments:
fix-gss-credentials-leak.patch (text/x-diff, 4.7K) download
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index e3bddf91203..fe9756da691 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -101,9 +101,9 @@ pg_GSS_continue(PGconn *conn, int payloadlen)
if (conn->Pfdebug)
pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
- /* Only try to acquire credentials if GSS delegation isn't disabled. */
- if (!pg_GSS_have_cred_cache(&conn->gcred))
- conn->gcred = GSS_C_NO_CREDENTIAL;
+ /* try to acquire credentials if we didn't already */
+ if (conn->gcred == GSS_C_NO_CREDENTIAL)
+ conn->gcred = pg_GSS_get_cred_cache();
if (conn->gssdelegation && conn->gssdelegation[0] == '1')
gss_flags |= GSS_C_DELEG_FLAG;
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 811cb0f2a50..9e17a8f1c6f 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -3281,7 +3281,8 @@ keep_going: /* We will come back to here until there is
}
if (conn->gcred == GSS_C_NO_CREDENTIAL)
{
- if (!pg_GSS_have_cred_cache(&conn->gcred))
+ conn->gcred = pg_GSS_get_cred_cache();
+ if (conn->gcred == GSS_C_NO_CREDENTIAL)
{
libpq_append_conn_error(conn,
"GSSAPI encryption required but no credential cache");
@@ -4847,15 +4848,16 @@ select_next_encryption_method(PGconn *conn, bool have_valid_connection)
if ((remaining_methods & ENC_GSSAPI) != 0)
{
/*
- * If GSSAPI encryption is enabled, then call pg_GSS_have_cred_cache()
- * which will return true if we can acquire credentials (and give us a
- * handle to use in conn->gcred), and then send a packet to the server
+ * If GSSAPI encryption is enabled, then call pg_GSS_get_cred_cache()
+ * to try to acquire credentials, and then send a packet to the server
* asking for GSSAPI Encryption (and skip past SSL negotiation and
* regular startup below).
*/
if (!conn->gctx)
{
- if (!pg_GSS_have_cred_cache(&conn->gcred))
+ if (conn->gcred == GSS_C_NO_CREDENTIAL)
+ conn->gcred = pg_GSS_get_cred_cache();
+ if (conn->gcred == GSS_C_NO_CREDENTIAL)
{
conn->allowed_enc_methods &= ~ENC_GSSAPI;
remaining_methods &= ~ENC_GSSAPI;
@@ -5020,6 +5022,10 @@ pqMakeEmptyPGconn(void)
conn->altsock = PGINVALID_SOCKET;
conn->Pfdebug = NULL;
+#ifdef ENABLE_GSS
+ conn->gcred = GSS_C_NO_CREDENTIAL; /* let's not assume this is 0 */
+#endif
+
/*
* We try to send at least 8K at a time, which is the usual size of pipe
* buffers on Unix systems. That way, when we are sending a large amount
diff --git a/src/interfaces/libpq/fe-gssapi-common.c b/src/interfaces/libpq/fe-gssapi-common.c
index e7776fe76e7..c23819a219c 100644
--- a/src/interfaces/libpq/fe-gssapi-common.c
+++ b/src/interfaces/libpq/fe-gssapi-common.c
@@ -55,10 +55,11 @@ pg_GSS_error(const char *mprefix, PGconn *conn,
}
/*
- * Check if we can acquire credentials at all (and yield them if so).
+ * Attempt to acquire credentials, and return the token if successful.
+ * Return GSS_C_NO_CREDENTIAL if not.
*/
-bool
-pg_GSS_have_cred_cache(gss_cred_id_t *cred_out)
+gss_cred_id_t
+pg_GSS_get_cred_cache(void)
{
OM_uint32 major,
minor;
@@ -67,12 +68,8 @@ pg_GSS_have_cred_cache(gss_cred_id_t *cred_out)
major = gss_acquire_cred(&minor, GSS_C_NO_NAME, 0, GSS_C_NO_OID_SET,
GSS_C_INITIATE, &cred, NULL, NULL);
if (major != GSS_S_COMPLETE)
- {
- *cred_out = NULL;
- return false;
- }
- *cred_out = cred;
- return true;
+ return GSS_C_NO_CREDENTIAL;
+ return cred;
}
/*
diff --git a/src/interfaces/libpq/fe-gssapi-common.h b/src/interfaces/libpq/fe-gssapi-common.h
index f89d9e071e8..769e9f5ae91 100644
--- a/src/interfaces/libpq/fe-gssapi-common.h
+++ b/src/interfaces/libpq/fe-gssapi-common.h
@@ -20,7 +20,7 @@
void pg_GSS_error(const char *mprefix, PGconn *conn,
OM_uint32 maj_stat, OM_uint32 min_stat);
-bool pg_GSS_have_cred_cache(gss_cred_id_t *cred_out);
+gss_cred_id_t pg_GSS_get_cred_cache(void);
int pg_GSS_load_servicename(PGconn *conn);
#endif
diff --git a/src/interfaces/libpq/fe-secure-gssapi.c b/src/interfaces/libpq/fe-secure-gssapi.c
index 5ae10af5678..c43ab0294df 100644
--- a/src/interfaces/libpq/fe-secure-gssapi.c
+++ b/src/interfaces/libpq/fe-secure-gssapi.c
@@ -640,9 +640,9 @@ pqsecure_open_gss(PGconn *conn)
if (conn->gssdelegation && conn->gssdelegation[0] == '1')
{
- /* Acquire credentials if possible */
+ /* Attempt to acquire credentials, if we didn't already */
if (conn->gcred == GSS_C_NO_CREDENTIAL)
- (void) pg_GSS_have_cred_cache(&conn->gcred);
+ conn->gcred = pg_GSS_get_cred_cache();
/*
* We have credentials and gssdelegation is enabled, so request
^ permalink raw reply [nested|flat] 17+ messages in thread
* RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
2026-02-16 20:44 BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed PG Bug reporting form <noreply@postgresql.org>
2026-02-17 17:01 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 17:54 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-02-17 18:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 19:45 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Andres Freund <andres@anarazel.de>
2026-10-29 18:40 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-10-29 20:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
@ 2026-10-29 21:52 ` Matt Carter <Matt.Carter@twosigma.com>
2026-10-30 16:55 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 17+ messages in thread
From: Matt Carter @ 2026-10-29 21:52 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
> Tom Lane wrote:
>> I propose changing its name altogether, as attached.
Thanks, Tom. I tested your patch against my GSSAPI-authenticated server (MIT krb5 1.20.1, OpenSSL 1.1). It applies to 18.3 with line offsets and builds without warnings. Using the reproducer from my previous message, these are the bytes per connection still allocated after warm-up (450 connections):
18.3 unpatched 17844
18.3 + your patch 108
18.3 + your patch, gssencmode=disable 42
GSSAPI authentication works as before. Under gdb, every gss_acquire_cred() now has a matching gss_release_cred():
- With gssencmode=disable there is one acquire per connection. Both gss_init_sec_context() calls share it, and pqDropConnection() releases it.
- With the default gssencmode, pqsecure_open_gss() releases the encryption probe's credential before the fallback. Authentication then acquires one more, and pqDropConnection() releases that.
I couldn't exercise gssencmode=require past negotiation, because my server's pg_hba.conf rejects GSSAPI-encrypted connections. It fails the same way with and without the patch.
We would benefit from back-patches to 16 and 17:
- Our Airflow deployments get libpq 17.11 from psycopg2-binary 2.9.13, which measures 17930 bytes/connection with the same reproducer.
- Our internal builds ship 16.x. There the fe-connect.c hunks won't apply as-is, but the pg_GSS_continue() change alone takes 16.7 from 17823 to 123 bytes per connection.
Matt
________________________________________
From: Tom Lane <tgl@sss.pgh.pa.us>
Sent: Thursday, 29 October 2026 20:47:18
To: Matt Carter
Cc: Andres Freund; pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
Matt Carter <Matt.Carter@twosigma.com> writes:
> Your suspicion was right, and I'm sorry for pointing at psycopg2 earlier. The leak is in libpq, and it needs GSSAPI *authentication*. Every connection to my server authenticates via "gss" in pg_hba.conf. My gssencmode=disable runs only turned off GSS encryption, so none of my tests ever removed it.
> pg_GSS_continue() runs once per GSS round trip, and Kerberos with mutual authentication takes two. On every call it does
> /* Only try to acquire credentials if GSS delegation isn't disabled. */
> if (!pg_GSS_have_cred_cache(&conn->gcred))
> conn->gcred = GSS_C_NO_CREDENTIAL;
> pg_GSS_have_cred_cache() calls gss_acquire_cred() and stores the result in *cred_out. It doesn't release the credential that is already there.
Ooops. It looks like this isn't the only place that makes that
mistake: select_next_encryption_method seems at risk too, or at least
if it's not, that's because of caller logic not anything it's doing
itself. I think that's partially because pg_GSS_have_cred_cache is
poorly named: it sounds like a status test not an action, and its API
is just bizarre. I propose changing its name altogether, as attached.
regards, tom lane
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
2026-02-16 20:44 BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed PG Bug reporting form <noreply@postgresql.org>
2026-02-17 17:01 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 17:54 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-02-17 18:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 19:45 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Andres Freund <andres@anarazel.de>
2026-10-29 18:40 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-10-29 20:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-10-29 21:52 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
@ 2026-10-30 16:55 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-10-30 17:25 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-10-30 19:00 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 2 replies; 17+ messages in thread
From: Tom Lane @ 2026-10-30 16:55 UTC (permalink / raw)
To: Matt Carter <Matt.Carter@twosigma.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
Matt Carter <Matt.Carter@twosigma.com> writes:
> Tom Lane wrote:
>> I propose changing its name altogether, as attached.
> Thanks, Tom. I tested your patch against my GSSAPI-authenticated server (MIT krb5 1.20.1, OpenSSL 1.1). It applies to 18.3 with line offsets and builds without warnings. Using the reproducer from my previous message, these are the bytes per connection still allocated after warm-up (450 connections):
> 18.3 unpatched 17844
> 18.3 + your patch 108
> 18.3 + your patch, gssencmode=disable 42
> GSSAPI authentication works as before.
Thanks for testing that, but after looking closer, I believe that
pg_GSS_continue has even more problems here. It will attempt to
acquire credentials whether or not GSS delegation is enabled,
contrary to the adjacent comment and unlike what happens in
pqsecure_open_gss. And if GSS delegation is enabled, it will
set the GSS_C_DELEG_FLAG whether or not it was able to acquire
credentials, which seems not very sane and again different from
the logic in pqsecure_open_gss. So I think what we really need
is more like the attached. This passes our own test suite, but
I wonder if you can exercise it in your own environment.
> We would benefit from back-patches to 16 and 17:
Yeah, this problem was visibly introduced in v16 when we added
GSS delegation support, so I anticipate back-patching that far.
regards, tom lane
Attachments:
v1-0001-libpq-handle-GSS-credentials-more-sanely.patch (text/x-diff, 6.4K) download
From e7509d1ec9f0d75e5539340981abf0d607d1fac4 Mon Sep 17 00:00:00 2001
From: Tom Lane <tgl@sss.pgh.pa.us>
Date: Fri, 30 Oct 2026 12:47:51 -0400
Subject: [PATCH v1] libpq: handle GSS credentials more sanely.
pg_GSS_have_cred_cache() always acquires new GSS credentials.
If we already had some, they were just leaked due to overwriting
conn->gcred, and that's a nontrivial amount of memory (around 16K
according to reporter). Unfortunately, pg_GSS_continue() did this
each time through, and it is typically called more than once in a
GSSAPI connection cycle. So repeated connections using GSS
authentication could leak substantial amounts of memory.
IMO the source of this problem is really the poorly-designed API
of pg_GSS_have_cred_cache(): it's unclear that it always gets new
credentials. Let's rename it and remove the unnecessary pointer
parameter to make the behavior clearer.
To add insult to injury, pg_GSS_continue() shouldn't even be
trying to acquire credentials unless GSS delegation is enabled,
which it is not by default; so the whole effort is often wasted.
Problem evidently introduced by 6633cfb21, so back-patch to v16.
Bug: #19411
Reported-by: Matt Carter <Matt.Carter@twosigma.com>
Diagnosed-by: Matt Carter <Matt.Carter@twosigma.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/19411-0440c8897a04b638@postgresql.org
Backpatch-through: 16
---
src/interfaces/libpq/fe-auth.c | 10 ++++++----
src/interfaces/libpq/fe-connect.c | 16 +++++++++++-----
src/interfaces/libpq/fe-gssapi-common.c | 15 ++++++---------
src/interfaces/libpq/fe-gssapi-common.h | 2 +-
src/interfaces/libpq/fe-secure-gssapi.c | 4 ++--
5 files changed, 26 insertions(+), 21 deletions(-)
diff --git a/src/interfaces/libpq/fe-auth.c b/src/interfaces/libpq/fe-auth.c
index e3bddf91203..4a55964875a 100644
--- a/src/interfaces/libpq/fe-auth.c
+++ b/src/interfaces/libpq/fe-auth.c
@@ -102,11 +102,13 @@ pg_GSS_continue(PGconn *conn, int payloadlen)
pqTraceOutputMessage(conn, conn->inBuffer + conn->inStart, false);
/* Only try to acquire credentials if GSS delegation isn't disabled. */
- if (!pg_GSS_have_cred_cache(&conn->gcred))
- conn->gcred = GSS_C_NO_CREDENTIAL;
-
if (conn->gssdelegation && conn->gssdelegation[0] == '1')
- gss_flags |= GSS_C_DELEG_FLAG;
+ {
+ if (conn->gcred == GSS_C_NO_CREDENTIAL)
+ conn->gcred = pg_GSS_get_cred_cache();
+ if (conn->gcred != GSS_C_NO_CREDENTIAL)
+ gss_flags |= GSS_C_DELEG_FLAG;
+ }
maj_stat = gss_init_sec_context(&min_stat,
conn->gcred,
diff --git a/src/interfaces/libpq/fe-connect.c b/src/interfaces/libpq/fe-connect.c
index 811cb0f2a50..9e17a8f1c6f 100644
--- a/src/interfaces/libpq/fe-connect.c
+++ b/src/interfaces/libpq/fe-connect.c
@@ -3281,7 +3281,8 @@ keep_going: /* We will come back to here until there is
}
if (conn->gcred == GSS_C_NO_CREDENTIAL)
{
- if (!pg_GSS_have_cred_cache(&conn->gcred))
+ conn->gcred = pg_GSS_get_cred_cache();
+ if (conn->gcred == GSS_C_NO_CREDENTIAL)
{
libpq_append_conn_error(conn,
"GSSAPI encryption required but no credential cache");
@@ -4847,15 +4848,16 @@ select_next_encryption_method(PGconn *conn, bool have_valid_connection)
if ((remaining_methods & ENC_GSSAPI) != 0)
{
/*
- * If GSSAPI encryption is enabled, then call pg_GSS_have_cred_cache()
- * which will return true if we can acquire credentials (and give us a
- * handle to use in conn->gcred), and then send a packet to the server
+ * If GSSAPI encryption is enabled, then call pg_GSS_get_cred_cache()
+ * to try to acquire credentials, and then send a packet to the server
* asking for GSSAPI Encryption (and skip past SSL negotiation and
* regular startup below).
*/
if (!conn->gctx)
{
- if (!pg_GSS_have_cred_cache(&conn->gcred))
+ if (conn->gcred == GSS_C_NO_CREDENTIAL)
+ conn->gcred = pg_GSS_get_cred_cache();
+ if (conn->gcred == GSS_C_NO_CREDENTIAL)
{
conn->allowed_enc_methods &= ~ENC_GSSAPI;
remaining_methods &= ~ENC_GSSAPI;
@@ -5020,6 +5022,10 @@ pqMakeEmptyPGconn(void)
conn->altsock = PGINVALID_SOCKET;
conn->Pfdebug = NULL;
+#ifdef ENABLE_GSS
+ conn->gcred = GSS_C_NO_CREDENTIAL; /* let's not assume this is 0 */
+#endif
+
/*
* We try to send at least 8K at a time, which is the usual size of pipe
* buffers on Unix systems. That way, when we are sending a large amount
diff --git a/src/interfaces/libpq/fe-gssapi-common.c b/src/interfaces/libpq/fe-gssapi-common.c
index e7776fe76e7..c23819a219c 100644
--- a/src/interfaces/libpq/fe-gssapi-common.c
+++ b/src/interfaces/libpq/fe-gssapi-common.c
@@ -55,10 +55,11 @@ pg_GSS_error(const char *mprefix, PGconn *conn,
}
/*
- * Check if we can acquire credentials at all (and yield them if so).
+ * Attempt to acquire credentials, and return the token if successful.
+ * Return GSS_C_NO_CREDENTIAL if not.
*/
-bool
-pg_GSS_have_cred_cache(gss_cred_id_t *cred_out)
+gss_cred_id_t
+pg_GSS_get_cred_cache(void)
{
OM_uint32 major,
minor;
@@ -67,12 +68,8 @@ pg_GSS_have_cred_cache(gss_cred_id_t *cred_out)
major = gss_acquire_cred(&minor, GSS_C_NO_NAME, 0, GSS_C_NO_OID_SET,
GSS_C_INITIATE, &cred, NULL, NULL);
if (major != GSS_S_COMPLETE)
- {
- *cred_out = NULL;
- return false;
- }
- *cred_out = cred;
- return true;
+ return GSS_C_NO_CREDENTIAL;
+ return cred;
}
/*
diff --git a/src/interfaces/libpq/fe-gssapi-common.h b/src/interfaces/libpq/fe-gssapi-common.h
index f89d9e071e8..769e9f5ae91 100644
--- a/src/interfaces/libpq/fe-gssapi-common.h
+++ b/src/interfaces/libpq/fe-gssapi-common.h
@@ -20,7 +20,7 @@
void pg_GSS_error(const char *mprefix, PGconn *conn,
OM_uint32 maj_stat, OM_uint32 min_stat);
-bool pg_GSS_have_cred_cache(gss_cred_id_t *cred_out);
+gss_cred_id_t pg_GSS_get_cred_cache(void);
int pg_GSS_load_servicename(PGconn *conn);
#endif
diff --git a/src/interfaces/libpq/fe-secure-gssapi.c b/src/interfaces/libpq/fe-secure-gssapi.c
index 5ae10af5678..c43ab0294df 100644
--- a/src/interfaces/libpq/fe-secure-gssapi.c
+++ b/src/interfaces/libpq/fe-secure-gssapi.c
@@ -640,9 +640,9 @@ pqsecure_open_gss(PGconn *conn)
if (conn->gssdelegation && conn->gssdelegation[0] == '1')
{
- /* Acquire credentials if possible */
+ /* Attempt to acquire credentials, if we didn't already */
if (conn->gcred == GSS_C_NO_CREDENTIAL)
- (void) pg_GSS_have_cred_cache(&conn->gcred);
+ conn->gcred = pg_GSS_get_cred_cache();
/*
* We have credentials and gssdelegation is enabled, so request
--
2.52.0
^ permalink raw reply [nested|flat] 17+ messages in thread
* RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
2026-02-16 20:44 BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed PG Bug reporting form <noreply@postgresql.org>
2026-02-17 17:01 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 17:54 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-02-17 18:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 19:45 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Andres Freund <andres@anarazel.de>
2026-10-29 18:40 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-10-29 20:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-10-29 21:52 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-10-30 16:55 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
@ 2026-10-30 17:25 ` Matt Carter <Matt.Carter@twosigma.com>
2026-10-30 17:36 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 17+ messages in thread
From: Matt Carter @ 2026-10-30 17:25 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
> Tom Lane wrote:
>> So I think what we really need is more like the attached. This passes
>> our own test suite, but I wonder if you can exercise it in your own
>> environment.
v1 works here. It applies to 18.3 with line offsets and builds without warnings. These are bytes per connection still allocated after warm-up (450 connections), against the same server (gss auth + SSL, MIT krb5 1.20.1):
18.3 unpatched 17844
v1, defaults 94
v1, gssencmode=disable 15
v1, gssdelegation=1 45
v1, gssdelegation=1 gssencmode=disable 46
A 3,000-connection run with defaults has a least-squares slope of 13.5 bytes/connection after warm-up. libpq 13.11 measures 17.5 on the same server, so v1 is at or below pre-16 behavior. Every connection authenticated with GSSAPI.
gdb on gss_init_sec_context() matches your description:
- With gssdelegation unset, the authentication path never calls gss_acquire_cred(). Both gss_init_sec_context() calls get GSS_C_NO_CREDENTIAL with req_flags = GSS_C_MUTUAL_FLAG.
- With gssdelegation=1, there is one gss_acquire_cred() per connection. Both calls share it with req_flags = GSS_C_MUTUAL_FLAG | GSS_C_DELEG_FLAG, and pqDropConnection() releases it.
- With the default gssencmode, the encryption probe's credential is acquired and released inside pqsecure_open_gss(), as before. GSS_C_DELEG_FLAG is set there only when gssdelegation=1.
One limit: this server has gss_accept_delegation = off, so pg_stat_gssapi.credentials_delegated is false with or without the patch. I could confirm that libpq requests delegation, with a forwardable TGT, but not that the server receives the credentials.
Thanks for back-patching to 16. I'm happy to test the back-branch versions too.
Matt
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
2026-02-16 20:44 BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed PG Bug reporting form <noreply@postgresql.org>
2026-02-17 17:01 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 17:54 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-02-17 18:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 19:45 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Andres Freund <andres@anarazel.de>
2026-10-29 18:40 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-10-29 20:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-10-29 21:52 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-10-30 16:55 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-10-30 17:25 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
@ 2026-10-30 17:36 ` Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 17+ messages in thread
From: Tom Lane @ 2026-10-30 17:36 UTC (permalink / raw)
To: Matt Carter <Matt.Carter@twosigma.com>; +Cc: Andres Freund <andres@anarazel.de>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
Matt Carter <Matt.Carter@twosigma.com> writes:
> Tom Lane wrote:
>> So I think what we really need is more like the attached. This passes
>> our own test suite, but I wonder if you can exercise it in your own
>> environment.
> v1 works here.
Great, that gives me enough confidence to proceed. Thanks for
following through on this problem!
regards, tom lane
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
2026-02-16 20:44 BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed PG Bug reporting form <noreply@postgresql.org>
2026-02-17 17:01 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 17:54 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-02-17 18:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 19:45 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Andres Freund <andres@anarazel.de>
2026-10-29 18:40 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-10-29 20:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-10-29 21:52 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-10-30 16:55 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
@ 2026-10-30 19:00 ` Jacob Champion <jacob.champion@enterprisedb.com>
2026-10-30 19:32 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 17+ messages in thread
From: Jacob Champion @ 2026-10-30 19:00 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Matt Carter <Matt.Carter@twosigma.com>; Andres Freund <andres@anarazel.de>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
On Fri, Oct 30, 2026 at 9:55 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Thanks for testing that, but after looking closer, I believe that
> pg_GSS_continue has even more problems here. It will attempt to
> acquire credentials whether or not GSS delegation is enabled,
> contrary to the adjacent comment and unlike what happens in
> pqsecure_open_gss. And if GSS delegation is enabled, it will
> set the GSS_C_DELEG_FLAG whether or not it was able to acquire
> credentials, which seems not very sane and again different from
> the logic in pqsecure_open_gss.
Hi! I don't have any more time today to dig, but I took a partial look.
+1 for the name change to pg_GSS_get_cred_cache().
> /* Only try to acquire credentials if GSS delegation isn't disabled. */
> - if (!pg_GSS_have_cred_cache(&conn->gcred))
> - conn->gcred = GSS_C_NO_CREDENTIAL;
> -
> if (conn->gssdelegation && conn->gssdelegation[0] == '1')
> - gss_flags |= GSS_C_DELEG_FLAG;
> + {
> + if (conn->gcred == GSS_C_NO_CREDENTIAL)
> + conn->gcred = pg_GSS_get_cred_cache();
> + if (conn->gcred != GSS_C_NO_CREDENTIAL)
> + gss_flags |= GSS_C_DELEG_FLAG;
> + }
This might be equivalent logic, but I'm not sure. The current code
always requests delegation if conn->gssdelegation is true (so if
pg_GSS_get_cred_cache()/gss_acquire_cred() fail, then delegation will
still be attempted with GSS_C_NO_CREDENTIAL). The new code requests
delegation only if gss_acquire_cred() succeeds -- if that fails but
the later GSS_C_NO_CREDENTIAL attempt succeeds for some weird
GSS-specific reason, shouldn't we still try to delegate as directed?
(Maybe if gss_acquire_cred() fails, the later GSS_C_NO_CREDENTIAL
attempt is also guaranteed to fail? But if that's true then this code
should look a lot different IMO.)
Separately from that:
> + if (conn->gcred == GSS_C_NO_CREDENTIAL)
> + conn->gcred = pg_GSS_get_cred_cache();
Are there any delegation cases where we were actually expecting to
overwrite the previous, valid, credentials? I deleted
pg_GSS_continue's pg_GSS_have_cred_cache() call entirely, and the
tests still passed on my machine, which makes me worried that maybe
there's an edge case that was never recorded. (Alternatively, maybe
the call is completely unnecessary.)
Thanks,
--Jacob
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
2026-02-16 20:44 BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed PG Bug reporting form <noreply@postgresql.org>
2026-02-17 17:01 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 17:54 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-02-17 18:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 19:45 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Andres Freund <andres@anarazel.de>
2026-10-29 18:40 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-10-29 20:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-10-29 21:52 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-10-30 16:55 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-10-30 19:00 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Jacob Champion <jacob.champion@enterprisedb.com>
@ 2026-10-30 19:32 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-10-30 20:09 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Jacob Champion <jacob.champion@enterprisedb.com>
0 siblings, 1 reply; 17+ messages in thread
From: Tom Lane @ 2026-10-30 19:32 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: Matt Carter <Matt.Carter@twosigma.com>; Andres Freund <andres@anarazel.de>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
Jacob Champion <jacob.champion@enterprisedb.com> writes:
> On Fri, Oct 30, 2026 at 9:55 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> /* Only try to acquire credentials if GSS delegation isn't disabled. */
>> - if (!pg_GSS_have_cred_cache(&conn->gcred))
>> - conn->gcred = GSS_C_NO_CREDENTIAL;
>> -
>> if (conn->gssdelegation && conn->gssdelegation[0] == '1')
>> - gss_flags |= GSS_C_DELEG_FLAG;
>> + {
>> + if (conn->gcred == GSS_C_NO_CREDENTIAL)
>> + conn->gcred = pg_GSS_get_cred_cache();
>> + if (conn->gcred != GSS_C_NO_CREDENTIAL)
>> + gss_flags |= GSS_C_DELEG_FLAG;
>> + }
> This might be equivalent logic, but I'm not sure.
You're right, it's intentionally not equivalent logic.
> The current code
> always requests delegation if conn->gssdelegation is true (so if
> pg_GSS_get_cred_cache()/gss_acquire_cred() fail, then delegation will
> still be attempted with GSS_C_NO_CREDENTIAL). The new code requests
> delegation only if gss_acquire_cred() succeeds -- if that fails but
> the later GSS_C_NO_CREDENTIAL attempt succeeds for some weird
> GSS-specific reason, shouldn't we still try to delegate as directed?
How can delegation succeed if there's nothing to delegate? Also,
the equivalent stanza in pqsecure_open_gss (fe-secure-gssapi.c,
lines 641-656 in master) behaves this way. I'm much more prepared
to believe that pqsecure_open_gss's behavior is intentional; for
one thing because it matches its comments, which pg_GSS_continue
didn't.
> Are there any delegation cases where we were actually expecting to
> overwrite the previous, valid, credentials?
Doubt it. There is nothing different about what a repeat call
to pg_GSS_get_cred_cache will do, so I'd expect it to fetch
credentials that are exactly like the old ones, modulo perhaps
a slightly newer expiration time.
> I deleted
> pg_GSS_continue's pg_GSS_have_cred_cache() call entirely, and the
> tests still passed on my machine, which makes me worried that maybe
> there's an edge case that was never recorded. (Alternatively, maybe
> the call is completely unnecessary.)
I wondered about that too. I tried running the src/test/kerberos
tests with --enable-coverage, and found that they reach every call
of pg_GSS_get_cred_cache *except* the one in pqsecure_open_gss ---
but that's because we already have credentials there, as evidenced by
the fact that the "gss_flags |= GSS_C_DELEG_FLAG" line is reached.
So we do seem to need pg_GSS_continue's call.
Also, if you look at 6633cfb21, this whole stanza in pg_GSS_continue
was introduced then; before that we never passed credentials nor
GSS_C_DELEG_FLAG to that call of gss_init_sec_context, or the other
one either. So I have to think that's the intended behavior in
non-delegation mode. In any case, it does not seem to make sense
to be passing different parameters to successive calls of
gss_init_sec_context.
But I'm mostly unfamiliar with the details of GSS, so it's possible
I'm all wet here.
regards, tom lane
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
2026-02-16 20:44 BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed PG Bug reporting form <noreply@postgresql.org>
2026-02-17 17:01 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 17:54 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-02-17 18:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 19:45 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Andres Freund <andres@anarazel.de>
2026-10-29 18:40 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-10-29 20:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-10-29 21:52 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-10-30 16:55 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-10-30 19:00 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Jacob Champion <jacob.champion@enterprisedb.com>
2026-10-30 19:32 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
@ 2026-10-30 20:09 ` Jacob Champion <jacob.champion@enterprisedb.com>
2026-10-30 21:04 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 17+ messages in thread
From: Jacob Champion @ 2026-10-30 20:09 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Matt Carter <Matt.Carter@twosigma.com>; Andres Freund <andres@anarazel.de>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
On Fri, Oct 30, 2026 at 12:32 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > The current code
> > always requests delegation if conn->gssdelegation is true (so if
> > pg_GSS_get_cred_cache()/gss_acquire_cred() fail, then delegation will
> > still be attempted with GSS_C_NO_CREDENTIAL). The new code requests
> > delegation only if gss_acquire_cred() succeeds -- if that fails but
> > the later GSS_C_NO_CREDENTIAL attempt succeeds for some weird
> > GSS-specific reason, shouldn't we still try to delegate as directed?
>
> How can delegation succeed if there's nothing to delegate?
IIUC, GSS_C_NO_CREDENTIAL doesn't actually mean "no credentials" to
gss_init_sec_context(), it means "get the default credentials from
somewhere magic." I don't know enough about the API to know whether
the gss_acquire_cred() call we're making separately is equivalent to
that magic behavior.
> Also,
> the equivalent stanza in pqsecure_open_gss (fe-secure-gssapi.c,
> lines 641-656 in master) behaves this way. I'm much more prepared
> to believe that pqsecure_open_gss's behavior is intentional; for
> one thing because it matches its comments, which pg_GSS_continue
> didn't.
Hmm, okay.
> > Are there any delegation cases where we were actually expecting to
> > overwrite the previous, valid, credentials?
>
> Doubt it. There is nothing different about what a repeat call
> to pg_GSS_get_cred_cache will do, so I'd expect it to fetch
> credentials that are exactly like the old ones, modulo perhaps
> a slightly newer expiration time.
I was worried about cases like pg_store_delegated_credential(), where
we manipulate the default creds in memory. But if we never do anything
like that in between client calls to pg_GSS_get_cred_cache(), then
it's probably not a concern.
> I tried running the src/test/kerberos
> tests with --enable-coverage, and found that they reach every call
> of pg_GSS_get_cred_cache *except* the one in pqsecure_open_gss ---
> but that's because we already have credentials there, as evidenced by
> the fact that the "gss_flags |= GSS_C_DELEG_FLAG" line is reached.
> So we do seem to need pg_GSS_continue's call.
Huh. I'll try to take a closer look when I can.
> In any case, it does not seem to make sense
> to be passing different parameters to successive calls of
> gss_init_sec_context.
Agreed [1]:
The initiator should not change any of the input
parameters to GSS_Init_sec_context() between calls to it during the
loop, with the exception of the input_token parameter, which will
contain a message from the acceptor after the initial call, and the
input_context_handle, which must be the result returned in the
output_context_handle of the previous call to GSS_Init_sec_context()
(GSS_C_NO_CONTEXT for the first call). RFC 2743 only
requires that the claimant_cred_handle argument remain constant over
all calls in the loop, but the other non-excepted arguments should
also remain fixed for reliable operation.
Thanks,
--Jacob
[1] https://www.rfc-editor.org/info/rfc7546/#section-3
^ permalink raw reply [nested|flat] 17+ messages in thread
* Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
2026-02-16 20:44 BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed PG Bug reporting form <noreply@postgresql.org>
2026-02-17 17:01 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 17:54 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-02-17 18:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 19:45 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Andres Freund <andres@anarazel.de>
2026-10-29 18:40 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-10-29 20:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-10-29 21:52 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-10-30 16:55 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-10-30 19:00 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Jacob Champion <jacob.champion@enterprisedb.com>
2026-10-30 19:32 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-10-30 20:09 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Jacob Champion <jacob.champion@enterprisedb.com>
@ 2026-10-30 21:04 ` Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 17+ messages in thread
From: Tom Lane @ 2026-10-30 21:04 UTC (permalink / raw)
To: Jacob Champion <jacob.champion@enterprisedb.com>; +Cc: Matt Carter <Matt.Carter@twosigma.com>; Andres Freund <andres@anarazel.de>; pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
Jacob Champion <jacob.champion@enterprisedb.com> writes:
> On Fri, Oct 30, 2026 at 12:32 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> How can delegation succeed if there's nothing to delegate?
> IIUC, GSS_C_NO_CREDENTIAL doesn't actually mean "no credentials" to
> gss_init_sec_context(), it means "get the default credentials from
> somewhere magic." I don't know enough about the API to know whether
> the gss_acquire_cred() call we're making separately is equivalent to
> that magic behavior.
Oh, that's very interesting!
If you look back at v14, there is just one call to
pg_GSS_have_cred_cache, and AFAICS it happens unconditionally if
we attempt GSS encryption, and we don't reach any other
GSS code if it fails.
In the code as it stands, my coverage result says it's possible to
reach pg_GSS_continue without having already obtained credentials, but
I don't quite see how. select_next_encryption_method doesn't look
like it will permit using ENC_GSSAPI unless we obtain credentials.
Too bad this code is so poorly documented. It's really pretty unclear
what the intent is.
regards, tom lane
^ permalink raw reply [nested|flat] 17+ messages in thread
* RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
2026-02-16 20:44 BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed PG Bug reporting form <noreply@postgresql.org>
2026-02-17 17:01 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 17:54 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-02-17 18:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
@ 2026-02-17 20:54 ` Matt Carter <Matt.Carter@twosigma.com>
2026-02-17 23:12 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
1 sibling, 1 reply; 17+ messages in thread
From: Matt Carter @ 2026-02-17 20:54 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Please try varying the connection type and encryption.
Good idea. Here's what I found:
Test 1:
Configuration: Original (SQLAlchemy + NullPool)
libpq: 16.0.7
SQLAlchemy: Yes
Leak Rate: 803 KB/min
Test 2:
Configuration: Pure psycopg2 (default)
libpq: 16.0.7
SQLAlchemy: NO
Leak Rate: 801 KB/min
Test 3:
Configuration: Pure psycopg2 + gssencmode=disable
libpq: 16.0.7
SQLAlchemy: NO
Leak Rate: 858 KB/min
Test 4:
Configuration: Pure psycopg2 + SSL only
libpq: 16.0.7
SQLAlchemy: NO
Leak Rate: 861 KB/min
Test 5:
Configuration: Pure psycopg2
libpq: 13.0.11
SQLAlchemy: NO
Leak Rate: 17 KB/min
So, it seems none of these changes avoids the leak:
- Removing SQLAlchemy
- Disabling GSSAPI encryption
- Using SSL-only
- Changing PostgreSQL DB Server version from 16 to 13
The only changes that I found that avoid the leak are:
- Changing libpq version from 16 to 13, or
- Changing psycopg version from 2 to 3.
Tom, since your C test showed no leak, the issue is likely in how psycopg2 calls libpq, not pure libpq itself.
I guess I should report this to the psycopg2 project.
^ permalink raw reply [nested|flat] 17+ messages in thread
* RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
2026-02-16 20:44 BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed PG Bug reporting form <noreply@postgresql.org>
2026-02-17 17:01 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 17:54 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
2026-02-17 18:47 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 20:54 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <Matt.Carter@twosigma.com>
@ 2026-02-17 23:12 ` Matt Carter <Matt.Carter@twosigma.com>
0 siblings, 0 replies; 17+ messages in thread
From: Matt Carter @ 2026-02-17 23:12 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-bugs@lists.postgresql.org <pgsql-bugs@lists.postgresql.org>
Matt Carter wrote:
> Tom, since your C test showed no leak, the issue is likely in how psycopg2 calls libpq, not pure libpq itself.
FYI, I have reported the issue to the psycopg project:
https://github.com/psycopg/psycopg2/issues/1827
Thanks you for your help, Tom. Your test results helped show that this is likely a psycopg2 issue.
Cheers,
Matt
^ permalink raw reply [nested|flat] 17+ messages in thread
end of thread, other threads:[~2026-10-30 21:04 UTC | newest]
Thread overview: 17+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-02-16 20:44 BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed PG Bug reporting form <noreply@postgresql.org>
2026-02-17 17:01 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 17:54 ` Matt Carter <Matt.Carter@twosigma.com>
2026-02-17 18:47 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 19:45 ` Andres Freund <andres@anarazel.de>
2026-10-29 18:40 ` Matt Carter <Matt.Carter@twosigma.com>
2026-10-29 20:47 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-10-29 21:52 ` Matt Carter <Matt.Carter@twosigma.com>
2026-10-30 16:55 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-10-30 17:25 ` Matt Carter <Matt.Carter@twosigma.com>
2026-10-30 17:36 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-10-30 19:00 ` Jacob Champion <jacob.champion@enterprisedb.com>
2026-10-30 19:32 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-10-30 20:09 ` Jacob Champion <jacob.champion@enterprisedb.com>
2026-10-30 21:04 ` Tom Lane <tgl@sss.pgh.pa.us>
2026-02-17 20:54 ` Matt Carter <Matt.Carter@twosigma.com>
2026-02-17 23:12 ` Matt Carter <Matt.Carter@twosigma.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox