public inbox for [email protected]
help / color / mirror / Atom feedRe: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
4+ messages / 3 participants
[nested] [flat]
* Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
@ 2026-02-17 18:47 Tom Lane <[email protected]>
2026-02-17 19:45 ` Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Andres Freund <[email protected]>
2026-02-17 20:54 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <[email protected]>
0 siblings, 2 replies; 4+ messages in thread
From: Tom Lane @ 2026-02-17 18:47 UTC (permalink / raw)
To: Matt Carter <[email protected]>; +Cc: [email protected] <[email protected]>
Matt Carter <[email protected]> writes:
> Tom Lane <[email protected]> wrote:
>> I think there are way too many moving parts here, and too few configuration details,
>> to allow assigning blame confidently.
> Thank you for taking the time to test this and for the feedback. Your C test showing no leak suggests the issue is specific to how psycopg2 uses libpq, not libpq itself. I apologize for not including enough environmental details. I used Kerberos/GSSAPI with SSL (TLS 1.2 connections). My connection string was: "postgresql://hostname/database" (no password, Kerberos auth).
> Your mention of "years ago libpq did leak memory while using GSSAPI encryption" is interesting because we ARE using GSSAPI/Kerberos authentication.
Interesting. I wondered about GSSAPI, but spinning up such an
environment is more work than I wanted to do on speculation.
> I can test with non-GSSAPI authentication to try to isolate that variable. I can also create a pure psycopg2 reproducer (without SQLAlchemy). I can also test whether disabling GSSAPI encryption (but keeping GSSAPI auth) changes the behavior. Would testing with GSSAPI authentication help narrow this down? I can also report this to the psycopg2 project if you think it's their issue.
Please try varying the connection type and encryption. I do suspect
this may be psycopg2's fault, but we lack enough data to pin blame
as yet.
regards, tom lane
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
2026-02-17 18:47 Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <[email protected]>
@ 2026-02-17 19:45 ` Andres Freund <[email protected]>
1 sibling, 0 replies; 4+ messages in thread
From: Andres Freund @ 2026-02-17 19:45 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Matt Carter <[email protected]>; [email protected] <[email protected]>
Hi,
On 2026-02-17 13:47:49 -0500, Tom Lane wrote:
> Matt Carter <[email protected]> writes:
> > Thank you for taking the time to test this and for the feedback. Your C test showing no leak suggests the issue is specific to how psycopg2 uses libpq, not libpq itself. I apologize for not including enough environmental details. I used Kerberos/GSSAPI with SSL (TLS 1.2 connections). My connection string was: "postgresql://hostname/database" (no password, Kerberos auth).
> > Your mention of "years ago libpq did leak memory while using GSSAPI encryption" is interesting because we ARE using GSSAPI/Kerberos authentication.
>
> Interesting. I wondered about GSSAPI, but spinning up such an
> environment is more work than I wanted to do on speculation.
Heh, understandable...
> > I can test with non-GSSAPI authentication to try to isolate that variable. I can also create a pure psycopg2 reproducer (without SQLAlchemy). I can also test whether disabling GSSAPI encryption (but keeping GSSAPI auth) changes the behavior. Would testing with GSSAPI authentication help narrow this down? I can also report this to the psycopg2 project if you think it's their issue.
>
> Please try varying the connection type and encryption. I do suspect
> this may be psycopg2's fault, but we lack enough data to pin blame
> as yet.
Matt, you could try analyzing the memory usage with heaptrack, it tends to be
pretty good at finding them even in uninstrumented builds, as long as enough
debug symbols for a backtrace are available. Often enough it'll pinpoint
where the leak is coming from quite easily (but note that it will report some
constant-sized leaks that are "intentional").
Greetings,
Andres
^ permalink raw reply [nested|flat] 4+ messages in thread
* RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
2026-02-17 18:47 Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <[email protected]>
@ 2026-02-17 20:54 ` Matt Carter <[email protected]>
2026-02-17 23:12 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <[email protected]>
1 sibling, 1 reply; 4+ messages in thread
From: Matt Carter @ 2026-02-17 20:54 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: [email protected] <[email protected]>
Tom Lane <[email protected]> wrote:
> Please try varying the connection type and encryption.
Good idea. Here's what I found:
Test 1:
Configuration: Original (SQLAlchemy + NullPool)
libpq: 16.0.7
SQLAlchemy: Yes
Leak Rate: 803 KB/min
Test 2:
Configuration: Pure psycopg2 (default)
libpq: 16.0.7
SQLAlchemy: NO
Leak Rate: 801 KB/min
Test 3:
Configuration: Pure psycopg2 + gssencmode=disable
libpq: 16.0.7
SQLAlchemy: NO
Leak Rate: 858 KB/min
Test 4:
Configuration: Pure psycopg2 + SSL only
libpq: 16.0.7
SQLAlchemy: NO
Leak Rate: 861 KB/min
Test 5:
Configuration: Pure psycopg2
libpq: 13.0.11
SQLAlchemy: NO
Leak Rate: 17 KB/min
So, it seems none of these changes avoids the leak:
- Removing SQLAlchemy
- Disabling GSSAPI encryption
- Using SSL-only
- Changing PostgreSQL DB Server version from 16 to 13
The only changes that I found that avoid the leak are:
- Changing libpq version from 16 to 13, or
- Changing psycopg version from 2 to 3.
Tom, since your C test showed no leak, the issue is likely in how psycopg2 calls libpq, not pure libpq itself.
I guess I should report this to the psycopg2 project.
^ permalink raw reply [nested|flat] 4+ messages in thread
* RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed
2026-02-17 18:47 Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <[email protected]>
2026-02-17 20:54 ` RE: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Matt Carter <[email protected]>
@ 2026-02-17 23:12 ` Matt Carter <[email protected]>
0 siblings, 0 replies; 4+ messages in thread
From: Matt Carter @ 2026-02-17 23:12 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: [email protected] <[email protected]>
Matt Carter wrote:
> Tom, since your C test showed no leak, the issue is likely in how psycopg2 calls libpq, not pure libpq itself.
FYI, I have reported the issue to the psycopg project:
https://github.com/psycopg/psycopg2/issues/1827
Thanks you for your help, Tom. Your test results helped show that this is likely a psycopg2 issue.
Cheers,
Matt
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2026-02-17 23:12 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-02-17 18:47 Re: BUG #19411: libpq 16.x exhibits a memory leak when connections are repeatedly created and destroyed Tom Lane <[email protected]>
2026-02-17 19:45 ` Andres Freund <[email protected]>
2026-02-17 20:54 ` Matt Carter <[email protected]>
2026-02-17 23:12 ` Matt Carter <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox