pg.ddx.io pgsql-general@postgresql.org mailing list archive
help / color / mirror / Atom feedvalgrind a background worker
8+ messages / 3 participants
[nested] [flat]
* valgrind a background worker
@ 2023-02-09 23:05 Jon Erdman <jon@thewickedtribe.net>
0 siblings, 1 reply; 8+ messages in thread
From: Jon Erdman @ 2023-02-09 23:05 UTC (permalink / raw)
To: pgsql-general
Hi,
I've got a background worker that has a slow memory leak in it
somewhere. How can I get it to start under valgrind to get a memcheck
output from it?
--
Jon Erdman (aka StuckMojo)
PostgreSQL Zealot
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: valgrind a background worker
@ 2023-02-10 15:03 Tom Lane <tgl@sss.pgh.pa.us>
parent: Jon Erdman <jon@thewickedtribe.net>
0 siblings, 2 replies; 8+ messages in thread
From: Tom Lane @ 2023-02-10 15:03 UTC (permalink / raw)
To: Jon Erdman <jon@thewickedtribe.net>; +Cc: pgsql-general
=?UTF-8?Q?Jon_Erdman?= <jon@thewickedtribe.net> writes:
> I've got a background worker that has a slow memory leak in it
> somewhere. How can I get it to start under valgrind to get a memcheck
> output from it?
You have to valgrind the whole cluster AFAIK. Basically, start
the postmaster under valgrind with --trace-children=yes.
For leak tracking you probably also want
--leak-check=full --track-origins=yes --read-var-info=yes
regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: valgrind a background worker
@ 2023-02-10 18:05 Jon Erdman <jon@thewickedtribe.net>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 8+ messages in thread
From: Jon Erdman @ 2023-02-10 18:05 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-general
Aha! Thanks Tom! This will be my first time using valgrind, so I was unaware of trace-children, and getting it to attach to forked stuff was exactly where I thought the difficulty would lie.
This is great, much appreciated!
I’m suspecting that the leak might be coming from initStringInfo(), as I see a palloc() in there and no associated pfree() in my background worker’s code, but looking at the elog backend code, it looks like maybe you only have to explicitly free buf if you relocate it larger?
--
Jon Erdman
> On Feb 10, 2023, at 9:03 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> =?UTF-8?Q?Jon_Erdman?= <jon@thewickedtribe.net> writes:
>> I've got a background worker that has a slow memory leak in it
>> somewhere. How can I get it to start under valgrind to get a memcheck
>> output from it?
>
> You have to valgrind the whole cluster AFAIK. Basically, start
> the postmaster under valgrind with --trace-children=yes.
> For leak tracking you probably also want
> --leak-check=full --track-origins=yes --read-var-info=yes
>
> regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: valgrind a background worker
@ 2023-02-10 19:56 Tom Lane <tgl@sss.pgh.pa.us>
parent: Jon Erdman <jon@thewickedtribe.net>
0 siblings, 0 replies; 8+ messages in thread
From: Tom Lane @ 2023-02-10 19:56 UTC (permalink / raw)
To: Jon Erdman <jon@thewickedtribe.net>; +Cc: pgsql-general
=?UTF-8?Q?Jon_Erdman?= <jon@thewickedtribe.net> writes:
> I’m suspecting that the leak might be coming from initStringInfo(), as I see a palloc() in there and no associated pfree() in my background worker’s code, but looking at the elog backend code, it looks like maybe you only have to explicitly free buf if you relocate it larger?
Usually it's more like "you need to pfree if you allocated in a long-lived
memory context". elog is working in ErrorContext which it expects will
be reset when the dust settles.
regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: valgrind a background worker
@ 2023-02-10 20:58 Jeffrey Walton <noloader@gmail.com>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 8+ messages in thread
From: Jeffrey Walton @ 2023-02-10 20:58 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Jon Erdman <jon@thewickedtribe.net>; pgsql-general
On Fri, Feb 10, 2023 at 10:04 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> =?UTF-8?Q?Jon_Erdman?= <jon@thewickedtribe.net> writes:
> > I've got a background worker that has a slow memory leak in it
> > somewhere. How can I get it to start under valgrind to get a memcheck
> > output from it?
>
> You have to valgrind the whole cluster AFAIK. Basically, start
> the postmaster under valgrind with --trace-children=yes.
> For leak tracking you probably also want
> --leak-check=full --track-origins=yes --read-var-info=yes
One additional comment... the program in question and PostgreSQL
should also be built with -g -O1 per
https://valgrind.org/docs/manual/quick-start.html . Otherwise, there's
a risk the line information will not be accurate or usable.
Jeff
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: valgrind a background worker
@ 2023-02-10 21:05 Tom Lane <tgl@sss.pgh.pa.us>
parent: Jeffrey Walton <noloader@gmail.com>
0 siblings, 2 replies; 8+ messages in thread
From: Tom Lane @ 2023-02-10 21:05 UTC (permalink / raw)
To: noloader@gmail.com; +Cc: Jon Erdman <jon@thewickedtribe.net>; pgsql-general
Jeffrey Walton <noloader@gmail.com> writes:
> On Fri, Feb 10, 2023 at 10:04 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> You have to valgrind the whole cluster AFAIK. Basically, start
>> the postmaster under valgrind with --trace-children=yes.
>> For leak tracking you probably also want
>> --leak-check=full --track-origins=yes --read-var-info=yes
> One additional comment... the program in question and PostgreSQL
> should also be built with -g -O1 per
> https://valgrind.org/docs/manual/quick-start.html . Otherwise, there's
> a risk the line information will not be accurate or usable.
Yeah. Also, you need to compile Postgres with -DUSE_VALGRIND
if you want valgrind to have any idea about palloc/pfree.
regards, tom lane
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: valgrind a background worker
@ 2023-02-11 03:08 Jon Erdman <jon@thewickedtribe.net>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 0 replies; 8+ messages in thread
From: Jon Erdman @ 2023-02-11 03:08 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; noloader@gmail.com <noloader@gmail.com>; +Cc: pgsql-general
On 2/10/23 3:05 PM, Tom Lane wrote:
> Jeffrey Walton <noloader@gmail.com> writes:
>> On Fri, Feb 10, 2023 at 10:04 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> You have to valgrind the whole cluster AFAIK. Basically, start
>>> the postmaster under valgrind with --trace-children=yes.
>>> For leak tracking you probably also want
>>> --leak-check=full --track-origins=yes --read-var-info=yes
>
>> One additional comment... the program in question and PostgreSQL
>> should also be built with -g -O1 per
>> https://valgrind.org/docs/manual/quick-start.html . Otherwise, there's
>> a risk the line information will not be accurate or usable.
>
> Yeah. Also, you need to compile Postgres with -DUSE_VALGRIND
> if you want valgrind to have any idea about palloc/pfree.
Thanks much both of you! I'll report back how it goes ;)
--
Jon Erdman (aka StuckMojo)
PostgreSQL Zealot
^ permalink raw reply [nested|flat] 8+ messages in thread
* Re: valgrind a background worker
@ 2023-03-16 06:38 Jon Erdman <jon@thewickedtribe.net>
parent: Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 0 replies; 8+ messages in thread
From: Jon Erdman @ 2023-03-16 06:38 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; noloader@gmail.com <noloader@gmail.com>; +Cc: pgsql-general
On 2/10/23 9:08 PM, Jon Erdman wrote:
> On 2/10/23 3:05 PM, Tom Lane wrote:
>> Jeffrey Walton <noloader@gmail.com> writes:
>>> On Fri, Feb 10, 2023 at 10:04 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>>> You have to valgrind the whole cluster AFAIK. Basically, start
>>>> the postmaster under valgrind with --trace-children=yes.
>>>> For leak tracking you probably also want
>>>> --leak-check=full --track-origins=yes --read-var-info=yes
>>
>>> One additional comment... the program in question and PostgreSQL
>>> should also be built with -g -O1 per
>>> https://valgrind.org/docs/manual/quick-start.html . Otherwise, there's
>>> a risk the line information will not be accurate or usable.
>>
>> Yeah. Also, you need to compile Postgres with -DUSE_VALGRIND
>> if you want valgrind to have any idea about palloc/pfree.
>
> Thanks much both of you! I'll report back how it goes ;)
FYI folks: once I got the build done properly (valgrind brew wouldn't
install on OSX) on linux, I was able to find and fix my leaks. They were
from not calling pfree on StringInfo.data.
--
Jon Erdman (aka StuckMojo)
PostgreSQL Zealot
^ permalink raw reply [nested|flat] 8+ messages in thread
end of thread, other threads:[~2023-03-16 06:38 UTC | newest]
Thread overview: 8+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-02-09 23:05 valgrind a background worker Jon Erdman <jon@thewickedtribe.net>
2023-02-10 15:03 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-02-10 18:05 ` Jon Erdman <jon@thewickedtribe.net>
2023-02-10 19:56 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-02-10 20:58 ` Jeffrey Walton <noloader@gmail.com>
2023-02-10 21:05 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-02-11 03:08 ` Jon Erdman <jon@thewickedtribe.net>
2023-03-16 06:38 ` Jon Erdman <jon@thewickedtribe.net>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox