pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
BUG #16321: Memory leaks in PostmasterMain
4+ messages / 4 participants
[nested] [flat]

* BUG #16321: Memory leaks in PostmasterMain
@ 2020-03-27 03:21 PG Bug reporting form <noreply@postgresql.org>
  2020-03-27 14:52 ` Re: BUG #16321: Memory leaks in PostmasterMain Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 1 reply; 4+ messages in thread

From: PG Bug reporting form @ 2020-03-27 03:21 UTC (permalink / raw)
  To: pgsql-bugs@lists.postgresql.org; +Cc: hghwng@gmail.com

The following bug has been logged on the website:

Bug reference:      16321
Logged by:          Hugh Wang
Email address:      hghwng@gmail.com
PostgreSQL version: 12.2
Operating system:   Linux
Description:        

The argument parsing duplicates strings, but never frees them.

For example, when you pass "-D $DATA_DIR" to postmaster, postmaster
duplicates the string here:
https://github.com/postgres/postgres/blob/master/src/backend/postmaster/postmaster.c#L698

The duplicated string is passed to `SelectConfigFiles`, which does
everything except freeing the string.



^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: BUG #16321: Memory leaks in PostmasterMain
  2020-03-27 03:21 BUG #16321: Memory leaks in PostmasterMain PG Bug reporting form <noreply@postgresql.org>
@ 2020-03-27 14:52 ` Tom Lane <tgl@sss.pgh.pa.us>
  2020-03-30 02:10   ` Re: BUG #16321: Memory leaks in PostmasterMain Hugh Wang <hghwng@gmail.com>
  0 siblings, 1 reply; 4+ messages in thread

From: Tom Lane @ 2020-03-27 14:52 UTC (permalink / raw)
  To: hghwng@gmail.com; +Cc: pgsql-bugs@lists.postgresql.org

PG Bug reporting form <noreply@postgresql.org> writes:
> The argument parsing duplicates strings, but never frees them.

This hardly amounts to enough of a problem to worry about.  The
string might be leaked, or it might not, but tracking whether it
is is more trouble than it's worth.  Generally we only worry about
memory leaks if they (a) can waste a lot of memory or (b) can
repeat, and thereby accumulate to waste a lot of memory.  Surely
neither one applies to postmaster argument parsing.

> For example, when you pass "-D $DATA_DIR" to postmaster, postmaster
> duplicates the string here:
> https://github.com/postgres/postgres/blob/master/src/backend/postmaster/postmaster.c#L698
> The duplicated string is passed to `SelectConfigFiles`, which does
> everything except freeing the string.

This is a great example of a case where the cure is likely to be
worse than the disease.  SelectConfigFiles surely has little business
freeing its input string (indeed, it couldn't do so without casting
away the "const").  On the other hand, the caller doesn't really
know whether SelectConfigFiles is going to stash away a copy of the
pointer; it wouldn't be unreasonable for it to do so.  So in order
to not perhaps-leak a few dozen bytes, we'd have to make that API
more complicated and more fragile.  It's not a win.

As for why we strdup the argument in the first place, see here:

https://www.postgresql.org/message-id/flat/20121008184026.GA28752%40momjian.us

			regards, tom lane





^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: BUG #16321: Memory leaks in PostmasterMain
  2020-03-27 03:21 BUG #16321: Memory leaks in PostmasterMain PG Bug reporting form <noreply@postgresql.org>
  2020-03-27 14:52 ` Re: BUG #16321: Memory leaks in PostmasterMain Tom Lane <tgl@sss.pgh.pa.us>
@ 2020-03-30 02:10   ` Hugh Wang <hghwng@gmail.com>
  2020-03-30 11:13     ` Re: BUG #16321: Memory leaks in PostmasterMain Francisco Olarte <folarte@peoplecall.com>
  0 siblings, 1 reply; 4+ messages in thread

From: Hugh Wang @ 2020-03-30 02:10 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-bugs@lists.postgresql.org

Hi Tom,

On Fri, Mar 27, 2020 at 2:52 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

> PG Bug reporting form <noreply@postgresql.org> writes:
> > The argument parsing duplicates strings, but never frees them.
>
> This hardly amounts to enough of a problem to worry about.  The
> string might be leaked, or it might not, but tracking whether it
> is is more trouble than it's worth.  Generally we only worry about
> memory leaks if they (a) can waste a lot of memory or (b) can
> repeat, and thereby accumulate to waste a lot of memory.  Surely
> neither one applies to postmaster argument parsing.
>

Your analysis is pretty educational! If the leak is small and has low
impact, then the leak itself is not important; yet fixing the bug brings
more complexity.

However, from the perspective of automated bug finding, I think removing
the bug is beneficial. I'm trying to find bugs in PostgreSQL with
sanitizers (the leak is reported by LeakSanitizer). If the bug cannot be
fixed, LeakSanitizer stops at this shallow point, which prevents detecting
more bugs in deep logic.


> > For example, when you pass "-D $DATA_DIR" to postmaster, postmaster
> > duplicates the string here:
> >
> https://github.com/postgres/postgres/blob/master/src/backend/postmaster/postmaster.c#L698
> > The duplicated string is passed to `SelectConfigFiles`, which does
> > everything except freeing the string.
>
> This is a great example of a case where the cure is likely to be
> worse than the disease.  SelectConfigFiles surely has little business
> freeing its input string (indeed, it couldn't do so without casting
> away the "const").  On the other hand, the caller doesn't really
> know whether SelectConfigFiles is going to stash away a copy of the
> pointer; it wouldn't be unreasonable for it to do so.  So in order
> to not perhaps-leak a few dozen bytes, we'd have to make that API
> more complicated and more fragile.  It's not a win.
>
> As for why we strdup the argument in the first place, see here:
>
>
> https://www.postgresql.org/message-id/flat/20121008184026.GA28752%40momjian.us
>
>                         regards, tom lane
>

Thanks,
Hugh

^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: BUG #16321: Memory leaks in PostmasterMain
  2020-03-27 03:21 BUG #16321: Memory leaks in PostmasterMain PG Bug reporting form <noreply@postgresql.org>
  2020-03-27 14:52 ` Re: BUG #16321: Memory leaks in PostmasterMain Tom Lane <tgl@sss.pgh.pa.us>
  2020-03-30 02:10   ` Re: BUG #16321: Memory leaks in PostmasterMain Hugh Wang <hghwng@gmail.com>
@ 2020-03-30 11:13     ` Francisco Olarte <folarte@peoplecall.com>
  0 siblings, 0 replies; 4+ messages in thread

From: Francisco Olarte @ 2020-03-30 11:13 UTC (permalink / raw)
  To: Hugh Wang <hghwng@gmail.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>

On Mon, Mar 30, 2020 at 4:11 AM Hugh Wang <hghwng@gmail.com> wrote:
> On Fri, Mar 27, 2020 at 2:52 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> PG Bug reporting form <noreply@postgresql.org> writes:
>> > The argument parsing duplicates strings, but never frees them.
...
> Your analysis is pretty educational! If the leak is small and has low impact, then the leak itself is not important; yet fixing the bug brings more complexity.

Bear in mind some of this cases are not leaks. You may need the string
live for the whole program lifetime and elect to free them by exiting.

I've had similar cases with a category of my programs which work by
reading many input files and building structures in memory, which I
then use to print some reports. After that I free them by exiting the
program, which is much faster, no leak there. There are a whole lot of
programs which do this, they memory use raises and raises while they
work, and they need all of them, and they free all at once by exiting,
its a valid way to free resources when the only resource is used
memory, the OS frees it on exit.

> However, from the perspective of automated bug finding, I think removing the bug is beneficial. I'm trying to find bugs in PostgreSQL with sanitizers (the leak is reported by LeakSanitizer). If the bug cannot be fixed, LeakSanitizer stops at this shallow point, which prevents detecting more bugs in deep logic.

This may not be a bug. Doesn't leak sanitizer have a way to mark some
data as never freed? A leak is something which you should track and
are not, which may eventually lead to your memory use raising. This
doesn't seem to be the case.


Francisco Olarte.





^ permalink  raw  reply  [nested|flat] 4+ messages in thread


end of thread, other threads:[~2020-03-30 11:13 UTC | newest]

Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-03-27 03:21 BUG #16321: Memory leaks in PostmasterMain PG Bug reporting form <noreply@postgresql.org>
2020-03-27 14:52 ` Tom Lane <tgl@sss.pgh.pa.us>
2020-03-30 02:10   ` Hugh Wang <hghwng@gmail.com>
2020-03-30 11:13     ` Francisco Olarte <folarte@peoplecall.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox