pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedProbable memory leak with ECPG and AIX
20+ messages / 6 participants
[nested] [flat]
* Probable memory leak with ECPG and AIX
@ 2021-12-10 14:40 Guillaume Lelarge <guillaume@lelarge.info>
2021-12-11 06:52 ` Re: Probable memory leak with ECPG and AIX Justin Pryzby <pryzby@telsasoft.com>
2021-12-12 07:34 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
0 siblings, 2 replies; 20+ messages in thread
From: Guillaume Lelarge @ 2021-12-10 14:40 UTC (permalink / raw)
To: pgsql-hackers; +Cc: Dr. Michael Meskes <michael.meskes@credativ.com>
Hello,
Our customer thinks he has found a memory leak on ECPG and AIX.
The code is quite simple. It declares a cursor, opens it, and fetches the
only line available in the table many times. After some time, the client
crashes with a segfault error. According to him, it consumed around 256MB.
What's weird is that it works great on Linux, but crashed on AIX. One
coworker thought it could be the compiler. Our customer used cc, but he
also tried with gcc, and got the same error.
The test case is attached (testcase.pgc is the ECPG code, testcase.sh is
what our customer used to precompile and compile his code). Do you have any
idea why that happens on AIX?
Two queries to create the table and populate it with a single record:
CREATE TABLE foo(
key integer PRIMARY KEY,
value character varying(20)
);
INSERT INTO foo values (1, 'one');
Thanks.
Regards.
--
Guillaume.
Attachments:
[application/octet-stream] testcase.pgc (946B, ../../CAECtzeWMVW=b8=DJje_4F=OG9BeYLh-phLcCxZ-FBpii+O=SBQ@mail.gmail.com/3-testcase.pgc)
download
[application/x-shellscript] testcase.sh (219B, ../../CAECtzeWMVW=b8=DJje_4F=OG9BeYLh-phLcCxZ-FBpii+O=SBQ@mail.gmail.com/4-testcase.sh)
download
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Probable memory leak with ECPG and AIX
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
@ 2021-12-11 06:52 ` Justin Pryzby <pryzby@telsasoft.com>
2021-12-11 07:49 ` Re: Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
1 sibling, 1 reply; 20+ messages in thread
From: Justin Pryzby @ 2021-12-11 06:52 UTC (permalink / raw)
To: Guillaume Lelarge <guillaume@lelarge.info>; +Cc: pgsql-hackers@postgresql.org, "Dr. Michael Meskes" <michael.meskes@credativ.com>
On Fri, Dec 10, 2021 at 03:40:50PM +0100, Guillaume Lelarge wrote:
> Hello,
>
> Our customer thinks he has found a memory leak on ECPG and AIX.
>
> The code is quite simple. It declares a cursor, opens it, and fetches the
> only line available in the table many times. After some time, the client
> crashes with a segfault error. According to him, it consumed around 256MB.
> What's weird is that it works great on Linux, but crashed on AIX. One
> coworker thought it could be the compiler. Our customer used cc, but he
> also tried with gcc, and got the same error.
A memory leak isn't the same as a segfault (although I don't know how AIX
responds to OOM).
Can you show that it's a memory leak ? Show RAM use increasing continuously
and linearly with loop count.
How many loops does it take to crash ?
Could you obtain a backtrace ?
--
Justin
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Probable memory leak with ECPG and AIX
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2021-12-11 06:52 ` Re: Probable memory leak with ECPG and AIX Justin Pryzby <pryzby@telsasoft.com>
@ 2021-12-11 07:49 ` Guillaume Lelarge <guillaume@lelarge.info>
0 siblings, 0 replies; 20+ messages in thread
From: Guillaume Lelarge @ 2021-12-11 07:49 UTC (permalink / raw)
To: Justin Pryzby <pryzby@telsasoft.com>; +Cc: pgsql-hackers; Dr. Michael Meskes <michael.meskes@credativ.com>
Le sam. 11 déc. 2021 à 07:52, Justin Pryzby <pryzby@telsasoft.com> a écrit :
> On Fri, Dec 10, 2021 at 03:40:50PM +0100, Guillaume Lelarge wrote:
> > Hello,
> >
> > Our customer thinks he has found a memory leak on ECPG and AIX.
> >
> > The code is quite simple. It declares a cursor, opens it, and fetches the
> > only line available in the table many times. After some time, the client
> > crashes with a segfault error. According to him, it consumed around
> 256MB.
> > What's weird is that it works great on Linux, but crashed on AIX. One
> > coworker thought it could be the compiler. Our customer used cc, but he
> > also tried with gcc, and got the same error.
>
> A memory leak isn't the same as a segfault (although I don't know how AIX
> responds to OOM).
>
> Can you show that it's a memory leak ? Show RAM use increasing
> continuously
> and linearly with loop count.
>
> How many loops does it take to crash ?
>
> Could you obtain a backtrace ?
>
>
Thanks. I'll try to get all these informations, but it won't be before
monday.
--
Guillaume.
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Probable memory leak with ECPG and AIX
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
@ 2021-12-12 07:34 ` Noah Misch <noah@leadboat.com>
2021-12-13 13:07 ` Re: Probable memory leak with ECPG and AIX talk to ben <blo.talkto@gmail.com>
1 sibling, 1 reply; 20+ messages in thread
From: Noah Misch @ 2021-12-12 07:34 UTC (permalink / raw)
To: Guillaume Lelarge <guillaume@lelarge.info>; +Cc: pgsql-hackers; Dr. Michael Meskes <michael.meskes@credativ.com>
On Fri, Dec 10, 2021 at 03:40:50PM +0100, Guillaume Lelarge wrote:
> After some time, the client
> crashes with a segfault error. According to him, it consumed around 256MB.
> What's weird is that it works great on Linux, but crashed on AIX.
That almost certainly means he's using a 32-bit binary with the default heap
size. To use more heap on AIX, build 64-bit or override the heap size. For
example, "env LDR_CNTRL=MAXDATA=0x80000000 ./a.out" gives 2GiB of heap. See
https://www.postgresql.org/docs/devel/installation-platform-notes.html#INSTALLATION-NOTES-AIX
for more ways to control heap size. While that documentation focuses on the
server, the same techniques apply to clients like your test program.
That said, I don't know why your test program reaches 256MB on AIX. On
GNU/Linux, it uses a lot less. What version of PostgreSQL provided your
client libraries?
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Probable memory leak with ECPG and AIX
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2021-12-12 07:34 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
@ 2021-12-13 13:07 ` talk to ben <blo.talkto@gmail.com>
2021-12-15 15:20 ` Re: Probable memory leak with ECPG and AIX Benoit Lobréau <benoit.lobreau@gmail.com>
0 siblings, 1 reply; 20+ messages in thread
From: talk to ben @ 2021-12-13 13:07 UTC (permalink / raw)
To: Noah Misch <noah@leadboat.com>; +Cc: Guillaume Lelarge <guillaume@lelarge.info>; pgsql-hackers; Dr. Michael Meskes <michael.meskes@credativ.com>
Hi,
(I work with Guillaume on this case.)
On Sun, Dec 12, 2021 at 8:34 AM Noah Misch <noah@leadboat.com> wrote:
> That almost certainly means he's using a 32-bit binary with the default
> heap
> size. To use more heap on AIX, build 64-bit or override the heap size.
> For
> example, "env LDR_CNTRL=MAXDATA=0x80000000 ./a.out" gives 2GiB of heap.
> See
>
> https://www.postgresql.org/docs/devel/installation-platform-notes.html#INSTALLATION-NOTES-AIX
> for more ways to control heap size. While that documentation focuses on
> the
> server, the same techniques apply to clients like your test program.
>
> That said, I don't know why your test program reaches 256MB on AIX. On
> GNU/Linux, it uses a lot less. What version of PostgreSQL provided your
> client libraries?
>
They use a 12.3 in production but have also tested on a 12.9 with the same
result.
We relayed your suggestion and will get back to you with the results.
Thanks for the input !
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Probable memory leak with ECPG and AIX
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2021-12-12 07:34 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2021-12-13 13:07 ` Re: Probable memory leak with ECPG and AIX talk to ben <blo.talkto@gmail.com>
@ 2021-12-15 15:20 ` Benoit Lobréau <benoit.lobreau@gmail.com>
2022-01-01 07:40 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
0 siblings, 1 reply; 20+ messages in thread
From: Benoit Lobréau @ 2021-12-15 15:20 UTC (permalink / raw)
To: Noah Misch <noah@leadboat.com>; +Cc: Guillaume Lelarge <guillaume@lelarge.info>; pgsql-hackers; Dr. Michael Meskes <michael.meskes@credativ.com>
Our client confirmed that the more he fetches the more memory is consumed.
The segfault was indeed caused by the absence of LDR_CNTRL.
The tests show that:
* without LDR_CNTRL, we reach 256Mb and segfault ;
* with LDR_CNTRL=MAXDATA=0x10000000, we reach 256Mo but there is no
segfault, the program just continues running ;
* with LDR_CNTRL=MAXDATA=0x80000000, we reach 2Go and there is no segfault
either, the program just continues running.
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Probable memory leak with ECPG and AIX
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2021-12-12 07:34 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2021-12-13 13:07 ` Re: Probable memory leak with ECPG and AIX talk to ben <blo.talkto@gmail.com>
2021-12-15 15:20 ` Re: Probable memory leak with ECPG and AIX Benoit Lobréau <benoit.lobreau@gmail.com>
@ 2022-01-01 07:40 ` Noah Misch <noah@leadboat.com>
2022-01-01 16:35 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 20+ messages in thread
From: Noah Misch @ 2022-01-01 07:40 UTC (permalink / raw)
To: Benoit Lobréau <benoit.lobreau@gmail.com>; +Cc: Guillaume Lelarge <guillaume@lelarge.info>; pgsql-hackers; Dr. Michael Meskes <michael.meskes@credativ.com>
On Wed, Dec 15, 2021 at 04:20:42PM +0100, Benoit Lobréau wrote:
> * with LDR_CNTRL=MAXDATA=0x10000000, we reach 256Mo but there is no
> segfault, the program just continues running ;
> * with LDR_CNTRL=MAXDATA=0x80000000, we reach 2Go and there is no segfault
> either, the program just continues running.
I get the same results. The leak arises because AIX freelocale() doesn't free
all memory allocated in newlocale(). The following program uses trivial
memory on GNU/Linux, but it leaks like you're seeing on AIX:
#include <locale.h>
int main(int argc, char **argv)
{
while (1)
freelocale(newlocale(LC_NUMERIC_MASK, "C", (locale_t) 0));
return 0;
}
If you have access to file an AIX bug, I recommend doing so. If we want
PostgreSQL to work around this, one idea is to have ECPG do this newlocale()
less often. For example, do it once per process or once per connection
instead of once per ecpg_do_prologue().
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Probable memory leak with ECPG and AIX
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2021-12-12 07:34 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2021-12-13 13:07 ` Re: Probable memory leak with ECPG and AIX talk to ben <blo.talkto@gmail.com>
2021-12-15 15:20 ` Re: Probable memory leak with ECPG and AIX Benoit Lobréau <benoit.lobreau@gmail.com>
2022-01-01 07:40 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
@ 2022-01-01 16:35 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-01-02 00:07 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
0 siblings, 1 reply; 20+ messages in thread
From: Tom Lane @ 2022-01-01 16:35 UTC (permalink / raw)
To: Noah Misch <noah@leadboat.com>; +Cc: Benoit Lobréau <benoit.lobreau@gmail.com>; Guillaume Lelarge <guillaume@lelarge.info>; pgsql-hackers; Dr. Michael Meskes <michael.meskes@credativ.com>
Noah Misch <noah@leadboat.com> writes:
> I get the same results. The leak arises because AIX freelocale() doesn't free
> all memory allocated in newlocale(). The following program uses trivial
> memory on GNU/Linux, but it leaks like you're seeing on AIX:
Bleah.
> If you have access to file an AIX bug, I recommend doing so. If we want
> PostgreSQL to work around this, one idea is to have ECPG do this newlocale()
> less often. For example, do it once per process or once per connection
> instead of once per ecpg_do_prologue().
It's worse than that: see also ECPGget_desc(). Seems like a case
could be made for doing something about this just on the basis
of cycles expended, never mind freelocale() bugs.
regards, tom lane
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Probable memory leak with ECPG and AIX
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2021-12-12 07:34 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2021-12-13 13:07 ` Re: Probable memory leak with ECPG and AIX talk to ben <blo.talkto@gmail.com>
2021-12-15 15:20 ` Re: Probable memory leak with ECPG and AIX Benoit Lobréau <benoit.lobreau@gmail.com>
2022-01-01 07:40 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-01-01 16:35 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
@ 2022-01-02 00:07 ` Noah Misch <noah@leadboat.com>
2022-03-25 09:40 ` Re: Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2022-04-18 04:16 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
0 siblings, 2 replies; 20+ messages in thread
From: Noah Misch @ 2022-01-02 00:07 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Benoit Lobréau <benoit.lobreau@gmail.com>; Guillaume Lelarge <guillaume@lelarge.info>; pgsql-hackers; Dr. Michael Meskes <michael.meskes@credativ.com>
On Sat, Jan 01, 2022 at 11:35:02AM -0500, Tom Lane wrote:
> Noah Misch <noah@leadboat.com> writes:
> > I get the same results. The leak arises because AIX freelocale() doesn't free
> > all memory allocated in newlocale(). The following program uses trivial
> > memory on GNU/Linux, but it leaks like you're seeing on AIX:
>
> Bleah.
>
> > If you have access to file an AIX bug, I recommend doing so. If we want
> > PostgreSQL to work around this, one idea is to have ECPG do this newlocale()
> > less often. For example, do it once per process or once per connection
> > instead of once per ecpg_do_prologue().
>
> It's worse than that: see also ECPGget_desc(). Seems like a case
> could be made for doing something about this just on the basis
> of cycles expended, never mind freelocale() bugs.
Agreed. Once per process seems best. I only hesitated before since it means
nothing will free this storage, which could be annoying in the context of
Valgrind and similar. However, ECPG already has bits of never-freed memory in
the form of pthread_key_create() calls having no pthread_key_delete(), so I
don't mind adding a bit more.
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Probable memory leak with ECPG and AIX
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2021-12-12 07:34 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2021-12-13 13:07 ` Re: Probable memory leak with ECPG and AIX talk to ben <blo.talkto@gmail.com>
2021-12-15 15:20 ` Re: Probable memory leak with ECPG and AIX Benoit Lobréau <benoit.lobreau@gmail.com>
2022-01-01 07:40 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-01-01 16:35 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-01-02 00:07 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
@ 2022-03-25 09:40 ` Guillaume Lelarge <guillaume@lelarge.info>
2022-03-25 13:25 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 20+ messages in thread
From: Guillaume Lelarge @ 2022-03-25 09:40 UTC (permalink / raw)
To: Noah Misch <noah@leadboat.com>; +Cc: Tom Lane <tgl@sss.pgh.pa.us>; Benoit Lobréau <benoit.lobreau@gmail.com>; pgsql-hackers; Dr. Michael Meskes <michael.meskes@credativ.com>
Le dim. 2 janv. 2022 à 01:07, Noah Misch <noah@leadboat.com> a écrit :
> On Sat, Jan 01, 2022 at 11:35:02AM -0500, Tom Lane wrote:
> > Noah Misch <noah@leadboat.com> writes:
> > > I get the same results. The leak arises because AIX freelocale()
> doesn't free
> > > all memory allocated in newlocale(). The following program uses
> trivial
> > > memory on GNU/Linux, but it leaks like you're seeing on AIX:
> >
> > Bleah.
> >
> > > If you have access to file an AIX bug, I recommend doing so. If we
> want
> > > PostgreSQL to work around this, one idea is to have ECPG do this
> newlocale()
> > > less often. For example, do it once per process or once per connection
> > > instead of once per ecpg_do_prologue().
> >
> > It's worse than that: see also ECPGget_desc(). Seems like a case
> > could be made for doing something about this just on the basis
> > of cycles expended, never mind freelocale() bugs.
>
> Agreed. Once per process seems best. I only hesitated before since it
> means
> nothing will free this storage, which could be annoying in the context of
> Valgrind and similar. However, ECPG already has bits of never-freed
> memory in
> the form of pthread_key_create() calls having no pthread_key_delete(), so I
> don't mind adding a bit more.
>
Did this get anywhere? Is there something we could do to make this move
forward?
--
Guillaume.
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Probable memory leak with ECPG and AIX
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2021-12-12 07:34 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2021-12-13 13:07 ` Re: Probable memory leak with ECPG and AIX talk to ben <blo.talkto@gmail.com>
2021-12-15 15:20 ` Re: Probable memory leak with ECPG and AIX Benoit Lobréau <benoit.lobreau@gmail.com>
2022-01-01 07:40 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-01-01 16:35 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-01-02 00:07 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-03-25 09:40 ` Re: Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
@ 2022-03-25 13:25 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-03-25 15:59 ` Re: Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
0 siblings, 1 reply; 20+ messages in thread
From: Tom Lane @ 2022-03-25 13:25 UTC (permalink / raw)
To: Guillaume Lelarge <guillaume@lelarge.info>; +Cc: Noah Misch <noah@leadboat.com>; Benoit Lobréau <benoit.lobreau@gmail.com>; pgsql-hackers; Dr. Michael Meskes <michael.meskes@credativ.com>
Guillaume Lelarge <guillaume@lelarge.info> writes:
> Did this get anywhere? Is there something we could do to make this move
> forward?
No. Write a patch?
regards, tom lane
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Probable memory leak with ECPG and AIX
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2021-12-12 07:34 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2021-12-13 13:07 ` Re: Probable memory leak with ECPG and AIX talk to ben <blo.talkto@gmail.com>
2021-12-15 15:20 ` Re: Probable memory leak with ECPG and AIX Benoit Lobréau <benoit.lobreau@gmail.com>
2022-01-01 07:40 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-01-01 16:35 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-01-02 00:07 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-03-25 09:40 ` Re: Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2022-03-25 13:25 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
@ 2022-03-25 15:59 ` Guillaume Lelarge <guillaume@lelarge.info>
0 siblings, 0 replies; 20+ messages in thread
From: Guillaume Lelarge @ 2022-03-25 15:59 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Noah Misch <noah@leadboat.com>; Benoit Lobréau <benoit.lobreau@gmail.com>; pgsql-hackers; Dr. Michael Meskes <michael.meskes@credativ.com>
Le ven. 25 mars 2022, 14:25, Tom Lane <tgl@sss.pgh.pa.us> a écrit :
> Guillaume Lelarge <guillaume@lelarge.info> writes:
> > Did this get anywhere? Is there something we could do to make this move
> > forward?
>
> No. Write a patch?
>
I wouldn't have asked if I could write such a patch :-)
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Probable memory leak with ECPG and AIX
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2021-12-12 07:34 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2021-12-13 13:07 ` Re: Probable memory leak with ECPG and AIX talk to ben <blo.talkto@gmail.com>
2021-12-15 15:20 ` Re: Probable memory leak with ECPG and AIX Benoit Lobréau <benoit.lobreau@gmail.com>
2022-01-01 07:40 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-01-01 16:35 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-01-02 00:07 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
@ 2022-04-18 04:16 ` Noah Misch <noah@leadboat.com>
2022-07-02 18:53 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
1 sibling, 1 reply; 20+ messages in thread
From: Noah Misch @ 2022-04-18 04:16 UTC (permalink / raw)
To: Guillaume Lelarge <guillaume@lelarge.info>; Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Benoit Lobréau <benoit.lobreau@gmail.com>; pgsql-hackers; Dr. Michael Meskes <michael.meskes@credativ.com>
On Sat, Jan 01, 2022 at 04:07:50PM -0800, Noah Misch wrote:
> On Sat, Jan 01, 2022 at 11:35:02AM -0500, Tom Lane wrote:
> > Noah Misch <noah@leadboat.com> writes:
> > > I get the same results. The leak arises because AIX freelocale() doesn't free
> > > all memory allocated in newlocale(). The following program uses trivial
> > > memory on GNU/Linux, but it leaks like you're seeing on AIX:
> >
> > Bleah.
> >
> > > If you have access to file an AIX bug, I recommend doing so. If we want
> > > PostgreSQL to work around this, one idea is to have ECPG do this newlocale()
> > > less often. For example, do it once per process or once per connection
> > > instead of once per ecpg_do_prologue().
> >
> > It's worse than that: see also ECPGget_desc(). Seems like a case
> > could be made for doing something about this just on the basis
> > of cycles expended, never mind freelocale() bugs.
>
> Agreed. Once per process seems best. I only hesitated before since it means
> nothing will free this storage, which could be annoying in the context of
> Valgrind and similar. However, ECPG already has bits of never-freed memory in
> the form of pthread_key_create() calls having no pthread_key_delete(), so I
> don't mind adding a bit more.
The comparison to pthread_key_create() wasn't completely fair. While POSIX
welcomes pthread_key_create() to fail with ENOMEM, the glibc implementation
appears not to allocate memory. Even so, I'm okay leaking one newlocale() per
process lifetime.
I had expected to use pthread_once() for the newlocale() call, but there would
be no useful way to report failure and try again later. Instead, I called
newlocale() while ECPGconnect() holds connections_mutex. See log message and
comments for details. I tested "./configure ac_cv_func_uselocale=no ..." and
tested the scenario of newlocale() failing every time.
Author: Noah Misch <noah@leadboat.com>
Commit: Noah Misch <noah@leadboat.com>
ecpglib: call newlocale() one per process.
ecpglib has been calling it once per SQL query and once per EXEC SQL GET
DESCRIPTOR. Instead, if newlocale() has not succeeded before, call it
while establishing a connection. This mitigates three problems:
- If newlocale() failed in EXEC SQL GET DESCRIPTOR, the command silently
proceeded without the intended locale change.
- On AIX, each newlocale()+freelocale() cycle leaked memory.
- newlocale() CPU usage may have been nontrivial.
Fail the connection attempt if newlocale() fails. Rerrange
ecpg_do_prologue() to validate the connection before its uselocale().
The sort of program that may regress is one running in an environment
where newlocale() fails. If that program establishes connections
without running SQL statements, it will stop working in response to this
change. I'm betting against the importance of such an ECPG use case.
Most SQL execution (any using ECPGdo()) has long required newlocale()
success, so there's little a connection could do without newlocale().
Back-patch to v10 (all supported versions).
Reviewed by FIXME. Reported by Guillaume Lelarge.
Discussion: https://postgr.es/m/20220101074055.GA54621@rfd.leadboat.com
diff --git a/src/interfaces/ecpg/ecpglib/connect.c b/src/interfaces/ecpg/ecpglib/connect.c
index 8dfcabe..2e4b003 100644
--- a/src/interfaces/ecpg/ecpglib/connect.c
+++ b/src/interfaces/ecpg/ecpglib/connect.c
@@ -10,6 +10,10 @@
#include "ecpgtype.h"
#include "sqlca.h"
+#ifdef HAVE_USELOCALE
+locale_t ecpg_clocale;
+#endif
+
#ifdef ENABLE_THREAD_SAFETY
static pthread_mutex_t connections_mutex = PTHREAD_MUTEX_INITIALIZER;
static pthread_key_t actual_connection_key;
@@ -502,6 +506,37 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
#ifdef ENABLE_THREAD_SAFETY
pthread_mutex_lock(&connections_mutex);
#endif
+#ifdef HAVE_USELOCALE
+ if (!ecpg_clocale)
+ {
+ ecpg_clocale = newlocale(LC_NUMERIC_MASK, "C", (locale_t) 0);
+ if (!ecpg_clocale)
+ {
+#ifdef ENABLE_THREAD_SAFETY
+ pthread_mutex_unlock(&connections_mutex);
+#endif
+ ecpg_raise(lineno, ECPG_OUT_OF_MEMORY,
+ ECPG_SQLSTATE_ECPG_OUT_OF_MEMORY, NULL);
+ if (host)
+ ecpg_free(host);
+ if (port)
+ ecpg_free(port);
+ if (options)
+ ecpg_free(options);
+ if (realname)
+ ecpg_free(realname);
+ if (dbname)
+ ecpg_free(dbname);
+ if (conn_keywords)
+ ecpg_free(conn_keywords);
+ if (conn_values)
+ ecpg_free(conn_values);
+ free(this);
+ return false;
+ }
+ }
+#endif
+
if (connection_name != NULL)
this->name = ecpg_strdup(connection_name, lineno);
else
diff --git a/src/interfaces/ecpg/ecpglib/descriptor.c b/src/interfaces/ecpg/ecpglib/descriptor.c
index 369c2f0..f1898de 100644
--- a/src/interfaces/ecpg/ecpglib/descriptor.c
+++ b/src/interfaces/ecpg/ecpglib/descriptor.c
@@ -486,9 +486,16 @@ ECPGget_desc(int lineno, const char *desc_name, int index,...)
/* since the database gives the standard decimal point */
/* (see comments in execute.c) */
#ifdef HAVE_USELOCALE
- stmt.clocale = newlocale(LC_NUMERIC_MASK, "C", (locale_t) 0);
- if (stmt.clocale != (locale_t) 0)
- stmt.oldlocale = uselocale(stmt.clocale);
+
+ /*
+ * To get here, the above PQnfields() test must have found nonzero
+ * fields. One needs a connection to create such a descriptor. (EXEC
+ * SQL SET DESCRIPTOR can populate the descriptor's "items", but it
+ * can't change the descriptor's PQnfields().) Any successful
+ * connection initializes ecpg_clocale.
+ */
+ Assert(ecpg_clocale);
+ stmt.oldlocale = uselocale(ecpg_clocale);
#else
#ifdef HAVE__CONFIGTHREADLOCALE
stmt.oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
@@ -504,8 +511,6 @@ ECPGget_desc(int lineno, const char *desc_name, int index,...)
#ifdef HAVE_USELOCALE
if (stmt.oldlocale != (locale_t) 0)
uselocale(stmt.oldlocale);
- if (stmt.clocale)
- freelocale(stmt.clocale);
#else
if (stmt.oldlocale)
{
diff --git a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
index 1a98dea..c438cfb 100644
--- a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
+++ b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
@@ -59,6 +59,10 @@ struct ECPGtype_information_cache
enum ARRAY_TYPE isarray;
};
+#ifdef HAVE_USELOCALE
+extern locale_t ecpg_clocale; /* LC_NUMERIC=C */
+#endif
+
/* structure to store one statement */
struct statement
{
@@ -73,7 +77,6 @@ struct statement
struct variable *inlist;
struct variable *outlist;
#ifdef HAVE_USELOCALE
- locale_t clocale;
locale_t oldlocale;
#else
char *oldlocale;
diff --git a/src/interfaces/ecpg/ecpglib/execute.c b/src/interfaces/ecpg/ecpglib/execute.c
index 6a7ef0b..2ebe665 100644
--- a/src/interfaces/ecpg/ecpglib/execute.c
+++ b/src/interfaces/ecpg/ecpglib/execute.c
@@ -101,10 +101,7 @@ free_statement(struct statement *stmt)
free_variable(stmt->outlist);
ecpg_free(stmt->command);
ecpg_free(stmt->name);
-#ifdef HAVE_USELOCALE
- if (stmt->clocale)
- freelocale(stmt->clocale);
-#else
+#ifndef HAVE_USELOCALE
ecpg_free(stmt->oldlocale);
#endif
ecpg_free(stmt);
@@ -1965,6 +1962,15 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
return false;
}
+#ifdef ENABLE_THREAD_SAFETY
+ ecpg_pthreads_init();
+#endif
+
+ con = ecpg_get_connection(connection_name);
+
+ if (!ecpg_init(con, connection_name, lineno))
+ return false;
+
stmt = (struct statement *) ecpg_alloc(sizeof(struct statement), lineno);
if (stmt == NULL)
@@ -1979,13 +1985,13 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
* treat that situation as if the function doesn't exist.
*/
#ifdef HAVE_USELOCALE
- stmt->clocale = newlocale(LC_NUMERIC_MASK, "C", (locale_t) 0);
- if (stmt->clocale == (locale_t) 0)
- {
- ecpg_do_epilogue(stmt);
- return false;
- }
- stmt->oldlocale = uselocale(stmt->clocale);
+
+ /*
+ * Since ecpg_init() succeeded, we have a connection. Any successful
+ * connection initializes ecpg_clocale.
+ */
+ Assert(ecpg_clocale);
+ stmt->oldlocale = uselocale(ecpg_clocale);
if (stmt->oldlocale == (locale_t) 0)
{
ecpg_do_epilogue(stmt);
@@ -2004,18 +2010,6 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
setlocale(LC_NUMERIC, "C");
#endif
-#ifdef ENABLE_THREAD_SAFETY
- ecpg_pthreads_init();
-#endif
-
- con = ecpg_get_connection(connection_name);
-
- if (!ecpg_init(con, connection_name, lineno))
- {
- ecpg_do_epilogue(stmt);
- return false;
- }
-
/*
* If statement type is ECPGst_prepnormal we are supposed to prepare the
* statement before executing them
Attachments:
[text/plain] newlocale-once-ecpg-v1.patch (6.7K, ../../20220418041645.GA1134228@rfd.leadboat.com/2-newlocale-once-ecpg-v1.patch)
download | inline diff:
Author: Noah Misch <noah@leadboat.com>
Commit: Noah Misch <noah@leadboat.com>
ecpglib: call newlocale() one per process.
ecpglib has been calling it once per SQL query and once per EXEC SQL GET
DESCRIPTOR. Instead, if newlocale() has not succeeded before, call it
while establishing a connection. This mitigates three problems:
- If newlocale() failed in EXEC SQL GET DESCRIPTOR, the command silently
proceeded without the intended locale change.
- On AIX, each newlocale()+freelocale() cycle leaked memory.
- newlocale() CPU usage may have been nontrivial.
Fail the connection attempt if newlocale() fails. Rerrange
ecpg_do_prologue() to validate the connection before its uselocale().
The sort of program that may regress is one running in an environment
where newlocale() fails. If that program establishes connections
without running SQL statements, it will stop working in response to this
change. I'm betting against the importance of such an ECPG use case.
Most SQL execution (any using ECPGdo()) has long required newlocale()
success, so there's little a connection could do without newlocale().
Back-patch to v10 (all supported versions).
Reviewed by FIXME. Reported by Guillaume Lelarge.
Discussion: https://postgr.es/m/20220101074055.GA54621@rfd.leadboat.com
diff --git a/src/interfaces/ecpg/ecpglib/connect.c b/src/interfaces/ecpg/ecpglib/connect.c
index 8dfcabe..2e4b003 100644
--- a/src/interfaces/ecpg/ecpglib/connect.c
+++ b/src/interfaces/ecpg/ecpglib/connect.c
@@ -10,6 +10,10 @@
#include "ecpgtype.h"
#include "sqlca.h"
+#ifdef HAVE_USELOCALE
+locale_t ecpg_clocale;
+#endif
+
#ifdef ENABLE_THREAD_SAFETY
static pthread_mutex_t connections_mutex = PTHREAD_MUTEX_INITIALIZER;
static pthread_key_t actual_connection_key;
@@ -502,6 +506,37 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
#ifdef ENABLE_THREAD_SAFETY
pthread_mutex_lock(&connections_mutex);
#endif
+#ifdef HAVE_USELOCALE
+ if (!ecpg_clocale)
+ {
+ ecpg_clocale = newlocale(LC_NUMERIC_MASK, "C", (locale_t) 0);
+ if (!ecpg_clocale)
+ {
+#ifdef ENABLE_THREAD_SAFETY
+ pthread_mutex_unlock(&connections_mutex);
+#endif
+ ecpg_raise(lineno, ECPG_OUT_OF_MEMORY,
+ ECPG_SQLSTATE_ECPG_OUT_OF_MEMORY, NULL);
+ if (host)
+ ecpg_free(host);
+ if (port)
+ ecpg_free(port);
+ if (options)
+ ecpg_free(options);
+ if (realname)
+ ecpg_free(realname);
+ if (dbname)
+ ecpg_free(dbname);
+ if (conn_keywords)
+ ecpg_free(conn_keywords);
+ if (conn_values)
+ ecpg_free(conn_values);
+ free(this);
+ return false;
+ }
+ }
+#endif
+
if (connection_name != NULL)
this->name = ecpg_strdup(connection_name, lineno);
else
diff --git a/src/interfaces/ecpg/ecpglib/descriptor.c b/src/interfaces/ecpg/ecpglib/descriptor.c
index 369c2f0..f1898de 100644
--- a/src/interfaces/ecpg/ecpglib/descriptor.c
+++ b/src/interfaces/ecpg/ecpglib/descriptor.c
@@ -486,9 +486,16 @@ ECPGget_desc(int lineno, const char *desc_name, int index,...)
/* since the database gives the standard decimal point */
/* (see comments in execute.c) */
#ifdef HAVE_USELOCALE
- stmt.clocale = newlocale(LC_NUMERIC_MASK, "C", (locale_t) 0);
- if (stmt.clocale != (locale_t) 0)
- stmt.oldlocale = uselocale(stmt.clocale);
+
+ /*
+ * To get here, the above PQnfields() test must have found nonzero
+ * fields. One needs a connection to create such a descriptor. (EXEC
+ * SQL SET DESCRIPTOR can populate the descriptor's "items", but it
+ * can't change the descriptor's PQnfields().) Any successful
+ * connection initializes ecpg_clocale.
+ */
+ Assert(ecpg_clocale);
+ stmt.oldlocale = uselocale(ecpg_clocale);
#else
#ifdef HAVE__CONFIGTHREADLOCALE
stmt.oldthreadlocale = _configthreadlocale(_ENABLE_PER_THREAD_LOCALE);
@@ -504,8 +511,6 @@ ECPGget_desc(int lineno, const char *desc_name, int index,...)
#ifdef HAVE_USELOCALE
if (stmt.oldlocale != (locale_t) 0)
uselocale(stmt.oldlocale);
- if (stmt.clocale)
- freelocale(stmt.clocale);
#else
if (stmt.oldlocale)
{
diff --git a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
index 1a98dea..c438cfb 100644
--- a/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
+++ b/src/interfaces/ecpg/ecpglib/ecpglib_extern.h
@@ -59,6 +59,10 @@ struct ECPGtype_information_cache
enum ARRAY_TYPE isarray;
};
+#ifdef HAVE_USELOCALE
+extern locale_t ecpg_clocale; /* LC_NUMERIC=C */
+#endif
+
/* structure to store one statement */
struct statement
{
@@ -73,7 +77,6 @@ struct statement
struct variable *inlist;
struct variable *outlist;
#ifdef HAVE_USELOCALE
- locale_t clocale;
locale_t oldlocale;
#else
char *oldlocale;
diff --git a/src/interfaces/ecpg/ecpglib/execute.c b/src/interfaces/ecpg/ecpglib/execute.c
index 6a7ef0b..2ebe665 100644
--- a/src/interfaces/ecpg/ecpglib/execute.c
+++ b/src/interfaces/ecpg/ecpglib/execute.c
@@ -101,10 +101,7 @@ free_statement(struct statement *stmt)
free_variable(stmt->outlist);
ecpg_free(stmt->command);
ecpg_free(stmt->name);
-#ifdef HAVE_USELOCALE
- if (stmt->clocale)
- freelocale(stmt->clocale);
-#else
+#ifndef HAVE_USELOCALE
ecpg_free(stmt->oldlocale);
#endif
ecpg_free(stmt);
@@ -1965,6 +1962,15 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
return false;
}
+#ifdef ENABLE_THREAD_SAFETY
+ ecpg_pthreads_init();
+#endif
+
+ con = ecpg_get_connection(connection_name);
+
+ if (!ecpg_init(con, connection_name, lineno))
+ return false;
+
stmt = (struct statement *) ecpg_alloc(sizeof(struct statement), lineno);
if (stmt == NULL)
@@ -1979,13 +1985,13 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
* treat that situation as if the function doesn't exist.
*/
#ifdef HAVE_USELOCALE
- stmt->clocale = newlocale(LC_NUMERIC_MASK, "C", (locale_t) 0);
- if (stmt->clocale == (locale_t) 0)
- {
- ecpg_do_epilogue(stmt);
- return false;
- }
- stmt->oldlocale = uselocale(stmt->clocale);
+
+ /*
+ * Since ecpg_init() succeeded, we have a connection. Any successful
+ * connection initializes ecpg_clocale.
+ */
+ Assert(ecpg_clocale);
+ stmt->oldlocale = uselocale(ecpg_clocale);
if (stmt->oldlocale == (locale_t) 0)
{
ecpg_do_epilogue(stmt);
@@ -2004,18 +2010,6 @@ ecpg_do_prologue(int lineno, const int compat, const int force_indicator,
setlocale(LC_NUMERIC, "C");
#endif
-#ifdef ENABLE_THREAD_SAFETY
- ecpg_pthreads_init();
-#endif
-
- con = ecpg_get_connection(connection_name);
-
- if (!ecpg_init(con, connection_name, lineno))
- {
- ecpg_do_epilogue(stmt);
- return false;
- }
-
/*
* If statement type is ECPGst_prepnormal we are supposed to prepare the
* statement before executing them
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Probable memory leak with ECPG and AIX
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2021-12-12 07:34 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2021-12-13 13:07 ` Re: Probable memory leak with ECPG and AIX talk to ben <blo.talkto@gmail.com>
2021-12-15 15:20 ` Re: Probable memory leak with ECPG and AIX Benoit Lobréau <benoit.lobreau@gmail.com>
2022-01-01 07:40 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-01-01 16:35 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-01-02 00:07 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-04-18 04:16 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
@ 2022-07-02 18:53 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-07-03 03:06 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
0 siblings, 1 reply; 20+ messages in thread
From: Tom Lane @ 2022-07-02 18:53 UTC (permalink / raw)
To: Noah Misch <noah@leadboat.com>; +Cc: Guillaume Lelarge <guillaume@lelarge.info>; Benoit Lobréau <benoit.lobreau@gmail.com>; pgsql-hackers; Dr. Michael Meskes <michael.meskes@credativ.com>
Noah Misch <noah@leadboat.com> writes:
> I had expected to use pthread_once() for the newlocale() call, but there would
> be no useful way to report failure and try again later. Instead, I called
> newlocale() while ECPGconnect() holds connections_mutex. See log message and
> comments for details. I tested "./configure ac_cv_func_uselocale=no ..." and
> tested the scenario of newlocale() failing every time.
This looks solid to me. The only nit I can find to pick is that I'd
have added one more comment, along the lines of
diff --git a/src/interfaces/ecpg/ecpglib/connect.c b/src/interfaces/ecpg/ecpglib/connect.c
index 9f958b822c..96f99ae072 100644
--- a/src/interfaces/ecpg/ecpglib/connect.c
+++ b/src/interfaces/ecpg/ecpglib/connect.c
@@ -508,6 +508,11 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
#ifdef ENABLE_THREAD_SAFETY
pthread_mutex_lock(&connections_mutex);
#endif
+
+ /*
+ * ... but first, make certain we have created ecpg_clocale. Rely on
+ * holding connections_mutex to ensure this is done by only one thread.
+ */
#ifdef HAVE_USELOCALE
if (!ecpg_clocale)
{
I've marked it RFC.
regards, tom lane
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Probable memory leak with ECPG and AIX
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2021-12-12 07:34 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2021-12-13 13:07 ` Re: Probable memory leak with ECPG and AIX talk to ben <blo.talkto@gmail.com>
2021-12-15 15:20 ` Re: Probable memory leak with ECPG and AIX Benoit Lobréau <benoit.lobreau@gmail.com>
2022-01-01 07:40 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-01-01 16:35 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-01-02 00:07 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-04-18 04:16 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-07-02 18:53 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
@ 2022-07-03 03:06 ` Noah Misch <noah@leadboat.com>
2022-07-03 03:37 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 20+ messages in thread
From: Noah Misch @ 2022-07-03 03:06 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Guillaume Lelarge <guillaume@lelarge.info>; Benoit Lobréau <benoit.lobreau@gmail.com>; pgsql-hackers; Dr. Michael Meskes <michael.meskes@credativ.com>
On Sat, Jul 02, 2022 at 02:53:34PM -0400, Tom Lane wrote:
> This looks solid to me. The only nit I can find to pick is that I'd
> have added one more comment, along the lines of
>
> diff --git a/src/interfaces/ecpg/ecpglib/connect.c b/src/interfaces/ecpg/ecpglib/connect.c
> index 9f958b822c..96f99ae072 100644
> --- a/src/interfaces/ecpg/ecpglib/connect.c
> +++ b/src/interfaces/ecpg/ecpglib/connect.c
> @@ -508,6 +508,11 @@ ECPGconnect(int lineno, int c, const char *name, const char *user, const char *p
> #ifdef ENABLE_THREAD_SAFETY
> pthread_mutex_lock(&connections_mutex);
> #endif
> +
> + /*
> + * ... but first, make certain we have created ecpg_clocale. Rely on
> + * holding connections_mutex to ensure this is done by only one thread.
> + */
> #ifdef HAVE_USELOCALE
> if (!ecpg_clocale)
> {
>
> I've marked it RFC.
Thanks for reviewing. Pushed with that comment. prairiedog complains[1]:
ld: common symbols not allowed with MH_DYLIB output format with the -multi_module option
connect.o definition of common _ecpg_clocale (size 4)
I bet this would fix it:
--- a/src/interfaces/ecpg/ecpglib/connect.c
+++ b/src/interfaces/ecpg/ecpglib/connect.c
@@ -11,7 +11,7 @@
#include "sqlca.h"
#ifdef HAVE_USELOCALE
-locale_t ecpg_clocale;
+locale_t ecpg_clocale = (locale_t) 0;
#endif
#ifdef ENABLE_THREAD_SAFETY
I hear[1] adding -fno-common to compiler options would also fix that. Still,
in the absence of other opinions, I'll just add the no-op initialization.
[1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prairiedog&dt=2022-07-03%2001%3A14%3A19
[2] https://gcc.gnu.org/legacy-ml/gcc/2005-06/msg00378.html
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Probable memory leak with ECPG and AIX
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2021-12-12 07:34 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2021-12-13 13:07 ` Re: Probable memory leak with ECPG and AIX talk to ben <blo.talkto@gmail.com>
2021-12-15 15:20 ` Re: Probable memory leak with ECPG and AIX Benoit Lobréau <benoit.lobreau@gmail.com>
2022-01-01 07:40 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-01-01 16:35 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-01-02 00:07 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-04-18 04:16 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-07-02 18:53 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-07-03 03:06 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
@ 2022-07-03 03:37 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-07-03 03:43 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
0 siblings, 1 reply; 20+ messages in thread
From: Tom Lane @ 2022-07-03 03:37 UTC (permalink / raw)
To: Noah Misch <noah@leadboat.com>; +Cc: Guillaume Lelarge <guillaume@lelarge.info>; Benoit Lobréau <benoit.lobreau@gmail.com>; pgsql-hackers; Dr. Michael Meskes <michael.meskes@credativ.com>
Noah Misch <noah@leadboat.com> writes:
> Thanks for reviewing. Pushed with that comment. prairiedog complains[1]:
> ld: common symbols not allowed with MH_DYLIB output format with the -multi_module option
> connect.o definition of common _ecpg_clocale (size 4)
Blah.
> I bet this would fix it:
> -locale_t ecpg_clocale;
> +locale_t ecpg_clocale = (locale_t) 0;
Hmm, I was considering suggesting that just on stylistic grounds,
but decided it was too nitpicky even for me.
Do you want me to test it on prairiedog?
> I hear[1] adding -fno-common to compiler options would also fix that.
I've got -fno-common turned on on my other macOS animals, but in
those cases I did it to detect bugs not fix them. I'm not sure
whether prairiedog's ancient toolchain has that switch at all,
or whether it behaves the same as in more recent platforms.
Still, that gcc.gnu.org message you cite is of the right era.
regards, tom lane
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Probable memory leak with ECPG and AIX
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2021-12-12 07:34 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2021-12-13 13:07 ` Re: Probable memory leak with ECPG and AIX talk to ben <blo.talkto@gmail.com>
2021-12-15 15:20 ` Re: Probable memory leak with ECPG and AIX Benoit Lobréau <benoit.lobreau@gmail.com>
2022-01-01 07:40 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-01-01 16:35 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-01-02 00:07 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-04-18 04:16 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-07-02 18:53 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-07-03 03:06 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-07-03 03:37 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
@ 2022-07-03 03:43 ` Noah Misch <noah@leadboat.com>
2022-07-03 03:45 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 20+ messages in thread
From: Noah Misch @ 2022-07-03 03:43 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Guillaume Lelarge <guillaume@lelarge.info>; Benoit Lobréau <benoit.lobreau@gmail.com>; pgsql-hackers; Dr. Michael Meskes <michael.meskes@credativ.com>
On Sat, Jul 02, 2022 at 11:37:08PM -0400, Tom Lane wrote:
> Noah Misch <noah@leadboat.com> writes:
> > Thanks for reviewing. Pushed with that comment. prairiedog complains[1]:
> > ld: common symbols not allowed with MH_DYLIB output format with the -multi_module option
> > connect.o definition of common _ecpg_clocale (size 4)
>
> Blah.
>
> > I bet this would fix it:
>
> > -locale_t ecpg_clocale;
> > +locale_t ecpg_clocale = (locale_t) 0;
>
> Hmm, I was considering suggesting that just on stylistic grounds,
> but decided it was too nitpicky even for me.
> Do you want me to test it on prairiedog?
Sure, if it's easy enough. If not, I'm 87% sure it will suffice.
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Probable memory leak with ECPG and AIX
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2021-12-12 07:34 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2021-12-13 13:07 ` Re: Probable memory leak with ECPG and AIX talk to ben <blo.talkto@gmail.com>
2021-12-15 15:20 ` Re: Probable memory leak with ECPG and AIX Benoit Lobréau <benoit.lobreau@gmail.com>
2022-01-01 07:40 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-01-01 16:35 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-01-02 00:07 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-04-18 04:16 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-07-02 18:53 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-07-03 03:06 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-07-03 03:37 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-07-03 03:43 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
@ 2022-07-03 03:45 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-07-03 03:59 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 20+ messages in thread
From: Tom Lane @ 2022-07-03 03:45 UTC (permalink / raw)
To: Noah Misch <noah@leadboat.com>; +Cc: Guillaume Lelarge <guillaume@lelarge.info>; Benoit Lobréau <benoit.lobreau@gmail.com>; pgsql-hackers; Dr. Michael Meskes <michael.meskes@credativ.com>
Noah Misch <noah@leadboat.com> writes:
> On Sat, Jul 02, 2022 at 11:37:08PM -0400, Tom Lane wrote:
>> Do you want me to test it on prairiedog?
> Sure, if it's easy enough. If not, I'm 87% sure it will suffice.
On it now, but it'll take a few minutes :-(
regards, tom lane
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Probable memory leak with ECPG and AIX
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2021-12-12 07:34 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2021-12-13 13:07 ` Re: Probable memory leak with ECPG and AIX talk to ben <blo.talkto@gmail.com>
2021-12-15 15:20 ` Re: Probable memory leak with ECPG and AIX Benoit Lobréau <benoit.lobreau@gmail.com>
2022-01-01 07:40 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-01-01 16:35 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-01-02 00:07 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-04-18 04:16 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-07-02 18:53 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-07-03 03:06 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-07-03 03:37 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-07-03 03:43 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-07-03 03:45 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
@ 2022-07-03 03:59 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-07-04 05:59 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
0 siblings, 1 reply; 20+ messages in thread
From: Tom Lane @ 2022-07-03 03:59 UTC (permalink / raw)
To: Noah Misch <noah@leadboat.com>; +Cc: Guillaume Lelarge <guillaume@lelarge.info>; Benoit Lobréau <benoit.lobreau@gmail.com>; pgsql-hackers; Dr. Michael Meskes <michael.meskes@credativ.com>
I wrote:
> On it now, but it'll take a few minutes :-(
Confirmed that either initializing ecpg_clocale or adding -fno-common
allows the ecpglib build to succeed. (Testing it beyond that would
require another hour or so to build the rest of the system, so I won't.)
As I said, I was already leaning to the idea that initializing the
variable explicitly is better style, so I recommend we do that.
regards, tom lane
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: Probable memory leak with ECPG and AIX
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2021-12-12 07:34 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2021-12-13 13:07 ` Re: Probable memory leak with ECPG and AIX talk to ben <blo.talkto@gmail.com>
2021-12-15 15:20 ` Re: Probable memory leak with ECPG and AIX Benoit Lobréau <benoit.lobreau@gmail.com>
2022-01-01 07:40 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-01-01 16:35 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-01-02 00:07 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-04-18 04:16 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-07-02 18:53 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-07-03 03:06 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-07-03 03:37 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-07-03 03:43 ` Re: Probable memory leak with ECPG and AIX Noah Misch <noah@leadboat.com>
2022-07-03 03:45 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
2022-07-03 03:59 ` Re: Probable memory leak with ECPG and AIX Tom Lane <tgl@sss.pgh.pa.us>
@ 2022-07-04 05:59 ` Noah Misch <noah@leadboat.com>
0 siblings, 0 replies; 20+ messages in thread
From: Noah Misch @ 2022-07-04 05:59 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: Guillaume Lelarge <guillaume@lelarge.info>; Benoit Lobréau <benoit.lobreau@gmail.com>; pgsql-hackers; Dr. Michael Meskes <michael.meskes@credativ.com>
On Sat, Jul 02, 2022 at 11:59:58PM -0400, Tom Lane wrote:
> Confirmed that either initializing ecpg_clocale or adding -fno-common
> allows the ecpglib build to succeed. (Testing it beyond that would
> require another hour or so to build the rest of the system, so I won't.)
>
> As I said, I was already leaning to the idea that initializing the
> variable explicitly is better style, so I recommend we do that.
Works for me. Pushed that way, and things have been clean.
^ permalink raw reply [nested|flat] 20+ messages in thread
end of thread, other threads:[~2022-07-04 05:59 UTC | newest]
Thread overview: 20+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-12-10 14:40 Probable memory leak with ECPG and AIX Guillaume Lelarge <guillaume@lelarge.info>
2021-12-11 06:52 ` Justin Pryzby <pryzby@telsasoft.com>
2021-12-11 07:49 ` Guillaume Lelarge <guillaume@lelarge.info>
2021-12-12 07:34 ` Noah Misch <noah@leadboat.com>
2021-12-13 13:07 ` talk to ben <blo.talkto@gmail.com>
2021-12-15 15:20 ` Benoit Lobréau <benoit.lobreau@gmail.com>
2022-01-01 07:40 ` Noah Misch <noah@leadboat.com>
2022-01-01 16:35 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-01-02 00:07 ` Noah Misch <noah@leadboat.com>
2022-03-25 09:40 ` Guillaume Lelarge <guillaume@lelarge.info>
2022-03-25 13:25 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-03-25 15:59 ` Guillaume Lelarge <guillaume@lelarge.info>
2022-04-18 04:16 ` Noah Misch <noah@leadboat.com>
2022-07-02 18:53 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-07-03 03:06 ` Noah Misch <noah@leadboat.com>
2022-07-03 03:37 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-07-03 03:43 ` Noah Misch <noah@leadboat.com>
2022-07-03 03:45 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-07-03 03:59 ` Tom Lane <tgl@sss.pgh.pa.us>
2022-07-04 05:59 ` Noah Misch <noah@leadboat.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox