pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedperl garbage collector
5+ messages / 3 participants
[nested] [flat]
* perl garbage collector
@ 2005-06-27 20:46 Jean-Max Reymond <jmreymond@gmail.com>
2005-06-27 21:11 ` Re: perl garbage collector Vivek Khera <vivek@khera.org>
2005-06-28 05:53 ` Re: perl garbage collector Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 2 replies; 5+ messages in thread
From: Jean-Max Reymond @ 2005-06-27 20:46 UTC (permalink / raw)
To: pgsql-performance
Hi,
I have a stored procedure written in perl and I doubt that perl's
garbage collector is working :-(
after a lot of work, postmaster has a size of 1100 Mb and I think
that the keyword "undef" has no effects.
Before tuning my procedure, does it exist a known issue, a workaround ?
--
Jean-Max Reymond
CKR Solutions Open Source
Nice France
http://www.ckr-solutions.com
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: perl garbage collector
2005-06-27 20:46 perl garbage collector Jean-Max Reymond <jmreymond@gmail.com>
@ 2005-06-27 21:11 ` Vivek Khera <vivek@khera.org>
1 sibling, 0 replies; 5+ messages in thread
From: Vivek Khera @ 2005-06-27 21:11 UTC (permalink / raw)
To: pgsql-performance
On Jun 27, 2005, at 4:46 PM, Jean-Max Reymond wrote:
> Hi,
> I have a stored procedure written in perl and I doubt that perl's
> garbage collector is working :-(
> after a lot of work, postmaster has a size of 1100 Mb and I think
> that the keyword "undef" has no effects.
> Before tuning my procedure, does it exist a known issue, a
> workaround ?
just because your application frees the memory doesn't mean that the
OS takes it back. in other words, don't confuse memory usage with
memory leakage.
Vivek Khera, Ph.D.
+1-301-869-4449 x806
Attachments:
smime.p7s (application/pkcs7-signature, 2.4K) download
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: perl garbage collector
2005-06-27 20:46 perl garbage collector Jean-Max Reymond <jmreymond@gmail.com>
@ 2005-06-28 05:53 ` Tom Lane <tgl@sss.pgh.pa.us>
2005-06-28 18:47 ` Re: perl garbage collector Jean-Max Reymond <jmreymond@gmail.com>
1 sibling, 1 reply; 5+ messages in thread
From: Tom Lane @ 2005-06-28 05:53 UTC (permalink / raw)
To: Jean-Max Reymond <jmreymond@gmail.com>; +Cc: pgsql-performance
Jean-Max Reymond <jmreymond@gmail.com> writes:
> I have a stored procedure written in perl and I doubt that perl's
> garbage collector is working :-(
> after a lot of work, postmaster has a size of 1100 Mb and I think
> that the keyword "undef" has no effects.
Check the PG list archives --- there's been previous discussion of
similar issues. I think we concluded that when Perl is built to use
its own private memory allocator, the results of that competing with
malloc are not very pretty :-(. You end up with a fragmented memory
map and no chance to give anything back to the OS.
regards, tom lane
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: perl garbage collector
2005-06-27 20:46 perl garbage collector Jean-Max Reymond <jmreymond@gmail.com>
2005-06-28 05:53 ` Re: perl garbage collector Tom Lane <tgl@sss.pgh.pa.us>
@ 2005-06-28 18:47 ` Jean-Max Reymond <jmreymond@gmail.com>
2005-06-28 22:10 ` Re: perl garbage collector Jean-Max Reymond <jmreymond@gmail.com>
0 siblings, 1 reply; 5+ messages in thread
From: Jean-Max Reymond @ 2005-06-28 18:47 UTC (permalink / raw)
To: pgsql-performance
2005/6/28, Tom Lane <tgl@sss.pgh.pa.us>:
> Jean-Max Reymond <jmreymond@gmail.com> writes:
> > I have a stored procedure written in perl and I doubt that perl's
> > garbage collector is working :-(
> > after a lot of work, postmaster has a size of 1100 Mb and I think
> > that the keyword "undef" has no effects.
>
> Check the PG list archives --- there's been previous discussion of
> similar issues. I think we concluded that when Perl is built to use
> its own private memory allocator, the results of that competing with
> malloc are not very pretty :-(. You end up with a fragmented memory
> map and no chance to give anything back to the OS.
thanks Tom for your advice. I have read the discussion but a small
test is very confusing for me.
Consider this function:
CREATE FUNCTION jmax() RETURNS integer
AS $_$use strict;
my $i=0;
for ($i=0; $i<10000;$i++) {
my $ch = "0123456789"x100000;
my $res = spi_exec_query("select * from xdb_child where
doc_id=100 and ele_id=3 ");
}
my $j=1;$_$
LANGUAGE plperlu SECURITY DEFINER;
ALTER FUNCTION public.jmax() OWNER TO postgres;
the line my $ch = "0123456789"x100000; is used to allocate 1Mb.
the line my $res = spi_exec_query("select * from xdb_child where
doc_id=100 and ele_id=3 limit 5"); simulates a query.
without spi_exec_quer, the used memory in postmaster is a constant.
So, I think that pl/perl manages correctly memory in this case.
with spi_exec_query, postmaster grows and grows until the end of the loop.
Si, it seems that spi_exec_query does not release all the memory after
each call.
For my application (in real life) afer millions of spi_exec_query, it
grows up to 1Gb :-(
--
Jean-Max Reymond
CKR Solutions Open Source
Nice France
http://www.ckr-solutions.com
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: perl garbage collector
2005-06-27 20:46 perl garbage collector Jean-Max Reymond <jmreymond@gmail.com>
2005-06-28 05:53 ` Re: perl garbage collector Tom Lane <tgl@sss.pgh.pa.us>
2005-06-28 18:47 ` Re: perl garbage collector Jean-Max Reymond <jmreymond@gmail.com>
@ 2005-06-28 22:10 ` Jean-Max Reymond <jmreymond@gmail.com>
0 siblings, 0 replies; 5+ messages in thread
From: Jean-Max Reymond @ 2005-06-28 22:10 UTC (permalink / raw)
To: pgsql-performance
2005/6/28, Jean-Max Reymond <jmreymond@gmail.com>:
> For my application (in real life) afer millions of spi_exec_query, it
> grows up to 1Gb :-(
OK, now in 2 lines:
CREATE FUNCTION jmax() RETURNS integer
AS $_$use strict;
for (my $i=0; $i<10000000;$i++) {
spi_exec_query("select 'foo'");
}
my $j=1;$_$
LANGUAGE plperlu SECURITY DEFINER
running this test and your postmaster eats a lot of memory.
it seems that there is a memory leak in spi_exec_query :-(
--
Jean-Max Reymond
CKR Solutions Open Source
Nice France
http://www.ckr-solutions.com
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2005-06-28 22:10 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2005-06-27 20:46 perl garbage collector Jean-Max Reymond <jmreymond@gmail.com>
2005-06-27 21:11 ` Vivek Khera <vivek@khera.org>
2005-06-28 05:53 ` Tom Lane <tgl@sss.pgh.pa.us>
2005-06-28 18:47 ` Jean-Max Reymond <jmreymond@gmail.com>
2005-06-28 22:10 ` Jean-Max Reymond <jmreymond@gmail.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox