From: jody brownell <jody.brownell@q1labs.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-performance@postgresql.org
Subject: Re: Postgres consuming way too much memory???
Date: Thu, 15 Jun 2006 13:12:37 -0300
Message-ID: <200606151312.37853.jody.brownell@q1labs.com> (raw)
In-Reply-To: <28419.1150386248@sss.pgh.pa.us>
References: <200606141618.37849.jody.brownell@q1labs.com>
<200606150901.19201.jody.brownell@q1labs.com>
<28419.1150386248@sss.pgh.pa.us>
Tom - that make sense... and fits the timeline of when the instability may have been introduced.
I use soft references in java to track these relationships. When the GC needs memory it will collect
objects referenced by soft references so I need to have this exception caught where my caches may get cleaned.
When the system is under load as it would be in this case, there references would be cleaned causing a large
number of exceptions in the pgplsql, subsequently causing the leak... hence the swift downward spiral.
The previous version of these routines used selects but due to volume of selects, performance suffered quite
a bit. I dont think I could revert now for production use... closing the connection maybe the workaround for
us for this release IF this is in fact what the problem is. Unfortunatly, I use the catch in about 20 similar
routines to reset sequences etc.... this may be painful :(
I will modify the routine to help isolate the problem. stay tuned.
BTW - the fix you mentioned .... is that targeted for 8.2? Is there a timeline for 8.2?
On Thursday 15 June 2006 12:44, Tom Lane wrote:
> "jody brownell" <jody.brownell@Q1Labs.com> writes:
> > BEGIN
> > INSERT into attacker_target_link (attacker_id, target_id) values (p_attacker, v_target);
> > v_returns_size := v_returns_size + 1;
> > v_returns[v_returns_size] := v_target;
>
> > EXCEPTION WHEN unique_violation THEN
> > -- do nothing... app cache may be out of date.
> > END;
>
> Hmm. There is a known problem that plpgsql leaks some memory when
> catching an exception:
> http://archives.postgresql.org/pgsql-hackers/2006-02/msg00885.php
>
> So if your problem case involves a whole lot of duplicates then that
> could explain the initial bloat. However, AFAIK that leakage is in
> a transaction-local memory context, so the space ought to be freed at
> transaction end. And Linux's malloc does know about giving space back
> to the kernel (unlike some platforms). So I'm not sure why you're
> seeing persistent bloat.
>
> Can you rewrite the function to not use an EXCEPTION block (perhaps
> a separate SELECT probe for each row --- note this won't be reliable
> if there are concurrent processes making insertions)? If so, does
> that fix the problem?
>
> regards, tom lane
>
>
>
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: jody.brownell@q1labs.com, tgl@sss.pgh.pa.us
Subject: Re: Postgres consuming way too much memory???
In-Reply-To: <200606151312.37853.jody.brownell@q1labs.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox