agora inbox for pljava-dev@postgresql.org
help / color / mirror / Atom feed[Pljava-dev] Portal object leak
6+ messages / 0 participants
[nested] [flat]
* [Pljava-dev] Portal object leak
@ 2007-01-13 03:54
0 siblings, 1 reply; 6+ messages in thread
From: @ 2007-01-13 03:54 UTC (permalink / raw)
I have a PL/Java trigger that leaks a large number of
org.postgresql.pljava.internal.Portal objects. The leak occurs under
PostgreSQL 8.1 with PL/Java 1.3.0 and under 8.2 with the version of
PL/Java that is included with the Windows installer. I get the same
results with multiple versions of Windows (XP and Server 2003) and with
1.5 and 1.6 JVMs.
So far, my attempts to isolate the leak have been fruitless (the code
path is about 500 lines). I can say that from a pure Java perspective,
there is no obvious reason for the leak. Furthermore, the only
references to the Portal objects are JNI global references so there is
no chain of references from our code to these objects.
Does anyone know what could cause PL/Java to create Portal objects but
not clean them up? At this point, I'm not even sure what I'm looking for
and the only option I have left is to rebuild or comment out the code
"one line at a time" until I can isolate the behavior.
I don't mind posting the code in question (it's a set of auditing
triggers and supporting methods) but, as I mentioned, there's quite a
bit of it.
Thanks,
-Ryan
^ permalink raw reply [nested|flat] 6+ messages in thread
* [Pljava-dev] Portal object leak
@ 2007-01-14 22:48
parent:
0 siblings, 1 reply; 6+ messages in thread
From: @ 2007-01-14 22:48 UTC (permalink / raw)
Found one source of the problem. Filed as http://gborg.postgresql.org/
project/pljava/bugs/bugupdate.php?1624
-Ryan
On Jan 12, 2007, at 7:54 PM, Ryan Holmes wrote:
> I have a PL/Java trigger that leaks a large number of
> org.postgresql.pljava.internal.Portal objects. The leak occurs under
> PostgreSQL 8.1 with PL/Java 1.3.0 and under 8.2 with the version of
> PL/Java that is included with the Windows installer. I get the same
> results with multiple versions of Windows (XP and Server 2003) and
> with
> 1.5 and 1.6 JVMs.
>
> So far, my attempts to isolate the leak have been fruitless (the code
> path is about 500 lines). I can say that from a pure Java perspective,
> there is no obvious reason for the leak. Furthermore, the only
> references to the Portal objects are JNI global references so there is
> no chain of references from our code to these objects.
>
> Does anyone know what could cause PL/Java to create Portal objects but
> not clean them up? At this point, I'm not even sure what I'm
> looking for
> and the only option I have left is to rebuild or comment out the code
> "one line at a time" until I can isolate the behavior.
>
> I don't mind posting the code in question (it's a set of auditing
> triggers and supporting methods) but, as I mentioned, there's quite a
> bit of it.
>
> Thanks,
> -Ryan
> _______________________________________________
> Pljava-dev mailing list
> Pljava-dev at gborg.postgresql.org
> http://gborg.postgresql.org/mailman/listinfo/pljava-dev
^ permalink raw reply [nested|flat] 6+ messages in thread
* [Pljava-dev] Portal object leak
@ 2007-01-16 19:57
parent:
0 siblings, 1 reply; 6+ messages in thread
From: @ 2007-01-16 19:57 UTC (permalink / raw)
Hi Ryan,
Sorry for late reply. I find myself constantly tied up in other activities these days. If
you have the time to create a patch for this I'd be happy to review it.
Kind Regards,
Thomas Hallgren
Ryan Holmes wrote:
> Found one source of the problem. Filed as http://gborg.postgresql.org/
> project/pljava/bugs/bugupdate.php?1624
>
> -Ryan
>
> On Jan 12, 2007, at 7:54 PM, Ryan Holmes wrote:
>
>> I have a PL/Java trigger that leaks a large number of
>> org.postgresql.pljava.internal.Portal objects. The leak occurs under
>> PostgreSQL 8.1 with PL/Java 1.3.0 and under 8.2 with the version of
>> PL/Java that is included with the Windows installer. I get the same
>> results with multiple versions of Windows (XP and Server 2003) and
>> with
>> 1.5 and 1.6 JVMs.
>>
>> So far, my attempts to isolate the leak have been fruitless (the code
>> path is about 500 lines). I can say that from a pure Java perspective,
>> there is no obvious reason for the leak. Furthermore, the only
>> references to the Portal objects are JNI global references so there is
>> no chain of references from our code to these objects.
>>
>> Does anyone know what could cause PL/Java to create Portal objects but
>> not clean them up? At this point, I'm not even sure what I'm
>> looking for
>> and the only option I have left is to rebuild or comment out the code
>> "one line at a time" until I can isolate the behavior.
>>
>> I don't mind posting the code in question (it's a set of auditing
>> triggers and supporting methods) but, as I mentioned, there's quite a
>> bit of it.
>>
>> Thanks,
>> -Ryan
>> _______________________________________________
>> Pljava-dev mailing list
>> Pljava-dev at gborg.postgresql.org
>> http://gborg.postgresql.org/mailman/listinfo/pljava-dev
^ permalink raw reply [nested|flat] 6+ messages in thread
* [Pljava-dev] Portal object leak
@ 2007-01-16 23:24
parent:
0 siblings, 2 replies; 6+ messages in thread
From: @ 2007-01-16 23:24 UTC (permalink / raw)
No problem at all, thank you for getting back to me.
If you think this is on the Java side, I'll be happy to try to debug
and fix it. But if it's a problem in the native code, my C/C++ skills
are, well let's just call them "nonexistent".
I've been assuming that the problem is in the native code because
heap analysis with jhat shows that the Portal objects are JNI global
references without any referrers. Of course, this could simply mean
that the PL/Java JDBC code is failing to call a native cleanup
method. I guess I'm asking for some basic pointers to what the
problem might be in theory before I jump in.
A few specific questions:
1.) With very little C experience, am I wasting my time trying to fix
the issue? I'm just asking for your best guess.
2.) What the heck is a Portal object anyway? Is it a handle to a
result set / cursor?
3.) I noticed there was an update to SPIStatement.java and
SPIDatabaseMetaData.java about 3.5 months ago with the commit message
"Fixed some issues with Meta-data." The updates look like they might
be relevant. Are these updates included in 1.3.0 and/or in the
version of PL/Java that ships with the PG 8.2 Windows installer? I
guess the real question is, am I debugging a problem that's already
been fixed? Since I'm on Windows here and haven't used MinGW before,
I haven't even tried compiling from HEAD.
4.) Would you be open to accepting payment to fix this yourself or do
you simply not have the time? We (my company) think PL/Java is an
ideal solution for our DB logic and it seems quite robust in all but
this one area, so dedicating some of my time and/or paying for a fix
is definitely on the table.
FYI, I've worked around most of the problem by using local caching to
avoid repeated calls to DatabaseMetaData.getPrimaryKeys() and
getImportedKeys(). This solves about 80% of the leak. The other 20%
(also in the form of Portal objects) is still somewhat of a mystery
to me. It seems to happen on JDBC query executions inside the
triggers but I haven't been able to reproduce it in a clean
environment, so I'll come back to it later.
Sorry for the long email and thank you for your time,
-Ryan
On Jan 16, 2007, at 11:57 AM, Thomas Hallgren wrote:
> Hi Ryan,
> Sorry for late reply. I find myself constantly tied up in other
> activities these days. If you have the time to create a patch for
> this I'd be happy to review it.
>
> Kind Regards,
> Thomas Hallgren
>
>
>
> Ryan Holmes wrote:
>> Found one source of the problem. Filed as http://
>> gborg.postgresql.org/ project/pljava/bugs/bugupdate.php?1624
>> -Ryan
>> On Jan 12, 2007, at 7:54 PM, Ryan Holmes wrote:
>>> I have a PL/Java trigger that leaks a large number of
>>> org.postgresql.pljava.internal.Portal objects. The leak occurs under
>>> PostgreSQL 8.1 with PL/Java 1.3.0 and under 8.2 with the version of
>>> PL/Java that is included with the Windows installer. I get the same
>>> results with multiple versions of Windows (XP and Server 2003)
>>> and with
>>> 1.5 and 1.6 JVMs.
>>>
>>> So far, my attempts to isolate the leak have been fruitless (the
>>> code
>>> path is about 500 lines). I can say that from a pure Java
>>> perspective,
>>> there is no obvious reason for the leak. Furthermore, the only
>>> references to the Portal objects are JNI global references so
>>> there is
>>> no chain of references from our code to these objects.
>>>
>>> Does anyone know what could cause PL/Java to create Portal
>>> objects but
>>> not clean them up? At this point, I'm not even sure what I'm
>>> looking for
>>> and the only option I have left is to rebuild or comment out the
>>> code
>>> "one line at a time" until I can isolate the behavior.
>>>
>>> I don't mind posting the code in question (it's a set of auditing
>>> triggers and supporting methods) but, as I mentioned, there's
>>> quite a
>>> bit of it.
>>>
>>> Thanks,
>>> -Ryan
>>> _______________________________________________
>>> Pljava-dev mailing list
>>> Pljava-dev at gborg.postgresql.org
>>> http://gborg.postgresql.org/mailman/listinfo/pljava-dev
>
^ permalink raw reply [nested|flat] 6+ messages in thread
* [Pljava-dev] Portal object leak
@ 2007-01-18 03:39
parent:
1 sibling, 0 replies; 6+ messages in thread
From: @ 2007-01-18 03:39 UTC (permalink / raw)
I'm giving up on this for now as I don't have the C skills required to
debug it. I did step through the PL/Java JDBC code to look for any
obvious problems (e.g. failing to call Portal.close()) and couldn't find
any.
My guess is that there is a general problem with Portal JNI global
reference cleanup as leaks seem to happen on certain regular queries in
addition to DatabaseMetaData requests.
Thanks,
-Ryan
Ryan Holmes wrote:
> No problem at all, thank you for getting back to me.
>
> If you think this is on the Java side, I'll be happy to try to debug
> and fix it. But if it's a problem in the native code, my C/C++ skills
> are, well let's just call them "nonexistent".
>
> I've been assuming that the problem is in the native code because
> heap analysis with jhat shows that the Portal objects are JNI global
> references without any referrers. Of course, this could simply mean
> that the PL/Java JDBC code is failing to call a native cleanup
> method. I guess I'm asking for some basic pointers to what the
> problem might be in theory before I jump in.
>
> A few specific questions:
>
> 1.) With very little C experience, am I wasting my time trying to fix
> the issue? I'm just asking for your best guess.
>
> 2.) What the heck is a Portal object anyway? Is it a handle to a
> result set / cursor?
>
> 3.) I noticed there was an update to SPIStatement.java and
> SPIDatabaseMetaData.java about 3.5 months ago with the commit message
> "Fixed some issues with Meta-data." The updates look like they might
> be relevant. Are these updates included in 1.3.0 and/or in the
> version of PL/Java that ships with the PG 8.2 Windows installer? I
> guess the real question is, am I debugging a problem that's already
> been fixed? Since I'm on Windows here and haven't used MinGW before,
> I haven't even tried compiling from HEAD.
>
> 4.) Would you be open to accepting payment to fix this yourself or do
> you simply not have the time? We (my company) think PL/Java is an
> ideal solution for our DB logic and it seems quite robust in all but
> this one area, so dedicating some of my time and/or paying for a fix
> is definitely on the table.
>
>
> FYI, I've worked around most of the problem by using local caching to
> avoid repeated calls to DatabaseMetaData.getPrimaryKeys() and
> getImportedKeys(). This solves about 80% of the leak. The other 20%
> (also in the form of Portal objects) is still somewhat of a mystery
> to me. It seems to happen on JDBC query executions inside the
> triggers but I haven't been able to reproduce it in a clean
> environment, so I'll come back to it later.
>
> Sorry for the long email and thank you for your time,
> -Ryan
>
> On Jan 16, 2007, at 11:57 AM, Thomas Hallgren wrote:
>
>
>> Hi Ryan,
>> Sorry for late reply. I find myself constantly tied up in other
>> activities these days. If you have the time to create a patch for
>> this I'd be happy to review it.
>>
>> Kind Regards,
>> Thomas Hallgren
>>
>>
>>
>> Ryan Holmes wrote:
>>
>>> Found one source of the problem. Filed as http://
>>> gborg.postgresql.org/ project/pljava/bugs/bugupdate.php?1624
>>> -Ryan
>>> On Jan 12, 2007, at 7:54 PM, Ryan Holmes wrote:
>>>
>>>> I have a PL/Java trigger that leaks a large number of
>>>> org.postgresql.pljava.internal.Portal objects. The leak occurs under
>>>> PostgreSQL 8.1 with PL/Java 1.3.0 and under 8.2 with the version of
>>>> PL/Java that is included with the Windows installer. I get the same
>>>> results with multiple versions of Windows (XP and Server 2003)
>>>> and with
>>>> 1.5 and 1.6 JVMs.
>>>>
>>>> So far, my attempts to isolate the leak have been fruitless (the
>>>> code
>>>> path is about 500 lines). I can say that from a pure Java
>>>> perspective,
>>>> there is no obvious reason for the leak. Furthermore, the only
>>>> references to the Portal objects are JNI global references so
>>>> there is
>>>> no chain of references from our code to these objects.
>>>>
>>>> Does anyone know what could cause PL/Java to create Portal
>>>> objects but
>>>> not clean them up? At this point, I'm not even sure what I'm
>>>> looking for
>>>> and the only option I have left is to rebuild or comment out the
>>>> code
>>>> "one line at a time" until I can isolate the behavior.
>>>>
>>>> I don't mind posting the code in question (it's a set of auditing
>>>> triggers and supporting methods) but, as I mentioned, there's
>>>> quite a
>>>> bit of it.
>>>>
>>>> Thanks,
>>>> -Ryan
>>>> _______________________________________________
>>>> Pljava-dev mailing list
>>>> Pljava-dev at gborg.postgresql.org
>>>> http://gborg.postgresql.org/mailman/listinfo/pljava-dev
>>>>
>
> _______________________________________________
> Pljava-dev mailing list
> Pljava-dev at gborg.postgresql.org
> http://gborg.postgresql.org/mailman/listinfo/pljava-dev
>
^ permalink raw reply [nested|flat] 6+ messages in thread
* [Pljava-dev] Portal object leak
@ 2007-01-22 11:16
parent:
1 sibling, 0 replies; 6+ messages in thread
From: @ 2007-01-22 11:16 UTC (permalink / raw)
Hi Ryan,
Some answers inline...
Ryan Holmes wrote:
> 1.) With very little C experience, am I wasting my time trying to fix
> the issue? I'm just asking for your best guess.
>
My best guess would be not doing it. The C code in there is fairly advanced.
> 2.) What the heck is a Portal object anyway? Is it a handle to a
> result set / cursor?
>
Yes, it's very similar.
> 3.) I noticed there was an update to SPIStatement.java and
> SPIDatabaseMetaData.java about 3.5 months ago with the commit message
> "Fixed some issues with Meta-data." The updates look like they might
> be relevant. Are these updates included in 1.3.0 and/or in the
> version of PL/Java that ships with the PG 8.2 Windows installer? I
> guess the real question is, am I debugging a problem that's already
> been fixed? Since I'm on Windows here and haven't used MinGW before,
> I haven't even tried compiling from HEAD.
>
They are not part of 1.3.0. They have to be part of anything that compiles with PostgreSQL 8.2.
> 4.) Would you be open to accepting payment to fix this yourself or do
> you simply not have the time? We (my company) think PL/Java is an
> ideal solution for our DB logic and it seems quite robust in all but
> this one area, so dedicating some of my time and/or paying for a fix
> is definitely on the table.
>
I'm incredibly busy at least until EclipseCon (5-8 March) is over. After that I might be
able to squeeze some time in so if you're not in a great hurry, let's postpone that
discussion until then.
I'm of course very open to contributions from anyone that would like to jump in an fix this.
Kind Regards,
Thomas Hallgren
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2007-01-22 11:16 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2007-01-13 03:54 [Pljava-dev] Portal object leak
2007-01-14 22:48 `
2007-01-16 19:57 `
2007-01-16 23:24 `
2007-01-18 03:39 `
2007-01-22 11:16 `
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox