From: ryan at hyperstep.com (Ryan Holmes) Date: Wed, 17 Jan 2007 19:39:35 -0800 Subject: [Pljava-dev] Portal object leak In-Reply-To: <401FA771-4DC4-4B7B-B905-021F7B4CC057@hyperstep.com> References: <45A85803.1070500@hyperstep.com> <4508612D-5A18-4C25-B97A-3FF9DCCAD26B@hyperstep.com> <45AD2E2D.4080507@tada.se> <401FA771-4DC4-4B7B-B905-021F7B4CC057@hyperstep.com> Message-ID: <45AEEBF7.9020505@hyperstep.com> I'm giving up on this for now as I don't have the C skills required to debug it. I did step through the PL/Java JDBC code to look for any obvious problems (e.g. failing to call Portal.close()) and couldn't find any. My guess is that there is a general problem with Portal JNI global reference cleanup as leaks seem to happen on certain regular queries in addition to DatabaseMetaData requests. Thanks, -Ryan Ryan Holmes wrote: > No problem at all, thank you for getting back to me. > > If you think this is on the Java side, I'll be happy to try to debug > and fix it. But if it's a problem in the native code, my C/C++ skills > are, well let's just call them "nonexistent". > > I've been assuming that the problem is in the native code because > heap analysis with jhat shows that the Portal objects are JNI global > references without any referrers. Of course, this could simply mean > that the PL/Java JDBC code is failing to call a native cleanup > method. I guess I'm asking for some basic pointers to what the > problem might be in theory before I jump in. > > A few specific questions: > > 1.) With very little C experience, am I wasting my time trying to fix > the issue? I'm just asking for your best guess. > > 2.) What the heck is a Portal object anyway? Is it a handle to a > result set / cursor? > > 3.) I noticed there was an update to SPIStatement.java and > SPIDatabaseMetaData.java about 3.5 months ago with the commit message > "Fixed some issues with Meta-data." The updates look like they might > be relevant. Are these updates included in 1.3.0 and/or in the > version of PL/Java that ships with the PG 8.2 Windows installer? I > guess the real question is, am I debugging a problem that's already > been fixed? Since I'm on Windows here and haven't used MinGW before, > I haven't even tried compiling from HEAD. > > 4.) Would you be open to accepting payment to fix this yourself or do > you simply not have the time? We (my company) think PL/Java is an > ideal solution for our DB logic and it seems quite robust in all but > this one area, so dedicating some of my time and/or paying for a fix > is definitely on the table. > > > FYI, I've worked around most of the problem by using local caching to > avoid repeated calls to DatabaseMetaData.getPrimaryKeys() and > getImportedKeys(). This solves about 80% of the leak. The other 20% > (also in the form of Portal objects) is still somewhat of a mystery > to me. It seems to happen on JDBC query executions inside the > triggers but I haven't been able to reproduce it in a clean > environment, so I'll come back to it later. > > Sorry for the long email and thank you for your time, > -Ryan > > On Jan 16, 2007, at 11:57 AM, Thomas Hallgren wrote: > > >> Hi Ryan, >> Sorry for late reply. I find myself constantly tied up in other >> activities these days. If you have the time to create a patch for >> this I'd be happy to review it. >> >> Kind Regards, >> Thomas Hallgren >> >> >> >> Ryan Holmes wrote: >> >>> Found one source of the problem. Filed as http:// >>> gborg.postgresql.org/ project/pljava/bugs/bugupdate.php?1624 >>> -Ryan >>> On Jan 12, 2007, at 7:54 PM, Ryan Holmes wrote: >>> >>>> I have a PL/Java trigger that leaks a large number of >>>> org.postgresql.pljava.internal.Portal objects. The leak occurs under >>>> PostgreSQL 8.1 with PL/Java 1.3.0 and under 8.2 with the version of >>>> PL/Java that is included with the Windows installer. I get the same >>>> results with multiple versions of Windows (XP and Server 2003) >>>> and with >>>> 1.5 and 1.6 JVMs. >>>> >>>> So far, my attempts to isolate the leak have been fruitless (the >>>> code >>>> path is about 500 lines). I can say that from a pure Java >>>> perspective, >>>> there is no obvious reason for the leak. Furthermore, the only >>>> references to the Portal objects are JNI global references so >>>> there is >>>> no chain of references from our code to these objects. >>>> >>>> Does anyone know what could cause PL/Java to create Portal >>>> objects but >>>> not clean them up? At this point, I'm not even sure what I'm >>>> looking for >>>> and the only option I have left is to rebuild or comment out the >>>> code >>>> "one line at a time" until I can isolate the behavior. >>>> >>>> I don't mind posting the code in question (it's a set of auditing >>>> triggers and supporting methods) but, as I mentioned, there's >>>> quite a >>>> bit of it. >>>> >>>> Thanks, >>>> -Ryan >>>> _______________________________________________ >>>> Pljava-dev mailing list >>>> Pljava-dev at gborg.postgresql.org >>>> http://gborg.postgresql.org/mailman/listinfo/pljava-dev >>>> > > _______________________________________________ > Pljava-dev mailing list > Pljava-dev at gborg.postgresql.org > http://gborg.postgresql.org/mailman/listinfo/pljava-dev >