pg.ddx.io pljava-dev@postgresql.org mailing list archive
help / color / mirror / Atom feed[Pljava-dev] null pointer exception in pl/java jdbc driver
11+ messages / 0 participants
[nested] [flat]
* [Pljava-dev] null pointer exception in pl/java jdbc driver
@ 2012-05-14 19:16
0 siblings, 2 replies; 11+ messages in thread
From: @ 2012-05-14 19:16 UTC (permalink / raw)
i'm having trouble chasing down what may be close to the last bug in my pl/java code, a null pointer when executing an update. are there restrictions on what i can do inside a function with this jdbc driver? at this point, the code has successfully done a select, found the rows it wants and issued an http request to a solr search engine; now it needs to delete the rows that triggered the successful update:
the method:
private static void removeUpdateRecords( Connection conn, String core, List<Integer> ids){
String idString = idsToString( ids);
try {
Statement update = conn.createStatement();
String updateString = "DELETE FROM solr_index_event WHERE core = '"
+ core + "' AND id IN (" + idString + ")";
Logger.getLogger(SolrPush.class.getName()).log( Level.INFO, "delete sql: " + updateString);
update.executeUpdate(updateString);
} catch (SQLException ex) {
Logger.getLogger(SolrPush.class.getName()).log(Level.SEVERE, "Failed to remove solr_index_event row, core: " + core + " ids: " + idString, ex);
throw new IllegalStateException("Failed to remove solr_index_event row core: " + core + " ids: " + idString, ex);
}
}
here is the logger output (i've verified i have both an update statement and a complete and correct seeming sql update statement -- and yes, com.example in the java package names is deliberate obfuscation my part).
INFO: 14 May 12 19:00:37 com.example.solr.SolrPush delete sql: DELETE FROM solr_index_event WHERE core = 'components' AND id IN (1651, 1652)
ERROR: 14 May 12 19:00:37 com.example.solr.SolrPush Exception in RetryPush
org.postgresql.pljava.internal.ServerException: 14 May 12 19:00:37 com.example.solr.SolrPush SQL Exception in retryPush
org.postgresql.pljava.internal.ServerException: 14 May 12 19:00:37 com.example.solr.SolrPush Failed to remove solr_index_event row, core: components ids: 1651, 1652
org.postgresql.pljava.internal.ServerException: java.lang.NullPointerException
at org.postgresql.pljava.internal.ExecutionPlan._execute(Native Method)
at org.postgresql.pljava.internal.ExecutionPlan.execute(ExecutionPlan.java:187)
at org.postgresql.pljava.jdbc.SPIStatement.executePlan(SPIStatement.java:132)
at org.postgresql.pljava.jdbc.SPIStatement.execute(SPIStatement.java:108)
at org.postgresql.pljava.jdbc.SPIStatement.executeUpdate(SPIStatement.java:193)
at com.example.solr.SolrPush.removeUpdateRecords(SolrPush.java:390)
at com.example.solr.SolrPush.doUpdates(SolrPush.java:197)
at com.example.solr.SolrPush.retryPush(SolrPush.java:144)
at org.postgresql.pljava.internal.Backend._log(Native Method)
at org.postgresql.pljava.internal.Backend.log(Backend.java:79)
at org.postgresql.pljava.internal.ELogHandler.publish(ELogHandler.java:100)
at java.util.logging.Logger.log(Logger.java:498)
at java.util.logging.Logger.doLog(Logger.java:520)
at java.util.logging.Logger.log(Logger.java:609)
at com.example.solr.SolrPush.removeUpdateRecords(SolrPush.java:392)
at com.example.solr.SolrPush.doUpdates(SolrPush.java:197)
at com.example.solr.SolrPush.retryPush(SolrPush.java:144)
at org.postgresql.pljava.internal.Backend._log(Native Method)
at org.postgresql.pljava.internal.Backend.log(Backend.java:79)
at org.postgresql.pljava.internal.ELogHandler.publish(ELogHandler.java:100)
at java.util.logging.Logger.log(Logger.java:498)
at java.util.logging.Logger.doLog(Logger.java:520)
at java.util.logging.Logger.log(Logger.java:609)
at com.example.solr.SolrPush.retryPush(SolrPush.java:170)
gforge=#
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pgfoundry.org/pipermail/pljava-dev/attachments/20120514/52575ee8/attachment.html;
^ permalink raw reply [nested|flat] 11+ messages in thread
* [Pljava-dev] null pointer exception in pl/java jdbc driver
@ 2012-05-17 19:34
parent:
1 sibling, 1 reply; 11+ messages in thread
From: @ 2012-05-17 19:34 UTC (permalink / raw)
does anyone have any thoughts on this? i haven't been able to work on it since late monday.
richard
-----Original Message-----
From: pljava-dev-bounces at pgfoundry.org on behalf of Welty, Richard
Sent: Mon 5/14/2012 3:16 PM
To: pljava-dev at pgfoundry.org
Subject: [Pljava-dev] null pointer exception in pl/java jdbc driver
i'm having trouble chasing down what may be close to the last bug in my pl/java code, a null pointer when executing an update. are there restrictions on what i can do inside a function with this jdbc driver? at this point, the code has successfully done a select, found the rows it wants and issued an http request to a solr search engine; now it needs to delete the rows that triggered the successful update:
the method:
private static void removeUpdateRecords( Connection conn, String core, List<Integer> ids){
String idString = idsToString( ids);
try {
Statement update = conn.createStatement();
String updateString = "DELETE FROM solr_index_event WHERE core = '"
+ core + "' AND id IN (" + idString + ")";
Logger.getLogger(SolrPush.class.getName()).log( Level.INFO, "delete sql: " + updateString);
update.executeUpdate(updateString);
} catch (SQLException ex) {
Logger.getLogger(SolrPush.class.getName()).log(Level.SEVERE, "Failed to remove solr_index_event row, core: " + core + " ids: " + idString, ex);
throw new IllegalStateException("Failed to remove solr_index_event row core: " + core + " ids: " + idString, ex);
}
}
here is the logger output (i've verified i have both an update statement and a complete and correct seeming sql update statement -- and yes, com.example in the java package names is deliberate obfuscation my part).
INFO: 14 May 12 19:00:37 com.example.solr.SolrPush delete sql: DELETE FROM solr_index_event WHERE core = 'components' AND id IN (1651, 1652)
ERROR: 14 May 12 19:00:37 com.example.solr.SolrPush Exception in RetryPush
org.postgresql.pljava.internal.ServerException: 14 May 12 19:00:37 com.example.solr.SolrPush SQL Exception in retryPush
org.postgresql.pljava.internal.ServerException: 14 May 12 19:00:37 com.example.solr.SolrPush Failed to remove solr_index_event row, core: components ids: 1651, 1652
org.postgresql.pljava.internal.ServerException: java.lang.NullPointerException
at org.postgresql.pljava.internal.ExecutionPlan._execute(Native Method)
at org.postgresql.pljava.internal.ExecutionPlan.execute(ExecutionPlan.java:187)
at org.postgresql.pljava.jdbc.SPIStatement.executePlan(SPIStatement.java:132)
at org.postgresql.pljava.jdbc.SPIStatement.execute(SPIStatement.java:108)
at org.postgresql.pljava.jdbc.SPIStatement.executeUpdate(SPIStatement.java:193)
at com.example.solr.SolrPush.removeUpdateRecords(SolrPush.java:390)
at com.example.solr.SolrPush.doUpdates(SolrPush.java:197)
at com.example.solr.SolrPush.retryPush(SolrPush.java:144)
at org.postgresql.pljava.internal.Backend._log(Native Method)
at org.postgresql.pljava.internal.Backend.log(Backend.java:79)
at org.postgresql.pljava.internal.ELogHandler.publish(ELogHandler.java:100)
at java.util.logging.Logger.log(Logger.java:498)
at java.util.logging.Logger.doLog(Logger.java:520)
at java.util.logging.Logger.log(Logger.java:609)
at com.example.solr.SolrPush.removeUpdateRecords(SolrPush.java:392)
at com.example.solr.SolrPush.doUpdates(SolrPush.java:197)
at com.example.solr.SolrPush.retryPush(SolrPush.java:144)
at org.postgresql.pljava.internal.Backend._log(Native Method)
at org.postgresql.pljava.internal.Backend.log(Backend.java:79)
at org.postgresql.pljava.internal.ELogHandler.publish(ELogHandler.java:100)
at java.util.logging.Logger.log(Logger.java:498)
at java.util.logging.Logger.doLog(Logger.java:520)
at java.util.logging.Logger.log(Logger.java:609)
at com.example.solr.SolrPush.retryPush(SolrPush.java:170)
gforge=#
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pgfoundry.org/pipermail/pljava-dev/attachments/20120517/cc037ad3/attachment.html;
^ permalink raw reply [nested|flat] 11+ messages in thread
* [Pljava-dev] null pointer exception in pl/java jdbc driver
@ 2012-05-17 23:55
parent:
0 siblings, 1 reply; 11+ messages in thread
From: @ 2012-05-17 23:55 UTC (permalink / raw)
"Welty, Richard" <rwelty at ltionline.com> writes:
> does anyone have any thoughts on this? i haven't been able to work on
> it since late monday.
Unfortunately I don't. I won't be able to take a proper look until next
week so all I can say is: good luck.
--
Johann Oskarsson http://www.2ndquadrant.com/ |[]
PostgreSQL Development, 24x7 Support, Training and Services --+--
|
Blog: http://my.opera.com/myrkraverk/blog/
^ permalink raw reply [nested|flat] 11+ messages in thread
* [Pljava-dev] null pointer exception in pl/java jdbc driver
@ 2012-05-18 11:37
parent:
0 siblings, 0 replies; 11+ messages in thread
From: @ 2012-05-18 11:37 UTC (permalink / raw)
Johann 'Myrkraverk' Oskarsson writes:
Sent: Thu 5/17/2012 7:55 PM
>"Welty, Richard" <rwelty at ltionline.com> writes:
>> does anyone have any thoughts on this? i haven't been able to work on
>> it since late monday.
>Unfortunately I don't. I won't be able to take a proper look until next
>week so all I can say is: good luck.
ok. it's not a crisis, and i'll be happy to provide as much information as i can
next week.
thanks,
richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pgfoundry.org/pipermail/pljava-dev/attachments/20120518/8679abcf/attachment.html;
^ permalink raw reply [nested|flat] 11+ messages in thread
* [Pljava-dev] null pointer exception in pl/java jdbc driver
@ 2012-05-24 09:07
parent:
1 sibling, 1 reply; 11+ messages in thread
From: @ 2012-05-24 09:07 UTC (permalink / raw)
"Welty, Richard" <rwelty at ltionline.com> writes:
> i'm having trouble chasing down what may be close to the last bug in
> my pl/ java code, a null pointer when executing an update. are there
> restrictions on what i can do inside a function with this jdbc driver?
> at this point, the code has successfully done a select, found the rows
> it wants and issued an http request to a solr search engine; now it
> needs to delete the rows that triggered the successful update:
>
> the method:
The following code works for me. Can you give us the steps you are
taking?
package com2ndQuadrant.pljava;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.Statement;
import java.sql.SQLException;
import java.util.List;
import java.util.LinkedList;
import java.util.logging.Logger;
public class Update
{
public static void delete( Connection c,
String key,
List< Integer > ids )
throws SQLException
{
String s = ids.toString();
s = s.substring( 1, s.length() - 1 );
Logger l = Logger.getAnonymousLogger();
l.fine( s );
String r = "delete from t where key = '"
+ key
+ "' and id in ( " + s + " ) ";
l.fine( r );
Statement stm = c.createStatement();
stm.execute( r );
}
public static void doStuff( String key )
throws SQLException
{
Connection c = DriverManager.getConnection
( "jdbc:default:connection" );
List< Integer > l = new LinkedList< Integer >();
l.add( 3 );
l.add( 5 );
l.add( 7 );
delete( c, key, l );
}
}
--
Johann Oskarsson http://www.2ndquadrant.com/ |[]
PostgreSQL Development, 24x7 Support, Training and Services --+--
|
Blog: http://my.opera.com/myrkraverk/blog/
^ permalink raw reply [nested|flat] 11+ messages in thread
* [Pljava-dev] null pointer exception in pl/java jdbc driver
@ 2012-05-24 13:44
parent:
0 siblings, 1 reply; 11+ messages in thread
From: @ 2012-05-24 13:44 UTC (permalink / raw)
Johann 'Myrkraverk' Oskarsson writes:
>"Welty, Richard" <rwelty at ltionline.com> writes:
>> i'm having trouble chasing down what may be close to the last bug in
>> my pl/ java code, a null pointer when executing an update. are there
>> restrictions on what i can do inside a function with this jdbc driver?
>> at this point, the code has successfully done a select, found the rows
>> it wants and issued an http request to a solr search engine; now it
>> needs to delete the rows that triggered the successful update:
>The following code works for me. Can you give us the steps you are
>taking?
the framework is a system for doing push notifications to an Apache Solr search engine when certain metadata in a PostgreSQL database is updated.
There is an intermediate table, solr_index_event. anytime an entity needs to be updated in the search engine because of an update/insert/delete in PostgreSQL, a row gets written to this table. the primary key is on two columns, the core (index) in the Solr engine containing the entity, and the unique id of the entity.
i'm debugging the retryPush method in this case, which will run as a cron job to scavenge for updates that failed in previous attempts. so at the top of retryPush, i have this select:
queryString
= "SELECT * FROM solr_index_event WHERE status = 'RETRY ' AND core = '" + core
+ "' AND (update_type = 'ADD ' OR update_type = 'UPDATE ')"
+ " FOR UPDATE";
(i have tried this both with and without "FOR UPDATE" with no difference in behavior.)
the select is giving me the correct list of entities. subsequent selects to get the data on the specific entities are providing the correct data; the xml for the Solr update request is being built properly, and Solr is receiving the updates (via an http POST) and processing them properly.
at the very end, i need to delete the rows where the retry succeeded, and that's the method that's failing with the NPE down in the PL/Java methods.
richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pgfoundry.org/pipermail/pljava-dev/attachments/20120524/61f48da8/attachment.html;
^ permalink raw reply [nested|flat] 11+ messages in thread
* [Pljava-dev] null pointer exception in pl/java jdbc driver
@ 2012-05-24 14:46
parent:
0 siblings, 1 reply; 11+ messages in thread
From: @ 2012-05-24 14:46 UTC (permalink / raw)
"Welty, Richard" <rwelty at ltionline.com> writes:
> Johann 'Myrkraverk' Oskarsson writes:
>
>
>>The following code works for me. Can you give us the steps you are
>>taking?
>
> the framework is a system for doing push notifications to an Apache Solr
> search engine when certain metadata in a PostgreSQL database is updated.
...
> at the very end, i need to delete the rows where the retry succeeded,
> and that's the method that's failing with the NPE down in the PL/Java
> methods.
And this is not an out of memory issue? What is your -Xmx?
Granted, if it is, there is prabably a missing error check somewhere in
the PL/Java code.
--
Johann Oskarsson http://www.2ndquadrant.com/ |[]
PostgreSQL Development, 24x7 Support, Training and Services --+--
|
Blog: http://my.opera.com/myrkraverk/blog/
^ permalink raw reply [nested|flat] 11+ messages in thread
* [Pljava-dev] null pointer exception in pl/java jdbc driver
@ 2012-05-24 15:45
parent:
0 siblings, 2 replies; 11+ messages in thread
From: @ 2012-05-24 15:45 UTC (permalink / raw)
Johann 'Myrkraverk' Oskarsson wrote:
>"Welty, Richard" <rwelty at ltionline.com> writes:
>> Johann 'Myrkraverk' Oskarsson writes:
>>>The following code works for me. Can you give us the steps you are
>>>taking?
>> the framework is a system for doing push notifications to an Apache Solr
>> search engine when certain metadata in a PostgreSQL database is updated.
...
>> at the very end, i need to delete the rows where the retry succeeded,
>> and that's the method that's failing with the NPE down in the PL/Java
>> methods.
>And this is not an out of memory issue? What is your -Xmx?
>Granted, if it is, there is prabably a missing error check somewhere in
>the PL/Java code.
-Xmx is the default. i'll try increasing it and see if that helps.
richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pgfoundry.org/pipermail/pljava-dev/attachments/20120524/f3a52256/attachment.html;
^ permalink raw reply [nested|flat] 11+ messages in thread
* [Pljava-dev] null pointer exception in pl/java jdbc driver
@ 2012-05-24 16:57
parent:
1 sibling, 1 reply; 11+ messages in thread
From: @ 2012-05-24 16:57 UTC (permalink / raw)
Richard Welty wrote:
>Johann 'Myrkraverk' Oskarsson wrote:
>>And this is not an out of memory issue? What is your -Xmx?
>>Granted, if it is, there is prabably a missing error check somewhere in
>>the PL/Java code.
>-Xmx is the default. i'll try increasing it and see if that help
it was 64MB (copied from the pl/java installation instructions). i increased it to 128MB, and then to 256MB but continue to get the NPE. (i restarted postgresql after each increase, which i assume is sufficient.)
richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pgfoundry.org/pipermail/pljava-dev/attachments/20120524/d4aa830b/attachment.html;
^ permalink raw reply [nested|flat] 11+ messages in thread
* [Pljava-dev] null pointer exception in pl/java jdbc driver
@ 2012-05-24 17:29
parent:
0 siblings, 0 replies; 11+ messages in thread
From: @ 2012-05-24 17:29 UTC (permalink / raw)
"Welty, Richard" <rwelty at ltionline.com> writes:
> it was 64MB (copied from the pl/java installation instructions). i
> increased it to 128MB, and then to 256MB but continue to get the
> NPE. (i restarted postgresql after each increase, which i assume is
> sufficient.)
Yes, it is.
In a new connection, before you run *any* PL/Java code, can you
SET log_min_messages = debug;
and if you're using psql it may help to
SET client_min_messages = debug;
and show us the results?
--
Johann Oskarsson http://www.2ndquadrant.com/ |[]
PostgreSQL Development, 24x7 Support, Training and Services --+--
|
Blog: http://my.opera.com/myrkraverk/blog/
^ permalink raw reply [nested|flat] 11+ messages in thread
* [Pljava-dev] null pointer exception in pl/java jdbc driver
@ 2012-05-25 12:40
parent:
1 sibling, 0 replies; 11+ messages in thread
From: @ 2012-05-25 12:40 UTC (permalink / raw)
the issue has been identified, this summary is to help anyone who comes along with a similar issue
in the future.
the root cause is there was a trigger being fired which should not have been. instead of triggering
after insert, it was triggering after insert or delete, and then attempting to operate on rows which
the retryPush function had just deleted from the table.
i only got an idea of what was going on after i ran the function from psql with these two debug
settings as requested by Johann:
SET log_min_messages = debug;
SET client_min_messages = debug;
until i set these and reviewed the output, i had no idea that the unwanted trigger was where
the error actually originated.
richard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.pgfoundry.org/pipermail/pljava-dev/attachments/20120525/0f655e0d/attachment.html;
^ permalink raw reply [nested|flat] 11+ messages in thread
end of thread, other threads:[~2012-05-25 12:40 UTC | newest]
Thread overview: 11+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2012-05-14 19:16 [Pljava-dev] null pointer exception in pl/java jdbc driver
2012-05-17 19:34 `
2012-05-17 23:55 `
2012-05-18 11:37 `
2012-05-24 09:07 `
2012-05-24 13:44 `
2012-05-24 14:46 `
2012-05-24 15:45 `
2012-05-24 16:57 `
2012-05-24 17:29 `
2012-05-25 12:40 `
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox