X-Original-To: pgsql-admin-postgresql.org@localhost.postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id 2DE6311F61E8 for ; Wed, 26 Apr 2006 12:45:05 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 38633-05 for ; Wed, 26 Apr 2006 12:44:46 -0300 (ADT) X-Greylist: domain auto-whitelisted by SQLgrey- Received: from nz-out-0102.google.com (nz-out-0102.google.com [64.233.162.201]) by postgresql.org (Postfix) with ESMTP id B4DDA11F60AF for ; Wed, 26 Apr 2006 12:44:45 -0300 (ADT) Received: by nz-out-0102.google.com with SMTP id o1so1327306nzf for ; Wed, 26 Apr 2006 08:44:42 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=Y2D6d1vQqkSmbD6VYJRxTs7veORohpHZnJqNcJvbnKjkLYJL3zN8rtpUltweHxNyx78W/hnS6KEPout08WuAAQpyxZf+kfs0YS6dYJr0NrX9Q7e5XtDXnHtDjrFkvJqEb2RzpTo4Wglku/lPEnGVJVZUZJxaDjfQA0Z5d7EbkY0= Received: by 10.36.134.13 with SMTP id h13mr1184582nzd; Wed, 26 Apr 2006 08:44:42 -0700 (PDT) Received: by 10.36.128.15 with HTTP; Wed, 26 Apr 2006 08:44:41 -0700 (PDT) Message-ID: <1e71f8880604260844l8a6949ana25cf35c3f4a5c6e@mail.gmail.com> Date: Wed, 26 Apr 2006 15:44:41 +0000 From: "Doug Lochart" To: pgsql-admin@postgresql.org Subject: Socket Timeouts and fatal errors (please help) MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=1.217 tagged_above=0 required=5 tests=BAYES_00, SPF_HELO_SOFTFAIL, SPF_SOFTFAIL X-Spam-Level: * X-Archive-Number: 200604/355 X-Sequence-Number: 21524 First let me divulge the pertinent info: database server: ----------------------------- Linux 2.6.8.1 postgres 8.1.0 webserver: ------------------- Linux 2.6.8.1 Apache 2.0.53, mod_jk, tomcat 5.0.28 jdbc-driver 8.1-404.jdbc3 libpq.so.4.1 We have Java client objects using jdbc, some JNI C code using libpq and some standalone c applications using libpq. Both Java and C apps are using connection pools. we have our max connections set to 100 in postgres. Our max pool size is 20 in the java realm and 10 in the C side. Our production server is not used heavily at the moment. What we are experiencing is the following: (java side) Sporadic (once or twice a month) Socket timeout exceptions like this:=20 The call in question is simply doing some data cleanup when a user session expires. Don't the the word 'Lock' worry you ... its all data. This call failed on a query. org.postgresql.util.PSQLException: An I/O error occured while sending to the backend. at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorIm= pl.java:214) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc= 2Statement.java:430) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(Abs= tractJdbc2Statement.java:332) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc= 2Statement.java:324) at com.capecomputing.mensana.dataaccess.MensanaConnection.getConnec= tion(MensanaConnection.java:116) at com.capecomputing.mensana.dataaccess.MensanaConnection.(Me= nsanaConnection.java:83) at com.capecomputing.mensana.dataaccess.DatabaseUpdateAction.getEnc= ryptedParameters(DatabaseUpdateAction.java:65) at com.capecomputing.mensana.dataaccess.DatabaseUpdateAction.create= EncryptedParamterList(DatabaseUpdateAction.java:39) at com.capecomputing.mensana.dataaccess.CleanupLockedPrepaidClientI= nvoicesAction.(CleanupLockedPrepaidClientInvoicesAction.java:33) at com.capecomputing.mensana.dataaccess.MensanaDataAccess.cleanupLo= ckedPrepaidInvoices(MensanaDataAccess.java:1685) at com.capecomputing.mensana.webportal.application.SessionEventList= ener.cleanupLockedPrepaidInvoices(SessionEventListener.java:62) ... Caused by: java.net.SocketException: Connection timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.io.BufferedInputStream.fill(BufferedInputStream.java:183) at java.io.BufferedInputStream.read(BufferedInputStream.java:201) at org.postgresql.core.PGStream.ReceiveChar(PGStream.java:254) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExe= cutorImpl.java:1163) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorIm= pl.java:188) ... 49 more On the JNI C side the error was like this: We retrieved a connection from the pool. Checked its status it was good. Performed a query an checked the result status and all that was returned was a fatal error. The JNI C side tried another connection from the pool and made another query, this one caused a tomcat crash (the query dumped this hs_err file) An unexpected exception has been detected in native code outside the VM. Unexpected Signal : 11 occurred at PC=3D0x400EA4D3 Function=3Dstrlen+0x33 Library=3D/lib/libc.so.6 ... The last message in the log was "about to call PQexec()" Also in the standalone C application we experienced the same fatal error message. One of the days when we noticed the java socket timeouts we had a power hit. We chalked it up to that but the last one was not. This never happens in our non prod environments ... probably because we are always bouncing the apps at least once a week for tweaks and tests. Does anyone have any ideas? I will post a sperate issue/question that may or may not be related to the issues I stated here. It will be titled pgadmin3 and tomcat hangs. thanks in advance Doug -- What profits a man if he gains the whole world yet loses his soul?