pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Koen Martens <pgsql@metro.cx>
To: pgsql-admin@postgresql.org
Subject: Random disconnects
Date: Thu, 23 Mar 2006 16:02:04 +0100
Message-ID: <4422B86C.5000703@metro.cx> (raw)

Hi All,

Just wondering whether someone might recognize the problem i am
having. First the intro: postgresql-8.1.0 running on FreeBSD
6.0-RELENG, on a dual xeon 3.0GHZ machine with 2gb memory/2gb swap.

The problem is we have been experiencing random disconnects from our
main application and a script. The application is a web spider
written in java that uses the postgresql db to store urls, and quite
a lot of them. The script is a perl script that coordinates several
of these crawlers, and has a low postgresql utilisation.

On the machines with the java app (there are two of them currently,
each with its own postgresql database) we use the latest jdbc:

postgresql-8.1-405.jdbc3.jar

So the errors we see are:

In java, this big fat exception:

org.postgresql.util.PSQLException: An I/O error occured while
sending to the backend.
Call stack:
org.postgresql.util.PSQLException: An I/O error occured while
sending to the backend.
	at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:214)
	at
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:437)
	at
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:353)
	at
org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:307)
	at
net.oneworld.issuemapper.CrawlToDBListener.dumpLinkToSpoolAndFreeUpMemory(CrawlToDBListener.java:214)
	at
net.oneworld.issuemapper.CrawlToDBListener.crawled(CrawlToDBListener.java:165)
	at websphinx.Crawler.sendLinkEvent(Crawler.java:1540)
	at websphinx.Crawler.process(Crawler.java:1910)
	at websphinx.Crawler.fetch(Crawler.java:1840)
	at websphinx.Worm.run(Crawler.java:1994)
Caused by: java.net.SocketException: Network is unreachable
	at java.net.SocketOutputStream.socketWrite0(Native Method)
	at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
	at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
	at
java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:66)
	at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:124)
	at org.postgresql.core.PGStream.flush(PGStream.java:502)
	at
org.postgresql.core.v3.QueryExecutorImpl.sendSync(QueryExecutorImpl.java:672)
	at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:187)
	... 9 more

Where at CrawlToDBListener.java:214 we just execute a prepared
statement that inserts the url data, which goes allright 99.99999%
of the time.


The perl error we get is:


DBI connect('dbname=issuemaps_a;host=192.168.0.2','crawler',...)
failed: server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
 at cbsd-lib.pl line 181
Can't call method "prepare" on an undefined value at cbsd-lib.pl
line 188, <SIZEFILEH> line 68.

Line 188 of cbsd-lib is a dbh->prepare line..



I guess the proper work-around is to wrap all database calls such
that when it detects a loss of connection, it will retry (which is
what i would've done in the first place if i was the one who had
written this software), or use dbh / jdbc persistent connections to
make this transparent.

But I think there might be some underlying problem and if so, i'd
like to know about it and fix it.

Gr,

Koen

-- 
K.F.J. Martens, Sonologic, http://www.sonologic.nl/
Networking, hosting, embedded systems, unix, artificial intelligence.
Public PGP key: http://www.metro.cx/pubkey-gmc.asc
Wondering about the funny attachment your mail program
can't read? Visit http://www.openpgp.org/



view thread (3+ messages)  latest in thread

Message-ID: <4422B86C.5000703@metro.cx>
Permalink:  ../4422B86C.5000703@metro.cx/
Also on:    postgresql.org/message-id/4422B86C.5000703@metro.cx

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-admin@postgresql.org
  Cc: pgsql@metro.cx
  Subject: Re: Random disconnects
  In-Reply-To: <4422B86C.5000703@metro.cx>
  References: <4422B86C.5000703@metro.cx>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox