pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Rich Cullingford <rculling@sysd.com>
To: pgsql-performance@postgresql.org
Subject: Java Out-of-memory errors on attempts to read tables with  millions of rows
Date: Mon, 14 Jul 2003 12:53:18 -0400
Message-ID: <3F12DFFE.3090207@sysd.com> (raw)

Greetings,
We have several tables (in a PG 7.3.3 database on RH Linux 7.3) with 2M+ 
rows (each row 300-400 bytes in length) that we SELECT into a JDBC 
ResultSet for display to the user. We expected that the driver would not 
actually transmit data from the database until the application began 
issuing getXXX() calls. (IIRC, this is the way the Oracle driver works, 
and we had created a buffering mechanism to use it.) Instead, the driver 
appears to be attempting to create the whole rowset in Java memory 
before returning, and the application runs out of memory. (Java has been 
configured to use up to 1.5G on the machine this occurs on.)

Now the SELECT is preceded by a COUNT of the rows that the same query 
would return, so perhaps that's what's causing the problem. But the 
question is, is this the way a ResultSet is supposed to work? Are there 
any configuration options available that modify this behavior? Are there 
commercial implementations of PG JDBC that don't have this problem? 
(Shame on me, but I have to ask. :)

Any help will be greatly appreciated!

                                 Rich Cullingford
                                 rculling@sysd.com




view thread (2+ messages)  latest in thread

Message-ID: <3F12DFFE.3090207@sysd.com>
Permalink:  ../3F12DFFE.3090207@sysd.com/
Also on:    postgresql.org/message-id/3F12DFFE.3090207@sysd.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-performance@postgresql.org
  Cc: rculling@sysd.com
  Subject: Re: Java Out-of-memory errors on attempts to read tables with  millions of rows
  In-Reply-To: <3F12DFFE.3090207@sysd.com>
  References: <3F12DFFE.3090207@sysd.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox