pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Evil Azrael <evilazrael@evilazrael.de>
To: Rich Cullingford <rculling@sysd.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Java Out-of-memory errors on attempts to read tables with millions of rows
Date: Mon, 14 Jul 2003 19:07:44 +0200
Message-ID: <71226588.20030714190744@evilazrael.de> (raw)
In-Reply-To: <3F12DFFE.3090207@sysd.com>
References: <3F12DFFE.3090207@sysd.com>

I think you want to use a Cursor for browsing the data.

Christoph Nelles


Am Montag, 14. Juli 2003 um 18:53 schrieben Sie:

RC> Greetings,
RC> We have several tables (in a PG 7.3.3 database on RH Linux 7.3) with 2M+ 
RC> rows (each row 300-400 bytes in length) that we SELECT into a JDBC 
RC> ResultSet for display to the user. We expected that the driver would not 
RC> actually transmit data from the database until the application began 
RC> issuing getXXX() calls. (IIRC, this is the way the Oracle driver works, 
RC> and we had created a buffering mechanism to use it.) Instead, the driver 
RC> appears to be attempting to create the whole rowset in Java memory 
RC> before returning, and the application runs out of memory. (Java has been 
RC> configured to use up to 1.5G on the machine this occurs on.)

RC> Now the SELECT is preceded by a COUNT of the rows that the same query 
RC> would return, so perhaps that's what's causing the problem. But the 
RC> question is, is this the way a ResultSet is supposed to work? Are there 
RC> any configuration options available that modify this behavior? Are there 
RC> commercial implementations of PG JDBC that don't have this problem? 
RC> (Shame on me, but I have to ask. :)

RC> Any help will be greatly appreciated!

RC>                                  Rich Cullingford
RC>                                  rculling@sysd.com


RC> ---------------------------(end of broadcast)---------------------------
RC> TIP 9: the planner will ignore your desire to choose an index scan if your
RC>       joining column's datatypes do not match



-- 
Mit freundlichen GrĂ¼ssen
Evil Azrael                            mailto:evilazrael@evilazrael.de




view thread (2+ messages)

Message-ID: <71226588.20030714190744@evilazrael.de>
Permalink:  ../71226588.20030714190744@evilazrael.de/
Also on:    postgresql.org/message-id/71226588.20030714190744@evilazrael.de

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-performance@postgresql.org
  Cc: evilazrael@evilazrael.de, rculling@sysd.com
  Subject: Re: Java Out-of-memory errors on attempts to read tables with millions of rows
  In-Reply-To: <71226588.20030714190744@evilazrael.de>
  References: <3F12DFFE.3090207@sysd.com> <71226588.20030714190744@evilazrael.de>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox