From: Evil Azrael <evilazrael@evilazrael.de>
To: Rich Cullingford <rculling@sysd.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Java Out-of-memory errors on attempts to read tables with millions of rows
Date: Mon, 14 Jul 2003 19:07:44 +0200
Message-ID: <71226588.20030714190744@evilazrael.de> (raw)
In-Reply-To: <3F12DFFE.3090207@sysd.com>
References: <3F12DFFE.3090207@sysd.com>
I think you want to use a Cursor for browsing the data.
Christoph Nelles
Am Montag, 14. Juli 2003 um 18:53 schrieben Sie:
RC> Greetings,
RC> We have several tables (in a PG 7.3.3 database on RH Linux 7.3) with 2M+
RC> rows (each row 300-400 bytes in length) that we SELECT into a JDBC
RC> ResultSet for display to the user. We expected that the driver would not
RC> actually transmit data from the database until the application began
RC> issuing getXXX() calls. (IIRC, this is the way the Oracle driver works,
RC> and we had created a buffering mechanism to use it.) Instead, the driver
RC> appears to be attempting to create the whole rowset in Java memory
RC> before returning, and the application runs out of memory. (Java has been
RC> configured to use up to 1.5G on the machine this occurs on.)
RC> Now the SELECT is preceded by a COUNT of the rows that the same query
RC> would return, so perhaps that's what's causing the problem. But the
RC> question is, is this the way a ResultSet is supposed to work? Are there
RC> any configuration options available that modify this behavior? Are there
RC> commercial implementations of PG JDBC that don't have this problem?
RC> (Shame on me, but I have to ask. :)
RC> Any help will be greatly appreciated!
RC> Rich Cullingford
RC> rculling@sysd.com
RC> ---------------------------(end of broadcast)---------------------------
RC> TIP 9: the planner will ignore your desire to choose an index scan if your
RC> joining column's datatypes do not match
--
Mit freundlichen GrĂ¼ssen
Evil Azrael mailto:evilazrael@evilazrael.de
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-performance@postgresql.org
Cc: evilazrael@evilazrael.de, rculling@sysd.com
Subject: Re: Java Out-of-memory errors on attempts to read tables with millions of rows
In-Reply-To: <71226588.20030714190744@evilazrael.de>
References: <3F12DFFE.3090207@sysd.com> <71226588.20030714190744@evilazrael.de>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox