From: Douglas Thomson <dougt@mugc.cc.monash.edu.au>
To: pgsql-interfaces@postgreSQL.org
Subject: Front end memory consumption in SELECT
Date: Sat, 20 Nov 1999 12:10:27 +1100 (EST)
Message-ID: <199911200110.MAA09758@mugca.cc.monash.edu.au> (raw)
I have a large table that I need to traverse in full. I currently
start with a simple unrestricted SELECT, and then fetch each and
every row one at a time. I thought that by fetching just one row at a
time I would not consume any significant amount of memory.
However, judging by the memory consumption of my front-end process,
it would seem that the SELECT is loading the entire table into memory
before I even fetch the first row! Can anyone confirm that this is in
fact what goes on?
If so, is there any way to avoid it? The obvious solution would seem
to be to use LIMIT and OFFSET to get just a few thousand rows at a
time, but will that suffer from a time overhead while the backend
skips over millions of rows to get to the ones it needs??
Thanks for any clues anyone can provide!
Doug.
P.S. If it matters, I am using the Perl interface. I am also running in
SERIALIZABLE mode...
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-interfaces@postgresql.org
Cc: dougt@mugc.cc.monash.edu.au
Subject: Re: Front end memory consumption in SELECT
In-Reply-To: <199911200110.MAA09758@mugca.cc.monash.edu.au>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox