From: Stephan Szabo <sszabo@megazone23.bigpanda.com>
To: Wilkinson Charlie E <Charlie.E.Wilkinson@irs.gov>
Cc: pgsql-sql@postgresql.org
Subject: Re: Working with very large datasets
Date: Tue, 11 Feb 2003 16:02:50 -0800 (PST)
Message-ID: <20030211160155.O19527-100000@megazone23.bigpanda.com> (raw)
In-Reply-To: <9DDDC002B736D711A7320004AC3699EB01C192C7@nct0010mb03.nc.no.irs.gov>
On Tue, 11 Feb 2003, Wilkinson Charlie E wrote:
> Greetings,
> Can anyone enlighten me or point me at resources concerning use of pgsql
> with
> very large datasets?
>
> My specific problem is this:
>
> I have two tables, one with about 100 million rows and one with about 22,000
> rows. My plan was to inner join the two tables on an integer key and output
> the 4 significant columns, excluding the keys. (Those with a better
> understanding
> of pgsql internals, feel free to laugh.) The result was a big angry psql
> that
> grew to 800+MB before I had to kill it.
Was it psql that grew to 800Mb or a backend? If the former, how many rows
do you expect that to return? You probably want to look into using
cursors rather than returning the entire result set at once.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-sql@postgresql.org
Cc: sszabo@megazone23.bigpanda.com, Charlie.E.Wilkinson@irs.gov
Subject: Re: Working with very large datasets
In-Reply-To: <20030211160155.O19527-100000@megazone23.bigpanda.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox