From: Josh Berkus <josh@agliodbs.com>
To: pgsql-sql@postgresql.org
Subject: Function won't complete
Date: Fri, 26 Apr 2002 14:57:20 -0700
Message-ID: <web-1386081@davinci.ethosmedia.com> (raw)
Folks,
I am currently testing a massive data update function, which takes a
large amount of raw data (in 3 tables comprising over 2 million records
total) and organizes and cleans that data in a series of successive
steps for the final relational database.
The problem is, it doesn't complete. About 1/3 of the way in to the
data updates, postgres gets stuck in an endless cycle of recycling
transaction logs and child processes and won't finish (current longest
abort time is 37 minutes).
I've upped the various memory parameters significantly ... working with
wal_files 32, wal_buffers 32, sort_mem 65536, shared_buffers 256, etc.
The CPU (1400mhz athalon) and memory (512Mb DDR RAM) are hardly being
taxed. The only hardware bottleneck I can see is that the system only
has one IDE drive (though a fast, large one). I've added many indexes.
Does anyone have an suggestions on making this process complete, short
of running out and buying a RAID SCSI array?
-Josh Berkus
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-sql@postgresql.org
Cc: josh@agliodbs.com
Subject: Re: Function won't complete
In-Reply-To: <web-1386081@davinci.ethosmedia.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox