Message-Id: <1166895168.15177.281812013@webmail.messagingengine.com>
From: "Jeremy Haile" <jhaile@fastmail.fm>
To: pgsql-performance@postgresql.org
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="ISO-8859-1"
MIME-Version: 1.0
References: <200612222215.kBMMFnB00280@momjian.us>
Subject: Re: URGENT: Out of disk space pg_xlog
In-Reply-To: <200612222215.kBMMFnB00280@momjian.us>
Date: Sat, 23 Dec 2006 12:32:48 -0500

> > Once you free some space on the data partition and restart, you should
> > be good to go --- there will be no loss of committed transactions, since
> > all the operations are in pg_xlog.  Might take a little while to replay
> > all that log though :-(
> 
> Amazing that all works.  What I did not see is confirmation from the
> user that the data directory filled up _before_ pg_xlog filled up.

After I freed up space on the pg_xlog partition and restarted, it took
some time to replay all of the log (15-20 minutes) and everything
recovered with no data corruption!  However, the theory about the data
partition filling up first didn't happen in my case.  The data partition
was (and still is) less than 50% utilized.  My pg_xlog files typically
run around 400MB, but with the long running update filled up the entire
10GB partition.  (which is now a 70 GB partition)  

So, I'm still not sure what caused the problem.  When I get back to work
(or maybe sooner), I'll take a look in the PG logs and post anything
that looks suspicious here.  Thanks for all of your comments and
suggestions.  Even though I haven't figured out the root of the problem
yet, they've been very informative.