DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=mime-version:date:message-id:subject:from:to:content-type;
	b=eCVdjqubiVL/LJBbm6JHBfGmm3KwzrRKe/kP6oDHdQn2xiMQcS7F61CSOxVOyPB4uV
	h8n+OKDuHwdMGHEmBnFtF7v5rA0uizYLbApXPsBkEDRkcQYn3UflYLGNqsrEx9E+mEul
	bOLek3J2wwJ2IQHTFzeGou3LX/l1TgtV9a9wU=
MIME-Version: 1.0
Date: Fri, 26 Feb 2010 14:12:45 -0600
Message-ID: <e09f34d01002261212s4e1ea86csca52f95a274fdfe2@mail.gmail.com>
Subject: db recovery after hd crash (could not open relation 1663/16385/16400:
	No such file or directory)
From: Roland Wells <roland.wells@gmail.com>
To: pgsql-admin@postgresql.org
Content-Type: text/plain; charset=ISO-8859-1

Hello all,

Background:
DB resides on a mirrored array, HD1 goes bad and before it can be
replaced, HD2 starts throwing hardware errors. We were able to get a
DD from HD2 and have restored the box quite successfully with the
exception of the pg db. There WAS corruption to the filesystem.  Since
the last backup, a significant number of records have been added to
the db. If there is a way to recover some data, it would likely be
easier to reconcile and fix manually than reenter. Box is FreeBSD 8,
postgresql version 8.1.

Progress so far:
On new hardware, installed pg8.1 and restored data directory and
attempted to start pg. After fixing some config differences, it
complained of a missing pg_clog/xxx file. After searching the archives
for hints, added a file of zero's and pg starts successfully. The db
in question exists with what appears to be intact structure. When
attempting to dump the db ran into the error: "could not open relation
1663/16385/16400: No such file or directory" and run into similar
errors (with different relation numbers) trying to select * from
certain tables.

I'm stuck here. Assuming the damage to filesystem, what is the best a
way forward? Does the relation error indicate damage to the data? or
to an index??? Is there a process I can do to try and salvage partial
data?

Thanks for any help/tips/suggestions.

Roland Wells