pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Thomas F. O'Connell <tfo@sitening.com>
To: pgsql-admin@postgresql.org
Subject: Re: Invalid Page Headers
Date: Tue, 18 Apr 2006 13:19:48 -0500
Message-ID: <A8661BBC-FC31-4AA1-BA22-6FAC15B32D74@sitening.com> (raw)
In-Reply-To: <B8F30540-C0A5-4CF7-8DB2-867DFF25E58D@sitening.com>
References: <B8F30540-C0A5-4CF7-8DB2-867DFF25E58D@sitening.com>


On Apr 18, 2006, at 12:30 PM, Thomas F. O'Connell wrote:

> So there are currently three separate relations exhibiting invalid  
> page errors.
>
> This box is a Debian 3.1 box running a custom Linux 2.6.10 #6 SMP  
> kernel. Postgres 8.1.3 was compiled from source. pgpool 3.0.1, also  
> built from source, is used by some parts of the application layer.  
> The system is running on an ext3 filesystem, WAL is on a 4-disk  
> RAID 10 running JFS, and data is on a 12-disk RAID 10 running JFS.  
> I'm not seeing any signs of apparent kernel or hardware errors in  
> the system and kernel logs.

I take back the lack of errors. megamgr is now reporting 5 (!) failed  
drives on a single channel in the RAID 10 for data. The RAID card is  
a MegaRAID SCSI 320-2X.

I would've expected the RAID to protect postgres from the possibility  
of data corruption, but I guess not.

In any event, we're working on replacing the failed drives. After the  
RAID is rebuilt, though, the focus will be on data. Is my best bet to  
restore the corrupted relations, or can I repair them somehow?

And I'm still concerned about whether postgres will recover if I stop  
it at this point, so I'm working on contingency plans for leaving  
postgres online, turning off the application, restoring the tables  
while nothing is accessing postgres, and then restarting the  
application. Is there a safer/better course of action available?

--
Thomas F. O'Connell
Database Architecture and Programming
Sitening, LLC

http://www.sitening.com/
3004 B Poston Avenue
Nashville, TN 37203-1314
615-260-0005 (cell)
615-469-5150 (office)
615-469-5151 (fax)



view thread (10+ messages)  latest in thread

Message-ID: <A8661BBC-FC31-4AA1-BA22-6FAC15B32D74@sitening.com>
Permalink:  ../A8661BBC-FC31-4AA1-BA22-6FAC15B32D74@sitening.com/
Also on:    postgresql.org/message-id/A8661BBC-FC31-4AA1-BA22-6FAC15B32D74@sitening.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-admin@postgresql.org
  Cc: tfo@sitening.com
  Subject: Re: Invalid Page Headers
  In-Reply-To: <A8661BBC-FC31-4AA1-BA22-6FAC15B32D74@sitening.com>
  References: <B8F30540-C0A5-4CF7-8DB2-867DFF25E58D@sitening.com> <A8661BBC-FC31-4AA1-BA22-6FAC15B32D74@sitening.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox