From: Achilleas Mantzios <achill@matrix.gatewaynet.com>
Organization: Dynacom Tankers Mgmt
To: "pgsql-admin@postgresql.org" <pgsql-admin@postgresql.org>
Subject: pgsql 8.3.3 "Invalid page header" leads to "FATAL:  segment too big"
Date: Thu, 22 Apr 2010 14:16:08 +0200
User-Agent: KMail/1.9.7
MIME-Version: 1.0
Content-Type: text/plain;
  charset="us-ascii"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Message-Id: <201004221516.08729.achill@matrix.gatewaynet.com>

Hello,
i have this serious problem in one of our remote vessels. (comm is done by minicom to the remote satelite modem)
I think that this server was under some sort of constant resets or hardware failures.
Initially,i had this problem:
ERROR:  invalid page header in block 672720 of relation "pg_toast_125716009"

This toast table corresponds to a table named "mail_message", 
                                Table "public.mail_message"
  Column   |       Type        |                         Modifiers
-----------+-------------------+-----------------------------------------------------------
 msgno     | mail_msgno_domain | not null default nextval('mail_msgno_sequence'::regclass)
 msgsource | bytea             |
Indexes:
    "mail_message_key" PRIMARY KEY, btree (msgno)

(obviously the TOAST table serves the msgsource varlena) the contents of which is not of vital importance.
I tried, REINDEXING, with no success, and after that, i tried resetting the said block on disk as per this
suggestion by Tom here: http://old.nabble.com/invalid-page-header-td11981154.html

i found the oid of the table:
SELECT tableoid,1 from pg_toast.pg_toast_125716009 order by chunk_id LIMIT 1
 tableoid  | ?column?
-----------+----------
 125716013 |        1

(and just to verify)
SELECT relname from pg_class where oid=125716013;
      relname
--------------------
 pg_toast_125716009

Then i did: (as i said i do not need the contents of msgsource - yet the contents of msgno are vital)
dd if=/dev/zero of=/usr/local/var/lib/pgsql/data/base/125714957/125716013 seek=672720 bs=8192 count=1 

However, after that, unfortunately i get constant postgresql server restarts with:
FATAL:  segment too big
server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
The connection to the server was lost. Attempting reset: Succeeded.

Is there anything i can do to savage the situation?

(one of) the hard part here is that i dont have neither physical nor network access to the server
(only ultra expensive unreliable satellite comms)

Thanks for any hints...

-- 
Achilleas Mantzios