pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedinvalid page header
11+ messages / 7 participants
[nested] [flat]
* invalid page header
@ 2003-09-25 19:01 Hornyak Laszlo <kocka@tigrasoft.hu>
2003-09-26 00:27 ` Re: invalid page header Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 11+ messages in thread
From: Hornyak Laszlo @ 2003-09-25 19:01 UTC (permalink / raw)
To: pgsql-admin
Hi folks!
I have a problem again, now with PostgreSQL 7.3.4. One of my 1.8 million
rows tables have damaged, it operated all the day and now it is off, we
don`t know the reason. When we try to dump or select from it, it says
'Invalid page header in block 12345 in prep'.
Does anybody know how to make our table healthy again?
I have read oin the manual about pg_resetxlog, it says it may fix the
problem, but I should use it only as last solution. What is the risk of
using it? May it damage other tables?
Thanks in advance,
Laszlo Hornyak
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: invalid page header
2003-09-25 19:01 invalid page header Hornyak Laszlo <kocka@tigrasoft.hu>
@ 2003-09-26 00:27 ` Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 11+ messages in thread
From: Tom Lane @ 2003-09-26 00:27 UTC (permalink / raw)
To: Hornyak Laszlo <kocka@tigrasoft.hu>; +Cc: pgsql-admin
Hornyak Laszlo <kocka@tigrasoft.hu> writes:
> I have a problem again, now with PostgreSQL 7.3.4. One of my 1.8 million
> rows tables have damaged, it operated all the day and now it is off, we
> don`t know the reason. When we try to dump or select from it, it says
> 'Invalid page header in block 12345 in prep'.
> Does anybody know how to make our table healthy again?
You could zero out that page (while the postmaster is stopped,
preferably) but you ought to first try to figure out what data you are
going to lose. Dumping the page with pg_filedump might provide some
clues. See this thread for a recent exercise in file-dumping:
http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&threadm=20030922162322.E12708%40quart...
Or you could just use dd; since the page header is corrupt, it's
unlikely that pg_filedump will be able to give you anything smarter
than a bare hex/ascii dump (-d mode), which dd and od could do as well.
regards, tom lane
^ permalink raw reply [nested|flat] 11+ messages in thread
* Invalid page header
@ 2004-07-22 17:29 Ian Burrell <imb@rentrak.com>
2004-07-22 18:24 ` Re: Invalid page header Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 11+ messages in thread
From: Ian Burrell @ 2004-07-22 17:29 UTC (permalink / raw)
To: pgsql-admin
I get the following error message when doing a select on a table:
ERROR: invalid page header in block 295 of relation "reported_titles"
I found some messages that said this means a block of this table is
corrupt. I found some suspicious lines in the server log just before:
ERROR: could not access status of transaction 3651584
DETAIL: could not open file "/usr/local/pgsql/data/pg_clog/0003": No
such file or directory
How do I fix this corruption? I have dumped as much of the databases as
I can including about half of this table. Only this table is corrupt.
What could cause the corruption? We are using custom C code. Could
bugs in this be causing it? Or is hardware problems more likely?
- Ian
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Invalid page header
2004-07-22 17:29 Invalid page header Ian Burrell <imb@rentrak.com>
@ 2004-07-22 18:24 ` Tom Lane <tgl@sss.pgh.pa.us>
2004-07-22 22:10 ` Re: Invalid page header Ian Burrell <imb@rentrak.com>
0 siblings, 1 reply; 11+ messages in thread
From: Tom Lane @ 2004-07-22 18:24 UTC (permalink / raw)
To: Ian Burrell <imb@rentrak.com>; +Cc: pgsql-admin
Ian Burrell <imb@rentrak.com> writes:
> I get the following error message when doing a select on a table:
> ERROR: invalid page header in block 295 of relation "reported_titles"
> How do I fix this corruption?
You can zap just the failed block by turning on "zero_damaged_pages";
that will at least allow you to recover the rest of the table. If you
want to try harder, you could look at the damaged page with pg_filedump
(http://sources.redhat.com/rhdb/) or a similar tool and try to intuit
how to fix it manually.
> What could cause the corruption? We are using custom C code. Could
> bugs in this be causing it? Or is hardware problems more likely?
Hmm. A scribble-on-memory kind of bug could cause this, but in my
experience it's unusual for coding errors to trash the disk buffers ---
that's a relatively small part of your address space, and usually a
memory clobber will crash the backend elsewhere before it hits a disk
buffer. (BTW, one reason we force a database restart after a backend
crash is in hopes of not letting any such clobber make it to disk. The
contents of shared disk buffers are simply thrown away in a restart.)
It would probably be worth your while to look at the damaged page with
pg_filedump before you zap it. The symptoms of hardware misfeasance and
software errors are enough different that you can often tell which
theory to believe by examining the bits.
regards, tom lane
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Invalid page header
2004-07-22 17:29 Invalid page header Ian Burrell <imb@rentrak.com>
2004-07-22 18:24 ` Re: Invalid page header Tom Lane <tgl@sss.pgh.pa.us>
@ 2004-07-22 22:10 ` Ian Burrell <imb@rentrak.com>
2004-07-22 22:30 ` Re: Invalid page header Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 11+ messages in thread
From: Ian Burrell @ 2004-07-22 22:10 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-admin
Tom Lane wrote:
>
> You can zap just the failed block by turning on "zero_damaged_pages";
> that will at least allow you to recover the rest of the table. If you
> want to try harder, you could look at the damaged page with pg_filedump
> (http://sources.redhat.com/rhdb/) or a similar tool and try to intuit
> how to fix it manually.
>
I zapped the damaged block. It didn't seem to effect the rows in the
table. My suspicion is that the page only contained deleted rows since
the table had many updates done recently.
> Hmm. A scribble-on-memory kind of bug could cause this, but in my
> experience it's unusual for coding errors to trash the disk buffers ---
> that's a relatively small part of your address space, and usually a
> memory clobber will crash the backend elsewhere before it hits a disk
> buffer. (BTW, one reason we force a database restart after a backend
> crash is in hopes of not letting any such clobber make it to disk. The
> contents of shared disk buffers are simply thrown away in a restart.)
>
> It would probably be worth your while to look at the damaged page with
> pg_filedump before you zap it. The symptoms of hardware misfeasance and
> software errors are enough different that you can often tell which
> theory to believe by examining the bits.
>
I used pg_filedump on a backup of the database files. The block looks
like it is mostly zero bytes with a few x02 bytes thrown to just be
confusing.
- Ian
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Invalid page header
2004-07-22 17:29 Invalid page header Ian Burrell <imb@rentrak.com>
2004-07-22 18:24 ` Re: Invalid page header Tom Lane <tgl@sss.pgh.pa.us>
2004-07-22 22:10 ` Re: Invalid page header Ian Burrell <imb@rentrak.com>
@ 2004-07-22 22:30 ` Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 11+ messages in thread
From: Tom Lane @ 2004-07-22 22:30 UTC (permalink / raw)
To: Ian Burrell <imb@rentrak.com>; +Cc: pgsql-admin
Ian Burrell <imb@rentrak.com> writes:
> Tom Lane wrote:
>> It would probably be worth your while to look at the damaged page with
>> pg_filedump before you zap it. The symptoms of hardware misfeasance and
>> software errors are enough different that you can often tell which
>> theory to believe by examining the bits.
> I used pg_filedump on a backup of the database files. The block looks
> like it is mostly zero bytes with a few x02 bytes thrown to just be
> confusing.
My interpretation of that would be a hardware glitch. A software
problem would be more likely to look like copying the wrong data
into the block, or possibly zeroing out the block when it shouldn't
--- but the sprinkling of x02's rules out a misaimed memset().
Time to break out the RAM and disk test programs ...
regards, tom lane
^ permalink raw reply [nested|flat] 11+ messages in thread
* Invalid Page Header
@ 2006-10-16 13:55 André José Guergolet <AGuergolet@compugraf.com.br>
0 siblings, 0 replies; 11+ messages in thread
From: André José Guergolet @ 2006-10-16 13:55 UTC (permalink / raw)
To: pgsql-admin
Hi everybody,
I have a Postgres Database Server 8.0 in a Windows 2000 Professional.
I need to migrate this server to 8.1.4 or 8.1.5 Posgresql Version, but, when I execute backup command from pgAdmin III or from a command line (pg_dump [parameters] ) I get this error:
invalid page header in block 4686 of relation "tb_Bilhete"
Details:
==========================
pg_dump: ERROR: invalid page header in block 4686 of relation "tb_Bilhete"
pg_dump: SQL command to dump the contents of table "tb_Bilhete" failed: PQendcopy() failed.
pg_dump: Error message from server: ERROR: invalid page header in block 4686 of relation "tb_Bilhete"
pg_dump: The command was: COPY public."tb_Bilhete" ("inIdBilhete", "dtDataHora", "inDuracao", "vcNumRefPrivado", "vcNumRefPublico", "vcNumPartPrivado", "vcNumPartPublico", "chDirecaoRef", "vcTronco", "chInterno", "chProcessado") TO stdout;
pg_dump: *** aborted because of error
Process returned exit code 1.
==========================
How Can I resolve this problem?
I have other backup, but I need the more recent backup.
I search at the internet about this problem, but I find solutions only for Linux System using Linux tools.
Thanks
André Guergolet
^ permalink raw reply [nested|flat] 11+ messages in thread
* Invalid page header
@ 2007-01-08 02:22 Robert Leftwich <pg@rtl.fmailbox.com>
2007-01-08 03:32 ` Re: Invalid page header Guido Barosio <gbarosio@gmail.com>
0 siblings, 1 reply; 11+ messages in thread
From: Robert Leftwich @ 2007-01-08 02:22 UTC (permalink / raw)
To: pgsql-admin
Along similar lines to other recent posts I've noticed in the logs some
"Invalid page header in block ### of relation ...". errors. Given that
we have two basically identical s/w setups (ubuntu 6.06 LTS, PG8.1.4)
running on different h/w and only 1 box is reporting these errors and
based on the feedback to other similar posts, it looks like a h/w issue.
I'd like to hear the lists collective wisdom on what would be the best
way to identify what is causing these errors, given that there are no
system-level errors being reported.
Robert
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Invalid page header
2007-01-08 02:22 Invalid page header Robert Leftwich <pg@rtl.fmailbox.com>
@ 2007-01-08 03:32 ` Guido Barosio <gbarosio@gmail.com>
0 siblings, 0 replies; 11+ messages in thread
From: Guido Barosio @ 2007-01-08 03:32 UTC (permalink / raw)
To: Robert Leftwich <pg@rtl.fmailbox.com>; +Cc: pgsql-admin
Robert,
Unfortunatelly yes, that error is suposed to be related with
hardware problems. In my case, a forced reindex sorted the problem for
a while.
smartmontools
badblocks
memtest86
Those are the tools I've used time ago to get the results for bad hardware.
Best wishes,
g.-
On 1/7/07, Robert Leftwich <pg@rtl.fmailbox.com> wrote:
> Along similar lines to other recent posts I've noticed in the logs some
> "Invalid page header in block ### of relation ...". errors. Given that
> we have two basically identical s/w setups (ubuntu 6.06 LTS, PG8.1.4)
> running on different h/w and only 1 box is reporting these errors and
> based on the feedback to other similar posts, it looks like a h/w issue.
> I'd like to hear the lists collective wisdom on what would be the best
> way to identify what is causing these errors, given that there are no
> system-level errors being reported.
>
> Robert
>
> ---------------------------(end of broadcast)---------------------------
> TIP 7: You can help support the PostgreSQL project by donating at
>
> http://www.postgresql.org/about/donate
>
--
Guido Barosio
-----------------------
http://www.globant.com
guido.barosio@globant.com
^ permalink raw reply [nested|flat] 11+ messages in thread
* Invalid page header
@ 2010-09-01 21:18 Ireneusz Pluta <ipluta@wp.pl>
2010-09-01 21:36 ` Re: Invalid page header Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 11+ messages in thread
From: Ireneusz Pluta @ 2010-09-01 21:18 UTC (permalink / raw)
To: pgsql-admin
Hello,
I have a server, 8.4.3, where I get intermittent and rather rare cases
of "invalid page headers". Quick search over the pg lists shows a
general advice to "check your hardware". Yes, I need to schedule a
downtime and perform some checks.
However, let me also share with you what I noticed and maybe you can
comment or suggest more than that.
As I said, I already had a few cases of invalid page header on that
server, but did not take an extensive care of them, as they always were
related to the same table, or its indexes. They could be easily dropped
and rebuilt, because that table depended on other tables. So I was happy
with doing just that. There were just a few such cases within 10 months
of lifetime of this server (and that was the actual reason I reported
autovacuum getting messed with invalid page header not taken care of for
a long time, earlier this year).
But the last time the invalid page header happened to another table,
which, actually, is a master source for many other tables in my
database, so I had to really take care of this case. What I have noticed
about this case was:
- this is a costantly growing table collecting raw information. The data
contained in the damaged page was accessed several times after its
insertion within a few hours, before finally a yet another access ended
with "invalid page header" error.
- there was exactly one page damaged. No other damages around. The
system is running on freebsd7.2, ufs with 16k block size, on a raid10
with 256 stripe size, if this matters
- when playing with pg_filedump I noticed that last pages of the table
are always initially reported as damaged, as they come, then, as newer
pages get allocated and filled, these initially bad pages "become
valid", as in the following example repeating the same pg_filedump.
[pgsql@gil ~]$ pg_filedump data/base/18319/36870.43 | grep -B9 -i
"invalid header" | grep ^Block
Block 7460 ********************************************************
Block 11457 ********************************************************
Block 11460 ********************************************************
Block 11461 ********************************************************
[pgsql@gil ~]$ pg_filedump data/base/18319/36870.43 | grep -B9 -i
"invalid header" | grep ^Block
Block 7460 ********************************************************
Block 11460 ********************************************************
Block 11461 ********************************************************
Block 11462 ********************************************************
[pgsql@gil ~]$ pg_filedump data/base/18319/36870.43 | grep -B9 -i
"invalid header" | grep ^Block
Block 7460 ********************************************************
Block 11461 ********************************************************
Block 11462 ********************************************************
Block 11463 ********************************************************
- Block 7460 above is the one which actually got currupted. In spite I
zeroed it with the zero_damaged_pages option it is still reported as invalid
Do the above remarks indicate that something else, other than
hard-to-find hardware issue, might be tracked in a more detailed way?
Thanks
Irek.
^ permalink raw reply [nested|flat] 11+ messages in thread
* Re: Invalid page header
2010-09-01 21:18 Invalid page header Ireneusz Pluta <ipluta@wp.pl>
@ 2010-09-01 21:36 ` Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 11+ messages in thread
From: Tom Lane @ 2010-09-01 21:36 UTC (permalink / raw)
To: Ireneusz Pluta <ipluta@wp.pl>; +Cc: pgsql-admin
Ireneusz Pluta <ipluta@wp.pl> writes:
> - when playing with pg_filedump I noticed that last pages of the table
> are always initially reported as damaged, as they come, then, as newer
> pages get allocated and filled, these initially bad pages "become
> valid", as in the following example repeating the same pg_filedump.
This doesn't seem terribly surprising. A newly-added page on disk will
be initially filled with zeroes, which I think pg_filedump will complain
about. It won't get overwritten with "valid" data until the page is
next written, either because of a checkpoint or because the buffer space
is needed for another page. pg_filedump can't see the state of the page
within the server's buffers, which is what counts here.
regards, tom lane
^ permalink raw reply [nested|flat] 11+ messages in thread
end of thread, other threads:[~2010-09-01 21:36 UTC | newest]
Thread overview: 11+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2003-09-25 19:01 invalid page header Hornyak Laszlo <kocka@tigrasoft.hu>
2003-09-26 00:27 ` Tom Lane <tgl@sss.pgh.pa.us>
2004-07-22 17:29 Invalid page header Ian Burrell <imb@rentrak.com>
2004-07-22 18:24 ` Re: Invalid page header Tom Lane <tgl@sss.pgh.pa.us>
2004-07-22 22:10 ` Re: Invalid page header Ian Burrell <imb@rentrak.com>
2004-07-22 22:30 ` Re: Invalid page header Tom Lane <tgl@sss.pgh.pa.us>
2006-10-16 13:55 Invalid Page Header André José Guergolet <AGuergolet@compugraf.com.br>
2007-01-08 02:22 Invalid page header Robert Leftwich <pg@rtl.fmailbox.com>
2007-01-08 03:32 ` Re: Invalid page header Guido Barosio <gbarosio@gmail.com>
2010-09-01 21:18 Invalid page header Ireneusz Pluta <ipluta@wp.pl>
2010-09-01 21:36 ` Re: Invalid page header Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox