From: Tom Lane <tgl@sss.pgh.pa.us>
To: Adam Witney <awitney@sgul.ac.uk>
Cc: pgsql-admin <pgsql-admin@postgresql.org>
Cc: Michael Fuhr <mike@fuhr.org>
Subject: Re: pg_dump error... Follow up
Date: Wed, 07 Sep 2005 13:06:26 -0400
Message-ID: <28205.1126112786@sss.pgh.pa.us> (raw)
In-Reply-To: <BF44BC99.4C70E%awitney@sgul.ac.uk>
References: <BF44BC99.4C70E%awitney@sgul.ac.uk>
Adam Witney <awitney@sgul.ac.uk> writes:
> I think I have found the offending row in measured_bioassay_base... The
> entry in its toast table looks like this
> bugasbase2=# select chunk_id, chunk_seq, length(chunk_data) from
> pg_toast.pg_toast_134401982 where chunk_id = 144391872;
> chunk_id | chunk_seq | length
> -----------+-----------+--------
> 144391872 | 0 | 1998
> 144391872 | 1 | 1998
> 144391872 | 2 | 1998
> 144391872 | 3 | 1998
> 144391872 | 4 | 1998
> 7625296 | 3292 | 24
> 7625297 | 3292 | 24
> 7625298 | 3292 | 24
> 7625299 | 3292 | 24
> 7625308 | 3292 | 19
This is pretty wacko, because as far as I can see there is nothing wrong
with the index at all --- in particular, no gap in the chunk id/seq
series here, according to pg_filedump. (The last two words of each
index item are the chunk_id and chunk_seq, and we are looking at
LSB-first data)
Item 102 -- Length: 16 Offset: 6544 (0x1990) Flags: USED
Block Id: 165390 linp Index: 4 Size: 16
Has Nulls: 0 Has Varwidths: 0
1990: 02000e86 04001000 c03e9b08 00000000 .........>......
Item 103 -- Length: 16 Offset: 6528 (0x1980) Flags: USED
Block Id: 165391 linp Index: 1 Size: 16
Has Nulls: 0 Has Varwidths: 0
1980: 02000f86 01001000 c03e9b08 01000000 .........>......
Item 104 -- Length: 16 Offset: 6512 (0x1970) Flags: USED
Block Id: 165391 linp Index: 2 Size: 16
Has Nulls: 0 Has Varwidths: 0
1970: 02000f86 02001000 c03e9b08 02000000 .........>......
Item 105 -- Length: 16 Offset: 6496 (0x1960) Flags: USED
Block Id: 165391 linp Index: 3 Size: 16
Has Nulls: 0 Has Varwidths: 0
1960: 02000f86 03001000 c03e9b08 03000000 .........>......
Item 106 -- Length: 16 Offset: 6480 (0x1950) Flags: USED
Block Id: 165391 linp Index: 4 Size: 16
Has Nulls: 0 Has Varwidths: 0
1950: 02000f86 04001000 c03e9b08 04000000 .........>......
Item 107 -- Length: 16 Offset: 6464 (0x1940) Flags: USED
Block Id: 165392 linp Index: 1 Size: 16
Has Nulls: 0 Has Varwidths: 0
1940: 02001086 01001000 c03e9b08 05000000 .........>......
Item 108 -- Length: 16 Offset: 6448 (0x1930) Flags: USED
Block Id: 165392 linp Index: 2 Size: 16
Has Nulls: 0 Has Varwidths: 0
1930: 02001086 02001000 c03e9b08 06000000 .........>......
Item 109 -- Length: 16 Offset: 6432 (0x1920) Flags: USED
Block Id: 165392 linp Index: 3 Size: 16
Has Nulls: 0 Has Varwidths: 0
1920: 02001086 03001000 c03e9b08 07000000 .........>......
Item 110 -- Length: 16 Offset: 6416 (0x1910) Flags: USED
Block Id: 165392 linp Index: 4 Size: 16
Has Nulls: 0 Has Varwidths: 0
1910: 02001086 04001000 c03e9b08 08000000 .........>......
What I think is going on is that you have a corrupted copy of this index
page sitting in shared buffers. It's not been marked dirty, else it'd
have been written out by a checkpoint and we could see the damage with
pg_filedump. This implies that the problem would go away by itself if
you simply shut down and restarted the database.
However, since you've been so helpful so far ;-) I wonder whether there
is any chance of getting an image of the bad page in memory before you
do that. We are presumably looking at the results of an unintentional
memory stomp somewhere in the code, and it's possible that a dump of the
damaged page would give a clue how it happened.
The only simple way I can think of for getting that info involves gdb,
but I don't suppose you are using a copy built with --enable-debug ...
anyone have other ideas?
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-admin@postgresql.org
Cc: tgl@sss.pgh.pa.us, awitney@sgul.ac.uk, mike@fuhr.org
Subject: Re: pg_dump error... Follow up
In-Reply-To: <28205.1126112786@sss.pgh.pa.us>
References: <BF44BC99.4C70E%awitney@sgul.ac.uk> <28205.1126112786@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox