pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedFATAL: cache lookup failed for access method 6881280
7+ messages / 3 participants
[nested] [flat]
* FATAL: cache lookup failed for access method 6881280
@ 2005-08-02 23:59 Brock Williams <brock@cotcomsol.com>
2005-08-03 00:37 ` Re: FATAL: cache lookup failed for access method 6881280 Chris Travers <chris@travelamericas.com>
2005-08-03 02:55 ` Re: FATAL: cache lookup failed for access method 6881280 Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 2 replies; 7+ messages in thread
From: Brock Williams @ 2005-08-02 23:59 UTC (permalink / raw)
To: pgsql-admin
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I'm getting this message when I try to access one of the databases in
my cluster:
FATAL: cache lookup failed for access method 6881280
I get the message any way I try to access it: remotely via TCP,
locally on the db server, as well as with a single-user mode postgres
command.
Any ideas what could be the cause, or what I could do to fix it? All
the other (30+) databases in the cluster appear to be perfectly fine.
Thanks,
Brock
- --
Brock Williams brock@cotcomsol.com
Cottonwood Computer Solutions, Inc.
www.cotcomsol.com 406-896-4910
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (FreeBSD)
iD8DBQFC8AjygMCVEEcrWuIRAr0rAJ0TJwvWDxxNSYq2nib2P2ix2IT9ggCeNixx
UvLi8JF08ga2PiBW6hr0DaU=
=Wum4
-----END PGP SIGNATURE-----
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: FATAL: cache lookup failed for access method 6881280
2005-08-02 23:59 FATAL: cache lookup failed for access method 6881280 Brock Williams <brock@cotcomsol.com>
@ 2005-08-03 00:37 ` Chris Travers <chris@travelamericas.com>
2005-08-03 00:00 ` Re: FATAL: cache lookup failed for access method 6881280 Brock Williams <brock@cotcomsol.com>
1 sibling, 1 reply; 7+ messages in thread
From: Chris Travers @ 2005-08-03 00:37 UTC (permalink / raw)
To: Brock Williams <brock@cotcomsol.com>; pgsql-admin
Have you tried restarting the Postmaster?
Best Wishes,
Chris Travers
Metatron Technology Consulting
Brock Williams wrote:
>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>I'm getting this message when I try to access one of the databases in
>my cluster:
>
>FATAL: cache lookup failed for access method 6881280
>
>I get the message any way I try to access it: remotely via TCP,
>locally on the db server, as well as with a single-user mode postgres
>command.
>
>Any ideas what could be the cause, or what I could do to fix it? All
>the other (30+) databases in the cluster appear to be perfectly fine.
>
>Thanks,
>
>Brock
>- --
>Brock Williams brock@cotcomsol.com
>Cottonwood Computer Solutions, Inc.
>www.cotcomsol.com 406-896-4910
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG v1.4.0 (FreeBSD)
>
>iD8DBQFC8AjygMCVEEcrWuIRAr0rAJ0TJwvWDxxNSYq2nib2P2ix2IT9ggCeNixx
>UvLi8JF08ga2PiBW6hr0DaU=
>=Wum4
>-----END PGP SIGNATURE-----
>
>---------------------------(end of broadcast)---------------------------
>TIP 5: don't forget to increase your free space map settings
>
>
>
>
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: FATAL: cache lookup failed for access method 6881280
2005-08-02 23:59 FATAL: cache lookup failed for access method 6881280 Brock Williams <brock@cotcomsol.com>
2005-08-03 00:37 ` Re: FATAL: cache lookup failed for access method 6881280 Chris Travers <chris@travelamericas.com>
@ 2005-08-03 00:00 ` Brock Williams <brock@cotcomsol.com>
0 siblings, 0 replies; 7+ messages in thread
From: Brock Williams @ 2005-08-03 00:00 UTC (permalink / raw)
To: ; +Cc: pgsql-admin
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Yep. I tried starting and stopping a couple of times. I also tried
stopping postmaster and starting a single-user mode postgres process.
Same result.
Brock
Chris Travers wrote:
> Have you tried restarting the Postmaster?
>
> Best Wishes,
> Chris Travers
> Metatron Technology Consulting
>
> Brock Williams wrote:
>
> I'm getting this message when I try to access one of the databases in
> my cluster:
>
> FATAL: cache lookup failed for access method 6881280
>
> I get the message any way I try to access it: remotely via TCP,
> locally on the db server, as well as with a single-user mode postgres
> command.
>
> Any ideas what could be the cause, or what I could do to fix it? All
> the other (30+) databases in the cluster appear to be perfectly fine.
>
> Thanks,
>
> Brock
>>
- ---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings
>>
>>
>>
> ---------------------------(end of broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (FreeBSD)
iD8DBQFC8AkegMCVEEcrWuIRApjZAJ9qvkzqIvVHKkgKyOfEl/Z3xr0JbACgtc0U
5CsANX2XfrkEkBoXRSPv298=
=3uBS
-----END PGP SIGNATURE-----
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: FATAL: cache lookup failed for access method 6881280
2005-08-02 23:59 FATAL: cache lookup failed for access method 6881280 Brock Williams <brock@cotcomsol.com>
@ 2005-08-03 02:55 ` Tom Lane <tgl@sss.pgh.pa.us>
2005-08-03 04:11 ` Re: FATAL: cache lookup failed for access method 6881280 Brock Williams <brock@cotcomsol.com>
1 sibling, 1 reply; 7+ messages in thread
From: Tom Lane @ 2005-08-03 02:55 UTC (permalink / raw)
To: Brock Williams <brock@cotcomsol.com>; +Cc: pgsql-admin
Brock Williams <brock@cotcomsol.com> writes:
> I'm getting this message when I try to access one of the databases in
> my cluster:
> FATAL: cache lookup failed for access method 6881280
Unless you are using a custom index access method you didn't tell us
about, this is pretty strong evidence of data corruption in the pg_class
system catalog (specifically some pg_class.relam entry).
If the corruption extends only as far as those four bytes, it'd be
pretty easy to poke the correct value back in ... but it seems likely
that this is just the tip of the iceberg :-(
Please get a copy of pg_filedump from
http://sources.redhat.com/rhdb/utilities.html
and show us a dump of the pg_class file from that database (I like the
format it produces with -i -f options; note pg_class is file 1259 in all
known PG versions).
regards, tom lane
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: FATAL: cache lookup failed for access method 6881280
2005-08-02 23:59 FATAL: cache lookup failed for access method 6881280 Brock Williams <brock@cotcomsol.com>
2005-08-03 02:55 ` Re: FATAL: cache lookup failed for access method 6881280 Tom Lane <tgl@sss.pgh.pa.us>
@ 2005-08-03 04:11 ` Brock Williams <brock@cotcomsol.com>
2005-08-03 14:32 ` Re: FATAL: cache lookup failed for access method 6881280 Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 7+ messages in thread
From: Brock Williams @ 2005-08-03 04:11 UTC (permalink / raw)
To: ; +Cc: pgsql-admin
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Tried to post directy to the list but I got a bounce back. Must be too
big...
Here it is posted to my web site:
http://www.cotcomsol.com/~brock/postgresql_debug.txt
Thanks for the help,
Brock
Tom Lane wrote:
> Brock Williams <brock@cotcomsol.com> writes:
>
>>I'm getting this message when I try to access one of the databases in
>>my cluster:
>
>
>>FATAL: cache lookup failed for access method 6881280
>
>
> Unless you are using a custom index access method you didn't tell us
> about, this is pretty strong evidence of data corruption in the pg_class
> system catalog (specifically some pg_class.relam entry).
>
> If the corruption extends only as far as those four bytes, it'd be
> pretty easy to poke the correct value back in ... but it seems likely
> that this is just the tip of the iceberg :-(
>
> Please get a copy of pg_filedump from
> http://sources.redhat.com/rhdb/utilities.html
> and show us a dump of the pg_class file from that database (I like the
> format it produces with -i -f options; note pg_class is file 1259 in all
> known PG versions).
>
> regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 5: don't forget to increase your free space map settings
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (FreeBSD)
iD8DBQFC8EPYgMCVEEcrWuIRAhCcAJ9RZEMrKFpbRfjHOVW1Vfj6eys+VgCfaKhA
2EV6MSgumAvDv+Lv0k2WrdI=
=IZnE
-----END PGP SIGNATURE-----
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: FATAL: cache lookup failed for access method 6881280
2005-08-02 23:59 FATAL: cache lookup failed for access method 6881280 Brock Williams <brock@cotcomsol.com>
2005-08-03 02:55 ` Re: FATAL: cache lookup failed for access method 6881280 Tom Lane <tgl@sss.pgh.pa.us>
2005-08-03 04:11 ` Re: FATAL: cache lookup failed for access method 6881280 Brock Williams <brock@cotcomsol.com>
@ 2005-08-03 14:32 ` Tom Lane <tgl@sss.pgh.pa.us>
2005-08-03 17:05 ` Re: FATAL: cache lookup failed for access method 6881280 Brock Williams <brock@cotcomsol.com>
0 siblings, 1 reply; 7+ messages in thread
From: Tom Lane @ 2005-08-03 14:32 UTC (permalink / raw)
To: Brock Williams <brock@cotcomsol.com>; +Cc: pgsql-admin
Brock Williams <brock@cotcomsol.com> writes:
> Tried to post directy to the list but I got a bounce back. Must be too
> big...
> Here it is posted to my web site:
> http://www.cotcomsol.com/~brock/postgresql_debug.txt
Hmm, this is quite odd. You have
Item 22 -- Length: 156 Offset: 4760 (0x1298) Flags: USED
XMIN: 1 CMIN: 0 XMAX: 0 CMAX|XVAC: 0
Block Id: 1 linp Index: 22 Attributes: 24 Size: 32
infomask: 0x0910 (HASOID|XMIN_COMMITTED|XMAX_INVALID)
1298: 01000000 00000000 00000000 00000000 ................
12a8: 00000100 16001800 10092000 4f410000 .......... .OA..
12b8: 70675f70 726f635f 70726f6e 616d655f pg_proc_proname_
12c8: 61726773 5f6e7370 5f696e64 65780000 args_nsp_index..
12d8: 00000000 00000000 00000000 00000000 ................
12e8: 00000000 00000000 00000000 00000000 ................
12f8: 0b000000 00000000 01000000 00006900 ..............i.
--------
1308: 01000000 00000000 94000000 00c0da44 ...............D
--------
1318: 01000000 00000000 00006900 04000000 ..........i.....
--------
1328: 00000000 00000000 00000000 ............
A non-broken 8.0 installation has
Item 22 -- Length: 156 Offset: 4760 (0x1298) Flags: USED
XMIN: 1 CMIN: 0 XMAX: 0 CMAX|XVAC: 0
Block Id: 1 linp Index: 22 Attributes: 24 Size: 32
infomask: 0x0910 (HASOID|XMIN_COMMITTED|XMAX_INVALID)
1298: 01000000 00000000 00000000 00000000 ................
12a8: 00000100 16001800 10092000 4f410000 .......... .OA..
12b8: 70675f70 726f635f 70726f6e 616d655f pg_proc_proname_
12c8: 61726773 5f6e7370 5f696e64 65780000 args_nsp_index..
12d8: 00000000 00000000 00000000 00000000 ................
12e8: 00000000 00000000 00000000 00000000 ................
12f8: 0b000000 00000000 01000000 93010000 ................
--------
1308: 4f410000 00000000 94000000 0060da44 OA...........`.D
--------
1318: 00000000 00000000 00006900 04000000 ..........i.....
--------
1328: 00000000 00000000 00000000 ............
where I underlined the parts that are different. The first different
word is the relam field, which leads directly to the reported error
message, and the other two changes are bad news as well. (Note: the
difference at offset 1314 is the reltuples field, which could
legitimately be different across installations, and your value matches
the other pg_proc indexes so it's presumably OK.)
I have no idea what happened here ... but if you can put those values
back the way they were you'll probably be all right. You'll need to
shut down the postmaster before editing the file (else it may have the
page cached in shared memory), and if there is a pg_internal.init file
in the same directory delete it.
regards, tom lane
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: FATAL: cache lookup failed for access method 6881280
2005-08-02 23:59 FATAL: cache lookup failed for access method 6881280 Brock Williams <brock@cotcomsol.com>
2005-08-03 02:55 ` Re: FATAL: cache lookup failed for access method 6881280 Tom Lane <tgl@sss.pgh.pa.us>
2005-08-03 04:11 ` Re: FATAL: cache lookup failed for access method 6881280 Brock Williams <brock@cotcomsol.com>
2005-08-03 14:32 ` Re: FATAL: cache lookup failed for access method 6881280 Tom Lane <tgl@sss.pgh.pa.us>
@ 2005-08-03 17:05 ` Brock Williams <brock@cotcomsol.com>
0 siblings, 0 replies; 7+ messages in thread
From: Brock Williams @ 2005-08-03 17:05 UTC (permalink / raw)
To: ; +Cc: pgsql-admin
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Tom:
Thanks for the help. I hex-edited the file back to what you
recommended, and it did in fact work. I did a dump/restore on the
database just to be safe as well.
Brock
Tom Lane wrote:
> Brock Williams <brock@cotcomsol.com> writes:
>
>>Tried to post directy to the list but I got a bounce back. Must be too
>>big...
>>Here it is posted to my web site:
>>http://www.cotcomsol.com/~brock/postgresql_debug.txt
>
>
> Hmm, this is quite odd. You have
>
> Item 22 -- Length: 156 Offset: 4760 (0x1298) Flags: USED
> XMIN: 1 CMIN: 0 XMAX: 0 CMAX|XVAC: 0
> Block Id: 1 linp Index: 22 Attributes: 24 Size: 32
> infomask: 0x0910 (HASOID|XMIN_COMMITTED|XMAX_INVALID)
>
> 1298: 01000000 00000000 00000000 00000000 ................
> 12a8: 00000100 16001800 10092000 4f410000 .......... .OA..
> 12b8: 70675f70 726f635f 70726f6e 616d655f pg_proc_proname_
> 12c8: 61726773 5f6e7370 5f696e64 65780000 args_nsp_index..
> 12d8: 00000000 00000000 00000000 00000000 ................
> 12e8: 00000000 00000000 00000000 00000000 ................
> 12f8: 0b000000 00000000 01000000 00006900 ..............i.
> --------
> 1308: 01000000 00000000 94000000 00c0da44 ...............D
> --------
> 1318: 01000000 00000000 00006900 04000000 ..........i.....
> --------
> 1328: 00000000 00000000 00000000 ............
>
> A non-broken 8.0 installation has
>
> Item 22 -- Length: 156 Offset: 4760 (0x1298) Flags: USED
> XMIN: 1 CMIN: 0 XMAX: 0 CMAX|XVAC: 0
> Block Id: 1 linp Index: 22 Attributes: 24 Size: 32
> infomask: 0x0910 (HASOID|XMIN_COMMITTED|XMAX_INVALID)
>
> 1298: 01000000 00000000 00000000 00000000 ................
> 12a8: 00000100 16001800 10092000 4f410000 .......... .OA..
> 12b8: 70675f70 726f635f 70726f6e 616d655f pg_proc_proname_
> 12c8: 61726773 5f6e7370 5f696e64 65780000 args_nsp_index..
> 12d8: 00000000 00000000 00000000 00000000 ................
> 12e8: 00000000 00000000 00000000 00000000 ................
> 12f8: 0b000000 00000000 01000000 93010000 ................
> --------
> 1308: 4f410000 00000000 94000000 0060da44 OA...........`.D
> --------
> 1318: 00000000 00000000 00006900 04000000 ..........i.....
> --------
> 1328: 00000000 00000000 00000000 ............
>
> where I underlined the parts that are different. The first different
> word is the relam field, which leads directly to the reported error
> message, and the other two changes are bad news as well. (Note: the
> difference at offset 1314 is the reltuples field, which could
> legitimately be different across installations, and your value matches
> the other pg_proc indexes so it's presumably OK.)
>
> I have no idea what happened here ... but if you can put those values
> back the way they were you'll probably be all right. You'll need to
> shut down the postmaster before editing the file (else it may have the
> page cached in shared memory), and if there is a pg_internal.init file
> in the same directory delete it.
>
> regards, tom lane
- --
Brock Williams brock@cotcomsol.com
Cottonwood Computer Solutions, Inc.
www.cotcomsol.com 406-896-4910
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (FreeBSD)
iD8DBQFC8PlMgMCVEEcrWuIRArLtAJ0fSPeN7z6a1XfiIuXqMEBizAS4wACgnLcp
hb+XBsKHmi7ToPf4p+D0Btg=
=hjNE
-----END PGP SIGNATURE-----
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2005-08-03 17:05 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2005-08-02 23:59 FATAL: cache lookup failed for access method 6881280 Brock Williams <brock@cotcomsol.com>
2005-08-03 00:37 ` Chris Travers <chris@travelamericas.com>
2005-08-03 00:00 ` Brock Williams <brock@cotcomsol.com>
2005-08-03 02:55 ` Re: FATAL: cache lookup failed for access method 6881280 Tom Lane <tgl@sss.pgh.pa.us>
2005-08-03 04:11 ` Re: FATAL: cache lookup failed for access method 6881280 Brock Williams <brock@cotcomsol.com>
2005-08-03 14:32 ` Re: FATAL: cache lookup failed for access method 6881280 Tom Lane <tgl@sss.pgh.pa.us>
2005-08-03 17:05 ` Re: FATAL: cache lookup failed for access method 6881280 Brock Williams <brock@cotcomsol.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox