pg.ddx.io pgsql-admin@postgresql.org mailing list archive
help / color / mirror / Atom feedQuestion on Postgres Index internals
4+ messages / 2 participants
[nested] [flat]
* Question on Postgres Index internals
@ 2011-08-18 21:46 A J <s5aly@yahoo.com>
2011-08-19 06:40 ` Re: Question on Postgres Index internals Craig Ringer <ringerc@ringerc.id.au>
0 siblings, 1 reply; 4+ messages in thread
From: A J @ 2011-08-18 21:46 UTC (permalink / raw)
To: pgsql-admin
How does Postgres detect corruption of index data files ? What is the exact mechanism by which it knows that the index is corrupt ? Can it happen that part of the index is corrupt but Postgres does not realize because those specific rows are not accessed (but other rows from that table are accessed)
Thanks for any inputs.
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Question on Postgres Index internals
2011-08-18 21:46 Question on Postgres Index internals A J <s5aly@yahoo.com>
@ 2011-08-19 06:40 ` Craig Ringer <ringerc@ringerc.id.au>
2011-08-21 15:07 ` Re: Question on Postgres Index internals A J <s5aly@yahoo.com>
0 siblings, 1 reply; 4+ messages in thread
From: Craig Ringer @ 2011-08-19 06:40 UTC (permalink / raw)
To: A J <s5aly@yahoo.com>; +Cc: pgsql-admin
On 19/08/2011 5:46 AM, A J wrote:
> How does Postgres detect corruption of index data files ?
If their structure doesn't make sense, they're corrupt. It doesn't
actively seek or detect corruption, because it shouldn't happen, and
cannot be reliably discovered if it has.
> What is the exact mechanism by which it knows that the index is corrupt ?
Numerous different ways, depending on the nature of the damage and the
type of index. Often it won't know the index is damaged at all.
> Can it happen that part of the index is corrupt but Postgres does not
> realize because those specific rows are not accessed (but other rows
> from that table are accessed)
>
Yes, that is possible. It is also possible that a damaged index could
cause issues like multiple instances of a value that's supposed to be
unique, or a foreign key reference to a non-existent tuple.
Index corruption should not happen. If you are facing index corruption,
either you have a hardware problem or you've hit a bug. The vast
majority of cases are hardware faults.
Perhaps it'd help if you'd supply a little more detail about the
background. Why you're asking this, and why you (presumably) suspect you
have index corruption.
--
Craig Ringer
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Question on Postgres Index internals
2011-08-18 21:46 Question on Postgres Index internals A J <s5aly@yahoo.com>
2011-08-19 06:40 ` Re: Question on Postgres Index internals Craig Ringer <ringerc@ringerc.id.au>
@ 2011-08-21 15:07 ` A J <s5aly@yahoo.com>
2011-08-26 04:14 ` Re: Question on Postgres Index internals Craig Ringer <ringerc@ringerc.id.au>
0 siblings, 1 reply; 4+ messages in thread
From: A J @ 2011-08-21 15:07 UTC (permalink / raw)
To: Craig Ringer <ringerc@ringerc.id.au>; +Cc: pgsql-admin
So I assume there is no redundant information stored such as checksum to validate the files against the stored checksums.
I don't have an active issue. Just trying to understand in detail how Postgres behaves under index corruption.
________________________________
From: Craig Ringer <ringerc@ringerc.id.au>
To: A J <s5aly@yahoo.com>
Cc: PG Admin <pgsql-admin@postgresql.org>
Sent: Friday, August 19, 2011 2:40 AM
Subject: Re: [ADMIN] Question on Postgres Index internals
On 19/08/2011 5:46 AM, A J wrote:
How does Postgres detect corruption of index data files ?
If their structure doesn't make sense, they're corrupt. It doesn't actively seek or detect corruption, because it shouldn't happen, and cannot be reliably discovered if it has.
What is the exact mechanism by which it knows that the index is corrupt ?
Numerous different ways, depending on the nature of the damage and the type of index. Often it won't know the index is damaged at all.
Can it happen that part of the index is corrupt but Postgres does not realize because those specific rows are not accessed (but other rows from that table are accessed)
>
>
Yes, that is possible. It is also possible that a damaged index could cause issues like multiple instances of a value that's supposed to be unique, or a foreign key reference to a non-existent tuple.
Index corruption should not happen. If you are facing index
corruption, either you have a hardware problem or you've hit a bug.
The vast majority of cases are hardware faults.
Perhaps it'd help if you'd supply a little more detail about the
background. Why you're asking this, and why you (presumably) suspect
you have index corruption.
--
Craig Ringer
^ permalink raw reply [nested|flat] 4+ messages in thread
* Re: Question on Postgres Index internals
2011-08-18 21:46 Question on Postgres Index internals A J <s5aly@yahoo.com>
2011-08-19 06:40 ` Re: Question on Postgres Index internals Craig Ringer <ringerc@ringerc.id.au>
2011-08-21 15:07 ` Re: Question on Postgres Index internals A J <s5aly@yahoo.com>
@ 2011-08-26 04:14 ` Craig Ringer <ringerc@ringerc.id.au>
0 siblings, 0 replies; 4+ messages in thread
From: Craig Ringer @ 2011-08-26 04:14 UTC (permalink / raw)
To: A J <s5aly@yahoo.com>; +Cc: pgsql-admin
On 21/08/2011 11:07 PM, A J wrote:
> So I assume there is no redundant information stored such as checksum
> to validate the files against the stored checksums.
Nope, there are no checksums.
It's the storage subsystem's job to make sure it doesn't mangle data on
disk and reads the data as it was written. It's the OS's job to make
sure nobody else overwrites or messes with that data while it's in RAM
cache or on disk.
Index corruption means you've hit a hardware fault, OS bug, or possibly
a PostgreSQL bug.
Only some kinds of corruption could be detected by application-level
checksums. If the OS or storage subsystem just failed to write a page,
Pg wouldn't be able to tell, for example. Checksums _would_ detect split
page writes, single-bit faults, corrupted disk sectors and the like and
could potentially be useful.
> I don't have an active issue. Just trying to understand in detail how
> Postgres behaves under index corruption.
Like most forms of fault behaviour, it is undefined in the "C standard"
sense of the meaning, though perhaps not quite as willing to just eat
data ;-)
--
Craig Ringer
^ permalink raw reply [nested|flat] 4+ messages in thread
end of thread, other threads:[~2011-08-26 04:14 UTC | newest]
Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2011-08-18 21:46 Question on Postgres Index internals A J <s5aly@yahoo.com>
2011-08-19 06:40 ` Craig Ringer <ringerc@ringerc.id.au>
2011-08-21 15:07 ` A J <s5aly@yahoo.com>
2011-08-26 04:14 ` Craig Ringer <ringerc@ringerc.id.au>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox