From: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
To: Michael Paquier <michael@paquier.xyz>
Cc: Michael Banck <michael.banck@credativ.de>
Cc: PostgreSQL Development <pgsql-hackers@postgresql.org>
Subject: Re: [patch] Fix checksum verification in base backups for zero page headers
Date: Thu, 22 Oct 2020 15:11:45 +0300
Message-ID: <036235b4-2461-c4b2-a448-7875d943ce1e@postgrespro.ru> (raw)
In-Reply-To: <20201022012519.GF1475@paquier.xyz>
References: <2439c2d6cee13d611e23ba651e5eb4ab0469bf99.camel@credativ.de>
<d041090c-ddf7-10d1-7d50-2a4ed03c53dd@postgrespro.ru>
<4b1c16216dcb60e1dad96142d25fd06fb77485bf.camel@credativ.de>
<6713b5e9d98c7db1608b43cd57b10be4a08dca67.camel@credativ.de>
<9489c221-e2ef-e7ad-d921-5ccad71b14de@postgrespro.ru>
<423deac52479a40624e775e6d2ab41c4e9f5b19f.camel@credativ.de>
<20201007081852.GC30037@paquier.xyz>
<608f3476-0598-2514-2c03-e05c7d2b0cbd@postgrespro.ru>
<20201020062432.GA30362@paquier.xyz>
<69b38451-e91f-95fc-c3ce-06eb778d36e0@postgrespro.ru>
<20201022012519.GF1475@paquier.xyz>
On 22.10.2020 04:25, Michael Paquier wrote:
> On Thu, Oct 22, 2020 at 12:47:03AM +0300, Anastasia Lubennikova wrote:
>> We can also read such pages via shared buffers to be 100% sure.
> Yeah, but this has its limits as well. One can use
> ignore_checksum_failure, but this can actually be very dangerous as
> you can finish by loading into shared buffers a page that has a header
> thought as sane but with a large portion of its page randomly
> corrupted, spreading corruption around and leading to more fancy
> logic failures in Postgres, with more panic from customers. Not using
> ignore_checksum_failure is safer, but it makes an analyze of the
> damages for a given file harder as things would stop at the first
> failure of a file with a seqscan. pg_prewarm can help here, but
> that's not the goal of the tool to do that either.
I was thinking about applying this only to pages with LSN > startLSN.
Most of such pages are valid and already in memory, because they were
changed just recently, so no need for pg_prewarm here. If such LSN
appeared because of a data corruption, page verification from inside
ReadBuffer() will report an error first. In proposed function, we can
handle this error in any fashion we want. Something like:
if (PageGetLSN(page) > startptr)
{
if (!read_page_via_buffercache())
//throw a warning about corrupted page
//handle checksum error as needed
else
//page is valid. No worries
}
--
Anastasia Lubennikova
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: a.lubennikova@postgrespro.ru, michael@paquier.xyz, michael.banck@credativ.de
Subject: Re: [patch] Fix checksum verification in base backups for zero page headers
In-Reply-To: <036235b4-2461-c4b2-a448-7875d943ce1e@postgrespro.ru>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox