Subject: Re: [patch] Fix checksum verification in base backups for zero page
 headers
To: Michael Paquier <michael@paquier.xyz>
Cc: Michael Banck <michael.banck@credativ.de>,
 PostgreSQL Development <pgsql-hackers@postgresql.org>
References: <2439c2d6cee13d611e23ba651e5eb4ab0469bf99.camel@credativ.de>
 <d041090c-ddf7-10d1-7d50-2a4ed03c53dd@postgrespro.ru>
 <4b1c16216dcb60e1dad96142d25fd06fb77485bf.camel@credativ.de>
 <6713b5e9d98c7db1608b43cd57b10be4a08dca67.camel@credativ.de>
 <9489c221-e2ef-e7ad-d921-5ccad71b14de@postgrespro.ru>
 <423deac52479a40624e775e6d2ab41c4e9f5b19f.camel@credativ.de>
 <20201007081852.GC30037@paquier.xyz>
 <608f3476-0598-2514-2c03-e05c7d2b0cbd@postgrespro.ru>
 <20201020062432.GA30362@paquier.xyz>
 <69b38451-e91f-95fc-c3ce-06eb778d36e0@postgrespro.ru>
 <20201022012519.GF1475@paquier.xyz>
From: Anastasia Lubennikova <a.lubennikova@postgrespro.ru>
Message-ID: <036235b4-2461-c4b2-a448-7875d943ce1e@postgrespro.ru>
Date: Thu, 22 Oct 2020 15:11:45 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <20201022012519.GF1475@paquier.xyz>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Precedence: bulk

On 22.10.2020 04:25, Michael Paquier wrote:
> On Thu, Oct 22, 2020 at 12:47:03AM +0300, Anastasia Lubennikova wrote:
>> We can also read such pages via shared buffers to be 100% sure.
> Yeah, but this has its limits as well.  One can use
> ignore_checksum_failure, but this can actually be very dangerous as
> you can finish by loading into shared buffers a page that has a header
> thought as sane but with a large portion of its page randomly
> corrupted, spreading corruption around and leading to more fancy
> logic failures in Postgres, with more panic from customers.  Not using
> ignore_checksum_failure is safer, but it makes an analyze of the
> damages for a given file harder as things would stop at the first
> failure of a file with a seqscan.  pg_prewarm can help here, but
> that's not the goal of the tool to do that either.
I was thinking about applying this only to pages with LSN > startLSN.

Most of such pages are valid and already in memory, because they were 
changed just recently, so no need for pg_prewarm here. If such LSN 
appeared because of a data corruption, page verification from inside 
ReadBuffer() will report an error first. In proposed function, we can 
handle this error in any fashion we want. Something like:

if (PageGetLSN(page) > startptr)
{
     if (!read_page_via_buffercache())

         //throw a warning about corrupted page
         //handle checksum error as needed
     else
         //page is valid. No worries
}

-- 
Anastasia Lubennikova
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company