public inbox for [email protected]  

help / color / mirror / Atom feed

From: Drouvot, Bertrand <[email protected]>
To: Bharath Rupireddy <[email protected]>
To: PostgreSQL Hackers <[email protected]>
Subject: Re: Add a new pg_walinspect function to extract FPIs from WAL records
Date: Tue, 10 Jan 2023 09:29:03 +0100
Message-ID: <[email protected]> (raw)
In-Reply-To: <CALj2ACXesN9DTjgsekM8fig7CxhhxQfQP4fCiSJgcmp9wrZOvA@mail.gmail.com>
References: <CALj2ACVCcvzd7WiWvD=6_7NBvVB_r6G0EGSxL4F8vosAi6Se4g@mail.gmail.com>
	<[email protected]>
	<CALj2ACX=QtDfj_7DzzRbV+s+oCG_YafY6PCp+cJ+4mZbrn-cnQ@mail.gmail.com>
	<CALj2ACXpw+8X2gqzkdVdVm68rsvfAbK=h50PLX7NuV8C1Qcxmg@mail.gmail.com>
	<CALj2ACXesN9DTjgsekM8fig7CxhhxQfQP4fCiSJgcmp9wrZOvA@mail.gmail.com>

Hi,

On 1/6/23 6:41 PM, Bharath Rupireddy wrote:
> On Fri, Jan 6, 2023 at 11:47 AM Bharath Rupireddy
> <[email protected]> wrote:
>>
>> On Thu, Jan 5, 2023 at 6:51 PM Bharath Rupireddy
>> <[email protected]> wrote:
>>>
>>>> I'm also wondering if it would make sense to extend the test coverage of it (and pg_waldump) to "validate" that both
>>>> extracted images are the same and matches the one modified right after the checkpoint.
>>>>
>>>> What do you think? (could be done later in another patch though).
>>>
>>> I think pageinspect can be used here. We can fetch the raw page from
>>> the table after the checkpoint and raw FPI from the WAL record logged
>>> as part of the update. I've tried to do so [1], but I see a slight
>>> difference in the raw output. The expectation is that they both be the
>>> same. It might be that the update operation logs the FPI with some
>>> more info set (prune_xid). I'll try to see why it is so.
>>>
>>> I'm attaching the v2 patch for further review.
>>>
>>> [1]
>>> SELECT * FROM page_header(:'page_from_table');
>>>      lsn    | checksum | flags | lower | upper | special | pagesize |
>>> version | prune_xid
>>> -----------+----------+-------+-------+-------+---------+----------+---------+-----------
>>>   0/1891D78 |        0 |     0 |    40 |  8064 |    8192 |     8192 |
>>>      4 |         0
>>> (1 row)
>>>
>>> SELECT * FROM page_header(:'page_from_wal');
>>>      lsn    | checksum | flags | lower | upper | special | pagesize |
>>> version | prune_xid
>>> -----------+----------+-------+-------+-------+---------+----------+---------+-----------
>>>   0/1891D78 |        0 |     0 |    44 |  8032 |    8192 |     8192 |
>>>      4 |       735
>>> (1 row)
>>
>> Ugh, v2 patch missed the new file added, I'm attaching v3 patch for
>> further review. Sorry for the noise.
> 
> I took a stab at how and what gets logged as FPI in WAL records:
> 
> Option 1:
> WAL record with FPI contains both the unmodified table page from the
> disk after checkpoint and new tuple (not applied to the unmodified
> page) and the recovery (redo) applies the new tuple to the unmodified
> page as part of recovery. A bit more WAL is needed to store both
> unmodified page and new tuple data in the WAL record and recovery can
> get slower a bit too as it needs to stitch the modified page.
> 
> Option 2:
> WAL record with FPI contains only the modified page (new tuple applied
> to the unmodified page from the disk after checkpoint) and the
> recovery (redo)  just returns the applied block as BLK_RESTORED.
> Recovery can get faster with this approach and less WAL is needed to
> store just the modified page.
> 
> My earlier understanding was that postgres does option (1), however, I
> was wrong, option (2) is what actually postgres has implemented for
> the obvious advantages specified.
> 
> I now made the tests a bit stricter in checking the FPI contents
> (tuple values) pulled from the WAL record with raw page contents
> pulled from the table using the pageinspect extension. Please see the
> attached v4 patch.
> 

Thanks for updating the patch!

+-- Compare FPI from WAL record and page from table, they must be same

I think "must be the same" or "must be identical" sounds better (but not 100% sure).

Except this nit, V4 looks good to me.

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

view thread (7+ messages)  latest in thread

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: [email protected]
  Cc: [email protected], [email protected], [email protected]
  Subject: Re: Add a new pg_walinspect function to extract FPIs from WAL records
  In-Reply-To: <[email protected]>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox