From: Fujii Masao <masao.fujii@oss.nttdata.com>
To: Andres Freund <andres@anarazel.de>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Pg Hackers <pgsql-hackers@lists.postgresql.org>
Cc: Bossart, Nathan <bossartn@amazon.com>
Cc: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Cc: 蔡梦娟(玊于) <mengjuan.cmj@alibaba-inc.com>
Cc: Jakub Wartak <Jakub.Wartak@tomtom.com>
Subject: Re: prevent immature WAL streaming
Date: Wed, 1 Sep 2021 11:34:34 +0900
Message-ID: <bb15b918-6dad-b01d-23b9-17b5dfa7e6a2@oss.nttdata.com> (raw)
In-Reply-To: <20210831155333.2vv2zdf7v4nhh2m2@alap3.anarazel.de>
References: <20210831042949.52eqp5xwbxgrfank@alap3.anarazel.de>
<202108311356.sl33wcpcz5x6@alvherre.pgsql>
<20210831155333.2vv2zdf7v4nhh2m2@alap3.anarazel.de>
On 2021/09/01 0:53, Andres Freund wrote:
> I was thinking that on a normal WAL write we'd do nothing. Instead we would
> have dedicated code at the end of recovery that, if the WAL ends in a partial
> record, changes the page following the "valid" portion of the WAL to indicate
> that an incomplete record is to be skipped.
Agreed!
> Of course, we need to be careful to not weaken WAL validity checking too
> much. How about the following:
>
> If we're "aborting" a continued record, we set XLP_FIRST_IS_ABORTED_PARTIAL on
> the page at which we do so (i.e. the page after the valid end of the WAL).
When do you expect that XLP_FIRST_IS_ABORTED_PARTIAL is set? It's set
when recovery finds a a partially-flushed segment-spanning record?
But maybe we cannot do that (i.e., cannot overwrite the page) because
the page that the flag is set in might have already been archived. No?
> I think we can just read the WAL and see if it ends with a partial
> record. It'd add a bit of complication to the error checking in xlogreader,
> because we'd likely want to treat verification from page headers a bit
> different from verification due to record data. But that seems doable.
Yes.
> Does this make sense?
Yes, I think!
Regards,
--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: masao.fujii@oss.nttdata.com, andres@anarazel.de, alvherre@alvh.no-ip.org, pgsql-hackers@lists.postgresql.org, bossartn@amazon.com, horikyota.ntt@gmail.com, mengjuan.cmj@alibaba-inc.com, Jakub.Wartak@tomtom.com
Subject: Re: prevent immature WAL streaming
In-Reply-To: <bb15b918-6dad-b01d-23b9-17b5dfa7e6a2@oss.nttdata.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox