From: Fujii Masao <masao.fujii@oss.nttdata.com>
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
To: jgdr@dalibo.com
Cc: pgsql-bugs@lists.postgresql.org
Cc: michael@paquier.xyz
Subject: Re: [BUG] non archived WAL removed during production crash recovery
Date: Thu, 2 Apr 2020 14:19:15 +0900
Message-ID: <03900fc5-fff4-109a-fe69-83de2c167929@oss.nttdata.com> (raw)
In-Reply-To: <20200402.130734.613319908534820379.horikyota.ntt@gmail.com>
References: <d0ef12c5-7e36-b9b3-c8ea-2e98d69d1317@oss.nttdata.com>
<20200401181735.11100908@firost>
<20200402.130443.1791362864704140391.horikyota.ntt@gmail.com>
<20200402.130734.613319908534820379.horikyota.ntt@gmail.com>
On 2020/04/02 13:07, Kyotaro Horiguchi wrote:
> Sorry, it was quite ambiguous.
>
> At Thu, 02 Apr 2020 13:04:43 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
>> At Wed, 1 Apr 2020 18:17:35 +0200, Jehan-Guillaume de Rorthais <jgdr@dalibo.com> wrote in
>>> Please, find in attachment a patch implementing this.
>>
>> The patch partially reintroduces the issue the patch have
>> fixed. Specifically a standby running a crash recovery wrongly marks a
>> WAL file as ".ready" if it is extant in pg_wal without accompanied by
>> .ready file.
>
> The patch partially reintroduces the issue the commit 78ea8b5daa have
> fixed. Specifically a standby running a crash recovery wrongly marks a
> WAL file as ".ready" if it is extant in pg_wal without accompanied by
> .ready file.
On second thought, I think that we should discuss what the desirable
behavior is before the implentation. Especially what's unclear to me
is whether to remove such WAL files in archive recovery case with
archive_mode=on. Those WAL files would be required when recovering
from the backup taken before that archive recovery happens.
So it seems unsafe to remove them in that case.
Therefore, IMO that the patch should change the code so that
no unarchived WAL files are removed not only in crash recovery
but also archive recovery. Thought?
Of course, this change would lead to the issue that the past unarchived
WAL files keep remaining in the case of warm-standby using archive
recovery. But this issue looks unavoidable. If users want to avoid that,
archive_mode should be set to always.
Also I'm a bit wondering if it's really safe to remove such unarchived
WAL files even in the standby case with archive_mode=on. I would need
more time to think that.
>> Perhaps checking '.ready' before the checking for archive-mode would
>> be sufficient.
>>
>>> Plus, I added a second commit to add one test in regard with this bug.
>>>
>>>> Another is to make the startup process remove .ready file if necessary.
>>>
>>> I'm not sure to understand this one.
I was thinking to make the startup process remove such unarchived WAL files
if archive_mode=on and StandbyModeRequested/ArchiveRecoveryRequested
is true.
Regards,
--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: masao.fujii@oss.nttdata.com, horikyota.ntt@gmail.com, jgdr@dalibo.com, pgsql-bugs@lists.postgresql.org, michael@paquier.xyz
Subject: Re: [BUG] non archived WAL removed during production crash recovery
In-Reply-To: <03900fc5-fff4-109a-fe69-83de2c167929@oss.nttdata.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox