Subject: Re: [BUG] non archived WAL removed during production crash recovery
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>, jgdr@dalibo.com
Cc: pgsql-bugs@lists.postgresql.org, michael@paquier.xyz
References: <20200402154915.4984cff2@firost>
 <3d543ef2-ae02-5e59-3bfe-b79d00dab360@oss.nttdata.com>
 <20200402174450.70de9ab5@firost>
 <20200403.101417.2239113628048167718.horikyota.ntt@gmail.com>
From: Fujii Masao <masao.fujii@oss.nttdata.com>
Message-ID: <c46a6ccd-02a5-18b0-bc32-556632933508@oss.nttdata.com>
Date: Fri, 3 Apr 2020 15:45:31 +0900
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0)
 Gecko/20100101 Thunderbird/68.6.0
MIME-Version: 1.0
In-Reply-To: <20200403.101417.2239113628048167718.horikyota.ntt@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Precedence: bulk


On 2020/04/03 10:14, Kyotaro Horiguchi wrote:
> At Thu, 2 Apr 2020 17:44:50 +0200, Jehan-Guillaume de Rorthais <jgdr@dalibo.com> wrote in
>> On Thu, 2 Apr 2020 23:55:46 +0900
>> Fujii Masao <masao.fujii@oss.nttdata.com> wrote:
>>
>> [...]
>>>> Well, it seems to me that this is what you suggested few paragraph away:
>>>>
>>>>     «.ready files are removed when either archive_mode!=always in standby
>>>> mode»
>>>
>>> Yes, so I'm fine with that as the first consensus because the behavior
>>> is obviously better than the current one. *If* the case where no WAL files
>>> should be removed is found, I'd just like to propose the additional patch.
>>
>> Do you mean to want to produce the next patch yourself?
> 
> No. Fujii-san is saying that he will address it, if the fix made in
> this thread is found to be imperfect later.
> 
> He suspects that WAL files should be preserved at least in certain
> cases even if the file persists forever, but the consensus here is to
> remove files under certain conditions so as not to no WAL file
> persists in pg_wal directory.

Yes.

> Feel free to propose the next version!

Yes!

BTW, now I'm thinking that the flag in shmem should be updated when
the startup process sets StandbyModeRequested to true at the beginning
of the recovery. That is,

- Add something like SharedStandbyModeRequested into XLogCtl. This field
    should be initialized with false;
- Set XLogCtl->SharedStandbyModeRequested to true when the startup
    process detects the standby.signal file and sets the local variable
    StandbyModeRequested to true.
- Make XLogArchiveCheckDone() use XLogCtl->SharedStandbyModeRequested
    to know whether the server is in standby mode or not.

Thought?

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION