Message-ID: <bd296352-77a6-2c46-5e3e-a548f74c2494@pgmasters.net>
Date: Thu, 4 Aug 2022 06:32:38 -0400
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.11.0
Subject: Re: Race between KeepFileRestoredFromArchive() and restartpoint
Content-Language: en-US
To: Kyotaro Horiguchi <horikyota.ntt@gmail.com>, noah@leadboat.com
Cc: pgsql-hackers@postgresql.org, don@seiler.us
References: <e1f1fb79-f668-0e7c-9841-763a023a7187@pgmasters.net>
 <20220802143727.GA3796043@rfd.leadboat.com>
 <20220804062456.GA3844351@rfd.leadboat.com>
 <20220804.170640.1425291966829818860.horikyota.ntt@gmail.com>
From: David Steele <david@pgmasters.net>
In-Reply-To: <20220804.170640.1425291966829818860.horikyota.ntt@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/bd296352-77a6-2c46-5e3e-a548f74c2494%40pgmasters.net>
Precedence: bulk

On 8/4/22 04:06, Kyotaro Horiguchi wrote:
> At Wed, 3 Aug 2022 23:24:56 -0700, Noah Misch <noah@leadboat.com> wrote in
>>>> I think in this case a HINT might be sufficient to at least keep people from
>>>> wasting time tracking down a problem that has already been fixed.
>>
>> Here's a patch to add that HINT.  I figure it's better to do this before next
>> week's minor releases.  In the absence of objections, I'll push this around
>> 2022-08-05 14:00 UTC.
> 
> +1

Looks good to me as well.

>>>> However, there is another issue [1] that might argue for a back patch if
>>>> this patch (as I believe) would fix the issue.
>>>
>>>> [1] https://www.postgresql.org/message-id/CAHJZqBDxWfcd53jm0bFttuqpK3jV2YKWx%3D4W7KxNB4zzt%2B%2BqFg%40mail.gmail.com
>>>
>>> That makes sense.  Each iteration of the restartpoint recycle loop has a 1/N
>>> chance of failing.  Recovery adds >N files between restartpoints.  Hence, the
>>> WAL directory grows without bound.  Is that roughly the theory in mind?
>>
>> On further reflection, I don't expect it to happen that way.  Each failure
>> message is LOG-level, so the remaining recycles still happen.  (The race
>> condition can yield an ERROR under PreallocXlogFiles(), but recycling is
>> already done at that point.)
> 
> I agree to this.

Hmmm, OK. We certainly have a fairly serious issue here, i.e. pg_wal 
growing without bound. Even if we are not sure what is causing it, how 
confident are we that the patches applied to v15 would fix it?

Regards,
-David