Date: Thu, 22 Apr 2021 01:09:19 +0200
From: Jehan-Guillaume de Rorthais <jgdr@dalibo.com>
To: Andres Freund <andres@anarazel.de>
Cc: Stephen Frost <sfrost@snowman.net>, Tom Lane <tgl@sss.pgh.pa.us>, Magnus
 Hagander <magnus@hagander.net>, Robert Haas <robertmhaas@gmail.com>, Thomas
 Munro <thomas.munro@gmail.com>, "pgsql-hackers@postgresql.org"
 <pgsql-hackers@postgresql.org>
Subject: Re: when the startup process doesn't
Message-ID: <20210422010919.4b655015@firost>
In-Reply-To: <20210421193605.cubavrq7q5bqkf4i@alap3.anarazel.de>
References: <465221.1618879471@sss.pgh.pa.us>
	<CA+TgmoaSELPiPqNcfNG_TyDe6=VVzfyh+YvhB6-GGsWoo=jc8Q@mail.gmail.com>
	<CABUevEwc_oP=kf6CYxNGM3KLMRBe3YBhM8+qu0oBbY_x89ifHw@mail.gmail.com>
	<20210420171720.56599607@firost>
	<CABUevEyTBiiDvjdKo5h=gj6W1uL6i4jOOV4nur0YZsuKvKRVXw@mail.gmail.com>
	<631849.1618942988@sss.pgh.pa.us>
	<20210420185150.GL20766@tamriel.snowman.net>
	<633337.1618945018@sss.pgh.pa.us>
	<20210420203642.nvxvolm37f2k6prj@alap3.anarazel.de>
	<20210421183624.GP20766@tamriel.snowman.net>
	<20210421193605.cubavrq7q5bqkf4i@alap3.anarazel.de>
Organization: Dalibo
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/20210422010919.4b655015%40firost>
Precedence: bulk

On Wed, 21 Apr 2021 12:36:05 -0700
Andres Freund <andres@anarazel.de> wrote:

>  [...]  
> 
> I don't think that concern equally applies for what I am proposing
> here. For one, we already have minRecoveryPoint in ControlData, and we
> already use it for the purpose of determining where we need to recover
> to, albeit only during crash recovery. Imo that's substantially
> different from adding actual recovery progress status information to the
> control file.

Just for the record, when I was talking about updating status of the startup
in the controldata, I was thinking about setting the last known LSN replayed.
Not some kind of variable string.

> 
> I also think that it'd actually be a significant reliability improvement
> if we maintained an approximate minRecoveryPoint during normal running:
> I've seen way too many cases where WAL files were lost / removed and
> crash recovery just started up happily. Only hitting problems months
> down the line. Yes, it'd obviously not bullet proof, since we'd not want
> to add a significant stream of new fsyncs, but IME such WAL files
> lost/removed issues tend not to be about a few hundred bytes of WAL but
> many segments missing.

Maybe setting this minRecoveryPoint once per segment would be enough, near
from the beginning of the WAL. At least, the recovery process would be
forced to actually replay until the very last known segment.

Regards,