Date: Thu, 03 Feb 2022 15:50:53 +0900 (JST)
Message-Id: <20220203.155053.170368585238406731.horikyota.ntt@gmail.com>
To: masao.fujii@oss.nttdata.com
Cc: bharath.rupireddyforpostgres@gmail.com, nathandbossart@gmail.com,
 sfrost@snowman.net, bossartn@amazon.com, rjuju123@gmail.com,
 michael@paquier.xyz, pgsql-hackers@lists.postgresql.org
Subject: Re: Add checkpoint and redo LSN to LogCheckpointEnd log message
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
In-Reply-To: <7bfad665-db9c-0c2a-2604-9f54763c5f9e@oss.nttdata.com>
References: <be548399-4c5e-77a7-2a95-fad224f1d24d@oss.nttdata.com>
	<CALj2ACVVMAGDvourutDvtDUOhTqS_yTtfrdCjtP75-RrQU0dFA@mail.gmail.com>
	<7bfad665-db9c-0c2a-2604-9f54763c5f9e@oss.nttdata.com>
User-Agent: Mew version 6.8 on Emacs 26.1
Mime-Version: 1.0
Content-Type: Text/Plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Archived-At: <https://www.postgresql.org/message-id/20220203.155053.170368585238406731.horikyota.ntt%40gmail.com>
Precedence: bulk

At Thu, 3 Feb 2022 13:59:03 +0900, Fujii Masao <masao.fujii@oss.nttdata.com> wrote in 
> 
> 
> On 2022/02/02 23:46, Bharath Rupireddy wrote:
> > On Tue, Feb 1, 2022 at 9:39 PM Fujii Masao
> > <masao.fujii@oss.nttdata.com> wrote:
> >> I found that CreateRestartPoint() already reported the redo lsn as
> >> follows after emitting the restartpoint log message. To avoid
> >> duplicated logging of the same information, we should update this
> >> code?
> >>
> >>          ereport((log_checkpoints ? LOG : DEBUG2),
> >>                          (errmsg("recovery restart point at %X/%X",
> >>                                          LSN_FORMAT_ARGS(lastCheckPoint.redo)),
> >>                           xtime ? errdetail("Last completed transaction was
> >>                           at log time %s.",
> >>                                                             timestamptz_to_str(xtime))
> >>                                                             : 0));
> >>
> >> This code reports lastCheckPoint.redo as redo lsn. OTOH, with the
> >> patch, LogCheckpointEnd() reports
> >> ControlFile->checkPointCopy.redo. They may be different, for example,
> >> when the current DB state is not DB_IN_ARCHIVE_RECOVERY. In this case,
> >> which lsn should we report as redo lsn?
> > Do we ever reach CreateRestartPoint when ControlFile->stat !=
> > DB_IN_ARCHIVE_RECOVERY? Assert(ControlFile->state ==
> > DB_IN_ARCHIVE_RECOVERY); in CreateRestartPoint doesn't fail any
> > regression tests.
> 
> ISTM that CreateRestartPoint() can reach the condition
> ControlFile->state != DB_IN_ARCHIVE_RECOVERY. Please imagine the case
> where CreateRestartPoint() has already started and calls
> CheckPointGuts(). If the standby server is promoted while
> CreateRestartPoint() is flushing the data to disk at CheckPointGuts(),
> the state would be updated to DB_IN_PRODUCTION and
> CreateRestartPoint() can see the state != DB_IN_ARCHIVE_RECOVERY
> later.

By the way a comment there:
> * this is a quick hack to make sure nothing really bad happens if somehow
> * we get here after the end-of-recovery checkpoint.

now looks a bit wrong since now it's normal that a restartpoint ends
after promotion.

> As far as I read the code, this case seems to be able to make the
> server unrecoverable. If this case happens, since pg_control is not
> updated, pg_control still indicates the REDO LSN of last valid
> restartpoint. But CreateRestartPoint() seems to delete old WAL files
> based on its "current" REDO LSN not pg_control's REDO LSN. That is,
> WAL files required for the crash recovery starting from pg_control's
> REDO LSN would be removed.

Seems right. (I didn't confirm the behavior, though	)

> If this understanding is right, to address this issue, probably we
> need to make CreateRestartPoint() do nothing (return immediately) when
> the state != DB_IN_ARCHIVE_RECOVERY?

On way to take. In that case should we log something like
"Restartpoint canceled" or something? 

By the way, restart point should start only while recoverying, and at
the timeof the start both checkpoint.redo and checkpoint LSN are
already past. We shouldn't update minRecovery point after promotion,
but is there any reason for not updating the checkPoint and
checkPointCopy?  If we update them after promotion, the
which-LSN-to-show problem would be gone.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center