Subject: Re: [PATCHES] Infrastructure changes for recovery
From: Simon Riggs <simon@2ndQuadrant.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: List pgsql-patches <pgsql-patches@postgresql.org>,
	pgsql-hackers <pgsql-hackers@postgresql.org>
In-Reply-To: <15776.1221594318@sss.pgh.pa.us>
References: <1218109459.4549.423.camel@ebony.2ndQuadrant>
	<1220213074.4371.173.camel@ebony.2ndQuadrant>
	<1905.1220895241@sss.pgh.pa.us>
	<1221080797.3913.768.camel@ebony.2ndQuadrant>
	<15776.1221594318@sss.pgh.pa.us>
Content-Type: text/plain
Date: Thu, 18 Sep 2008 09:05:45 +0100
Message-Id: <1221725145.3913.2315.camel@ebony.2ndQuadrant>
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit


On Tue, 2008-09-16 at 15:45 -0400, Tom Lane wrote:
> Simon Riggs <simon@2ndQuadrant.com> wrote:
> > Testing takes a while on this, I probably won't complete it until
> > Friday. So enclosed patch is for eyeballs only at this stage.
> 
> What's the status on that patch?

Having some trouble trying to get a clean state change from recovery to
normal mode.

Startup needs to be able to write WAL at the end of recovery so it can
write a ShutdownCheckpoint, yet must not be allowed to write WAL before
that. Other services are still not fully up yet. So every other process
apart from Startup musn't write WAL until Startup has fully completed
its actions, which is sometime later.

bgwriter needs to know about the impending state change so it can finish
off any checkpoint its currently working on. But then can't start doing
normal processing yet either. So it must have a third state that is
between recovery and normal processing. When normal processing is
reached, it *must* write WAL immediately from that point onwards, yet
using the new timeline that startup decides upon.

So the idea of a single database-wide boolean state seems impossible. We
need a local canInsertWAL flag that is set at different times in
different processes.

Global states changes are now

not started
started - postmaster process then startup process
safestoppingpoint - bgwriter starts
startup_does_shutdown_checkpoint - bgwriter finishes up, just waits,
   startup is now allowed to insert WAL for checkpoint record
startup completes StartupXLog - bgwriter begins normal processing
startup exits - postmaster in normal state

We *might* solve this by making the final checkpoint that Startup writes
into an online checkpoint. That would then allow a straight and safe
transition for bgwriter from just one state to the other. But that
leaves some other ugliness that I don't want to deal with, 'cos there's
other fish frying.

Feels like I should shutdown the bgwriter after recovery and then allow
it to be cranked up again after normal processing starts, and do all of
this through postmaster state changes. That way bgwriter doesn't need to
do a dynamic state change.

Comments anyone?

-- 
 Simon Riggs           www.2ndQuadrant.com
 PostgreSQL Training, Services and Support