Subject: Re: [PATCHES] Infrastructure changes for recovery
From: Simon Riggs <simon@2ndQuadrant.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: List pgsql-patches <pgsql-patches@postgresql.org>,
	pgsql-hackers <pgsql-hackers@postgresql.org>
In-Reply-To: <11540.1222697639@sss.pgh.pa.us>
References: <1218109459.4549.423.camel@ebony.2ndQuadrant>
	<1220213074.4371.173.camel@ebony.2ndQuadrant>
	<1905.1220895241@sss.pgh.pa.us>
	<1221080797.3913.768.camel@ebony.2ndQuadrant>
	<15776.1221594318@sss.pgh.pa.us>
	<1221725145.3913.2315.camel@ebony.2ndQuadrant>
	<16992.1221743196@sss.pgh.pa.us>
	<1221745847.3913.2408.camel@ebony.2ndQuadrant>
	<18549.1221746983@sss.pgh.pa.us>
	<1222121161.4445.248.camel@ebony.2ndQuadrant>
	<1222184541.4445.400.camel@ebony.2ndQuadrant>
	<28973.1222381719@sss.pgh.pa.us>
	<1222424415.4445.917.camel@ebony.2ndQuadrant>
	<9794.1222624945@sss.pgh.pa.us>
	<1222649659.4445.1061.camel@ebony.2ndQuadrant>
	<22856.1222650961@sss.pgh.pa.us>
	<1222691596.4445.1188.camel@ebony.2ndQuadrant>
	<10307.1222692406@sss.pgh.pa.us>
	<1222696408.4445.1229.camel@ebony.2ndQuadrant>
	<11540.1222697639@sss.pgh.pa.us>
Content-Type: text/plain
Date: Mon, 29 Sep 2008 16:06:44 +0100
Message-Id: <1222700804.4445.1271.camel@ebony.2ndQuadrant>
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit


On Mon, 2008-09-29 at 10:13 -0400, Tom Lane wrote:
> Simon Riggs <simon@2ndQuadrant.com> writes:
> > I think we can get away with writing the LSN value to disk, as you
> > suggested, but only every so often. No need to do it after every WAL
> > record, just consistently every so often, so it gives us a point at
> > which we know we are safe.
> 
> Huh?  How does that make you safe?  What you need to know is the max
> LSN that could possibly be on disk.
> 
> Hmm, actually we could get away with tying this to fetching WAL files
> from the archive.  When switching to a new WAL file, write out the
> *ending* WAL address of that file to pg_control.  Then process the WAL
> records in it.  Whether or not any of the affected pages get to disk,
> we know that there is no LSN on disk exceeding what we already put in
> pg_control.  If we crash and restart, we'll have to get to the end
> of this file before we start letting backends in; which might be further
> than we actually got before the crash, but not too much further because
> we already know the whole WAL file is available.
> 
> Or is that the same thing you were saying?  The detail about using
> the end address seems fairly critical, and you didn't mention it...

Same! Just said safe point was "LSN + 1", and since end = next start.

Looks we've got a solution, no matter how it's described. (I actually
have a more detailed proof of safety using snapshots/MVCC considerations
so I wasn't overly worried but what we've discussed is much easier to
understand and agree. Proof of safety is all we need, and this simpler
proof is more secure.)

Don't want to make it per file though. Big systems can whizz through WAL
files very quickly, so we either make it a big number e.g. 255 files per
xlogid, or we make it settable (and recorded in pg_control).

-- 
 Simon Riggs           www.2ndQuadrant.com
 PostgreSQL Training, Services and Support