From: Andres Freund <andres@2ndquadrant.com>
Organization: 2ndquadrant.com
To: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
Subject: Re: WAL format changes
Date: Mon, 18 Jun 2012 22:57:57 +0200
User-Agent: KMail/1.13.7 (Linux/3.4.0-andres; KDE/4.8.3; x86_64; ; )
Cc: pgsql-hackers@postgresql.org,
 Robert Haas <robertmhaas@gmail.com>
References: <4FDA5136.6080206@enterprisedb.com>
	<201206182045.53962.andres@2ndquadrant.com>
	<4FDF7F54.5060406@enterprisedb.com>
In-Reply-To: <4FDF7F54.5060406@enterprisedb.com>
MIME-Version: 1.0
Content-Type: Text/Plain;
  charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Message-Id: <201206182257.58019.andres@2ndquadrant.com>

On Monday, June 18, 2012 09:19:48 PM Heikki Linnakangas wrote:
> On 18.06.2012 21:45, Andres Freund wrote:
> > On Monday, June 18, 2012 08:32:54 PM Heikki Linnakangas wrote:
> >> On 18.06.2012 21:13, Andres Freund wrote:
> >>> On Monday, June 18, 2012 08:08:14 PM Heikki Linnakangas wrote:
> >>>> The page header contains an XLogRecPtr (LSN), so if we change it we'll
> >>>> have to deal with pg_upgrade. I guess we could still keep XLogRecPtr
> >>>> around as the on-disk representation, and convert between the 64-bit
> >>>> integer and XLogRecPtr in PageGetLSN/PageSetLSN. I can try that out -
> >>>> many xlog calculations would admittedly be simpler if it was an
> >>>> uint64.
> >>> 
> >>> I am out of my depth here, not having read any of the relevant code,
> >>> but couldn't we simply replace the lsn from disk with
> >>> InvalidXLogRecPtr without marking the page dirty?
> >>> 
> >>> There is the valid argument that you would loose some information when
> >>> pages with hint bits are written out again, but on the other hand you
> >>> would also gain the information that it was a hint-bit write...
> >> 
> >> Sorry, I don't understand that. Where would you "replace the LSN from
> >> disk with InvalidXLogRecPtr" ? (and no, it probably won't work ;-) )
> > 
> > In ReadBuffer_common or such, after reading a page from disk and
> > verifying that the page has a valid header it seems to be possible to
> > replace pd_lsn *in memory* by InvalidXLogRecPtr without marking the page
> > dirty.
> > If the page isn't modified the lsn on disk won't be changed so you don't
> > loose debugging information in that case. If will be zero if it has been
> > written by a hint-bit write and thats arguable a win.
> 
> We use the LSN to decide whether a full-page image need to be xlogged if
> the page is modified. If you reset LSN every time you read in a page,
> you'll be doing full page writes every time a page is read from disk and
> modified, whether or not it's the first time the page is modified after
> the last checkpoint.
Yea, I somehow disregarded that hint-bit writes would make a problem with full 
page writes in that case. Normal writes wouldn't be a problem...
This should be fixable but the result would be too ugly. So its either 
converting the on-disk representation or keeping the duplicated 
representation.

pd_lsn seems to be well enough abstracted, doing the conversion in 
PageSet/GetLSN seems to be easy enough.

Andres
-- 
 Andres Freund	                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services