Message-ID: <461ECA4E.9010101@oss.ntt.co.jp>
Date: Fri, 13 Apr 2007 09:09:50 +0900
From: Koichi Suzuki <suzuki.koichi@oss.ntt.co.jp>
User-Agent: Thunderbird 1.5.0.10 (Windows/20070221)
MIME-Version: 1.0
To: Zeugswetter Andreas ADI SD <ZeugswetterA@spardat.at>
CC: Hannu Krosing <hannu@skype.net>, "Joshua D. Drake" <jd@commandprompt.com>,
	Tom Lane <tgl@sss.pgh.pa.us>, Simon Riggs <simon@2ndquadrant.com>,
	josh@agliodbs.com,  pgsql-hackers@postgresql.org,
	pgsql-patches@postgresql.org
Subject: Re: [HACKERS] Full page writes improvement, code update
References: <460886E8.6040907@oss.ntt.co.jp>	 <4609CAC3.7060008@oss.ntt.co.jp>	
	<1175072848.4386.323.camel@silverbirch.site>	
	<200703291145.28574.josh@agliodbs.com>	
	<1175200117.4386.616.camel@silverbirch.site>	
	<460C611E.6090800@oss.ntt.co.jp> <27693.1176226131@sss.pgh.pa.us>	
	<461C3557.6040201@oss.ntt.co.jp> <461C3732.20006@commandprompt.com>
	<1176269612.3962.11.camel@localhost.localdomain>
	<461D9090.60808@oss.ntt.co.jp>
	<E1539E0ED7043848906A8FF995BDA57901E7BD64@m0143.s-mxs.net>
In-Reply-To: <E1539E0ED7043848906A8FF995BDA57901E7BD64@m0143.s-mxs.net>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

Sorry, inline reply.

Zeugswetter Andreas ADI SD wrote:

> 
> Yup, this is a good summary.
> 
> You say you need to remove the optimization that avoids 
> the logging of a new tuple because the full page image exists.
> I think we must already have the info in WAL which tuple inside the full
> page image
> is new (the one for which we avoided the WAL entry for).
> 
> How about this:
> Leave current WAL as it is and only add the not removeable flag to full
> pages.
> pg_compresslog then replaces the full page image with a record for the
> one tuple that is changed.
> I tend to think it is not worth the increased complexity only to save
> bytes in the uncompressed WAL though.

It is essentially what my patch proposes.  My patch includes flag to 
full page writes which "can be" removed.

> Another point about pg_decompresslog:
> 
> Why do you need a pg_decompresslog ? Imho pg_compresslog should already
> do the replacing of the
> full_page with the dummy entry. Then pg_decompresslog could be a simple
> gunzip, or whatever compression was used,
> but no logic.

Just removing full page writes does not work.   If we shift the rest of 
the WAL, then LSN becomes inconsistent in compressed archive logs which 
pg_compresslog produces.   For recovery, we have to restore LSN as the 
original WAL.   Pg_decompresslog restores removed full page writes as a 
dumm records so that recovery redo functions won't be confused.

Regards;

> 
> Andreas
> 


-- 
-------------
Koichi Suzuki