Message-ID: <462BFB11.7000409@oss.ntt.co.jp>
Date: Mon, 23 Apr 2007 09:17:21 +0900
From: Koichi Suzuki <suzuki.koichi@oss.ntt.co.jp>
User-Agent: Thunderbird 1.5.0.10 (Windows/20070221)
MIME-Version: 1.0
To: Zeugswetter Andreas ADI SD <ZeugswetterA@spardat.at>
CC: Hannu Krosing <hannu@skype.net>, "Joshua D. Drake" <jd@commandprompt.com>,
	Tom Lane <tgl@sss.pgh.pa.us>, Simon Riggs <simon@2ndquadrant.com>,
	josh@agliodbs.com,  pgsql-hackers@postgresql.org,
	pgsql-patches@postgresql.org
Subject: Re: [HACKERS] Full page writes improvement, code update
References: <460886E8.6040907@oss.ntt.co.jp>	 <4609CAC3.7060008@oss.ntt.co.jp>	
	<1175072848.4386.323.camel@silverbirch.site>	
	<200703291145.28574.josh@agliodbs.com>	
	<1175200117.4386.616.camel@silverbirch.site>	
	<460C611E.6090800@oss.ntt.co.jp> <27693.1176226131@sss.pgh.pa.us>	
	<461C3557.6040201@oss.ntt.co.jp> <461C3732.20006@commandprompt.com>
	<1176269612.3962.11.camel@localhost.localdomain>
	<461D9090.60808@oss.ntt.co.jp>
	<E1539E0ED7043848906A8FF995BDA57901E7BD64@m0143.s-mxs.net>
	<461ECA4E.9010101@oss.ntt.co.jp>
	<E1539E0ED7043848906A8FF995BDA57901E7BEBE@m0143.s-mxs.net>
	<462856EF.1030005@oss.ntt.co.jp>
	<E1539E0ED7043848906A8FF995BDA57901F3FC4B@m0143.s-mxs.net>
In-Reply-To: <E1539E0ED7043848906A8FF995BDA57901F3FC4B@m0143.s-mxs.net>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Hi,

I don't insist the name and the default of the GUC parameter.  I'm 
afraid wal_fullpage_optimization = on (default) makes some confusion 
because the default behavior becomes a bit different on WAL itself.

I'd like to have some more opinion on this.

Zeugswetter Andreas ADI SD wrote:
>> With DBT-2 benchmark, I've already compared the amount of WAL.   The 
>> result was as follows:
>>
>> Amount of WAL after 60min. run of DBT-2 benchmark 
>> wal_add_optimization_info = off (default) 3.13GB
> 
> how about wal_fullpage_optimization = on (default)
>  
>> wal_add_optimization_info = on (new case) 3.17GB -> can be 
>> optimized to 0.31GB by pg_compresslog.
>>
>> So the difference will be around a couple of percents.   I think this
> is 
>> very good figure.
>>
>> For information,
>> DB Size: 12.35GB (120WH)
>> Checkpoint timeout: 60min.  Checkpoint occured only once in the run.
> 
> Unfortunately I think DBT-2 is not a good benchmark to test the disabled
> wal optimization.
> The test should contain some larger rows (maybe some updates on large
> toasted values), and maybe more frequent checkpoints. Actually the poor
> ratio between full pages and normal WAL content in this benchmark is
> strange to begin with.
> Tom fixed a bug recently, and it would be nice to see the new ratio. 
> 
> Have you read Tom's comment on not really having to be able to
> reconstruct all record types from the full page image ? I think that
> sounded very promising (e.g. start out with only heap insert/update). 
> 
> Then:
> - we would not need the wal optimization switch (the full page flag
> would always be added depending only on backup)
> - pg_compresslog would only remove such "full page" images where it
> knows how to reconstruct a "normal" WAL record from
> - with time and effort pg_compresslog would be able to compress [nearly]
> all record types's full images (no change in backend)
> 
>> I don't think replacing LSN works fine.  For full recovery to 
>> the current time, we need both archive log and WAL.  
>> Replacing LSN will make archive log LSN inconsistent with 
>> WAL's LSN and the recovery will not work.
> 
> WAL recovery would have had to be modified (decouple LSN from WAL
> position during recovery).
> An "archive log" would have been a valid WAL (with appropriate LSN
> advance records). 
>  
>> Reconstruction to regular WAL is proposed as 
>> pg_decompresslog.  We should be careful enough not to make 
>> redo routines confused with the dummy full page writes, as 
>> Simon suggested.  So far, it works fine.
> 
> Yes, Tom didn't like "LSN replacing" eighter. I withdraw my concern
> regarding pg_decompresslog.
> 
> Your work in this area is extremely valuable and I hope my comments are
> not discouraging.
> 
> Thank you
> Andreas
> 


-- 
-------------
Koichi Suzuki