thread-index: AcdwtH+8LetBb550SRmdK9Xf0XqJHg==
Content-Class: urn:content-classes:message
Importance: normal
Subject: Re: Full page writes improvement, code update
From: "Simon Riggs" <simon@2ndquadrant.com>
To: "Koichi Suzuki" <suzuki.koichi@oss.ntt.co.jp>
Cc: <pgsql-hackers@postgresql.org>,
	<pgsql-patches@postgresql.org>
In-Reply-To: <460886E8.6040907@oss.ntt.co.jp>
References: <460886E8.6040907@oss.ntt.co.jp>
Content-Type: text/plain;
	charset="iso-8859-1"
Date: Tue, 27 Mar 2007 22:07:49 +0100
Message-ID: <1175029670.4386.222.camel@silverbirch.site>
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit

On Tue, 2007-03-27 at 11:52 +0900, Koichi Suzuki wrote:

> Here's an update of a code to improve full page writes as proposed in
> 
> http://archives.postgresql.org/pgsql-hackers/2007-01/msg01491.php
> and
> http://archives.postgresql.org/pgsql-patches/2007-01/msg00607.php
> 
> Update includes some modification for error handling in archiver and
> restoration command.
> 
> In the previous threads, I posted several evaluation and shown that we
> can keep all the full page writes needed for full XLOG crash recovery,
> while compressing archive log size considerably better than gzip, with
> less CPU consumption.  I've found no further objection for this proposal
> but still would like to hear comments/opinions/advices.

Koichi-san,

Looks interesting. I like the small amount of code to do this.

A few thoughts:

- Not sure why we need "full_page_compress", why not just mark them
always? That harms noone. (Did someone else ask for that? If so, keep
it)

- OTOH I'd like to see an explicit parameter set during recovery since
you're asking the main recovery path to act differently in case a single
bit is set/unset. If you are using that form of recovery, we should say
so explicitly, to keep everybody else safe.

- I'd rather mark just the nonremovable blocks. But no real difference

- We definitely don't want an normal elog in a XLogInsert critical
section, especially at DEBUG1 level

- diff -c format is easier and the standard

- pg_logarchive and pg_logrestore should be called by a name that
reflects what they actually do. Possibly pg_compresslog and
pg_decompresslog etc.. I've not reviewed those programs, but:

- Not sure why we have to compress away page headers. Touch as little as
you can has always been my thinking with recovery code.

- I'm very uncomfortable with touching the LSN. Maybe I misunderstand?

- Have you thought about how pg_standby would integrate with this
option? Can you please?

- I'll do some docs for this after Freeze, if you'd like. I have some
other changes to make there, so I can do this at the same time.

-- 
  Simon Riggs             
  EnterpriseDB   http://www.enterprisedb.com