From: Koichi Suzuki <suzuki.koichi@oss.ntt.co.jp>
To: Zeugswetter Andreas ADI SD <ZeugswetterA@spardat.at>
Cc: Hannu Krosing <hannu@skype.net>
Cc: Joshua D. Drake <jd@commandprompt.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Simon Riggs <simon@2ndquadrant.com>
Cc: josh@agliodbs.com
Cc: pgsql-hackers@postgresql.org
Cc: pgsql-patches@postgresql.org
Subject: Re: [HACKERS] Full page writes improvement, code update
Date: Fri, 13 Apr 2007 09:09:50 +0900
Message-ID: <461ECA4E.9010101@oss.ntt.co.jp> (raw)
In-Reply-To: <E1539E0ED7043848906A8FF995BDA57901E7BD64@m0143.s-mxs.net>
References: <460886E8.6040907@oss.ntt.co.jp>
<4609CAC3.7060008@oss.ntt.co.jp>
<1175072848.4386.323.camel@silverbirch.site>
<200703291145.28574.josh@agliodbs.com>
<1175200117.4386.616.camel@silverbirch.site>
<460C611E.6090800@oss.ntt.co.jp>
<27693.1176226131@sss.pgh.pa.us>
<461C3557.6040201@oss.ntt.co.jp>
<461C3732.20006@commandprompt.com>
<1176269612.3962.11.camel@localhost.localdomain>
<461D9090.60808@oss.ntt.co.jp>
<E1539E0ED7043848906A8FF995BDA57901E7BD64@m0143.s-mxs.net>
Hi,
Sorry, inline reply.
Zeugswetter Andreas ADI SD wrote:
>
> Yup, this is a good summary.
>
> You say you need to remove the optimization that avoids
> the logging of a new tuple because the full page image exists.
> I think we must already have the info in WAL which tuple inside the full
> page image
> is new (the one for which we avoided the WAL entry for).
>
> How about this:
> Leave current WAL as it is and only add the not removeable flag to full
> pages.
> pg_compresslog then replaces the full page image with a record for the
> one tuple that is changed.
> I tend to think it is not worth the increased complexity only to save
> bytes in the uncompressed WAL though.
It is essentially what my patch proposes. My patch includes flag to
full page writes which "can be" removed.
> Another point about pg_decompresslog:
>
> Why do you need a pg_decompresslog ? Imho pg_compresslog should already
> do the replacing of the
> full_page with the dummy entry. Then pg_decompresslog could be a simple
> gunzip, or whatever compression was used,
> but no logic.
Just removing full page writes does not work. If we shift the rest of
the WAL, then LSN becomes inconsistent in compressed archive logs which
pg_compresslog produces. For recovery, we have to restore LSN as the
original WAL. Pg_decompresslog restores removed full page writes as a
dumm records so that recovery redo functions won't be confused.
Regards;
>
> Andreas
>
--
-------------
Koichi Suzuki
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: suzuki.koichi@oss.ntt.co.jp, ZeugswetterA@spardat.at, hannu@skype.net, jd@commandprompt.com, tgl@sss.pgh.pa.us, simon@2ndquadrant.com, josh@agliodbs.com, pgsql-patches@postgresql.org
Subject: Re: [HACKERS] Full page writes improvement, code update
In-Reply-To: <461ECA4E.9010101@oss.ntt.co.jp>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox