Message-ID: <4E7CB603.2060200@enterprisedb.com>
Date: Fri, 23 Sep 2011 19:38:27 +0300
From: Heikki Linnakangas <heikki.linnakangas@enterprisedb.com>
User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; fi-FI;
	rv:1.9.2.21) Gecko/20110831 Icedove/3.1.13
MIME-Version: 1.0
To: Magnus Hagander <magnus@hagander.net>
CC: Alvaro Herrera <alvherre@commandprompt.com>,
	Pg Hackers <pgsql-hackers@postgresql.org>,
	Robert Haas <robertmhaas@gmail.com>,
	Linas Virbalas <linas.virbalas@continuent.com>,
	Euler Taveira de Oliveira <euler@timbira.com>,
	Florian Pflug <fgp@phlo.org>
Subject: Re: Hot Backup with rsync fails at pg_clog if under load
References: <CAA25A88.1E7B5%linas.virbalas@continuent.com>
	<1316793461-sup-7812@alvh.no-ip.org>
	<CABUevEzH3q=q8XMZ+murfoyE=pvWko6UqO9t8JkfqVNJ2QpVjg@mail.gmail.com>
In-Reply-To: 
 <CABUevEzH3q=q8XMZ+murfoyE=pvWko6UqO9t8JkfqVNJ2QpVjg@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

On 23.09.2011 19:03, Magnus Hagander wrote:
> On Sep 23, 2011 5:59 PM, "Alvaro Herrera"<alvherre@commandprompt.com>
> wrote:
>>
>>
>> Excerpts from Linas Virbalas's message of vie sep 23 09:47:20 -0300 2011:
>>> On 9/23/11 12:05 PM, "Heikki Linnakangas"
>>> <heikki.linnakangas@enterprisedb.com>  wrote:
>>
>>> But on the standby its size is the old one (thus, it seems, that the
> size
>>> changed after the rsync transfer and before the pg_stop_backup() was
>>> called):
>>>
>>> ls -l pg_clog/
>>> total 8
>>> -rw------- 1 postgres postgres 8192 Sep 23 14:31 0000
>>
>> Sounds like rsync is caching the file size at the start of the run, and
>> then copying that many bytes, ignoring the growth that occurred after it
>> started.
>
> That pretty much matches what Daniel does when he got the same failure case.
>
> Is this not allowed? Shouldn't wal replay fix this?

That's OK. The effect is the same as if rsync had copied the file at the 
start.

What's not OK is to store the file as empty or truncated file in the 
backup, when the file is deleted from pg_clog while rsync is running. 
The file must have length >= the length the file had when backup 
started, or the file must be omitted from the backup altogether.

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com