Date: Wed, 28 Feb 2018 14:22:02 -0500
From: Dianne Skoll <dfs@roaringpenguin.com>
To: pgsql-admin@lists.postgresql.org
Subject: Re: Reliable WAL file shipping over unreliable network
Message-ID: <20180228142202.427f459a@hydrogen.roaringpenguin.com>
In-Reply-To: <FB6AA142-81F4-4238-8456-E965555CDA44@icloud.com>
References: <dd116172-3ddf-ee56-57f3-6e18d49abf8e@shopzeus.com>
	<400AC9CD-1E43-422C-A689-70BDBAFF231E@elevated-dev.com>
	<ee3ae4be-ab24-8443-82c1-95bcd4f86a63@shopzeus.com>
	<debd4205-e27a-cd9d-9169-6748e65083a2@shopzeus.com>
	<BA4A1866-95B0-49C6-8CD1-3D1B60A7837E@elevated-dev.com>
	<CF30D15A-BD42-42A7-9AD3-BB69AF9E8047@icloud.com>
	<D5B11CF5-E816-435B-BFB8-1226E23C046A@elevated-dev.com>
	<37C5A42B-4CDD-4F01-8F1E-1B185C1819B9@icloud.com>
	<20180228132620.2c882f84@hydrogen.roaringpenguin.com>
	<33220469-5BB2-4A40-8F56-8F4F392C95B3@icloud.com>
	<20180228135220.1cd46347@hydrogen.roaringpenguin.com>
	<FB6AA142-81F4-4238-8456-E965555CDA44@icloud.com>
Organization: Roaring Penguin Software Inc.
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Precedence: bulk

On Wed, 28 Feb 2018 14:12:39 -0500
Rui DeSousa <rui.desousa@icloud.com> wrote:

> You are dealing with a race condition. Postgres will continually
> attempt to archive the same file over and over until rsync foolishly
> returns success.  

What do you mean by "rsync foolishly returns success"?

> You are making the assumption that rsync removed the file?

Rsync does not create the file with the final name until it
has successfully transferred it completely.  It copies it to a temporary
file and then if (and only if) the transfer succeeded, then it atomically
renames the temporary file to the final filename.

The only failure mode if metadata updates are lost is that the file
will appear not to exist when it really should, not the other way
around.  You can never, ever have the file existing with the proper
filename until a transfer has succeeded.

However, see my other post about a test.  The -c option doesn't actually
help if a file changes while rsync is running.

Regards,

Dianne.