pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Dianne Skoll <dfs@roaringpenguin.com>
To: pgsql-admin@lists.postgresql.org
Subject: Re: Reliable WAL file shipping over unreliable network
Date: Wed, 28 Feb 2018 14:22:02 -0500
Message-ID: <20180228142202.427f459a@hydrogen.roaringpenguin.com> (raw)
In-Reply-To: <FB6AA142-81F4-4238-8456-E965555CDA44@icloud.com>
References: <dd116172-3ddf-ee56-57f3-6e18d49abf8e@shopzeus.com>
	<400AC9CD-1E43-422C-A689-70BDBAFF231E@elevated-dev.com>
	<ee3ae4be-ab24-8443-82c1-95bcd4f86a63@shopzeus.com>
	<debd4205-e27a-cd9d-9169-6748e65083a2@shopzeus.com>
	<BA4A1866-95B0-49C6-8CD1-3D1B60A7837E@elevated-dev.com>
	<CF30D15A-BD42-42A7-9AD3-BB69AF9E8047@icloud.com>
	<D5B11CF5-E816-435B-BFB8-1226E23C046A@elevated-dev.com>
	<37C5A42B-4CDD-4F01-8F1E-1B185C1819B9@icloud.com>
	<20180228132620.2c882f84@hydrogen.roaringpenguin.com>
	<33220469-5BB2-4A40-8F56-8F4F392C95B3@icloud.com>
	<20180228135220.1cd46347@hydrogen.roaringpenguin.com>
	<FB6AA142-81F4-4238-8456-E965555CDA44@icloud.com>

On Wed, 28 Feb 2018 14:12:39 -0500
Rui DeSousa <rui.desousa@icloud.com> wrote:

> You are dealing with a race condition. Postgres will continually
> attempt to archive the same file over and over until rsync foolishly
> returns success.  

What do you mean by "rsync foolishly returns success"?

> You are making the assumption that rsync removed the file?

Rsync does not create the file with the final name until it
has successfully transferred it completely.  It copies it to a temporary
file and then if (and only if) the transfer succeeded, then it atomically
renames the temporary file to the final filename.

The only failure mode if metadata updates are lost is that the file
will appear not to exist when it really should, not the other way
around.  You can never, ever have the file existing with the proper
filename until a transfer has succeeded.

However, see my other post about a test.  The -c option doesn't actually
help if a file changes while rsync is running.

Regards,

Dianne.




view thread (59+ messages)  latest in thread

Message-ID: <20180228142202.427f459a@hydrogen.roaringpenguin.com>
Permalink:  ../20180228142202.427f459a@hydrogen.roaringpenguin.com/
Also on:    postgresql.org/message-id/20180228142202.427f459a@hydrogen.roaringpenguin.com

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-admin@postgresql.org
  Cc: dfs@roaringpenguin.com, pgsql-admin@lists.postgresql.org
  Subject: Re: Reliable WAL file shipping over unreliable network
  In-Reply-To: <20180228142202.427f459a@hydrogen.roaringpenguin.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox