Content-Type: text/plain;
	charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\))
Subject: Re: Reliable WAL file shipping over unreliable network
From: scott ribe <scott_ribe@elevated-dev.com>
In-Reply-To: <954e01b6-b240-eb9a-5feb-8efd202fef5a@shopzeus.com>
Date: Wed, 28 Feb 2018 11:12:03 -0700
Cc: pgsql-admin@lists.postgresql.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <9BE48599-5417-42BF-A7D3-554590878841@elevated-dev.com>
References: <dd116172-3ddf-ee56-57f3-6e18d49abf8e@shopzeus.com>
 <400AC9CD-1E43-422C-A689-70BDBAFF231E@elevated-dev.com>
 <ee3ae4be-ab24-8443-82c1-95bcd4f86a63@shopzeus.com>
 <debd4205-e27a-cd9d-9169-6748e65083a2@shopzeus.com>
 <BA4A1866-95B0-49C6-8CD1-3D1B60A7837E@elevated-dev.com>
 <CF30D15A-BD42-42A7-9AD3-BB69AF9E8047@icloud.com>
 <954e01b6-b240-eb9a-5feb-8efd202fef5a@shopzeus.com>
To: =?utf-8?Q?Nagy_L=C3=A1szl=C3=B3_Zsolt?= <gandalf@shopzeus.com>
Precedence: bulk

There seem to be 2 fundamental misunderstandings here:

1) That other processes cannot see data written to a file until it is =
flushed to disk; this is not true; while file data is still in file =
cache, it is visible to other processes.

2) That rsync writes the file on the destination directly; it does not; =
it writes into a temporary file and renames that file when it is =
complete.

> On Feb 28, 2018, at 10:53 AM, Nagy L=C3=A1szl=C3=B3 Zsolt =
<gandalf@shopzeus.com> wrote:
>=20
>=20
>>=20
>> Just use "-ac=E2=80=9D;  you want -c option to ensure no data =
corruption during the transfer.  Do not delete the file; let Postgres =
manage that.
>>=20
>> Here is a snippet from I script I use for archiving.  You also want =
to make your script returns failure or success correctly. =20
>>=20
>> # SSH Command and options
>> SSH_CMD=3D"ssh -o ServerAliveInterval=3D20 $ARCH_SERVER"
>> STS=3D3
>>=20
>> OUTPUT=3D$(rsync -ac --rsync-path=3D"mkdir -p $ARCH_DIR && rsync" =
$XLOGFILE $ARCH_SERVER:$ARCH_DIR/$WALFILE)
>> if [ $? =3D=3D 0 ]; then=20
>>    STS=3D0
>> fi
>>=20
>> exit $STS
>>=20

--
Scott Ribe
https://www.linkedin.com/in/scottribe/
(303) 722-0567