From: scott ribe <scott_ribe@elevated-dev.com>
To: Nagy László Zsolt <gandalf@shopzeus.com>
Cc: pgsql-admin@lists.postgresql.org
Subject: Re: Reliable WAL file shipping over unreliable network
Date: Wed, 28 Feb 2018 11:12:03 -0700
Message-ID: <9BE48599-5417-42BF-A7D3-554590878841@elevated-dev.com> (raw)
In-Reply-To: <954e01b6-b240-eb9a-5feb-8efd202fef5a@shopzeus.com>
References: <dd116172-3ddf-ee56-57f3-6e18d49abf8e@shopzeus.com>
<400AC9CD-1E43-422C-A689-70BDBAFF231E@elevated-dev.com>
<ee3ae4be-ab24-8443-82c1-95bcd4f86a63@shopzeus.com>
<debd4205-e27a-cd9d-9169-6748e65083a2@shopzeus.com>
<BA4A1866-95B0-49C6-8CD1-3D1B60A7837E@elevated-dev.com>
<CF30D15A-BD42-42A7-9AD3-BB69AF9E8047@icloud.com>
<954e01b6-b240-eb9a-5feb-8efd202fef5a@shopzeus.com>
There seem to be 2 fundamental misunderstandings here:
1) That other processes cannot see data written to a file until it is flushed to disk; this is not true; while file data is still in file cache, it is visible to other processes.
2) That rsync writes the file on the destination directly; it does not; it writes into a temporary file and renames that file when it is complete.
> On Feb 28, 2018, at 10:53 AM, Nagy László Zsolt <gandalf@shopzeus.com> wrote:
>
>
>>
>> Just use "-ac”; you want -c option to ensure no data corruption during the transfer. Do not delete the file; let Postgres manage that.
>>
>> Here is a snippet from I script I use for archiving. You also want to make your script returns failure or success correctly.
>>
>> # SSH Command and options
>> SSH_CMD="ssh -o ServerAliveInterval=20 $ARCH_SERVER"
>> STS=3
>>
>> OUTPUT=$(rsync -ac --rsync-path="mkdir -p $ARCH_DIR && rsync" $XLOGFILE $ARCH_SERVER:$ARCH_DIR/$WALFILE)
>> if [ $? == 0 ]; then
>> STS=0
>> fi
>>
>> exit $STS
>>
--
Scott Ribe
https://www.linkedin.com/in/scottribe/
(303) 722-0567
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: scott_ribe@elevated-dev.com, gandalf@shopzeus.com, pgsql-admin@lists.postgresql.org
Subject: Re: Reliable WAL file shipping over unreliable network
In-Reply-To: <9BE48599-5417-42BF-A7D3-554590878841@elevated-dev.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox