From: Rui DeSousa <rui.desousa@icloud.com>
To: Stephen Frost <sfrost@snowman.net>
Cc: scott ribe <scott_ribe@elevated-dev.com>
Cc: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
Cc: Dianne Skoll <dfs@roaringpenguin.com>
Cc: pgsql-admin@lists.postgresql.org
Subject: Re: Reliable WAL file shipping over unreliable network
Date: Fri, 02 Mar 2018 13:22:53 -0500
Message-ID: <BB8E0E71-D65F-409F-B9AD-DDF2237CDF99@icloud.com> (raw)
In-Reply-To: <20180302180911.GO2416@tamriel.snowman.net>
References: <20180228132620.2c882f84@hydrogen.roaringpenguin.com>
<33220469-5BB2-4A40-8F56-8F4F392C95B3@icloud.com>
<21AF0CB4-0873-4960-BA14-E72FA08B352E@icloud.com>
<5a49c387-e4f4-f70c-537c-9c338c80b687@catalyst.net.nz>
<9DDA6F85-077E-4F8D-8708-632CC2907D3C@icloud.com>
<06dd8b81-8b1a-ae5c-8ddd-1556b5946176@catalyst.net.nz>
<6056907B-551F-4ED0-804B-0FCACD39EB23@icloud.com>
<ea674584-85bc-3f63-c182-0d45123bb3eb@catalyst.net.nz>
<988AB20C-9FDE-4197-BF3F-C54D80C2ED78@elevated-dev.com>
<E6AB850C-D05E-405B-8D4E-DE18E128F402@icloud.com>
<20180302180911.GO2416@tamriel.snowman.net>
Not a problem; as you should be archiving your WALs to multiple sites simultaneously. You’re DR site should not rely on any resources stored at the primary site.
> On Mar 2, 2018, at 1:09 PM, Stephen Frost <sfrost@snowman.net> wrote:
>
> Greetings,
>
> * Rui DeSousa (rui.desousa@icloud.com) wrote:
>>> On Mar 1, 2018, at 12:21 AM, scott ribe <scott_ribe@elevated-dev.com> wrote:
>>> The false report of success is not good, but it's not the root problem.
>>
>> A false success if a problem; especially in this use case as the source WAL file will be deleted by Postgres before it was truly successful. While monitoring is nice to avoid the issue it is not a fix for the issue.
>>
>> I personally cannot recommend the use of rsync in this application for two reasons.
>
> There's a bigger problem that I'm amazed hasn't been brought up already-
> the rsync, copy, cp, whatever, may finish just fine and then the system
> that the WAL file was copied to crashes and you end up losing a bunch of
> WAL because none of these methods actually sync's the WAL to disk.
>
> No archive_command should ever return success until the WAL is actually
> written out to permanent storage and sync'd. That's what tools like
> pgBackRest do and is part of the reason why people really shouldn't try
> to hack up their own archive_command solution but should be using well
> tested existing solutions.
>
> Thanks!
>
> Stephen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: rui.desousa@icloud.com, sfrost@snowman.net, scott_ribe@elevated-dev.com, mark.kirkwood@catalyst.net.nz, dfs@roaringpenguin.com, pgsql-admin@lists.postgresql.org
Subject: Re: Reliable WAL file shipping over unreliable network
In-Reply-To: <BB8E0E71-D65F-409F-B9AD-DDF2237CDF99@icloud.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox