From: Achilleas Mantzios <achill@matrix.gatewaynet.com>
To: pgsql-admin@lists.postgresql.org
Subject: Re: Backup solution over unreliable network
Date: Fri, 30 Nov 2018 20:49:39 +0200
Message-ID: <9956da38-bcb6-6768-d245-887df9e0fb93@matrix.gatewaynet.com> (raw)
In-Reply-To: <20801752-C311-4F70-9D03-08C5E711645C@mac.com>
References: <fb7e7296-c60e-c2cc-93d5-9c2451e9a2a5@matrix.gatewaynet.com>
<20801752-C311-4F70-9D03-08C5E711645C@mac.com>
On 30/11/18 8:22 μ.μ., Evan Bauer wrote:
> Achilleas,
>
> I may be over-simplifying your situation, but have you considered breaking the problem into two pieces? First backing up to a local drive and then using rsync to move those files over the unreliable network to the remote site.
>
> Like other who have responded, I can heartily recommend pgbackrest. But if the network stinks, then I’d break the problem in two and leave PostgreSQL out of the network equation.
Pretty good idea, but :
1) those rsync transfers have to be somehow db aware, otherwise lots of
things might break, checksums, order of WALs, etc. There would be the
need to write a whole solution and end up ... getting one of the
established solutions
2) the rsync part would go basically unattended, meaning no smart
software would be taking care of it, monitoring it, sending alerts, etc.
Also we had our issues with rsync in the past with unreliable networks
like getting error messages for which google returns one or no results
(no pgsql stuff, just system scripts) . No wonder more and more PgSQL
backup solutions move away from rsync.
Am I missing something or exaggerating?
>
> Cheers,
>
> Evan
>
> Sent from my iPhone
>
>> On Nov 30, 2018, at 05:17, Achilleas Mantzios <achill@matrix.gatewaynet.com> wrote:
>>
>> Hello,
>> we've been running our backup solution for the last 5 months to a second site which has an unreliable network connection. We had problems with barman, since it doesn't support backup resume, also no option to disable the replication slot, in the sense, that it is better to sacrifice the backup rather than fill up the primary with WALs and bring the primary down. Another issue is now supporting entirely backing up from the secondary. With barman this is not possible, streaming (or archiving) must originate from the primary.So I want to ask two things here :
>> - Backing up to a remote site over an unreliable channel is a limited use case by itself, it is useful for local PITR restores on specific tables/data, or in case the whole primary suffers a disaster. Is there any other benefit that would justify building a solution for it?
>> - I have only read the best reviews about PgBackRest, can PgBackRest address those issues?
>>
>> Thank you!
>>
>> --
>> Achilleas Mantzios
>> IT DEV Lead
>> IT DEPT
>> Dynacom Tankers Mgmt
>>
>>
>
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: achill@matrix.gatewaynet.com, pgsql-admin@lists.postgresql.org
Subject: Re: Backup solution over unreliable network
In-Reply-To: <9956da38-bcb6-6768-d245-887df9e0fb93@matrix.gatewaynet.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox