From: Stephen Frost <sfrost@snowman.net>
To: Achilleas Mantzios <achill@matrix.gatewaynet.com>
Cc: pgsql-admin@lists.postgresql.org
Subject: Re: Backup solution over unreliable network
Date: Fri, 30 Nov 2018 07:06:13 -0500
Message-ID: <20181130120613.GW3415@tamriel.snowman.net> (raw)
In-Reply-To: <fb7e7296-c60e-c2cc-93d5-9c2451e9a2a5@matrix.gatewaynet.com>
References: <fb7e7296-c60e-c2cc-93d5-9c2451e9a2a5@matrix.gatewaynet.com>
Greetings,
* Achilleas Mantzios (achill@matrix.gatewaynet.com) wrote:
> we've been running our backup solution for the last 5 months to a second
> site which has an unreliable network connection. We had problems with
> barman, since it doesn't support backup resume, also no option to disable
> the replication slot, in the sense, that it is better to sacrifice the
> backup rather than fill up the primary with WALs and bring the primary down.
> Another issue is now supporting entirely backing up from the secondary. With
> barman this is not possible, streaming (or archiving) must originate from
> the primary.So I want to ask two things here :
> - Backing up to a remote site over an unreliable channel is a limited use
> case by itself, it is useful for local PITR restores on specific
> tables/data, or in case the whole primary suffers a disaster. Is there any
> other benefit that would justify building a solution for it?
Please don't build your own solution, it's really quite difficult to get
backups done correctly.
> - I have only read the best reviews about PgBackRest, can PgBackRest address those issues?
Glad to hear you've read good reviews about pgbackrest. As for
addressing these issues, pgbackrest has:
- Backup resume
- Max WAL lag (in other words, you can have it simply start throwing WAL
away if it can't archive it, rather than allowing the primary to run
out of disk space)
- Backup using the replica, primairly (note that this, currently,
requires access to the primary, but the bulk of the data comes from
the replica)
- Incremental/differential backup
- Parallel backup/resume and parallel archiving/fetching
- Backup verification- we checksum every file backed up and verify those
checksums on a resume, and we make sure that every WAL file needed to
restore the backup has made it into the archive.
- Delta restore
Which I believe covers most of the use-cases you brought up.
When we first implemented backup using the replica we had concerns
regarding doing a 'full' replica-based backup, and we didn't really see
there being a lot of demand for such a use-case (the replica has access
to the primary in general if it's a streaming replica, after all...),
but we might be open to revisiting that.
Thanks!
Stephen
Attachments:
signature.asc (application/pgp-signature, 818B) download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: sfrost@snowman.net, achill@matrix.gatewaynet.com, pgsql-admin@lists.postgresql.org
Subject: Re: Backup solution over unreliable network
In-Reply-To: <20181130120613.GW3415@tamriel.snowman.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox