Date: Fri, 30 Nov 2018 10:29:31 -0500
From: Stephen Frost <sfrost@snowman.net>
To: Achilleas Mantzios <achill@matrix.gatewaynet.com>
Cc: pgsql-admin@lists.postgresql.org
Subject: Re: Backup solution over unreliable network
Message-ID: <20181130152931.GY3415@tamriel.snowman.net>
References: <fb7e7296-c60e-c2cc-93d5-9c2451e9a2a5@matrix.gatewaynet.com>
 <20181130120613.GW3415@tamriel.snowman.net>
 <6834c573-2a52-dd63-0264-5fc7258e7857@matrix.gatewaynet.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="ID5h4+vKj8m6mVzk"
Content-Disposition: inline
In-Reply-To: <6834c573-2a52-dd63-0264-5fc7258e7857@matrix.gatewaynet.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Precedence: bulk


--ID5h4+vKj8m6mVzk
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Greetings,

* Achilleas Mantzios (achill@matrix.gatewaynet.com) wrote:
> On 30/11/18 2:06 =CE=BC.=CE=BC., Stephen Frost wrote:
> >>- I have only read the best reviews about PgBackRest, can PgBackRest ad=
dress those issues?
> >Glad to hear you've read good reviews about pgbackrest.  As for
> >addressing these issues, pgbackrest has:
> >
> >- Backup resume
> >- Max WAL lag (in other words, you can have it simply start throwing WAL
> >   away if it can't archive it, rather than allowing the primary to run
> >   out of disk space)
>=20
> This is just superb! In our case we had the following architecture (now b=
arman is defunct) :
>=20
> Primary (consistent snapshots with pg_start/stop_backup)+ --> reliable ne=
t (archive_command via rsync) --> WAL repository
> =C2=A0=C2=A0 | (async streaming replication)
> =C2=A0=C2=A0 | (reliable net)
> =C2=A0=C2=A0 V
> Standby --> unreliable net (barman via method rsync + barman streaming fr=
om standby ***) --> remote cloud provider site(barman)
>=20
> So Primary and Standby are in the same cloud provider over consistent
> (mostly) network, whereas the barman (remote recovery) site communicates
> over internet. We would like to keep the old functionality (or even add a
> new PgBackRest node in the main cloud provider, so the question is : is
> there a way for archive-push to two different stanzas? Or delegate the
> archive-push to work from the Standby ?

We've had a few folks using pgbackrest to push to two stanzas by way of
basically doing 'pgbackrest --stanza=3Da archive-push && pgbackrest
--stanza=3Db archive-push' and with that it does work, and you could
combine that with the max WAL setting, potentially, but it's not a
solution that I'm really a fan of.  That's definitely a use-case we've
been thinking about though and have plans to support in the future,
but there are other things we're tackling now and so multi-repo hasn't
been a priority.

We've also considered supporting archive-mode=3Dalways and being able to
have the standby also push WAL and while we may support that in the
future, I'd say it's farther down on the list than multi-repo support.
As I recall, David Steele also had some specific technical concerns
around how to handle two systems pushing into the same WAL archive.
Having archive-mode=3Dalways be allowed if it's going to an independent
repo is an interesting thought though and might be simpler to do.

Thanks!

Stephen

--ID5h4+vKj8m6mVzk
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCgAGBQJcAVdbAAoJEO1sijiDR2RVb9IP/im7maK/xxs9RCGFDYOkyt5B
lk1OHq8OVPlqUNi8YjfCoyyjmNeXcNVINp0R9Cic1IWfqXe3XEv/TIIN2gCmwCKw
eXn5tPDRif/vSzGbTmSdcMCNe7Id9038xNGw/OgOhopVHY2T4hRgG5bamE1D1tmz
781iAZggbcOZ60Oo0daXogB8TowOb20WwIYGBmTD/RP51MnrhcRUAbJ7KFof+Hkk
9FRhq6sCcY3MU2C5BgUfRs4njWl1cv4PwsIQCEfBewWBruCd2CuPg5KbxA+LKJre
lboPDc2ddSzLGwE/HUIUPPzgjhfJq9k8RFoeqqkMQJPwasoAdY4Wye0DZCKSayRI
sLLiSJW6eP6vYNE2L6+6XMuRFPSp69+dovaGWrW6aqdfkf2+aPC5tsBHIkg5FZL4
Sos/jzhLtPp1O/M0ShbrpoMsZCeagpTf8A3fgwT+1BjYJI5f1wjg5JUsIpXjiWMw
RDgM1cdIzZo19aZGmgk3CFnheIVy59ZrrPUDl59owxtPYo3ay1Tbkm7Uw/s7N8Xy
z1ogf9/WA0vj/hh7occqRU0ACa0RQpw2HbD0jh3JR0IOaIea1SYkGSLxqi164QAP
WmwsS761uGCGB0AZAGTzdYA9lvVlAScZjy/oZnOcc8mAiXi/RiDce0NzFHKzY5I3
jxW9deZx8Ec91XuTPt5h
=4cwA
-----END PGP SIGNATURE-----

--ID5h4+vKj8m6mVzk--