Date: Fri, 10 Jun 2022 12:33:25 -0400
From: Stephen Frost <sfrost@snowman.net>
To: Tim <timfosho@gmail.com>
Cc: david@pgmasters.net, posgres support <pgsql-admin@postgresql.org>
Subject: Re: pg_basebackup and pg_receivewal timing, missing WAL files
Message-ID: <20220610163325.GW9030@tamriel.snowman.net>
References: 
 <CAKhLO5itE_xp7KX7Vxnho8Uzczgwt8iUD9bVtKNPamBOst=hPA@mail.gmail.com>
 <20220609183641.GN9030@tamriel.snowman.net>
 <CAKhLO5gY4rUS65RUD_OGyHQTxoOa1-akLBWjh-CDQiVDJknecw@mail.gmail.com>
 <20220609212409.GR9030@tamriel.snowman.net>
 <477f38d6-df05-60a2-2266-e54a4f232ac6@pgmasters.net>
 <3268b3bf-1fea-a891-ffa8-7f3907b7b1d0@pgmasters.net>
 <CAKhLO5jTyY_DJv=dDQH4QqmBt4LLmK0EuXcZy84_JsEg6opY0A@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="V1B6tgkYnQOXc079"
Content-Disposition: inline
In-Reply-To: 
 <CAKhLO5jTyY_DJv=dDQH4QqmBt4LLmK0EuXcZy84_JsEg6opY0A@mail.gmail.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Archived-At: 
 <https://www.postgresql.org/message-id/20220610163325.GW9030%40tamriel.snowman.net>
Precedence: bulk


--V1B6tgkYnQOXc079
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Greetings,

* Tim (timfosho@gmail.com) wrote:
> > However, you can use repo-retention-archive and
> > repo-retention-archive-type to change this behavior, see
> >
> > https://pgbackrest.org/configuration.html#section-repository/option-rep=
o-retention-archive
> > .
> > Basically, if you need to keep a number of full backups for compliance
> > you can set repo-retention-archive-type=3Dfull and
> > repo-retention-archive-type=3D1 in that repo and WAL in between the
> > backups will only be kept after the last backup. WAL required to make
> > backups consistent is always kept until the backup expires.
>=20
> So as I understand it, it would keep WALs from the most current backup up
> to the most current WALs.
> In my case, the only purpose of this 2nd repo is to keep monthly backups
> that are 2-6 months old (and occasionally test restores)
> for legal purposes.
> The  most recent backup in this repo would still be 2 months old, so it
> would be keeping an unnecessary 2 months of WALs.

You don't need to have the second repo be enabled all the time, just
enable it when you do the next full backup into the new repo, and
disable it afterwards, so you don't continue to push WAL to it.

> We've had requests for repo-retention-archive-type=3D0, i.e. only archive
> > WAL during a backup, but we have not had time to implement that. Some
> > users have implemented a work-around based on checking the backup lock.
> > There is an issue that describes the method but I was not able to find
> > it after a brief search. This is safer than it sounds because pgbackrest
> > will check that all WAL required to make the backup consistent reached
> > the archive. So, if the wrapper is broken you'll get errors.
>=20
> I'm guessing you're referring to: optional wal archiving =B7 Issue #900 =
=B7
> pgbackrest/pgbackrest (github.com)
> <https://github.com/pgbackrest/pgbackrest/issues/900>,

This was from before we had multi-repo support and I'd think simply
enabling/disabling that second repo would be the simpler approach.

I see that it sounds like you're going to take a look at trying that
approach- would be great to get your feedback on how hard it was and how
well it did or didn't work out.

Thanks,

Stephen

--V1B6tgkYnQOXc079
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCgAGBQJio3JVAAoJEO1sijiDR2RVLM0QAMMQ2IOx92cuUX1dLk3To5fJ
StY03eyxLC0a1jlQ7XyHdCMTCp6zc3CpqtlXoMxkKXQq9OO2/rkYESl6IUTmhqK4
pijHfLZhc2wNsGuRNdniH5JHSW2HDxUVB5e4jpZXwRrFRsuiwj7Z6d6rDE4GP2HY
xUGSRwGQDtl4HbSNE6KaChzxRaL/iGqnvgQCzrIz3cLRd+sosAUmKNmp5hPgSCOt
wlvgbRq0hVKeBw2t3mQ7sNBA3wcONh2rbpvN1PhIgnRZI/8i+AsV75Ssr2mzCjTf
4IuMkWDqg+7lz//o1Bb5gNF74f2LCLtVjqybH2LAvH3arbITuNDbPGkLxyC7facA
Sc0gBvAuvJ2NE76tNkx7jcdaWRPOMBHMJTpsqwaLi2uvPt9FEN/HmqYVodOS0Fot
t4LYP2wyFJ6LsbQI98NKhY4l7fmnclaEZsfP6A3VnNkjxDe+W4qfPaeRGFoFXJrM
8Jwy6RTxbMGfMqXMaqiFV0zvaBxsOfTx4ErShjGl7zBOKUtK8xflpzOWH7VmWIQG
8GNxXmikUo7kkJ2mLBIinulbu3raWlBe/RvukiflLYl31zGKlOeoEvQ6WR4CIqmG
vH7pILSHfyIrFpke2GWrbbDA9NnRwfEsPCLx7Yau56CAmKuMkKkddp5mYFxfHMPU
4rz1gmAxR3AzRMoJcYK8
=t8iD
-----END PGP SIGNATURE-----

--V1B6tgkYnQOXc079--