From: David Steele <david@pgmasters.net>
To: Stephen Frost <sfrost@snowman.net>
To: Tim <timfosho@gmail.com>
Cc: posgres support <pgsql-admin@postgresql.org>
Subject: Re: pg_basebackup and pg_receivewal timing, missing WAL files
Date: Thu, 9 Jun 2022 18:34:11 -0400
Message-ID: <477f38d6-df05-60a2-2266-e54a4f232ac6@pgmasters.net> (raw)
In-Reply-To: <20220609212409.GR9030@tamriel.snowman.net>
References: <CAKhLO5itE_xp7KX7Vxnho8Uzczgwt8iUD9bVtKNPamBOst=hPA@mail.gmail.com>
<20220609183641.GN9030@tamriel.snowman.net>
<CAKhLO5gY4rUS65RUD_OGyHQTxoOa1-akLBWjh-CDQiVDJknecw@mail.gmail.com>
<20220609212409.GR9030@tamriel.snowman.net>
On 6/9/22 5:24 PM, Stephen Frost wrote:
>
> * Tim (timfosho@gmail.com) wrote:
>> We currently use pgbackrest as our primary backup tool actually, and it
>> works great! The issue I ran into with trying to set it up that way with
>> pgbackrest, is when configuring the second repo, pgbackrest will have to
>> archive all WAL files to it, going back to the oldest backup. And we have
>
> Eh? Why is that? I think there's some confusion here regarding how
> pgbackrest multi-repo works. We won't archive back to the 'oldest
> backup' when you add a second repo, we'll just archive *new* WAL to it
> after it's been configured.
By default pgbackrest will keep all WAL since the start of the oldest
backup in a repo. Each repo can have its own retention settings so this
might be a *different* oldest WAL per repo.
However, you can use repo-retention-archive and
repo-retention-archive-type to change this behavior, see
https://pgbackrest.org/configuration.html#section-repository/option-repo-retention-archive.
Basically, if you need to keep a number of full backups for compliance
you can set repo-retention-archive-type=full and
repo-retention-archive-type=1 in that repo and WAL in between the
backups will only be kept after the last backup. WAL required to make
backups consistent is always kept until the backup expires.
We've had requests for repo-retention-archive-type=0, i.e. only archive
WAL during a backup, but we have not had time to implement that. Some
users have implemented a work-around based on checking the backup lock.
There is an issue that describes the method but I was not able to find
it after a brief search. This is safer than it sounds because pgbackrest
will check that all WAL required to make the backup consistent reached
the archive. So, if the wrapper is broken you'll get errors.
Regards,
--
-David
david@pgmasters.net
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: david@pgmasters.net, sfrost@snowman.net, timfosho@gmail.com
Subject: Re: pg_basebackup and pg_receivewal timing, missing WAL files
In-Reply-To: <477f38d6-df05-60a2-2266-e54a4f232ac6@pgmasters.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox