From: Stephen Frost <sfrost@snowman.net>
To: Tim <timfosho@gmail.com>
Cc: david@pgmasters.net, posgres support <pgsql-admin@postgresql.org>
Subject: Re: pg_basebackup and pg_receivewal timing, missing WAL files
Date: Fri, 10 Jun 2022 12:33:25 -0400
Message-ID: <20220610163325.GW9030@tamriel.snowman.net> (raw)
In-Reply-To: <CAKhLO5jTyY_DJv=dDQH4QqmBt4LLmK0EuXcZy84_JsEg6opY0A@mail.gmail.com>
References: <CAKhLO5itE_xp7KX7Vxnho8Uzczgwt8iUD9bVtKNPamBOst=hPA@mail.gmail.com>
<20220609183641.GN9030@tamriel.snowman.net>
<CAKhLO5gY4rUS65RUD_OGyHQTxoOa1-akLBWjh-CDQiVDJknecw@mail.gmail.com>
<20220609212409.GR9030@tamriel.snowman.net>
<477f38d6-df05-60a2-2266-e54a4f232ac6@pgmasters.net>
<3268b3bf-1fea-a891-ffa8-7f3907b7b1d0@pgmasters.net>
<CAKhLO5jTyY_DJv=dDQH4QqmBt4LLmK0EuXcZy84_JsEg6opY0A@mail.gmail.com>
Greetings,
* Tim (timfosho@gmail.com) wrote:
> > However, you can use repo-retention-archive and
> > repo-retention-archive-type to change this behavior, see
> >
> > https://pgbackrest.org/configuration.html#section-repository/option-repo-retention-archive
> > .
> > Basically, if you need to keep a number of full backups for compliance
> > you can set repo-retention-archive-type=full and
> > repo-retention-archive-type=1 in that repo and WAL in between the
> > backups will only be kept after the last backup. WAL required to make
> > backups consistent is always kept until the backup expires.
>
> So as I understand it, it would keep WALs from the most current backup up
> to the most current WALs.
> In my case, the only purpose of this 2nd repo is to keep monthly backups
> that are 2-6 months old (and occasionally test restores)
> for legal purposes.
> The most recent backup in this repo would still be 2 months old, so it
> would be keeping an unnecessary 2 months of WALs.
You don't need to have the second repo be enabled all the time, just
enable it when you do the next full backup into the new repo, and
disable it afterwards, so you don't continue to push WAL to it.
> We've had requests for repo-retention-archive-type=0, i.e. only archive
> > WAL during a backup, but we have not had time to implement that. Some
> > users have implemented a work-around based on checking the backup lock.
> > There is an issue that describes the method but I was not able to find
> > it after a brief search. This is safer than it sounds because pgbackrest
> > will check that all WAL required to make the backup consistent reached
> > the archive. So, if the wrapper is broken you'll get errors.
>
> I'm guessing you're referring to: optional wal archiving · Issue #900 ·
> pgbackrest/pgbackrest (github.com)
> <https://github.com/pgbackrest/pgbackrest/issues/900;,
This was from before we had multi-repo support and I'd think simply
enabling/disabling that second repo would be the simpler approach.
I see that it sounds like you're going to take a look at trying that
approach- would be great to get your feedback on how hard it was and how
well it did or didn't work out.
Thanks,
Stephen
Attachments:
signature.asc (application/pgp-signature, 818B) download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: sfrost@snowman.net, timfosho@gmail.com
Subject: Re: pg_basebackup and pg_receivewal timing, missing WAL files
In-Reply-To: <20220610163325.GW9030@tamriel.snowman.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox