From: Stephen Frost <sfrost@snowman.net>
To: Victor Sudakov <vas@sibptus.ru>
Cc: pgsql-admin@lists.postgresql.org
Subject: Re: wal-g (https://github.com/wal-g/wal-g) reliability
Date: Mon, 8 Feb 2021 16:28:40 -0500
Message-ID: <20210208212840.GW27507@tamriel.snowman.net> (raw)
In-Reply-To: <YCDKyYzrv4HQZz9t@admin.sibptus.ru>
References: <YB0VuB8RJAKvTI6+@admin.sibptus.ru>
<CANNMO++t7hLG2Uk-M61ohXGNhBYszSPL8vn=zP09tcRv15QB4A@mail.gmail.com>
<YCDKyYzrv4HQZz9t@admin.sibptus.ru>
Greetings,
* Victor Sudakov (vas@sibptus.ru) wrote:
> Nikolay Samokhvalov wrote:
> > In short, it's reliable and battle-tested. It's used in companies such as
> > Yandex.Cloud and GitLab.com, successfully.
>
> Nikolay, don't you mind another question?
>
> I'm used to the fact that "pg_basebackup -X" creates a self-sufficient
> backup of a cluster which can be started right away as it contains all
> the WAL files required for recovery. `touch recovery.signal` is never necessary,
> and `touch standby.signal` is optional (when you do PITR etc).
>
> It's not the case with wal-g, the result of the `wal-g backup-fetch`
> command requires `touch recovery.signal` and a restore_command
> configured to fetch WALs from the wal-g storage.
>
> I have also noticed that wal-g keeps pg_control in a separate tar
> archive, and keeps a lot of metadata.
>
> The questions are:
>
> 1. If the metadata in the wal-g storage ever becomes corrupt, will I be
> able to restore the database manually from the archives in
> $WALG_*_PREFIX/{basebackups,wal}_005/ ?
>
> 2. Is there a `wal-g backup-fetch` option for truly self-sufficient
> restoration?
Interesting that you ask this! I say that because it's actually a case
that pgbackrest contemplated and explicitly added support for-
specifically, if you set --archive-copy (or archive-copy=true) and
disable compression (or decompress everything before you start PG), then
you can just start PG from the backup and it'll perform the necessary
recovery from the WAL and start up. We considered this an interesting
use-case and used it extensively and have maintained support for it.
All that said, of course, this limits the ability to do typical PITR,
but at least it gives you a consistent backup which you can start PG
from.
Thanks,
Stephen
Attachments:
signature.asc (application/pgp-signature, 818B) download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: sfrost@snowman.net, vas@sibptus.ru, pgsql-admin@lists.postgresql.org
Subject: Re: wal-g (https://github.com/wal-g/wal-g) reliability
In-Reply-To: <20210208212840.GW27507@tamriel.snowman.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox