From: Stephen Frost <sfrost@snowman.net>
To: Ron Johnson <ronljohnsonjr@gmail.com>
Cc: Pgsql-admin <pgsql-admin@lists.postgresql.org>
Subject: Re: pg_basebackup --wal-method=fetch
Date: Thu, 8 Feb 2024 17:21:29 -0500
Message-ID: <ZcVT6e9AU9XgP/37@tamriel.snowman.net> (raw)
In-Reply-To: <CANzqJaADTqhZXGZ8gxTj_0SdQ85rowLnuXHA6FKs19ir28x_3w@mail.gmail.com>
References: <CANzqJaDR-LZn2jQKOzb10-jXOdGrDtgf9=99_kKzLN3+0pWfAA@mail.gmail.com>
<CABUevEy38dnyRaJNgbextVVWjC72K-v0EUO+W00L7+ZfqVoqVA@mail.gmail.com>
<CANzqJaB7m2V+uTvu1rOY-2toye1ZtS3xp-GeZ9=bxO+r93wHow@mail.gmail.com>
<ZcVKf0vByWDMVH5M@tamriel.snowman.net>
<CANzqJaADTqhZXGZ8gxTj_0SdQ85rowLnuXHA6FKs19ir28x_3w@mail.gmail.com>
Greetings,
* Ron Johnson (ronljohnsonjr@gmail.com) wrote:
> On Thu, Feb 8, 2024 at 4:41 PM Stephen Frost <sfrost@snowman.net> wrote:
> > * Ron Johnson (ronljohnsonjr@gmail.com) wrote:
> > > On Thu, Feb 8, 2024 at 12:48 PM Magnus Hagander <magnus@hagander.net>
> > wrote:
> > > > On Thu, Feb 8, 2024, 17:05 Ron Johnson <ronljohnsonjr@gmail.com>
> > wrote:
> > > >> (PG 14, if it matters.)
> > > >>
> > > >> What's the purpose of fetch mode, as opposed to streaming mode? Is
> > it a
> > > >> legacy of bygone days that just hasn't been deprecated, or is there
> > > >> something I don't understand from reading
> > > >> https://www.postgresql.org/docs/14/app-pgbasebackup.html?
> > > >
> > > > Your backup can go to a single file with it, which it can't do in
> > > > streaming. Which means it can also be sent through a pipe.
> > >
> > > But isn't the whole purpose of pg_basebackup (running it on Node B, when
> > > the database instance is Node A)?
> >
> > Something seems missing from this question?
>
> The word "streaming".
> Should be "But isn't streaming the whole purpose of pg_basebackup"?
I'm a bit confused on this point still as if the whole purpose of
pg_basebackup is to be streaming ... then we should be defaulting to
fetch mode still?
> I use PgBackRest, though, and can't imagine single-threading any
> reasonably-sized database. In fact, one of the tasks on my mental TODO
> list is to research how to use PgBackRest to initialize a replica instance
> prior to starting Streaming Replication.
I use pgbackrest too. ;) That said, one of the longest poles in the
tent when it comes to dealing with backups is compression- and there are
tools like pigz that allow you to multi-thread a single stream across
many cores, so it isn't necessarily the case that using fetch mode or
pg_basebackup generally makes everything have to be completely
single-process. Of course, pgbackrest has a lot of other features that
make it a great tool to use.
In terms of using pgbackrest to initialize a replica ... that's
basically running 'pgbackrest restore --type=standby'? There's really
not much more to it than that. pgbackrest will set up the restored
system to replay from the WAL in the archive, you'd just need to
configure primary_conninfo so that the replica will attempt to connect
to the primary once it's caught up with all of the WAL in the archive.
> > > It also needs one connection instead of two to the server, if that's
> > > > limited.
> > >
> > > It's 2024, not 2011. Who can't spare an extra connection?
> >
> > Changing max_wal_senders requires a database-wide restart, so..
>
> To not have some wiggle room is poor planning
Doesn't change reality though.
> > If so, why? If that's not the angle,
> > then what is? Would you suggest some better documentation of the
> > option? I'm sure a proposal to improve the docs would be welcome, if
> > there's something confusing about them and this option.
>
> A hint as to the use-case for the non-default "streaming" option would be
> enlightening.
I mentioned a couple of them above and there is an example of streaming
in the documentation today:
######
To create a backup of a single-tablespace local database and compress
this with bzip2:
$ pg_basebackup -D - -Ft -X fetch | bzip2 > backup.tar.bz2
######
Would it make things more clear if this was an example that sent data to
a tape device instead of through bzip2..?
Thanks!
Stephen
Attachments:
[application/pgp-signature] signature.asc (832B, ../ZcVT6e9AU9XgP%2F37@tamriel.snowman.net/2-signature.asc)
download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: sfrost@snowman.net, ronljohnsonjr@gmail.com, pgsql-admin@lists.postgresql.org
Subject: Re: pg_basebackup --wal-method=fetch
In-Reply-To: <ZcVT6e9AU9XgP/37@tamriel.snowman.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox