Subject: Basebackup

From: Rajesh Kumar <[email-masked]>
Date: Tue, 3 Sep 2024 18:01:16 +0530

How different is basebackup from new server with primary host and rsync
from primary,  move to new server and start cluster?

---

From: Kashif Zeeshan <[email-masked]>
Date: Tue, 3 Sep 2024 17:37:02 +0500

Hi Rajesh

Are you talking above backup taken with pg_basebackup with WAL Archiving
enabled?
If yes then you can start a server on it on the new host.

Regards
Kashif Zeeshan

---

From: rams nalabolu <[email-masked]>
Date: Tue, 3 Sep 2024 09:34:42 -0500

The difference is that offloading resource consumption on primary while
taking backup and copy it to new server.
So it’s better to run it from new server

---

From: ajit wangkhem <[email-masked]>
Date: Tue, 3 Sep 2024 20:15:23 +0530

To overcome performance used Peer to Peer of both machines with CAT6A cable
so that you get 10GBPS speed and this does not hamer primary server because
separate network used for replication and also if possible used replication
slot. This was already tested in 1.6 TB data. If you have problem in
storage than used ( --format=tar --gzip) in pg_basebackup and then
decompress it.

Regards,
Ajit W

---

From: Laurenz Albe <[email-masked]>
Date: Tue, 03 Sep 2024 16:52:55 +0200

On Tue, 2024-09-03 at 18:01 +0530, Rajesh Kumar wrote:
> How different is basebackup from new server with primary host and rsync from primary,  move to new server and start cluster?

The main difference is that you have to stop the server if you want to use "rsync",
otherwise the backup will be inconsistent.

A backup taken with "pg_basebackup" will be recovered when you start it.

You *can* perform an online file system backup with "rsync", but then you will
have to use the low-level backup API:
https://www.postgresql.org/docs/current/continuous-archiving.html#BACKUP-LOWLEVEL-BASE-BACKUP

Yours,
Laurenz Albe

---

From: Asad Ali <[email-masked]>
Date: Wed, 4 Sep 2024 13:19:27 +0500

Hi Rajesh,

Here are a few differences between basebackup and rsync.

*pg_basebackup* is easier, as it's a single command with PostgreSQL
support. Data consistency is automatically ensured by postgresql. It
transfers the entire database, potentially high network bandwidth is
required. It is less flexible, due to mostly predefined options.

*rsync *is more complex, it requires multiple manual steps. This method
requires stopping or ensuring the primary server's data directory is in a
consistent state before starting the synchronization, Which can cause
downtime or performance impact. It requires careful handling to ensure data
consistency.  It is more efficient with bandwidth, as it only transfers
changes.  It is more flexible with control over what gets copied.

*You can use basebackup* if you want a straightforward, reliable way to
create a base backup. If you are looking for a method that integrates
directly with PostgreSQL's replication capabilities.
Network bandwidth and performance impact on the primary server are not
critical concerns. If you want an automated process that requires less
manual intervention.

*You can use rsync* if you have specific requirements for how the data
should be copied (e.g., selective file copying). If you need more control
over the synchronization process or want to optimize for minimal network
usage. If you have a large dataset and want to minimize the data transfer
by only copying changes after an initial sync. If you are comfortable with
handling data consistency and can manage the additional complexity.

Regards,
Asad Ali

---

From: Rajesh Kumar <[email-masked]>
Date: Wed, 4 Sep 2024 15:14:41 +0530

Thanks Asad. I have two openshift environments in different data centers. I
want to move one data from one DC to another Data center.

---

From: vrms <[email-masked]>
Date: Mon, 9 Sep 2024 21:21:46 +0200

hi Rajesh,

On 9/4/24 11:44 AM, Rajesh Kumar wrote:
>
> ... I have two openshift environments in different data centers. I 
it is not 100% clear what exactly you want to achieve.
But ... if you want to migrate your postgres from location A to location 
B (that is what it somehow sounds like) I guess ...

 1. setting up a replica on the target (B) via pg_basebackup
 2. stopping your application and the old main (A)
 3. promoting the replica (B) to become the new main
 4. re-directing your application to the target (B)

could be quite efficient.

>     *pg_basebackup* is easier, as it's a single command with
>     PostgreSQL support. Data consistency is automatically ensured by
>     postgresql. It transfers the entire database, potentially high
>     network bandwidth is required. It is less flexible, due to mostly
>     predefined options.
>
>     *rsync *is more complex, it requires multiple manual steps. This
>     method requires stopping or ensuring the primary server's data
>     directory is in a consistent state before starting the
>     synchronization, Which can cause downtime or performance impact.
>     It requires careful handling to ensure data consistency. It is
>     more efficient with bandwidth, as it only transfers changes.  It
>     is more flexible with control over what gets copied.
>
>     *You can use basebackup* if you want a straightforward, reliable
>     way to create a base backup. If you are looking for a method that
>     integrates directly with PostgreSQL's replication capabilities.
>     Network bandwidth and performance impact on the primary server are
>     not critical concerns. If you want an automated process that
>     requires less manual intervention.
>
>     *You can use rsync* if you have specific requirements for how the
>     data should be copied (e.g., selective file copying). If you need
>     more control over the synchronization process or want to optimize
>     for minimal network usage. If you have a large dataset and want to
>     minimize the data transfer by only copying changes after an
>     initial sync. If you are comfortable with handling data
>     consistency and can manage the additional complexity.
>
>     Regards,
>     Asad Ali
>
>
>     On Tue, Sep 3, 2024 at 5:31 PM Rajesh Kumar
>     <[email-masked]> wrote:
>
>         How different is basebackup from new server with primary host
>         and rsync from primary,  move to new server and start cluster?
>

---

From: Ron Johnson <[email-masked]>
Date: Mon, 9 Sep 2024 15:28:54 -0400

Physical replication using PgBackrest would probably be even faster,
because parallelism will better use the network bandwidth.

On Mon, Sep 9, 2024 at 3:18 PM vrms <[email-masked]> wrote:

> ... I have two openshift environments in different data centers. I want to
> move one data from one DC to another Data center.
>
> But ... if you want to migrate your postgres from location A to location B
> (that is what it somehow sounds like) I guess ...
>

-- 
Death to America, and butter sauce.
Iraq lobster!