public inbox for [email protected]help / color / mirror / Atom feed
Basebackup 9+ messages / 8 participants [nested] [flat]
* Basebackup @ 2024-09-03 12:31 Rajesh Kumar <[email protected]> 0 siblings, 4 replies; 9+ messages in thread From: Rajesh Kumar @ 2024-09-03 12:31 UTC (permalink / raw) To: Pgsql-admin <[email protected]> How different is basebackup from new server with primary host and rsync from primary, move to new server and start cluster? ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Basebackup @ 2024-09-03 12:37 Kashif Zeeshan <[email protected]> parent: Rajesh Kumar <[email protected]> 3 siblings, 0 replies; 9+ messages in thread From: Kashif Zeeshan @ 2024-09-03 12:37 UTC (permalink / raw) To: Rajesh Kumar <[email protected]>; +Cc: Pgsql-admin <[email protected]> Hi Rajesh Are you talking above backup taken with pg_basebackup with WAL Archiving enabled? If yes then you can start a server on it on the new host. Regards Kashif Zeeshan On Tue, Sep 3, 2024 at 5:31 PM Rajesh Kumar <[email protected]> wrote: > How different is basebackup from new server with primary host and rsync > from primary, move to new server and start cluster? > ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Basebackup @ 2024-09-03 14:34 rams nalabolu <[email protected]> parent: Rajesh Kumar <[email protected]> 3 siblings, 1 reply; 9+ messages in thread From: rams nalabolu @ 2024-09-03 14:34 UTC (permalink / raw) To: Rajesh Kumar <[email protected]>; +Cc: Pgsql-admin <[email protected]> The difference is that offloading resource consumption on primary while taking backup and copy it to new server. So it’s better to run it from new server On Tue, Sep 3, 2024 at 7:31 AM Rajesh Kumar <[email protected]> wrote: > How different is basebackup from new server with primary host and rsync > from primary, move to new server and start cluster? > ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Basebackup @ 2024-09-03 14:45 ajit wangkhem <[email protected]> parent: rams nalabolu <[email protected]> 0 siblings, 0 replies; 9+ messages in thread From: ajit wangkhem @ 2024-09-03 14:45 UTC (permalink / raw) To: Rajesh Kumar <[email protected]>; +Cc: Pgsql-admin <[email protected]>; rams nalabolu <[email protected]> To overcome performance used Peer to Peer of both machines with CAT6A cable so that you get 10GBPS speed and this does not hamer primary server because separate network used for replication and also if possible used replication slot. This was already tested in 1.6 TB data. If you have problem in storage than used ( --format=tar --gzip) in pg_basebackup and then decompress it. Regards, Ajit W On Tue, Sep 3, 2024 at 8:05 PM rams nalabolu <[email protected]> wrote: > The difference is that offloading resource consumption on primary while > taking backup and copy it to new server. > So it’s better to run it from new server > > > On Tue, Sep 3, 2024 at 7:31 AM Rajesh Kumar <[email protected]> > wrote: > >> How different is basebackup from new server with primary host and rsync >> from primary, move to new server and start cluster? >> > ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Basebackup @ 2024-09-03 14:52 Laurenz Albe <[email protected]> parent: Rajesh Kumar <[email protected]> 3 siblings, 0 replies; 9+ messages in thread From: Laurenz Albe @ 2024-09-03 14:52 UTC (permalink / raw) To: Rajesh Kumar <[email protected]>; Pgsql-admin <[email protected]> On Tue, 2024-09-03 at 18:01 +0530, Rajesh Kumar wrote: > How different is basebackup from new server with primary host and rsync from primary, move to new server and start cluster? The main difference is that you have to stop the server if you want to use "rsync", otherwise the backup will be inconsistent. A backup taken with "pg_basebackup" will be recovered when you start it. You *can* perform an online file system backup with "rsync", but then you will have to use the low-level backup API: https://www.postgresql.org/docs/current/continuous-archiving.html#BACKUP-LOWLEVEL-BASE-BACKUP Yours, Laurenz Albe ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Basebackup @ 2024-09-04 08:19 Asad Ali <[email protected]> parent: Rajesh Kumar <[email protected]> 3 siblings, 1 reply; 9+ messages in thread From: Asad Ali @ 2024-09-04 08:19 UTC (permalink / raw) To: Rajesh Kumar <[email protected]>; +Cc: Pgsql-admin <[email protected]> Hi Rajesh, Here are a few differences between basebackup and rsync. *pg_basebackup* is easier, as it's a single command with PostgreSQL support. Data consistency is automatically ensured by postgresql. It transfers the entire database, potentially high network bandwidth is required. It is less flexible, due to mostly predefined options. *rsync *is more complex, it requires multiple manual steps. This method requires stopping or ensuring the primary server's data directory is in a consistent state before starting the synchronization, Which can cause downtime or performance impact. It requires careful handling to ensure data consistency. It is more efficient with bandwidth, as it only transfers changes. It is more flexible with control over what gets copied. *You can use basebackup* if you want a straightforward, reliable way to create a base backup. If you are looking for a method that integrates directly with PostgreSQL's replication capabilities. Network bandwidth and performance impact on the primary server are not critical concerns. If you want an automated process that requires less manual intervention. *You can use rsync* if you have specific requirements for how the data should be copied (e.g., selective file copying). If you need more control over the synchronization process or want to optimize for minimal network usage. If you have a large dataset and want to minimize the data transfer by only copying changes after an initial sync. If you are comfortable with handling data consistency and can manage the additional complexity. Regards, Asad Ali On Tue, Sep 3, 2024 at 5:31 PM Rajesh Kumar <[email protected]> wrote: > How different is basebackup from new server with primary host and rsync > from primary, move to new server and start cluster? > ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Basebackup @ 2024-09-04 09:44 Rajesh Kumar <[email protected]> parent: Asad Ali <[email protected]> 0 siblings, 1 reply; 9+ messages in thread From: Rajesh Kumar @ 2024-09-04 09:44 UTC (permalink / raw) To: Asad Ali <[email protected]>; +Cc: Pgsql-admin <[email protected]> Thanks Asad. I have two openshift environments in different data centers. I want to move one data from one DC to another Data center. On Wed, 4 Sept 2024, 13:49 Asad Ali, <[email protected]> wrote: > Hi Rajesh, > > Here are a few differences between basebackup and rsync. > > *pg_basebackup* is easier, as it's a single command with PostgreSQL > support. Data consistency is automatically ensured by postgresql. It > transfers the entire database, potentially high network bandwidth is > required. It is less flexible, due to mostly predefined options. > > *rsync *is more complex, it requires multiple manual steps. This method > requires stopping or ensuring the primary server's data directory is in a > consistent state before starting the synchronization, Which can cause > downtime or performance impact. It requires careful handling to ensure data > consistency. It is more efficient with bandwidth, as it only transfers > changes. It is more flexible with control over what gets copied. > > *You can use basebackup* if you want a straightforward, reliable way to > create a base backup. If you are looking for a method that integrates > directly with PostgreSQL's replication capabilities. > Network bandwidth and performance impact on the primary server are not > critical concerns. If you want an automated process that requires less > manual intervention. > > *You can use rsync* if you have specific requirements for how the data > should be copied (e.g., selective file copying). If you need more control > over the synchronization process or want to optimize for minimal network > usage. If you have a large dataset and want to minimize the data transfer > by only copying changes after an initial sync. If you are comfortable with > handling data consistency and can manage the additional complexity. > > Regards, > Asad Ali > > > On Tue, Sep 3, 2024 at 5:31 PM Rajesh Kumar <[email protected]> > wrote: > >> How different is basebackup from new server with primary host and rsync >> from primary, move to new server and start cluster? >> > ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Basebackup @ 2024-09-09 19:21 vrms <[email protected]> parent: Rajesh Kumar <[email protected]> 0 siblings, 1 reply; 9+ messages in thread From: vrms @ 2024-09-09 19:21 UTC (permalink / raw) To: [email protected] hi Rajesh, On 9/4/24 11:44 AM, Rajesh Kumar wrote: > > ... I have two openshift environments in different data centers. I > want to move one data from one DC to another Data center. > it is not 100% clear what exactly you want to achieve. But ... if you want to migrate your postgres from location A to location B (that is what it somehow sounds like) I guess ... 1. setting up a replica on the target (B) via pg_basebackup 2. stopping your application and the old main (A) 3. promoting the replica (B) to become the new main 4. re-directing your application to the target (B) could be quite efficient. > > On Wed, 4 Sept 2024, 13:49 Asad Ali, <[email protected]> wrote: > > Hi Rajesh, > > Here are a few differences between basebackup and rsync. > > *pg_basebackup* is easier, as it's a single command with > PostgreSQL support. Data consistency is automatically ensured by > postgresql. It transfers the entire database, potentially high > network bandwidth is required. It is less flexible, due to mostly > predefined options. > > *rsync *is more complex, it requires multiple manual steps. This > method requires stopping or ensuring the primary server's data > directory is in a consistent state before starting the > synchronization, Which can cause downtime or performance impact. > It requires careful handling to ensure data consistency. It is > more efficient with bandwidth, as it only transfers changes. It > is more flexible with control over what gets copied. > > *You can use basebackup* if you want a straightforward, reliable > way to create a base backup. If you are looking for a method that > integrates directly with PostgreSQL's replication capabilities. > Network bandwidth and performance impact on the primary server are > not critical concerns. If you want an automated process that > requires less manual intervention. > > *You can use rsync* if you have specific requirements for how the > data should be copied (e.g., selective file copying). If you need > more control over the synchronization process or want to optimize > for minimal network usage. If you have a large dataset and want to > minimize the data transfer by only copying changes after an > initial sync. If you are comfortable with handling data > consistency and can manage the additional complexity. > > Regards, > Asad Ali > > > On Tue, Sep 3, 2024 at 5:31 PM Rajesh Kumar > <[email protected]> wrote: > > How different is basebackup from new server with primary host > and rsync from primary, move to new server and start cluster? > ^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Basebackup @ 2024-09-09 19:28 Ron Johnson <[email protected]> parent: vrms <[email protected]> 0 siblings, 0 replies; 9+ messages in thread From: Ron Johnson @ 2024-09-09 19:28 UTC (permalink / raw) To: Pgsql-admin <[email protected]> Physical replication using PgBackrest would probably be even faster, because parallelism will better use the network bandwidth. On Mon, Sep 9, 2024 at 3:18 PM vrms <[email protected]> wrote: > hi Rajesh, > > On 9/4/24 11:44 AM, Rajesh Kumar wrote: > > ... I have two openshift environments in different data centers. I want to > move one data from one DC to another Data center. > > it is not 100% clear what exactly you want to achieve. > But ... if you want to migrate your postgres from location A to location B > (that is what it somehow sounds like) I guess ... > > 1. setting up a replica on the target (B) via pg_basebackup > 2. stopping your application and the old main (A) > 3. promoting the replica (B) to become the new main > 4. re-directing your application to the target (B) > > could be quite efficient. > > > On Wed, 4 Sept 2024, 13:49 Asad Ali, <[email protected]> wrote: > >> Hi Rajesh, >> >> Here are a few differences between basebackup and rsync. >> >> *pg_basebackup* is easier, as it's a single command with PostgreSQL >> support. Data consistency is automatically ensured by postgresql. It >> transfers the entire database, potentially high network bandwidth is >> required. It is less flexible, due to mostly predefined options. >> >> *rsync *is more complex, it requires multiple manual steps. This method >> requires stopping or ensuring the primary server's data directory is in a >> consistent state before starting the synchronization, Which can cause >> downtime or performance impact. It requires careful handling to ensure data >> consistency. It is more efficient with bandwidth, as it only transfers >> changes. It is more flexible with control over what gets copied. >> >> *You can use basebackup* if you want a straightforward, reliable way to >> create a base backup. If you are looking for a method that integrates >> directly with PostgreSQL's replication capabilities. >> Network bandwidth and performance impact on the primary server are not >> critical concerns. If you want an automated process that requires less >> manual intervention. >> >> *You can use rsync* if you have specific requirements for how the data >> should be copied (e.g., selective file copying). If you need more control >> over the synchronization process or want to optimize for minimal network >> usage. If you have a large dataset and want to minimize the data transfer >> by only copying changes after an initial sync. If you are comfortable with >> handling data consistency and can manage the additional complexity. >> >> Regards, >> Asad Ali >> >> >> On Tue, Sep 3, 2024 at 5:31 PM Rajesh Kumar <[email protected]> >> wrote: >> >>> How different is basebackup from new server with primary host and rsync >>> from primary, move to new server and start cluster? >>> >> -- Death to America, and butter sauce. Iraq lobster! ^ permalink raw reply [nested|flat] 9+ messages in thread
end of thread, other threads:[~2024-09-09 19:28 UTC | newest] Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-09-03 12:31 Basebackup Rajesh Kumar <[email protected]> 2024-09-03 12:37 ` Kashif Zeeshan <[email protected]> 2024-09-03 14:34 ` rams nalabolu <[email protected]> 2024-09-03 14:45 ` ajit wangkhem <[email protected]> 2024-09-03 14:52 ` Laurenz Albe <[email protected]> 2024-09-04 08:19 ` Asad Ali <[email protected]> 2024-09-04 09:44 ` Rajesh Kumar <[email protected]> 2024-09-09 19:21 ` vrms <[email protected]> 2024-09-09 19:28 ` Ron Johnson <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox