public inbox for [email protected]
help / color / mirror / Atom feedBasebackup
9+ messages / 8 participants
[nested] [flat]
* Basebackup
@ 2024-09-03 12:31 Rajesh Kumar <[email protected]>
2024-09-03 12:37 ` Re: Basebackup Kashif Zeeshan <[email protected]>
2024-09-03 14:34 ` Re: Basebackup rams nalabolu <[email protected]>
2024-09-03 14:52 ` Re: Basebackup Laurenz Albe <[email protected]>
2024-09-04 08:19 ` Re: Basebackup Asad Ali <[email protected]>
0 siblings, 4 replies; 9+ messages in thread
From: Rajesh Kumar @ 2024-09-03 12:31 UTC (permalink / raw)
To: Pgsql-admin <[email protected]>
How different is basebackup from new server with primary host and rsync
from primary, move to new server and start cluster?
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Basebackup
2024-09-03 12:31 Basebackup Rajesh Kumar <[email protected]>
@ 2024-09-03 12:37 ` Kashif Zeeshan <[email protected]>
3 siblings, 0 replies; 9+ messages in thread
From: Kashif Zeeshan @ 2024-09-03 12:37 UTC (permalink / raw)
To: Rajesh Kumar <[email protected]>; +Cc: Pgsql-admin <[email protected]>
Hi Rajesh
Are you talking above backup taken with pg_basebackup with WAL Archiving
enabled?
If yes then you can start a server on it on the new host.
Regards
Kashif Zeeshan
On Tue, Sep 3, 2024 at 5:31 PM Rajesh Kumar <[email protected]>
wrote:
> How different is basebackup from new server with primary host and rsync
> from primary, move to new server and start cluster?
>
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Basebackup
2024-09-03 12:31 Basebackup Rajesh Kumar <[email protected]>
@ 2024-09-03 14:34 ` rams nalabolu <[email protected]>
2024-09-03 14:45 ` Re: Basebackup ajit wangkhem <[email protected]>
3 siblings, 1 reply; 9+ messages in thread
From: rams nalabolu @ 2024-09-03 14:34 UTC (permalink / raw)
To: Rajesh Kumar <[email protected]>; +Cc: Pgsql-admin <[email protected]>
The difference is that offloading resource consumption on primary while
taking backup and copy it to new server.
So it’s better to run it from new server
On Tue, Sep 3, 2024 at 7:31 AM Rajesh Kumar <[email protected]>
wrote:
> How different is basebackup from new server with primary host and rsync
> from primary, move to new server and start cluster?
>
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Basebackup
2024-09-03 12:31 Basebackup Rajesh Kumar <[email protected]>
2024-09-03 14:34 ` Re: Basebackup rams nalabolu <[email protected]>
@ 2024-09-03 14:45 ` ajit wangkhem <[email protected]>
0 siblings, 0 replies; 9+ messages in thread
From: ajit wangkhem @ 2024-09-03 14:45 UTC (permalink / raw)
To: Rajesh Kumar <[email protected]>; +Cc: Pgsql-admin <[email protected]>; rams nalabolu <[email protected]>
To overcome performance used Peer to Peer of both machines with CAT6A cable
so that you get 10GBPS speed and this does not hamer primary server because
separate network used for replication and also if possible used replication
slot. This was already tested in 1.6 TB data. If you have problem in
storage than used ( --format=tar --gzip) in pg_basebackup and then
decompress it.
Regards,
Ajit W
On Tue, Sep 3, 2024 at 8:05 PM rams nalabolu <[email protected]> wrote:
> The difference is that offloading resource consumption on primary while
> taking backup and copy it to new server.
> So it’s better to run it from new server
>
>
> On Tue, Sep 3, 2024 at 7:31 AM Rajesh Kumar <[email protected]>
> wrote:
>
>> How different is basebackup from new server with primary host and rsync
>> from primary, move to new server and start cluster?
>>
>
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Basebackup
2024-09-03 12:31 Basebackup Rajesh Kumar <[email protected]>
@ 2024-09-03 14:52 ` Laurenz Albe <[email protected]>
3 siblings, 0 replies; 9+ messages in thread
From: Laurenz Albe @ 2024-09-03 14:52 UTC (permalink / raw)
To: Rajesh Kumar <[email protected]>; Pgsql-admin <[email protected]>
On Tue, 2024-09-03 at 18:01 +0530, Rajesh Kumar wrote:
> How different is basebackup from new server with primary host and rsync from primary, move to new server and start cluster?
The main difference is that you have to stop the server if you want to use "rsync",
otherwise the backup will be inconsistent.
A backup taken with "pg_basebackup" will be recovered when you start it.
You *can* perform an online file system backup with "rsync", but then you will
have to use the low-level backup API:
https://www.postgresql.org/docs/current/continuous-archiving.html#BACKUP-LOWLEVEL-BASE-BACKUP
Yours,
Laurenz Albe
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Basebackup
2024-09-03 12:31 Basebackup Rajesh Kumar <[email protected]>
@ 2024-09-04 08:19 ` Asad Ali <[email protected]>
2024-09-04 09:44 ` Re: Basebackup Rajesh Kumar <[email protected]>
3 siblings, 1 reply; 9+ messages in thread
From: Asad Ali @ 2024-09-04 08:19 UTC (permalink / raw)
To: Rajesh Kumar <[email protected]>; +Cc: Pgsql-admin <[email protected]>
Hi Rajesh,
Here are a few differences between basebackup and rsync.
*pg_basebackup* is easier, as it's a single command with PostgreSQL
support. Data consistency is automatically ensured by postgresql. It
transfers the entire database, potentially high network bandwidth is
required. It is less flexible, due to mostly predefined options.
*rsync *is more complex, it requires multiple manual steps. This method
requires stopping or ensuring the primary server's data directory is in a
consistent state before starting the synchronization, Which can cause
downtime or performance impact. It requires careful handling to ensure data
consistency. It is more efficient with bandwidth, as it only transfers
changes. It is more flexible with control over what gets copied.
*You can use basebackup* if you want a straightforward, reliable way to
create a base backup. If you are looking for a method that integrates
directly with PostgreSQL's replication capabilities.
Network bandwidth and performance impact on the primary server are not
critical concerns. If you want an automated process that requires less
manual intervention.
*You can use rsync* if you have specific requirements for how the data
should be copied (e.g., selective file copying). If you need more control
over the synchronization process or want to optimize for minimal network
usage. If you have a large dataset and want to minimize the data transfer
by only copying changes after an initial sync. If you are comfortable with
handling data consistency and can manage the additional complexity.
Regards,
Asad Ali
On Tue, Sep 3, 2024 at 5:31 PM Rajesh Kumar <[email protected]>
wrote:
> How different is basebackup from new server with primary host and rsync
> from primary, move to new server and start cluster?
>
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Basebackup
2024-09-03 12:31 Basebackup Rajesh Kumar <[email protected]>
2024-09-04 08:19 ` Re: Basebackup Asad Ali <[email protected]>
@ 2024-09-04 09:44 ` Rajesh Kumar <[email protected]>
2024-09-09 19:21 ` Re: Basebackup vrms <[email protected]>
0 siblings, 1 reply; 9+ messages in thread
From: Rajesh Kumar @ 2024-09-04 09:44 UTC (permalink / raw)
To: Asad Ali <[email protected]>; +Cc: Pgsql-admin <[email protected]>
Thanks Asad. I have two openshift environments in different data centers. I
want to move one data from one DC to another Data center.
On Wed, 4 Sept 2024, 13:49 Asad Ali, <[email protected]> wrote:
> Hi Rajesh,
>
> Here are a few differences between basebackup and rsync.
>
> *pg_basebackup* is easier, as it's a single command with PostgreSQL
> support. Data consistency is automatically ensured by postgresql. It
> transfers the entire database, potentially high network bandwidth is
> required. It is less flexible, due to mostly predefined options.
>
> *rsync *is more complex, it requires multiple manual steps. This method
> requires stopping or ensuring the primary server's data directory is in a
> consistent state before starting the synchronization, Which can cause
> downtime or performance impact. It requires careful handling to ensure data
> consistency. It is more efficient with bandwidth, as it only transfers
> changes. It is more flexible with control over what gets copied.
>
> *You can use basebackup* if you want a straightforward, reliable way to
> create a base backup. If you are looking for a method that integrates
> directly with PostgreSQL's replication capabilities.
> Network bandwidth and performance impact on the primary server are not
> critical concerns. If you want an automated process that requires less
> manual intervention.
>
> *You can use rsync* if you have specific requirements for how the data
> should be copied (e.g., selective file copying). If you need more control
> over the synchronization process or want to optimize for minimal network
> usage. If you have a large dataset and want to minimize the data transfer
> by only copying changes after an initial sync. If you are comfortable with
> handling data consistency and can manage the additional complexity.
>
> Regards,
> Asad Ali
>
>
> On Tue, Sep 3, 2024 at 5:31 PM Rajesh Kumar <[email protected]>
> wrote:
>
>> How different is basebackup from new server with primary host and rsync
>> from primary, move to new server and start cluster?
>>
>
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Basebackup
2024-09-03 12:31 Basebackup Rajesh Kumar <[email protected]>
2024-09-04 08:19 ` Re: Basebackup Asad Ali <[email protected]>
2024-09-04 09:44 ` Re: Basebackup Rajesh Kumar <[email protected]>
@ 2024-09-09 19:21 ` vrms <[email protected]>
2024-09-09 19:28 ` Re: Basebackup Ron Johnson <[email protected]>
0 siblings, 1 reply; 9+ messages in thread
From: vrms @ 2024-09-09 19:21 UTC (permalink / raw)
To: [email protected]
hi Rajesh,
On 9/4/24 11:44 AM, Rajesh Kumar wrote:
>
> ... I have two openshift environments in different data centers. I
> want to move one data from one DC to another Data center.
>
it is not 100% clear what exactly you want to achieve.
But ... if you want to migrate your postgres from location A to location
B (that is what it somehow sounds like) I guess ...
1. setting up a replica on the target (B) via pg_basebackup
2. stopping your application and the old main (A)
3. promoting the replica (B) to become the new main
4. re-directing your application to the target (B)
could be quite efficient.
>
> On Wed, 4 Sept 2024, 13:49 Asad Ali, <[email protected]> wrote:
>
> Hi Rajesh,
>
> Here are a few differences between basebackup and rsync.
>
> *pg_basebackup* is easier, as it's a single command with
> PostgreSQL support. Data consistency is automatically ensured by
> postgresql. It transfers the entire database, potentially high
> network bandwidth is required. It is less flexible, due to mostly
> predefined options.
>
> *rsync *is more complex, it requires multiple manual steps. This
> method requires stopping or ensuring the primary server's data
> directory is in a consistent state before starting the
> synchronization, Which can cause downtime or performance impact.
> It requires careful handling to ensure data consistency. It is
> more efficient with bandwidth, as it only transfers changes. It
> is more flexible with control over what gets copied.
>
> *You can use basebackup* if you want a straightforward, reliable
> way to create a base backup. If you are looking for a method that
> integrates directly with PostgreSQL's replication capabilities.
> Network bandwidth and performance impact on the primary server are
> not critical concerns. If you want an automated process that
> requires less manual intervention.
>
> *You can use rsync* if you have specific requirements for how the
> data should be copied (e.g., selective file copying). If you need
> more control over the synchronization process or want to optimize
> for minimal network usage. If you have a large dataset and want to
> minimize the data transfer by only copying changes after an
> initial sync. If you are comfortable with handling data
> consistency and can manage the additional complexity.
>
> Regards,
> Asad Ali
>
>
> On Tue, Sep 3, 2024 at 5:31 PM Rajesh Kumar
> <[email protected]> wrote:
>
> How different is basebackup from new server with primary host
> and rsync from primary, move to new server and start cluster?
>
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: Basebackup
2024-09-03 12:31 Basebackup Rajesh Kumar <[email protected]>
2024-09-04 08:19 ` Re: Basebackup Asad Ali <[email protected]>
2024-09-04 09:44 ` Re: Basebackup Rajesh Kumar <[email protected]>
2024-09-09 19:21 ` Re: Basebackup vrms <[email protected]>
@ 2024-09-09 19:28 ` Ron Johnson <[email protected]>
0 siblings, 0 replies; 9+ messages in thread
From: Ron Johnson @ 2024-09-09 19:28 UTC (permalink / raw)
To: Pgsql-admin <[email protected]>
Physical replication using PgBackrest would probably be even faster,
because parallelism will better use the network bandwidth.
On Mon, Sep 9, 2024 at 3:18 PM vrms <[email protected]> wrote:
> hi Rajesh,
>
> On 9/4/24 11:44 AM, Rajesh Kumar wrote:
>
> ... I have two openshift environments in different data centers. I want to
> move one data from one DC to another Data center.
>
> it is not 100% clear what exactly you want to achieve.
> But ... if you want to migrate your postgres from location A to location B
> (that is what it somehow sounds like) I guess ...
>
> 1. setting up a replica on the target (B) via pg_basebackup
> 2. stopping your application and the old main (A)
> 3. promoting the replica (B) to become the new main
> 4. re-directing your application to the target (B)
>
> could be quite efficient.
>
>
> On Wed, 4 Sept 2024, 13:49 Asad Ali, <[email protected]> wrote:
>
>> Hi Rajesh,
>>
>> Here are a few differences between basebackup and rsync.
>>
>> *pg_basebackup* is easier, as it's a single command with PostgreSQL
>> support. Data consistency is automatically ensured by postgresql. It
>> transfers the entire database, potentially high network bandwidth is
>> required. It is less flexible, due to mostly predefined options.
>>
>> *rsync *is more complex, it requires multiple manual steps. This method
>> requires stopping or ensuring the primary server's data directory is in a
>> consistent state before starting the synchronization, Which can cause
>> downtime or performance impact. It requires careful handling to ensure data
>> consistency. It is more efficient with bandwidth, as it only transfers
>> changes. It is more flexible with control over what gets copied.
>>
>> *You can use basebackup* if you want a straightforward, reliable way to
>> create a base backup. If you are looking for a method that integrates
>> directly with PostgreSQL's replication capabilities.
>> Network bandwidth and performance impact on the primary server are not
>> critical concerns. If you want an automated process that requires less
>> manual intervention.
>>
>> *You can use rsync* if you have specific requirements for how the data
>> should be copied (e.g., selective file copying). If you need more control
>> over the synchronization process or want to optimize for minimal network
>> usage. If you have a large dataset and want to minimize the data transfer
>> by only copying changes after an initial sync. If you are comfortable with
>> handling data consistency and can manage the additional complexity.
>>
>> Regards,
>> Asad Ali
>>
>>
>> On Tue, Sep 3, 2024 at 5:31 PM Rajesh Kumar <[email protected]>
>> wrote:
>>
>>> How different is basebackup from new server with primary host and rsync
>>> from primary, move to new server and start cluster?
>>>
>>
--
Death to America, and butter sauce.
Iraq lobster!
^ permalink raw reply [nested|flat] 9+ messages in thread
end of thread, other threads:[~2024-09-09 19:28 UTC | newest]
Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2024-09-03 12:31 Basebackup Rajesh Kumar <[email protected]>
2024-09-03 12:37 ` Kashif Zeeshan <[email protected]>
2024-09-03 14:34 ` rams nalabolu <[email protected]>
2024-09-03 14:45 ` ajit wangkhem <[email protected]>
2024-09-03 14:52 ` Laurenz Albe <[email protected]>
2024-09-04 08:19 ` Asad Ali <[email protected]>
2024-09-04 09:44 ` Rajesh Kumar <[email protected]>
2024-09-09 19:21 ` vrms <[email protected]>
2024-09-09 19:28 ` Ron Johnson <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox