Subject: Re: Export operation efficiency in read replica

From: Siraj G <[email-masked]>
Date: Thu, 20 Mar 2025 18:28:32 +0530

Hello Laurenz

As per my understanding coming to a proper conclusion wrt RPO with export
operation is challenging. Eg., the export started at x and ended at z, the
time stamp here for many data sets is different. Moreover, I do not think
there is an incremental way available for export, correct?

Please correct me if my understanding is wrong.

Although I do agree the primary choice of backup to be storage based, we
wanted to have export backups as well as a secondary.

Concerning the impact taking export on read only.. Would you think we may
run into recovery issues on the replica side or anything that would prevent
the operation from being successful?

Regards
Siraj

On Thu, Mar 20, 2025 at 5:29 PM Laurenz Albe <[email-masked]>
wrote:

> On Thu, 2025-03-20 at 17:22 +0530, Siraj G wrote:
> > I have a DB with 1TB in size serving needs of one of our critical
> > applications. I have a requirement to take export of the DB on a
> > daily basis, but want to carry out this operation in read replica.
> > The postgresql version is: 16.6
> >
> > What would be the RPO of such backup?
>
> Depends on the speed of disk and network and on what an RPO is.
>
> > What would be the impact on the READ REPLICA with a long running
> > export operation?
>
> Potentially severe.
>
> You could look into storage technologies that allow you to take
> a snapshot and clone it.
>
> Yours,
> Laurenz Albe
>

---

From: Adrian Klaver <[email-masked]>
Date: Thu, 20 Mar 2025 07:04:55 -0700

On 3/20/25 05:58, Siraj G wrote:
> Hello Laurenz
> 
> As per my understanding coming to a proper conclusion wrt RPO 

You still have not defined what RPO is.

-- 
Adrian Klaver
[email-masked]

---

From: Laurenz Albe <[email-masked]>
Date: Thu, 20 Mar 2025 17:03:51 +0100

On Thu, 2025-03-20 at 18:28 +0530, Siraj G wrote:
> As per my understanding coming to a proper conclusion wrt RPO with
> export operation is challenging. Eg., the export started at x and
> ended at z, the time stamp here for many data sets is different.
> Moreover, I do not think there is an incremental way available for export, correct?

You are talking about "pg_dump", right?
A dump is always consistent, no matter how long it takes.

But if you want to take a daily dump of a 1TB database, you
are doing something wrong.  You should change the requirements.

Yours,
Laurenz Albe

---

From: Guillaume Lelarge <[email-masked]>
Date: Fri, 21 Mar 2025 14:52:55 +0100

> You still have not defined what RPO is.

I guess the OP is talking about Recovery Point Objective, which is one 
of two important parameters WRT to disaster recovery. It's the maximum 
data loss you agree on with your backup solution. That mostly depends on 
the database context, something we can't tell. And according to how much 
you agree to lose (one minute of activity? one hour?), you then can 
choose between pg_dump or PITR backups.

But with a 1TB-database, I wouldn't dare using pg_dump. PITR backup is 
the only option.

-- 
Guillaume Lelarge
Consultant
https://dalibo.com

---

From: Siraj G <[email-masked]>
Date: Fri, 21 Mar 2025 19:37:46 +0530

Thank you everyone!