agora inbox for pgsql-performance@postgresql.org
help / color / mirror / Atom feedPostgres DB backup is taking too much time
7+ messages / 5 participants
[nested] [flat]
* Postgres DB backup is taking too much time
@ 2026-05-03 09:42 masheed ullah <masheedullah@gmail.com>
2026-05-03 17:16 ` Re: Postgres DB backup is taking too much time manish yadav <manishy174@yahoo.co.in>
2026-05-04 00:20 ` Re: Postgres DB backup is taking too much time Greg Sabino Mullane <htamfids@gmail.com>
2026-05-04 09:29 ` Re: Postgres DB backup is taking too much time Ilya Anfimov <ilan@tzirechnoy.com>
0 siblings, 3 replies; 7+ messages in thread
From: masheed ullah @ 2026-05-03 09:42 UTC (permalink / raw)
To: pgsql-performance@lists.postgresql.org
Hi,
We have a database of 20TB, and it's taking almost 24 hours to complete.
While full restore takes 28 hours. Although we have HA enabled with primary
and multiple replica's.
*So we need a solution/tool to reduce the RTO, incase of disk failure/
Human error for data/tables deletion.*
We are using Barman for backup.
My question is "Are there any tools like ZDLRA appliances for Oracle" to
reduce the RTO to less than 10 hours".
Best Regards,
Khattak
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Postgres DB backup is taking too much time
2026-05-03 09:42 Postgres DB backup is taking too much time masheed ullah <masheedullah@gmail.com>
@ 2026-05-03 17:16 ` manish yadav <manishy174@yahoo.co.in>
2 siblings, 0 replies; 7+ messages in thread
From: manish yadav @ 2026-05-03 17:16 UTC (permalink / raw)
To: pgsql-performance@lists.postgresql.org <pgsql-performance@lists.postgresql.org>; masheed ullah <masheedullah@gmail.com>
You may try barman with rsync option and adding parallelism i.e. parallel_jobs =6 (based on the CPU core available on the barman machine)
Thanks and Regards,
Manish Yadav
On Sunday 3 May, 2026 at 03:12:45 PM IST, masheed ullah <masheedullah@gmail.com> wrote:
Hi,
We have a database of 20TB, and it's taking almost 24 hours to complete. While full restore takes 28 hours. Although we have HA enabled with primary and multiple replica's.
So we need a solution/tool to reduce the RTO, incase of disk failure/ Human error for data/tables deletion.
We are using Barman for backup.
My question is "Are there any tools like ZDLRA appliances for Oracle" to reduce the RTO to less than 10 hours".
Best Regards,
Khattak
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Postgres DB backup is taking too much time
2026-05-03 09:42 Postgres DB backup is taking too much time masheed ullah <masheedullah@gmail.com>
@ 2026-05-04 00:20 ` Greg Sabino Mullane <htamfids@gmail.com>
2 siblings, 0 replies; 7+ messages in thread
From: Greg Sabino Mullane @ 2026-05-04 00:20 UTC (permalink / raw)
To: masheed ullah <masheedullah@gmail.com>; +Cc: pgsql-performance@lists.postgresql.org
(This is more suitable for the pgsql-general list, you may get more/better
replies there)
There is nothing equivalent to ZDLRA in the Postgres world, but there are
certainly ways to reduce RTO. You are using replicas, and a tool that saves
WALs, so that really gets you 99% of the way there. Tools like pgBackrest
can do incremental backups, and parallel, delta-only restores, which can
wildly reduce both backup and restore times. RTO for Postgres can be
measured in seconds (e.g. Patroni handling a disk failure) to minutes (e.g.
someone dropped a table). Also see the recovery_min_apply_delay parameter.
If you give us a more specific use case / disaster scenario, we can give
more options and advice.
Cheers,
Greg
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Postgres DB backup is taking too much time
2026-05-03 09:42 Postgres DB backup is taking too much time masheed ullah <masheedullah@gmail.com>
@ 2026-05-04 09:29 ` Ilya Anfimov <ilan@tzirechnoy.com>
2026-05-04 09:58 ` Re: Postgres DB backup is taking too much time masheed ullah <masheedullah@gmail.com>
2 siblings, 1 reply; 7+ messages in thread
From: Ilya Anfimov @ 2026-05-04 09:29 UTC (permalink / raw)
To: pgsql-performance@lists.postgresql.org
On Sun, May 03, 2026 at 12:42:21PM +0300, masheed ullah wrote:
> Hi,
> We have a database of 20TB, and it's taking almost 24 hours to complete.
> While full restore takes 28 hours. Although we have HA enabled with
> primary and multiple replica's.
> So we need a solution/tool to reduce the RTO, incase of disk failure/
> Human error for data/tables deletion.
>
> We are using Barman for backup.
First, you'd need engineers to measure -- what's going on.
Who is the slowest part? The network (most probable)? The post-
gres replication process? The disk that postgres reads? The bar-
man itself? Receiver disks/S3 daemon/etc? TLS of one of the
above?
Measures is the key. Then the solutions may be searched.
btw, a modern postgres can definitely supply 1GB/s to barman on a
modern system. This is not a much, and probably could be im-
proved by special tricks like rsync or manual prefetch or incre-
mental copies -- however, it's less than 6 hours estimate and
therefore is fine for you.
> My question is "Are there any tools like ZDLRA appliances for Oracle" to
> reduce the RTO to less than 10 hours".
> Best Regards,
> Khattak
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Postgres DB backup is taking too much time
2026-05-03 09:42 Postgres DB backup is taking too much time masheed ullah <masheedullah@gmail.com>
2026-05-04 09:29 ` Re: Postgres DB backup is taking too much time Ilya Anfimov <ilan@tzirechnoy.com>
@ 2026-05-04 09:58 ` masheed ullah <masheedullah@gmail.com>
2026-05-04 10:10 ` Re: Postgres DB backup is taking too much time Gerald Drouillard <gerald@drouillard.ca>
2026-05-04 11:11 ` Re: Postgres DB backup is taking too much time Ilya Anfimov <ilan@tzirechnoy.com>
0 siblings, 2 replies; 7+ messages in thread
From: masheed ullah @ 2026-05-04 09:58 UTC (permalink / raw)
To: Ilya Anfimov <ilan@tzirechnoy.com>; pgsql-performance@lists.postgresql.org
Thank you every one for your output.
Although pgBeckRest has no more support.
Should I suggest starting testing backup and restore with pgBectRest?
I did not find any 3rd Party tool.
On Mon, May 4, 2026 at 12:29 PM Ilya Anfimov <ilan@tzirechnoy.com> wrote:
> On Sun, May 03, 2026 at 12:42:21PM +0300, masheed ullah wrote:
> > Hi,
> > We have a database of 20TB, and it's taking almost 24 hours to
> complete.
> > While full restore takes 28 hours. Although we have HA enabled with
> > primary and multiple replica's.
> > So we need a solution/tool to reduce the RTO, incase of disk failure/
> > Human error for data/tables deletion.
> >
> > We are using Barman for backup.
>
> First, you'd need engineers to measure -- what's going on.
> Who is the slowest part? The network (most probable)? The post-
> gres replication process? The disk that postgres reads? The bar-
> man itself? Receiver disks/S3 daemon/etc? TLS of one of the
> above?
> Measures is the key. Then the solutions may be searched.
>
> btw, a modern postgres can definitely supply 1GB/s to barman on a
> modern system. This is not a much, and probably could be im-
> proved by special tricks like rsync or manual prefetch or incre-
> mental copies -- however, it's less than 6 hours estimate and
> therefore is fine for you.
>
> > My question is "Are there any tools like ZDLRA appliances for Oracle"
> to
> > reduce the RTO to less than 10 hours".
> > Best Regards,
> > Khattak
>
>
>
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Postgres DB backup is taking too much time
2026-05-03 09:42 Postgres DB backup is taking too much time masheed ullah <masheedullah@gmail.com>
2026-05-04 09:29 ` Re: Postgres DB backup is taking too much time Ilya Anfimov <ilan@tzirechnoy.com>
2026-05-04 09:58 ` Re: Postgres DB backup is taking too much time masheed ullah <masheedullah@gmail.com>
@ 2026-05-04 10:10 ` Gerald Drouillard <gerald@drouillard.ca>
1 sibling, 0 replies; 7+ messages in thread
From: Gerald Drouillard @ 2026-05-04 10:10 UTC (permalink / raw)
To: masheed ullah <masheedullah@gmail.com>; +Cc: Ilya Anfimov <ilan@tzirechnoy.com>; pgsql-performance@lists.postgresql.org
You could setup a replication to another PG server that is setup on a COW
file system like zfs.
Then you can take snapshots on the replica.
On Mon, May 4, 2026 at 5:59 AM masheed ullah <masheedullah@gmail.com> wrote:
> Thank you every one for your output.
>
> Although pgBeckRest has no more support.
> Should I suggest starting testing backup and restore with pgBectRest?
> I did not find any 3rd Party tool.
>
> On Mon, May 4, 2026 at 12:29 PM Ilya Anfimov <ilan@tzirechnoy.com> wrote:
>
>> On Sun, May 03, 2026 at 12:42:21PM +0300, masheed ullah wrote:
>> > Hi,
>> > We have a database of 20TB, and it's taking almost 24 hours to
>> complete.
>> > While full restore takes 28 hours. Although we have HA enabled with
>> > primary and multiple replica's.
>> > So we need a solution/tool to reduce the RTO, incase of disk failure/
>> > Human error for data/tables deletion.
>> >
>> > We are using Barman for backup.
>>
>> First, you'd need engineers to measure -- what's going on.
>> Who is the slowest part? The network (most probable)? The post-
>> gres replication process? The disk that postgres reads? The bar-
>> man itself? Receiver disks/S3 daemon/etc? TLS of one of the
>> above?
>> Measures is the key. Then the solutions may be searched.
>>
>> btw, a modern postgres can definitely supply 1GB/s to barman on a
>> modern system. This is not a much, and probably could be im-
>> proved by special tricks like rsync or manual prefetch or incre-
>> mental copies -- however, it's less than 6 hours estimate and
>> therefore is fine for you.
>>
>> > My question is "Are there any tools like ZDLRA appliances for
>> Oracle" to
>> > reduce the RTO to less than 10 hours".
>> > Best Regards,
>> > Khattak
>>
>>
>>
>
>
>
>
Attachments:
[application/x-shellscript] pg-zfs-snapshot.sh (14.8K, ../../CAC_Y+MA5Lxk=GpHf14ZQoJmOUjBj5YREYUvhNdDni9=4-y=+ig@mail.gmail.com/3-pg-zfs-snapshot.sh)
download
^ permalink raw reply [nested|flat] 7+ messages in thread
* Re: Postgres DB backup is taking too much time
2026-05-03 09:42 Postgres DB backup is taking too much time masheed ullah <masheedullah@gmail.com>
2026-05-04 09:29 ` Re: Postgres DB backup is taking too much time Ilya Anfimov <ilan@tzirechnoy.com>
2026-05-04 09:58 ` Re: Postgres DB backup is taking too much time masheed ullah <masheedullah@gmail.com>
@ 2026-05-04 11:11 ` Ilya Anfimov <ilan@tzirechnoy.com>
1 sibling, 0 replies; 7+ messages in thread
From: Ilya Anfimov @ 2026-05-04 11:11 UTC (permalink / raw)
To: pgsql-performance@lists.postgresql.org
On Mon, May 04, 2026 at 12:58:54PM +0300, masheed ullah wrote:
> Thank you every one for your output.
>
> Although pgBeckRest has no more support.
> Should I suggest starting testing backup and restore with pgBectRest?
1) Yes, it may be a good training to set up pgBackRest.
And compare it to barman.
I don't think it would magically solve any of your problems,
however gaining experience seems beneficial overall.
2) pgBackRest is in a kind of uncertain position right now.
A week ago pgBackRest author and the main maintainer said he
stops maintaining pgBackRest.
Probably, some time later the situation would stabilize somehow
-- and probably, some fork would gain acceptance (or the author
would get back to maintaining) -- but now the situation is a bit
clumsy.
> I did not find any 3rd Party tool.
> On Mon, May 4, 2026 at 12:29 PM Ilya Anfimov <ilan@tzirechnoy.com> wrote:
>
> On Sun, May 03, 2026 at 12:42:21PM +0300, masheed ullah wrote:
> > Hi,
> > We have a database of 20TB, and it's taking almost 24 hours to
> complete.
> > While full restore takes 28 hours. Although we have HA enabled with
> > primary and multiple replica's.
> > So we need a solution/tool to reduce the RTO, incase of disk
> failure/
> > Human error for data/tables deletion.
> >
> > We are using Barman for backup.
>
> First, you'd need engineers to measure -- what's going on.
> Who is the slowest part? The network (most probable)? The post-
> gres replication process? The disk that postgres reads? The bar-
> man itself? Receiver disks/S3 daemon/etc? TLS of one of the
> above?
> Measures is the key. Then the solutions may be searched.
>
> btw, a modern postgres can definitely supply 1GB/s to barman on a
> modern system. This is not a much, and probably could be im-
> proved by special tricks like rsync or manual prefetch or incre-
> mental copies -- however, it's less than 6 hours estimate and
> therefore is fine for you.
>
> > My question is "Are there any tools like ZDLRA appliances for
> Oracle" to
> > reduce the RTO to less than 10 hours".
> > Best Regards,
> > Khattak
^ permalink raw reply [nested|flat] 7+ messages in thread
end of thread, other threads:[~2026-05-04 11:11 UTC | newest]
Thread overview: 7+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2026-05-03 09:42 Postgres DB backup is taking too much time masheed ullah <masheedullah@gmail.com>
2026-05-03 17:16 ` manish yadav <manishy174@yahoo.co.in>
2026-05-04 00:20 ` Greg Sabino Mullane <htamfids@gmail.com>
2026-05-04 09:29 ` Ilya Anfimov <ilan@tzirechnoy.com>
2026-05-04 09:58 ` masheed ullah <masheedullah@gmail.com>
2026-05-04 10:10 ` Gerald Drouillard <gerald@drouillard.ca>
2026-05-04 11:11 ` Ilya Anfimov <ilan@tzirechnoy.com>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox