public inbox for [email protected]
help / color / mirror / Atom feedPGbackrest very slow!
10+ messages / 7 participants
[nested] [flat]
* PGbackrest very slow!
@ 2025-10-23 08:59 Pär Mattsson <[email protected]>
2025-10-23 09:54 ` Re: PGbackrest very slow! Ilya Kosmodemiansky <[email protected]>
2025-10-23 13:14 ` Re: PGbackrest very slow! Scott Ribe <[email protected]>
0 siblings, 2 replies; 10+ messages in thread
From: Pär Mattsson @ 2025-10-23 08:59 UTC (permalink / raw)
To: pgsql-admin
Hi!
I have 3 node postgres cluster with repmgr and pgpool.
Pgbackrest is configed like this
We use pgbackrest with this config
=======================================
[global]
backup-standby=y
repo1-path=/postgresql/backup
repo1-retention-full=2
start-fast=y
[PG]
pg1-path=/postgresql/data
pg2-host=node2
pg2-host-user=postgres
pg2-path=/postgresql/data
pg3-host=node3
pg3-host-user=postgres
pg3-path=/postgresql/data
=================================================
I run the backuplike this
/usr/bin/pgbackrest --log-level-console=detail --stanza=PG --type=full
backup
The backup of 50GB took 40 min.
It is not a network issue. I tested to transfer 2GB between two host and it
took 4 sec.
I think backup-standby=y slows down the backup out of all minds.
Br Pär
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: PGbackrest very slow!
2025-10-23 08:59 PGbackrest very slow! Pär Mattsson <[email protected]>
@ 2025-10-23 09:54 ` Ilya Kosmodemiansky <[email protected]>
1 sibling, 0 replies; 10+ messages in thread
From: Ilya Kosmodemiansky @ 2025-10-23 09:54 UTC (permalink / raw)
To: Pär Mattsson <[email protected]>; +Cc: pgsql-admin
Hi Pär,
On Thu, Oct 23, 2025 at 10:59 AM Pär Mattsson <[email protected]> wrote:
>
> Hi!
> I have 3 node postgres cluster with repmgr and pgpool.
>
> Pgbackrest is configed like this
> We use pgbackrest with this config
> =======================================
It is a quite basic config,
> It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec.
>
>
> I think backup-standby=y slows down the backup out of all minds.
>
so if you are sure it is not a network issue, your guess could be
right. You can try to compare with backup from primary and if it is
faster, it could be archiving process on standby which slows you down.
You can try to improve it with more processes [1] and with better zst
compression [2]. Async archiving [3] could also help
>
> Br Pär
>
>
[1] https://pgbackrest.org/configuration.html#section-general/option-process-max
[2] https://pgbackrest.org/configuration.html#section-general/option-compress
[3] https://pgbackrest.org/configuration.html#section-archive/option-archive-async
best regards,
Ilya Kosmodemiansky
CEO, Founder
Data Egret GmbH
Your remote PostgreSQL DBA team
T.: +49 6821 919 3297
[email protected]
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: PGbackrest very slow!
2025-10-23 08:59 PGbackrest very slow! Pär Mattsson <[email protected]>
@ 2025-10-23 13:14 ` Scott Ribe <[email protected]>
2025-10-23 13:51 ` Re: PGbackrest very slow! Pär Mattsson <[email protected]>
2025-10-23 14:21 ` Re: PGbackrest very slow! Ron Johnson <[email protected]>
1 sibling, 2 replies; 10+ messages in thread
From: Scott Ribe @ 2025-10-23 13:14 UTC (permalink / raw)
To: Pär Mattsson <[email protected]>; +Cc: pgsql-admin
> On Oct 23, 2025, at 2:59 AM, Pär Mattsson <[email protected]> wrote:
>
> ...
> The backup of 50GB took 40 min.
> It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec.
>
> I think backup-standby=y slows down the backup out of all minds.
>
> Br Pär
pgbackrest defaults to gzip compression, which is slow--try lz4 and see what happens
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: PGbackrest very slow!
2025-10-23 08:59 PGbackrest very slow! Pär Mattsson <[email protected]>
2025-10-23 13:14 ` Re: PGbackrest very slow! Scott Ribe <[email protected]>
@ 2025-10-23 13:51 ` Pär Mattsson <[email protected]>
2025-10-24 02:15 ` Re: PGbackrest very slow! DINESH NAIR <[email protected]>
1 sibling, 1 reply; 10+ messages in thread
From: Pär Mattsson @ 2025-10-23 13:51 UTC (permalink / raw)
To: Scott Ribe <[email protected]>; +Cc: pgsql-admin
Thanks I will test that!
Br Pär
Mvh Pär
________________________________
Från: Scott Ribe <[email protected]>
Skickat: Thursday, October 23, 2025 3:14:20 PM
Till: Pär Mattsson <[email protected]>
Kopia: pgsql-admin <[email protected]>
Ämne: Re: PGbackrest very slow!
> On Oct 23, 2025, at 2:59 AM, Pär Mattsson <[email protected]> wrote:
>
> ...
> The backup of 50GB took 40 min.
> It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec.
>
> I think backup-standby=y slows down the backup out of all minds.
>
> Br Pär
pgbackrest defaults to gzip compression, which is slow--try lz4 and see what happens
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: PGbackrest very slow!
2025-10-23 08:59 PGbackrest very slow! Pär Mattsson <[email protected]>
2025-10-23 13:14 ` Re: PGbackrest very slow! Scott Ribe <[email protected]>
2025-10-23 13:51 ` Re: PGbackrest very slow! Pär Mattsson <[email protected]>
@ 2025-10-24 02:15 ` DINESH NAIR <[email protected]>
2025-10-24 12:36 ` Re: PGbackrest very slow! Pär Mattsson <[email protected]>
0 siblings, 1 reply; 10+ messages in thread
From: DINESH NAIR @ 2025-10-24 02:15 UTC (permalink / raw)
To: Pär Mattsson <[email protected]>; Scott Ribe <[email protected]>; +Cc: pgsql-admin
Hi,
Kindly try changing below mentioned parameters.
Parallel-process= -- based on the CPU CORE
compress-type=lz4
compress-level= -- based on the CPU CORE
Thanks & Regards
Dinesh Nair
________________________________
From: Pär Mattsson <[email protected]>
Sent: Thursday, October 23, 2025 7:21 PM
To: Scott Ribe <[email protected]>
Cc: pgsql-admin <[email protected]>
Subject: Re: PGbackrest very slow!
You don't often get email from [email protected]. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification;
Caution: This email was sent from an external source. Please verify the sender’s identity before clicking links or opening attachments.
Thanks I will test that!
Br Pär
Mvh Pär
________________________________
Från: Scott Ribe <[email protected]>
Skickat: Thursday, October 23, 2025 3:14:20 PM
Till: Pär Mattsson <[email protected]>
Kopia: pgsql-admin <[email protected]>
Ämne: Re: PGbackrest very slow!
> On Oct 23, 2025, at 2:59 AM, Pär Mattsson <[email protected]> wrote:
>
> ...
> The backup of 50GB took 40 min.
> It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec.
>
> I think backup-standby=y slows down the backup out of all minds.
>
> Br Pär
pgbackrest defaults to gzip compression, which is slow--try lz4 and see what happens
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: PGbackrest very slow!
2025-10-23 08:59 PGbackrest very slow! Pär Mattsson <[email protected]>
2025-10-23 13:14 ` Re: PGbackrest very slow! Scott Ribe <[email protected]>
2025-10-23 13:51 ` Re: PGbackrest very slow! Pär Mattsson <[email protected]>
2025-10-24 02:15 ` Re: PGbackrest very slow! DINESH NAIR <[email protected]>
@ 2025-10-24 12:36 ` Pär Mattsson <[email protected]>
0 siblings, 0 replies; 10+ messages in thread
From: Pär Mattsson @ 2025-10-24 12:36 UTC (permalink / raw)
To: DINESH NAIR <[email protected]>; +Cc: Scott Ribe <[email protected]>; pgsql-admin
Hi!
[global]
repo1-path=/postgresql/backup
repo1-retention-full=2
start-fast=y
process-max=4
When I deleted backup-standby=y and increased process-max=4 , then the
backup took 7 min of 50GB.
I will test all your tips! Thanks alot!
Br Pär
Den fre 24 okt. 2025 kl 04:15 skrev DINESH NAIR <
[email protected]>:
> Hi,
>
> Kindly try changing below mentioned parameters.
>
>
>
> Parallel-process= -- based on the CPU CORE
> compress-type=lz4
> compress-level= -- based on the CPU CORE
>
>
> Thanks & Regards
>
> Dinesh Nair
> ------------------------------
> *From:* Pär Mattsson <[email protected]>
> *Sent:* Thursday, October 23, 2025 7:21 PM
> *To:* Scott Ribe <[email protected]>
> *Cc:* pgsql-admin <[email protected]>
> *Subject:* Re: PGbackrest very slow!
>
> You don't often get email from [email protected]. Learn why this
> is important <https://aka.ms/LearnAboutSenderIdentification;
> Caution: This email was sent from an external source. Please verify the
> sender’s identity before clicking links or opening attachments.
> Thanks I will test that!
> Br Pär
>
> Mvh Pär
> ------------------------------
> *Från:* Scott Ribe <[email protected]>
> *Skickat:* Thursday, October 23, 2025 3:14:20 PM
> *Till:* Pär Mattsson <[email protected]>
> *Kopia:* pgsql-admin <[email protected]>
> *Ämne:* Re: PGbackrest very slow!
>
> > On Oct 23, 2025, at 2:59 AM, Pär Mattsson <[email protected]>
> wrote:
> >
> > ...
> > The backup of 50GB took 40 min.
> > It is not a network issue. I tested to transfer 2GB between two host and
> it took 4 sec.
> >
> > I think backup-standby=y slows down the backup out of all minds.
> >
> > Br Pär
>
> pgbackrest defaults to gzip compression, which is slow--try lz4 and see
> what happens
>
>
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: PGbackrest very slow!
2025-10-23 08:59 PGbackrest very slow! Pär Mattsson <[email protected]>
2025-10-23 13:14 ` Re: PGbackrest very slow! Scott Ribe <[email protected]>
@ 2025-10-23 14:21 ` Ron Johnson <[email protected]>
2025-10-23 14:25 ` Re: PGbackrest very slow! Ken Marshall <[email protected]>
2025-10-23 17:54 ` Re: PGbackrest very slow! adnan salam <[email protected]>
1 sibling, 2 replies; 10+ messages in thread
From: Ron Johnson @ 2025-10-23 14:21 UTC (permalink / raw)
To: Pgsql-admin <[email protected]>
On Thu, Oct 23, 2025 at 9:14 AM Scott Ribe <[email protected]>
wrote:
> > On Oct 23, 2025, at 2:59 AM, Pär Mattsson <[email protected]>
> wrote:
> >
> > ...
> > The backup of 50GB took 40 min.
> > It is not a network issue. I tested to transfer 2GB between two host and
> it took 4 sec.
> >
> > I think backup-standby=y slows down the backup out of all minds.
> >
> > Br Pär
>
> pgbackrest defaults to gzip compression, which is slow--try lz4 and see
> what happens
>
lz4 is faster, but not 1440x faster.
--
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: PGbackrest very slow!
2025-10-23 08:59 PGbackrest very slow! Pär Mattsson <[email protected]>
2025-10-23 13:14 ` Re: PGbackrest very slow! Scott Ribe <[email protected]>
2025-10-23 14:21 ` Re: PGbackrest very slow! Ron Johnson <[email protected]>
@ 2025-10-23 14:25 ` Ken Marshall <[email protected]>
1 sibling, 0 replies; 10+ messages in thread
From: Ken Marshall @ 2025-10-23 14:25 UTC (permalink / raw)
To: Ron Johnson <[email protected]>; +Cc: Pgsql-admin <[email protected]>
On Thu, Oct 23, 2025 at 10:21:37AM -0400, Ron Johnson wrote:
> On Thu, Oct 23, 2025 at 9:14 AM Scott Ribe <[email protected]>
> wrote:
>
> > > On Oct 23, 2025, at 2:59 AM, Pär Mattsson <[email protected]>
> > wrote:
> > >
> > > ...
> > > The backup of 50GB took 40 min.
> > > It is not a network issue. I tested to transfer 2GB between two host and
> > it took 4 sec.
> > >
> > > I think backup-standby=y slows down the backup out of all minds.
> > >
> > > Br Pär
> >
> > pgbackrest defaults to gzip compression, which is slow--try lz4 and see
> > what happens
> >
>
> lz4 is faster, but not 1440x faster.
>
But it might be 24x faster, using 2s/GB. :)
--Ken
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: PGbackrest very slow!
2025-10-23 08:59 PGbackrest very slow! Pär Mattsson <[email protected]>
2025-10-23 13:14 ` Re: PGbackrest very slow! Scott Ribe <[email protected]>
2025-10-23 14:21 ` Re: PGbackrest very slow! Ron Johnson <[email protected]>
@ 2025-10-23 17:54 ` adnan salam <[email protected]>
2025-10-23 18:07 ` Re: PGbackrest very slow! Pär Mattsson <[email protected]>
1 sibling, 1 reply; 10+ messages in thread
From: adnan salam @ 2025-10-23 17:54 UTC (permalink / raw)
To: [email protected]; Pgsql-admin <[email protected]>
Hi,
Increase the Parallel processes for pg back rest backup.
Regards,Adnan
Yahoo Mail: Search, organise, conquer
On Thu, 23 Oct 2025 at 19:22, Ron Johnson<[email protected]> wrote: On Thu, Oct 23, 2025 at 9:14 AM Scott Ribe <[email protected]> wrote:
> On Oct 23, 2025, at 2:59 AM, Pär Mattsson <[email protected]> wrote:
>
> ...
> The backup of 50GB took 40 min.
> It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec.
>
> I think backup-standby=y slows down the backup out of all minds.
>
> Br Pär
pgbackrest defaults to gzip compression, which is slow--try lz4 and see what happens
lz4 is faster, but not 1440x faster.
--
Death to <Redacted>, and butter sauce.Don't boil me, I'm still alive.
<Redacted> lobster!
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: PGbackrest very slow!
2025-10-23 08:59 PGbackrest very slow! Pär Mattsson <[email protected]>
2025-10-23 13:14 ` Re: PGbackrest very slow! Scott Ribe <[email protected]>
2025-10-23 14:21 ` Re: PGbackrest very slow! Ron Johnson <[email protected]>
2025-10-23 17:54 ` Re: PGbackrest very slow! adnan salam <[email protected]>
@ 2025-10-23 18:07 ` Pär Mattsson <[email protected]>
0 siblings, 0 replies; 10+ messages in thread
From: Pär Mattsson @ 2025-10-23 18:07 UTC (permalink / raw)
To: adnan salam <[email protected]>; [email protected] <[email protected]>; Pgsql-admin <[email protected]>
Thanks
Mvh Pär
________________________________
Från: adnan salam <[email protected]>
Skickat: Thursday, October 23, 2025 7:54:40 PM
Till: [email protected] <[email protected]>; Pgsql-admin <[email protected]>
Ämne: Re: PGbackrest very slow!
Hi,
Increase the Parallel processes for pg back rest backup.
Regards,
Adnan
Yahoo Mail: Search, organise, conquer<https://mail.onelink.me/107872968?pid=nativeplacement&c=US_Acquisition_YMktg_315_SearchOrgConque...;
On Thu, 23 Oct 2025 at 19:22, Ron Johnson
<[email protected]> wrote:
On Thu, Oct 23, 2025 at 9:14 AM Scott Ribe <[email protected]<mailto:[email protected]>> wrote:
> On Oct 23, 2025, at 2:59 AM, Pär Mattsson <[email protected]<mailto:[email protected]>> wrote:
>
> ...
> The backup of 50GB took 40 min.
> It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec.
>
> I think backup-standby=y slows down the backup out of all minds.
>
> Br Pär
pgbackrest defaults to gzip compression, which is slow--try lz4 and see what happens
lz4 is faster, but not 1440x faster.
--
Death to <Redacted>, and butter sauce.
Don't boil me, I'm still alive.
<Redacted> lobster!
^ permalink raw reply [nested|flat] 10+ messages in thread
end of thread, other threads:[~2025-10-24 12:36 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2025-10-23 08:59 PGbackrest very slow! Pär Mattsson <[email protected]>
2025-10-23 09:54 ` Ilya Kosmodemiansky <[email protected]>
2025-10-23 13:14 ` Scott Ribe <[email protected]>
2025-10-23 13:51 ` Pär Mattsson <[email protected]>
2025-10-24 02:15 ` DINESH NAIR <[email protected]>
2025-10-24 12:36 ` Pär Mattsson <[email protected]>
2025-10-23 14:21 ` Ron Johnson <[email protected]>
2025-10-23 14:25 ` Ken Marshall <[email protected]>
2025-10-23 17:54 ` adnan salam <[email protected]>
2025-10-23 18:07 ` Pär Mattsson <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox