public inbox for [email protected]help / color / mirror / Atom feed
PGbackrest very slow! 10+ messages / 7 participants [nested] [flat]
* PGbackrest very slow! @ 2025-10-23 08:59 Pär Mattsson <[email protected]> 0 siblings, 2 replies; 10+ messages in thread From: Pär Mattsson @ 2025-10-23 08:59 UTC (permalink / raw) To: pgsql-admin Hi! I have 3 node postgres cluster with repmgr and pgpool. Pgbackrest is configed like this We use pgbackrest with this config ======================================= [global] backup-standby=y repo1-path=/postgresql/backup repo1-retention-full=2 start-fast=y [PG] pg1-path=/postgresql/data pg2-host=node2 pg2-host-user=postgres pg2-path=/postgresql/data pg3-host=node3 pg3-host-user=postgres pg3-path=/postgresql/data ================================================= I run the backuplike this /usr/bin/pgbackrest --log-level-console=detail --stanza=PG --type=full backup The backup of 50GB took 40 min. It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec. I think backup-standby=y slows down the backup out of all minds. Br Pär ^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: PGbackrest very slow! @ 2025-10-23 09:54 Ilya Kosmodemiansky <[email protected]> parent: Pär Mattsson <[email protected]> 1 sibling, 0 replies; 10+ messages in thread From: Ilya Kosmodemiansky @ 2025-10-23 09:54 UTC (permalink / raw) To: Pär Mattsson <[email protected]>; +Cc: pgsql-admin Hi Pär, On Thu, Oct 23, 2025 at 10:59 AM Pär Mattsson <[email protected]> wrote: > > Hi! > I have 3 node postgres cluster with repmgr and pgpool. > > Pgbackrest is configed like this > We use pgbackrest with this config > ======================================= It is a quite basic config, > It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec. > > > I think backup-standby=y slows down the backup out of all minds. > so if you are sure it is not a network issue, your guess could be right. You can try to compare with backup from primary and if it is faster, it could be archiving process on standby which slows you down. You can try to improve it with more processes [1] and with better zst compression [2]. Async archiving [3] could also help > > Br Pär > > [1] https://pgbackrest.org/configuration.html#section-general/option-process-max [2] https://pgbackrest.org/configuration.html#section-general/option-compress [3] https://pgbackrest.org/configuration.html#section-archive/option-archive-async best regards, Ilya Kosmodemiansky CEO, Founder Data Egret GmbH Your remote PostgreSQL DBA team T.: +49 6821 919 3297 [email protected] ^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: PGbackrest very slow! @ 2025-10-23 13:14 Scott Ribe <[email protected]> parent: Pär Mattsson <[email protected]> 1 sibling, 2 replies; 10+ messages in thread From: Scott Ribe @ 2025-10-23 13:14 UTC (permalink / raw) To: Pär Mattsson <[email protected]>; +Cc: pgsql-admin > On Oct 23, 2025, at 2:59 AM, Pär Mattsson <[email protected]> wrote: > > ... > The backup of 50GB took 40 min. > It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec. > > I think backup-standby=y slows down the backup out of all minds. > > Br Pär pgbackrest defaults to gzip compression, which is slow--try lz4 and see what happens ^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: PGbackrest very slow! @ 2025-10-23 13:51 Pär Mattsson <[email protected]> parent: Scott Ribe <[email protected]> 1 sibling, 1 reply; 10+ messages in thread From: Pär Mattsson @ 2025-10-23 13:51 UTC (permalink / raw) To: Scott Ribe <[email protected]>; +Cc: pgsql-admin Thanks I will test that! Br Pär Mvh Pär ________________________________ Från: Scott Ribe <[email protected]> Skickat: Thursday, October 23, 2025 3:14:20 PM Till: Pär Mattsson <[email protected]> Kopia: pgsql-admin <[email protected]> Ämne: Re: PGbackrest very slow! > On Oct 23, 2025, at 2:59 AM, Pär Mattsson <[email protected]> wrote: > > ... > The backup of 50GB took 40 min. > It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec. > > I think backup-standby=y slows down the backup out of all minds. > > Br Pär pgbackrest defaults to gzip compression, which is slow--try lz4 and see what happens ^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: PGbackrest very slow! @ 2025-10-23 14:21 Ron Johnson <[email protected]> parent: Scott Ribe <[email protected]> 1 sibling, 2 replies; 10+ messages in thread From: Ron Johnson @ 2025-10-23 14:21 UTC (permalink / raw) To: Pgsql-admin <[email protected]> On Thu, Oct 23, 2025 at 9:14 AM Scott Ribe <[email protected]> wrote: > > On Oct 23, 2025, at 2:59 AM, Pär Mattsson <[email protected]> > wrote: > > > > ... > > The backup of 50GB took 40 min. > > It is not a network issue. I tested to transfer 2GB between two host and > it took 4 sec. > > > > I think backup-standby=y slows down the backup out of all minds. > > > > Br Pär > > pgbackrest defaults to gzip compression, which is slow--try lz4 and see > what happens > lz4 is faster, but not 1440x faster. -- Death to <Redacted>, and butter sauce. Don't boil me, I'm still alive. <Redacted> lobster! ^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: PGbackrest very slow! @ 2025-10-23 14:25 Ken Marshall <[email protected]> parent: Ron Johnson <[email protected]> 1 sibling, 0 replies; 10+ messages in thread From: Ken Marshall @ 2025-10-23 14:25 UTC (permalink / raw) To: Ron Johnson <[email protected]>; +Cc: Pgsql-admin <[email protected]> On Thu, Oct 23, 2025 at 10:21:37AM -0400, Ron Johnson wrote: > On Thu, Oct 23, 2025 at 9:14 AM Scott Ribe <[email protected]> > wrote: > > > > On Oct 23, 2025, at 2:59 AM, Pär Mattsson <[email protected]> > > wrote: > > > > > > ... > > > The backup of 50GB took 40 min. > > > It is not a network issue. I tested to transfer 2GB between two host and > > it took 4 sec. > > > > > > I think backup-standby=y slows down the backup out of all minds. > > > > > > Br Pär > > > > pgbackrest defaults to gzip compression, which is slow--try lz4 and see > > what happens > > > > lz4 is faster, but not 1440x faster. > But it might be 24x faster, using 2s/GB. :) --Ken ^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: PGbackrest very slow! @ 2025-10-23 17:54 adnan salam <[email protected]> parent: Ron Johnson <[email protected]> 1 sibling, 1 reply; 10+ messages in thread From: adnan salam @ 2025-10-23 17:54 UTC (permalink / raw) To: [email protected]; Pgsql-admin <[email protected]> Hi, Increase the Parallel processes for pg back rest backup. Regards,Adnan Yahoo Mail: Search, organise, conquer On Thu, 23 Oct 2025 at 19:22, Ron Johnson<[email protected]> wrote: On Thu, Oct 23, 2025 at 9:14 AM Scott Ribe <[email protected]> wrote: > On Oct 23, 2025, at 2:59 AM, Pär Mattsson <[email protected]> wrote: > > ... > The backup of 50GB took 40 min. > It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec. > > I think backup-standby=y slows down the backup out of all minds. > > Br Pär pgbackrest defaults to gzip compression, which is slow--try lz4 and see what happens lz4 is faster, but not 1440x faster. -- Death to <Redacted>, and butter sauce.Don't boil me, I'm still alive. <Redacted> lobster! ^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: PGbackrest very slow! @ 2025-10-23 18:07 Pär Mattsson <[email protected]> parent: adnan salam <[email protected]> 0 siblings, 0 replies; 10+ messages in thread From: Pär Mattsson @ 2025-10-23 18:07 UTC (permalink / raw) To: adnan salam <[email protected]>; [email protected] <[email protected]>; Pgsql-admin <[email protected]> Thanks Mvh Pär ________________________________ Från: adnan salam <[email protected]> Skickat: Thursday, October 23, 2025 7:54:40 PM Till: [email protected] <[email protected]>; Pgsql-admin <[email protected]> Ämne: Re: PGbackrest very slow! Hi, Increase the Parallel processes for pg back rest backup. Regards, Adnan Yahoo Mail: Search, organise, conquer<https://mail.onelink.me/107872968?pid=nativeplacement&c=US_Acquisition_YMktg_315_SearchOrgConque...; On Thu, 23 Oct 2025 at 19:22, Ron Johnson <[email protected]> wrote: On Thu, Oct 23, 2025 at 9:14 AM Scott Ribe <[email protected]<mailto:[email protected]>> wrote: > On Oct 23, 2025, at 2:59 AM, Pär Mattsson <[email protected]<mailto:[email protected]>> wrote: > > ... > The backup of 50GB took 40 min. > It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec. > > I think backup-standby=y slows down the backup out of all minds. > > Br Pär pgbackrest defaults to gzip compression, which is slow--try lz4 and see what happens lz4 is faster, but not 1440x faster. -- Death to <Redacted>, and butter sauce. Don't boil me, I'm still alive. <Redacted> lobster! ^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: PGbackrest very slow! @ 2025-10-24 02:15 DINESH NAIR <[email protected]> parent: Pär Mattsson <[email protected]> 0 siblings, 1 reply; 10+ messages in thread From: DINESH NAIR @ 2025-10-24 02:15 UTC (permalink / raw) To: Pär Mattsson <[email protected]>; Scott Ribe <[email protected]>; +Cc: pgsql-admin Hi, Kindly try changing below mentioned parameters. Parallel-process= -- based on the CPU CORE compress-type=lz4 compress-level= -- based on the CPU CORE Thanks & Regards Dinesh Nair ________________________________ From: Pär Mattsson <[email protected]> Sent: Thursday, October 23, 2025 7:21 PM To: Scott Ribe <[email protected]> Cc: pgsql-admin <[email protected]> Subject: Re: PGbackrest very slow! You don't often get email from [email protected]. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification; Caution: This email was sent from an external source. Please verify the sender’s identity before clicking links or opening attachments. Thanks I will test that! Br Pär Mvh Pär ________________________________ Från: Scott Ribe <[email protected]> Skickat: Thursday, October 23, 2025 3:14:20 PM Till: Pär Mattsson <[email protected]> Kopia: pgsql-admin <[email protected]> Ämne: Re: PGbackrest very slow! > On Oct 23, 2025, at 2:59 AM, Pär Mattsson <[email protected]> wrote: > > ... > The backup of 50GB took 40 min. > It is not a network issue. I tested to transfer 2GB between two host and it took 4 sec. > > I think backup-standby=y slows down the backup out of all minds. > > Br Pär pgbackrest defaults to gzip compression, which is slow--try lz4 and see what happens ^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: PGbackrest very slow! @ 2025-10-24 12:36 Pär Mattsson <[email protected]> parent: DINESH NAIR <[email protected]> 0 siblings, 0 replies; 10+ messages in thread From: Pär Mattsson @ 2025-10-24 12:36 UTC (permalink / raw) To: DINESH NAIR <[email protected]>; +Cc: Scott Ribe <[email protected]>; pgsql-admin Hi! [global] repo1-path=/postgresql/backup repo1-retention-full=2 start-fast=y process-max=4 When I deleted backup-standby=y and increased process-max=4 , then the backup took 7 min of 50GB. I will test all your tips! Thanks alot! Br Pär Den fre 24 okt. 2025 kl 04:15 skrev DINESH NAIR < [email protected]>: > Hi, > > Kindly try changing below mentioned parameters. > > > > Parallel-process= -- based on the CPU CORE > compress-type=lz4 > compress-level= -- based on the CPU CORE > > > Thanks & Regards > > Dinesh Nair > ------------------------------ > *From:* Pär Mattsson <[email protected]> > *Sent:* Thursday, October 23, 2025 7:21 PM > *To:* Scott Ribe <[email protected]> > *Cc:* pgsql-admin <[email protected]> > *Subject:* Re: PGbackrest very slow! > > You don't often get email from [email protected]. Learn why this > is important <https://aka.ms/LearnAboutSenderIdentification; > Caution: This email was sent from an external source. Please verify the > sender’s identity before clicking links or opening attachments. > Thanks I will test that! > Br Pär > > Mvh Pär > ------------------------------ > *Från:* Scott Ribe <[email protected]> > *Skickat:* Thursday, October 23, 2025 3:14:20 PM > *Till:* Pär Mattsson <[email protected]> > *Kopia:* pgsql-admin <[email protected]> > *Ämne:* Re: PGbackrest very slow! > > > On Oct 23, 2025, at 2:59 AM, Pär Mattsson <[email protected]> > wrote: > > > > ... > > The backup of 50GB took 40 min. > > It is not a network issue. I tested to transfer 2GB between two host and > it took 4 sec. > > > > I think backup-standby=y slows down the backup out of all minds. > > > > Br Pär > > pgbackrest defaults to gzip compression, which is slow--try lz4 and see > what happens > > ^ permalink raw reply [nested|flat] 10+ messages in thread
end of thread, other threads:[~2025-10-24 12:36 UTC | newest] Thread overview: 10+ messages (download: mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2025-10-23 08:59 PGbackrest very slow! Pär Mattsson <[email protected]> 2025-10-23 09:54 ` Ilya Kosmodemiansky <[email protected]> 2025-10-23 13:14 ` Scott Ribe <[email protected]> 2025-10-23 13:51 ` Pär Mattsson <[email protected]> 2025-10-24 02:15 ` DINESH NAIR <[email protected]> 2025-10-24 12:36 ` Pär Mattsson <[email protected]> 2025-10-23 14:21 ` Ron Johnson <[email protected]> 2025-10-23 14:25 ` Ken Marshall <[email protected]> 2025-10-23 17:54 ` adnan salam <[email protected]> 2025-10-23 18:07 ` Pär Mattsson <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox