Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1sjYXW-001mKy-Hg for pgsql-general@arkaria.postgresql.org; Thu, 29 Aug 2024 06:21:50 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1sjYXU-00FqFL-MD for pgsql-general@arkaria.postgresql.org; Thu, 29 Aug 2024 06:21:49 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1sjYXU-00FqFD-6t for pgsql-general@lists.postgresql.org; Thu, 29 Aug 2024 06:21:48 +0000 Received: from mail-yw1-x1132.google.com ([2607:f8b0:4864:20::1132]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1sjYXS-001y5t-7b for pgsql-general@postgresql.org; Thu, 29 Aug 2024 06:21:47 +0000 Received: by mail-yw1-x1132.google.com with SMTP id 00721157ae682-6d0e7dfab60so3836547b3.3 for ; Wed, 28 Aug 2024 23:21:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1724912505; x=1725517305; darn=postgresql.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=ZoRmoPQtOcMRooXdc98fiB/0HyMZt/dSdDBNi5z+lYU=; b=l4dsdvle32pedT9KfcBeWqHjk1guTIDDflWKb2DXSbcNoSf+2fxW8No3n6Sf5KxOJ+ b4LMAmvWSNUrFb+95yw+uDqJC7LLShBwc9HgjenTMaG0mTejrg4z358qCjHb/4Bc3/Mx 8cYPkyIUBSF5dn3xtOfeF/vCfYPuu8V9PZ0AlyLieRSN8hpvdkLdTmuR8G13bqJd7Xrm +wBPaQTgsh3XsILmxK1WtKVKi7uhnd9r/Sw+0cATN0O9KD8Ujxa+9DBITiLnoAaSRMU1 Y/1L9/iWfSMipeUxrKP5jAKqR8KzKPcJg5k2WCYT/fWfXgvzpbdEccOlwyqEk6gVWMA5 /2+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1724912505; x=1725517305; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ZoRmoPQtOcMRooXdc98fiB/0HyMZt/dSdDBNi5z+lYU=; b=XSAscwHynT3gNeWA8UDWlV6EGYB6E8UKhfJwYkTbh6wywHyBUpN/tPhfeUF8rjZvNl wYC2ALhxCJeMUlzvd+1rVMKotm609t8cJXBWJNXFWwvv6gy3xm1SazTcP4NP5jRSYuFL 7oQjIil7998EwhP4y3wehn1H/fuGOGhH5ElCqk+vM8tLM+cwsOXiffJg6Ym9Q6ksjqmb 8Q+Sri+110jPx5y8Vbl0qGzXpHZURw8bBrwLVSOo79p4rNC2yFqNZqZa9J6zGlSH7R95 06/LYFaGy+asbL2hWXFk+MsC8+BwGoKjROFF1JxLbr0RH0nERRRBzHzodDarOH1iZZJW gTxA== X-Gm-Message-State: AOJu0YzC7BR6i8T1b/z9QL4exoGAi+C046pX2LMe49xMFx/Pi8y1KJy3 3AbMotmLNlbaHErTNZD22G6dR1AqV4YqxM4yBTh/HhMW4+l48fnWZizQ2AxDpU/Vd2N3+mBc55i u03EGA8enkQmaWK1qWrQNSh2FryufiF9Z X-Google-Smtp-Source: AGHT+IFgUhn7uvb5AvKE2f66Pnf/dih1gxF5PqG4vXT9d8wxirr+Y9wSdsOzGOGvI80Q0X+dgOr8LNV090TNdk9d54U= X-Received: by 2002:a05:690c:397:b0:699:7d04:c7b4 with SMTP id 00721157ae682-6d2778788c1mr18429757b3.31.1724912505160; Wed, 28 Aug 2024 23:21:45 -0700 (PDT) MIME-Version: 1.0 From: KK CHN Date: Thu, 29 Aug 2024 11:51:54 +0530 Message-ID: Subject: PgBackRest Full backup and N/W reliability To: pgsql-general Content-Type: multipart/alternative; boundary="000000000000238fa00620cc7de4" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --000000000000238fa00620cc7de4 Content-Type: text/plain; charset="UTF-8" List, I am doing a full backup using PgBackRest from a production server to Reposerver. My connection is IPSec VPN over ILL ( 8 Mbps link) between the Production DB Server and the remote RepoServer. I understood the bottleneck of 8 Mbps between servers. (Server NICs 10Gbps and switch) Query : I have started the backup command and it is running (may go for hours and days as link speed is minimal) . If the link disconnected or Network error happens before completion of the backup command Definitely the option is to reissue the backup command again. If so, does the backup process start again from scratch ? or it resumes from where the backup process is stopped ? If it starts from scratch I am afraid that I can''t complete the initial full backup never :( Or is there a work around if the network connectivity is lost in between ? Any suggestions much appreciated Thank you , Krishane [root@dbtest pgbackrest]# sudo -u postgres pgbackrest --stanza=Repo --log-level-console=info backup 2024-08-29 10:55:27.729 P00 INFO: backup command begin 2.52.1: --delta --exec-id=523103-56943986 --log-level-console=info --log-level-file=debug --pg1-host=10.15.0.202 --pg1-host-user=enterprisedb --pg1-path=/data/edb/as16/data --pg-version-force=16 --process-max=5 --repo1-block --repo1-bundle --repo1-cipher-pass= --repo1-cipher-type=aes-256-cbc --repo1-path=/data/DB_BKUPS --repo1-retention-diff=2 --repo1-retention-full=2 --stanza=Repo --start-fast WARN: no prior backup exists, incr backup has been changed to full 2024-08-29 10:55:30.589 P00 INFO: execute non-exclusive backup start: backup begins after the requested immediate checkpoint completes 2024-08-29 10:55:31.543 P00 INFO: backup start archive = 00000001000000850000004C, lsn = 85/4C0007F8 2024-08-29 10:55:31.543 P00 INFO: check archive for prior segment 00000001000000850000004B ON Repo Server: [root@dbtest backup]# date Thursday 29 August 2024 10:58:08 AM IST [root@dbtest backup]# du -h 165M ./Repo 165M [root@dbtest backup]# date Thursday 29 August 2024 11:37:03 AM IST [root@dbtest backup]# du -h 1.9G ./Repo 1.9G ON Production Server /data/edb/as16/data directory size is 500 GB --000000000000238fa00620cc7de4 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
List,=C2=A0

I am doing a full backup=C2= =A0 using PgBackRest from a production server=C2=A0to Reposerver.=C2=A0=C2= =A0

My connection is=C2=A0 IPSec VPN over ILL=C2= =A0 ( 8 Mbps link) between the=C2=A0 =C2=A0Production DB Server and=C2=A0 t= he remote RepoServer.

I understood the=C2=A0 bottl= eneck of 8 Mbps between servers. (Server NICs 10Gbps and=C2=A0switch)

Query : I have started the=C2=A0 backup command=C2= =A0 and it is running (may go for hours and days as link speed is minimal) = .
If the link disconnected or Network error=C2=A0 happens before = completion of the backup command=C2=A0 Definitely the option is to reissue = the=C2=A0 backup command=C2=A0 again.=C2=A0

If so,= does the backup process start=C2=A0 again from=C2=A0 scratch ?=C2=A0 =C2= =A0or it resumes from=C2=A0 where the backup process is stopped=C2=A0 =C2= =A0?=C2=A0

If it starts from scratch I am afraid t= hat=C2=A0 I can''t=C2=A0complete the initial full backup never :(= =C2=A0

=C2=A0=C2=A0Or is there a work around if th= e network=C2=A0connectivity is lost in between=C2=A0 ?

=

Any suggestions much appreciated

Thank you ,
Krishane
=C2=A0 =C2=A0=C2=A0
[root@db= test pgbackrest]# sudo -u postgres pgbackrest --stanza=3DRepo --log-level-c= onsole=3Dinfo backup

2024-08-29 10:55:27.729 P00 =C2=A0 INFO:= backup command begin 2.52.1: --delta --exec-id=3D523103-56943986 --log-lev= el-console=3Dinfo --log-level-file=3Ddebug --pg1-host=3D10.15.0.202 --pg1-h= ost-user=3Denterprisedb --pg1-path=3D/data/edb/as16/data --pg-version-force= =3D16 --process-max=3D5 --repo1-block --repo1-bundle --repo1-cipher-pass=3D= <redacted> --repo1-cipher-type=3Daes-256-cbc --repo1-path=3D/data/DB_= BKUPS=C2=A0 --repo1-retention-diff=3D2 --repo1-retention-full=3D2 --stanza= =3DRepo --start-fast
WARN: no prior backup exists, incr backup has been = changed to full
2024-08-29 10:55:30.589 P00 =C2=A0 INFO: execute non-exc= lusive backup start: backup begins after the requested immediate checkpoint= completes
2024-08-29 10:55:31.543 P00 =C2=A0 INFO: backup start archive= =3D 00000001000000850000004C, lsn =3D 85/4C0007F8
2024-08-29 10:55:31.5= 43 P00 =C2=A0 INFO: check archive for prior segment 00000001000000850000004= B


ON Repo Server:=C2=A0
[root@dbtest backup]# date=C2=A0 =C2=A0
Thursday 29 August 2024 10:58= :08 AM IST
[root@dbtest backup]# du -h
165M =C2=A0 =C2=A0.= /Repo
165M=C2=A0=C2=A0

[root@dbtest backup]= # date
Thursday 29 August 2024 11:37:03 AM IST
[root@dbtest backup]# = du -h
1.9G =C2=A0 =C2=A0./Repo
1.9G=C2=A0

ON=C2=A0 Production Server=C2=A0 =C2=A0 /data/edb/a= s16/data=C2=A0 =C2=A0 directory size is=C2=A0 =C2=A0500 GB


--000000000000238fa00620cc7de4--