MIME-Version: 1.0
References: <SJ1PR19MB6162FA7FE28CBA7C952147FBBAF92@SJ1PR19MB6162.namprd19.prod.outlook.com>
 <705876869cf3692a7a89e097195b4e502e3348da.camel@cybertec.at>
In-Reply-To: <705876869cf3692a7a89e097195b4e502e3348da.camel@cybertec.at>
From: Ron Johnson <ronljohnsonjr@gmail.com>
Date: Thu, 6 Jun 2024 12:25:38 -0400
Message-ID: <CANzqJaBEGsHxMn4xE2n37pBDvRwMDRFVrvJSJaJmXCYDKW0FXw@mail.gmail.com>
Subject: Re: Logical replication type- WAL recovery fails and changes the size
 of wal segment in archivedir
To: Laurenz Albe <laurenz.albe@cybertec.at>
Cc: Meera Nair <mnair@commvault.com>, 
	"pgsql-general@lists.postgresql.org" <pgsql-general@lists.postgresql.org>, 
	Punit Pranesh Koujalgi <pkoujalgi@commvault.com>, B Ganesh Kishan <bkishan@commvault.com>
Content-Type: multipart/alternative; boundary="000000000000d82765061a3b2279"
Archived-At: <https://www.postgresql.org/message-id/CANzqJaBEGsHxMn4xE2n37pBDvRwMDRFVrvJSJaJmXCYDKW0FXw%40mail.gmail.com>
Precedence: bulk

--000000000000d82765061a3b2279
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, Jun 5, 2024 at 6:26=E2=80=AFAM Laurenz Albe <laurenz.albe@cybertec.=
at>
wrote:

> On Wed, 2024-06-05 at 06:36 +0000, Meera Nair wrote:
> > 2024-06-05 11:41:32.369 IST [54369] LOG:  restored log file
> "00000005000000010000006A" from archive
> > 2024-06-05 11:41:33.112 IST [54369] LOG:  restored log file
> "00000005000000010000006B" from archive
> > cp: cannot stat =E2=80=98/home/pgsql/wmaster/00000005000000010000006C=
=E2=80=99: No such
> file or directory
> > 2024-06-05 11:41:33.167 IST [54369] LOG:  redo done at 1/6B000100
> > 2024-06-05 11:41:33.172 IST [54369] FATAL:  archive file
> "00000005000000010000006B" has wrong size: 0 instead of 16777216
> > 2024-06-05 11:41:33.173 IST [54367] LOG:  startup process (PID 54369)
> exited with exit code 1
> > 2024-06-05 11:41:33.173 IST [54367] LOG:  terminating any other active
> server processes
> > 2024-06-05 11:41:33.174 IST [54375] FATAL:  archive command was
> terminated by signal 3: Quit
> > 2024-06-05 11:41:33.174 IST [54375] DETAIL:  The failed archive command
> was: cp pg_wal/00000005000000010000006B
> /home/pgsql/wmaster/00000005000000010000006B
> > 2024-06-05 11:41:33.175 IST [54367] LOG:  archiver process (PID 54375)
> exited with exit code 1
> > 2024-06-05 11:41:33.177 IST [54367] LOG:  database system is shut down
> >
> > Here =E2=80=98/home/pgsql/wmaster=E2=80=99 is my archivedir (the folder=
 where WAL
> segments are restored from)
> >
> > Before attempting start, size of
> > 00000005000000010000006B file was 16 MB.
> > After failing to detect 00000005000000010000006C, there is a FATAL erro=
r
> saying wrong size for 00000005000000010000006B
> > Now the size of 00000005000000010000006B is observed as 2 MB. Size of
> all other WAL segments remain 16 MB.
> >
> > -rw------- 1 postgres postgres  2359296 Jun  5 11:34
> 00000005000000010000006B
>
> That looks like you have "archive_mode =3D always", and "archive_command"
> writes
> back to the archive.  Don't do that.
>

In fact, don't write your own PITR backup process.  Use something like
PgBackRest or BarMan.

--000000000000d82765061a3b2279
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">On Wed, Jun 5, 2024 at 6:26=E2=80=AFAM La=
urenz Albe &lt;<a href=3D"mailto:laurenz.albe@cybertec.at">laurenz.albe@cyb=
ertec.at</a>&gt; wrote:<br></div><div class=3D"gmail_quote"><blockquote cla=
ss=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid =
rgb(204,204,204);padding-left:1ex">On Wed, 2024-06-05 at 06:36 +0000, Meera=
 Nair wrote:<br>
&gt; 2024-06-05 11:41:32.369 IST [54369] LOG:=C2=A0 restored log file &quot=
;00000005000000010000006A&quot; from archive<br>
&gt; 2024-06-05 11:41:33.112 IST [54369] LOG:=C2=A0 restored log file &quot=
;00000005000000010000006B&quot; from archive<br>
&gt; cp: cannot stat =E2=80=98/home/pgsql/wmaster/00000005000000010000006C=
=E2=80=99: No such file or directory<br>
&gt; 2024-06-05 11:41:33.167 IST [54369] LOG:=C2=A0 redo done at 1/6B000100=
=C2=A0=C2=A0=C2=A0<br>
&gt; 2024-06-05 11:41:33.172 IST [54369] FATAL:=C2=A0 archive file &quot;00=
000005000000010000006B&quot; has wrong size: 0 instead of 16777216<br>
&gt; 2024-06-05 11:41:33.173 IST [54367] LOG:=C2=A0 startup process (PID 54=
369) exited with exit code 1<br>
&gt; 2024-06-05 11:41:33.173 IST [54367] LOG:=C2=A0 terminating any other a=
ctive server processes<br>
&gt; 2024-06-05 11:41:33.174 IST [54375] FATAL:=C2=A0 archive command was t=
erminated by signal 3: Quit<br>
&gt; 2024-06-05 11:41:33.174 IST [54375] DETAIL:=C2=A0 The failed archive c=
ommand was: cp pg_wal/00000005000000010000006B /home/pgsql/wmaster/00000005=
000000010000006B<br>
&gt; 2024-06-05 11:41:33.175 IST [54367] LOG:=C2=A0 archiver process (PID 5=
4375) exited with exit code 1<br>
&gt; 2024-06-05 11:41:33.177 IST [54367] LOG:=C2=A0 database system is shut=
 down<br>
&gt; =C2=A0<br>
&gt; Here =E2=80=98/home/pgsql/wmaster=E2=80=99 is my archivedir (the folde=
r where WAL segments are restored from)<br>
&gt; =C2=A0<br>
&gt; Before attempting start, size of <br>
&gt; 00000005000000010000006B file was 16 MB.<br>
&gt; After failing to detect 00000005000000010000006C, there is a FATAL err=
or saying wrong size for 00000005000000010000006B<br>
&gt; Now the size of 00000005000000010000006B is observed as 2 MB. Size of =
all other WAL segments remain 16 MB.<br>
&gt; =C2=A0<br>
&gt; -rw------- 1 postgres postgres=C2=A0 2359296 Jun=C2=A0 5 11:34 0000000=
5000000010000006B<br>
<br>
That looks like you have &quot;archive_mode =3D always&quot;, and &quot;arc=
hive_command&quot; writes<br>
back to the archive.=C2=A0 Don&#39;t do that.<br></blockquote><div>=C2=A0</=
div><div>In fact, don&#39;t write your own PITR backup process.=C2=A0 Use s=
omething like PgBackRest or BarMan.</div><div><br></div></div></div>

--000000000000d82765061a3b2279--