MIME-Version: 1.0
References: <PUrgXj3toL_CVxytc7fzYQNlY5FiUllklbS9UL1wU5ptiJAC2ChrF64EwO8neCKU9QnET5Zjv1HpMLFAgQPJQxOzM9f49jjkV56gEpUOYw8=@protonmail.com>
In-Reply-To: <PUrgXj3toL_CVxytc7fzYQNlY5FiUllklbS9UL1wU5ptiJAC2ChrF64EwO8neCKU9QnET5Zjv1HpMLFAgQPJQxOzM9f49jjkV56gEpUOYw8=@protonmail.com>
From: Stefan Fercot <stefan.fercot@dataegret.com>
Date: Fri, 2 Aug 2024 08:47:02 +0200
Message-ID: <CAPzjO7U4oM8pkLOPrMq=ugWHqjfo-a1hdm6rd2Y+y+Xqt_Tkdw@mail.gmail.com>
Subject: Re: Recovery of .partial WAL segments
To: "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>
Content-Type: multipart/alternative; boundary="00000000000093a61c061eadb22b"
Archived-At: <https://www.postgresql.org/message-id/CAPzjO7U4oM8pkLOPrMq%3DugWHqjfo-a1hdm6rd2Y%2By%2BXqt_Tkdw%40mail.gmail.com>
Precedence: bulk

--00000000000093a61c061eadb22b
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi,

I've added a CF entry for this patch:
https://commitfest.postgresql.org/49/5148/

Not sure why CFbot CI fails on macOS/Windows while it works with the Github
CI on my fork (
https://cirrus-ci.com/github/pgstef/postgres/partial-walseg-recovery).

Many thanks in advance for your feedback and thoughts about this patch,
Kind Regards,
--
Stefan FERCOT
Data Egret (https://dataegret.com)

On Thu, Aug 1, 2024 at 10:23=E2=80=AFPM Stefan Fercot <stefan.fercot@proton=
mail.com>
wrote:

> Dear hackers,
>
> Generating a ".partial" WAL segment is pretty common nowadays (using
> pg_receivewal or during standby promotion).
> However, we currently don't do anything with it unless the user manually
> removes that ".partial" extension.
>
> The 028_pitr_timelines tests are highlighting that fact: with test data
> being being in 000000020000000000000003 and
> 000000010000000000000003.partial, a recovery following the latest timelin=
e
> (2) will succeed but fail if we follow the current timeline (1).
>
> By simply trying to fetch the ".partial" file in XLogFileRead, we can
> easily recover more data and also cover that (current timeline) recovery
> case.
>
> So, this proposed patch makes XLogFileRead try to restore ".partial" WAL
> archives and adds a test to 028_pitr_timelines using current
> recovery_target_timeline.
>
> As far as I've seen, the current pg_receivewal tests only seem to cover
> the archives generation but not actually trying to recover using it. I
> wasn't sure it was interesting to add such tests right now, so I didn't
> considered it for this patch.
>
> Many thanks in advance for your feedback and thoughts about this,
> Kind Regards,
> --
> Stefan FERCOT
> Data Egret (https://dataegret.com)

--00000000000093a61c061eadb22b
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><div>Hi,</div><div><br></div><div>I&#39;v=
e added a CF entry for this patch: <a href=3D"https://commitfest.postgresql=
.org/49/5148/">https://commitfest.postgresql.org/49/5148/</a></div><div><br=
></div><div>Not sure why CFbot CI fails on macOS/Windows while it works wit=
h the Github CI on my fork (<a href=3D"https://cirrus-ci.com/github/pgstef/=
postgres/partial-walseg-recovery">https://cirrus-ci.com/github/pgstef/postg=
res/partial-walseg-recovery</a>).</div><div><br></div><div>Many thanks in a=
dvance for your feedback and thoughts about this patch,<br>
Kind Regards,<br>
--<br>
Stefan FERCOT<br>
Data Egret (<a href=3D"https://dataegret.com" rel=3D"noreferrer" target=3D"=
_blank">https://dataegret.com</a>)</div></div><br><div class=3D"gmail_quote=
"><div dir=3D"ltr" class=3D"gmail_attr">On Thu, Aug 1, 2024 at 10:23=E2=80=
=AFPM Stefan Fercot &lt;<a href=3D"mailto:stefan.fercot@protonmail.com">ste=
fan.fercot@protonmail.com</a>&gt; wrote:<br></div><blockquote class=3D"gmai=
l_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,20=
4,204);padding-left:1ex">Dear hackers,<br>
<br>
Generating a &quot;.partial&quot; WAL segment is pretty common nowadays (us=
ing pg_receivewal or during standby promotion).<br>
However, we currently don&#39;t do anything with it unless the user manuall=
y removes that &quot;.partial&quot; extension.<br>
<br>
The 028_pitr_timelines tests are highlighting that fact: with test data bei=
ng being in 000000020000000000000003 and 000000010000000000000003.partial, =
a recovery following the latest timeline (2) will succeed but fail if we fo=
llow the current timeline (1).<br>
<br>
By simply trying to fetch the &quot;.partial&quot; file in XLogFileRead, we=
 can easily recover more data and also cover that (current timeline) recove=
ry case.<br>
<br>
So, this proposed patch makes XLogFileRead try to restore &quot;.partial&qu=
ot; WAL archives and adds a test to 028_pitr_timelines using current recove=
ry_target_timeline.<br>
<br>
As far as I&#39;ve seen, the current pg_receivewal tests only seem to cover=
 the archives generation but not actually trying to recover using it. I was=
n&#39;t sure it was interesting to add such tests right now, so I didn&#39;=
t considered it for this patch.<br>
<br>
Many thanks in advance for your feedback and thoughts about this,<br>
Kind Regards,<br>
--<br>
Stefan FERCOT<br>
Data Egret (<a href=3D"https://dataegret.com" rel=3D"noreferrer" target=3D"=
_blank">https://dataegret.com</a>)</blockquote></div></div>

--00000000000093a61c061eadb22b--