Feedback-ID: i0c21471d:Fastmail
User-Agent: Cyrus-JMAP/3.11.0-alpha0-386-g4cb8e397f9-fm-20240415.001-g4cb8e397
MIME-Version: 1.0
Message-Id: <2e86cf53-c806-49d2-9a10-1a1257776d1f@app.fastmail.com>
In-Reply-To: 
 <CAA4eK1KPoLUUGzjw9eEQd56LUAf2QQj8BqMcsoSe6vycB5CYUQ@mail.gmail.com>
References: <9a9c7e65-c47b-4400-a63e-ac6169c0860e@app.fastmail.com>
 <e8564096-3c4f-4990-ac7d-ba089739b878@app.fastmail.com>
 <CANhcyEW6-dH28gLbFc5XpDTJ6JPizU+t5g-aKUWJBf5W_Zriqw@mail.gmail.com>
 <2f0b73f3-e7c0-4e95-9712-1993442a6b64@app.fastmail.com>
 <CAA4eK1JtnzqYNQ_S+mj_XsPfgk7szxegKiVQq4-8fY1wUcUnpg@mail.gmail.com>
 <34637e7f-0330-420d-8f45-1d022962d2fe@app.fastmail.com>
 <7a970912-0b77-4942-84f7-2c9ca0bc05a5@eisentraut.org>
 <d546c4bb-92d1-4e2d-898f-48234b12ed25@app.fastmail.com>
 <CANhcyEWdTTO71v+GFYUTsOOUvHus1qccg88bRmsU9KykJqjSrQ@mail.gmail.com>
 <44de4e04-db22-40ed-8ba9-46387dc8bf83@app.fastmail.com>
 <3fa9ef0f-b277-4c13-850a-8ccc04de1406@eisentraut.org>
 <CALj2ACV85cn-cTNCTP=bPehYo9d2ddvZtfGhGq==0K+65SvufQ@mail.gmail.com>
 <65b841a8-0c2f-4953-b9cd-240fce84b90b@eisentraut.org>
 <TYCPR01MB12077907C052EC329C45AE198F5362@TYCPR01MB12077.jpnprd01.prod.outlook.com>
 <c8e92bcb-69c3-477e-93d4-6f39e030cab0@app.fastmail.com>
 <47524383-6fd1-4b13-8c4e-fb918b1d35ba@enterprisedb.com>
 <bbbeed26-41f9-468c-ac24-ad99f429ccd7@app.fastmail.com>
 <CAA4eK1KPoLUUGzjw9eEQd56LUAf2QQj8BqMcsoSe6vycB5CYUQ@mail.gmail.com>
Date: Mon, 29 Apr 2024 08:53:19 -0300
From: "Euler Taveira" <euler@eulerto.com>
To: "Amit Kapila" <amit.kapila16@gmail.com>
Cc: "Tomas Vondra" <tomas.vondra@enterprisedb.com>,
 "kuroda.hayato@fujitsu.com" <kuroda.hayato@fujitsu.com>,
 "Peter Eisentraut" <peter@eisentraut.org>,
 "Bharath Rupireddy" <bharath.rupireddyforpostgres@gmail.com>,
 "Shlok Kyal" <shlok.kyal.oss@gmail.com>,
 "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>,
 "Michael Paquier" <michael@paquier.xyz>,
 "Andres Freund" <andres@anarazel.de>,
 "Ashutosh Bapat" <ashutosh.bapat.oss@gmail.com>,
 =?UTF-8?Q?Fabr=C3=ADzio_de_Royes_Mello?= <fabriziomello@gmail.com>,
 "vignesh C" <vignesh21@gmail.com>
Subject: Re: speed up a logical replica setup
Content-Type: multipart/alternative;
 boundary=a5b8760bebfa46ef8dce28cc7a0f5d3f
Archived-At: <https://www.postgresql.org/message-id/2e86cf53-c806-49d2-9a10-1a1257776d1f%40app.fastmail.com>
Precedence: bulk

--a5b8760bebfa46ef8dce28cc7a0f5d3f
Content-Type: text/plain;charset=utf-8
Content-Transfer-Encoding: quoted-printable

On Mon, Apr 29, 2024, at 6:56 AM, Amit Kapila wrote:
> On Wed, Mar 27, 2024 at 1:47=E2=80=AFAM Euler Taveira <euler@eulerto.c=
om> wrote:
> >
> > On Tue, Mar 26, 2024, at 4:12 PM, Tomas Vondra wrote:
> >
> > Perhaps I'm missing something, but why is NUM_CONN_ATTEMPTS even nee=
ded?
> > Why isn't recovery_timeout enough to decide if wait_for_end_recovery=
()
> > waited long enough?
> >
> >
> > It was an attempt to decoupled a connection failure (that keeps stre=
aming the
> > WAL) from recovery timeout. The NUM_CONN_ATTEMPTS guarantees that if=
 the primary
> > is gone during the standby recovery process, there is a way to bail =
out.
> >
>=20
> I think we don't need to check primary if the WAL corresponding to
> consistent_lsn is already present on the standby. Shouldn't we first
> check that? Once we ensure that the required WAL is copied, just
> checking server_is_in_recovery() should be sufficient. I feel that
> will be a direct way of ensuring what is required rather than
> indirectly verifying the same (by checking pg_stat_wal_receiver) as we
> are doing currently.

How would you check it? WAL file? During recovery, you are not allowed t=
o use
pg_current_wal_lsn.

Tomas suggested to me off-list that we should adopt a simple solution in
wait_for_end_recovery: wait for recovery_timeout without additional chec=
ks
(which means remove the pg_stat_wal_receiver logic).  When we have addit=
ional
information that we can reliably use in this function, we can add it. He=
nce, it
is also easy to adjust the PG_TEST_TIMEOUT_DEFAULT to have stable tests.


--
Euler Taveira
EDB   https://www.enterprisedb.com/

--a5b8760bebfa46ef8dce28cc7a0f5d3f
Content-Type: text/html;charset=utf-8
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE html><html><head><title></title><style type=3D"text/css">p.Mso=
Normal,p.MsoNoSpacing{margin:0}</style></head><body><div>On Mon, Apr 29,=
 2024, at 6:56 AM, Amit Kapila wrote:<br></div><blockquote type=3D"cite"=
 id=3D"qt" style=3D""><div>On Wed, Mar 27, 2024 at 1:47=E2=80=AFAM Euler=
 Taveira &lt;<a href=3D"mailto:euler@eulerto.com">euler@eulerto.com</a>&=
gt; wrote:<br></div><div>&gt;<br></div><div>&gt; On Tue, Mar 26, 2024, a=
t 4:12 PM, Tomas Vondra wrote:<br></div><div>&gt;<br></div><div>&gt; Per=
haps I'm missing something, but why is NUM_CONN_ATTEMPTS even needed?<br=
></div><div>&gt; Why isn't recovery_timeout enough to decide if wait_for=
_end_recovery()<br></div><div>&gt; waited long enough?<br></div><div>&gt=
;<br></div><div>&gt;<br></div><div>&gt; It was an attempt to decoupled a=
 connection failure (that keeps streaming the<br></div><div>&gt; WAL) fr=
om recovery timeout. The NUM_CONN_ATTEMPTS guarantees that if the primar=
y<br></div><div>&gt; is gone during the standby recovery process, there =
is a way to bail out.<br></div><div>&gt;<br></div><div><br></div><div>I =
think we don't need to check primary if the WAL corresponding to<br></di=
v><div>consistent_lsn is already present on the standby. Shouldn't we fi=
rst<br></div><div>check that? Once we ensure that the required WAL is co=
pied, just<br></div><div>checking server_is_in_recovery() should be suff=
icient. I feel that<br></div><div>will be a direct way of ensuring what =
is required rather than<br></div><div>indirectly verifying the same (by =
checking pg_stat_wal_receiver) as we<br></div><div>are doing currently.<=
br></div></blockquote><div><br></div><div>How would you check it? WAL fi=
le? During recovery, you are not allowed to use<br></div><div>pg_current=
_wal_lsn.<br></div><div><br></div><div>Tomas suggested to me off-list th=
at we should adopt a simple solution in<br></div><div>wait_for_end_recov=
ery: wait for recovery_timeout without additional checks<br></div><div>(=
which means remove the pg_stat_wal_receiver logic).&nbsp; When we have a=
dditional<br></div><div>information that we can reliably use in this fun=
ction, we can add it. Hence, it<br></div><div>is also easy to adjust the=
 PG_TEST_TIMEOUT_DEFAULT to have stable tests.<br></div><div><br></div><=
div><br></div><div id=3D"sig112409080"><div class=3D"signature">--<br></=
div><div class=3D"signature">Euler Taveira<br></div><div class=3D"signat=
ure">EDB&nbsp;&nbsp; <a href=3D"https://www.enterprisedb.com/">https://w=
ww.enterprisedb.com/</a><br></div></div><div><br></div></body></html>
--a5b8760bebfa46ef8dce28cc7a0f5d3f--