Feedback-ID: i0c21471d:Fastmail
User-Agent: Cyrus-JMAP/3.11.0-alpha0-480-g515a2f54a-fm-20240515.001-g515a2f54
MIME-Version: 1.0
Message-Id: <5da4b286-e570-4341-8262-92d763f6b1a4@app.fastmail.com>
In-Reply-To: 
 <CAA4eK1JaQcAZ=n513-4VDW4ujE9T9f007fczNCpVDbg1smhT4g@mail.gmail.com>
References: <9a9c7e65-c47b-4400-a63e-ac6169c0860e@app.fastmail.com>
 <e8564096-3c4f-4990-ac7d-ba089739b878@app.fastmail.com>
 <CANhcyEW6-dH28gLbFc5XpDTJ6JPizU+t5g-aKUWJBf5W_Zriqw@mail.gmail.com>
 <2f0b73f3-e7c0-4e95-9712-1993442a6b64@app.fastmail.com>
 <CAA4eK1JtnzqYNQ_S+mj_XsPfgk7szxegKiVQq4-8fY1wUcUnpg@mail.gmail.com>
 <34637e7f-0330-420d-8f45-1d022962d2fe@app.fastmail.com>
 <7a970912-0b77-4942-84f7-2c9ca0bc05a5@eisentraut.org>
 <d546c4bb-92d1-4e2d-898f-48234b12ed25@app.fastmail.com>
 <CANhcyEWdTTO71v+GFYUTsOOUvHus1qccg88bRmsU9KykJqjSrQ@mail.gmail.com>
 <44de4e04-db22-40ed-8ba9-46387dc8bf83@app.fastmail.com>
 <3fa9ef0f-b277-4c13-850a-8ccc04de1406@eisentraut.org>
 <CALj2ACV85cn-cTNCTP=bPehYo9d2ddvZtfGhGq==0K+65SvufQ@mail.gmail.com>
 <65b841a8-0c2f-4953-b9cd-240fce84b90b@eisentraut.org>
 <TYCPR01MB12077907C052EC329C45AE198F5362@TYCPR01MB12077.jpnprd01.prod.outlook.com>
 <c8e92bcb-69c3-477e-93d4-6f39e030cab0@app.fastmail.com>
 <47524383-6fd1-4b13-8c4e-fb918b1d35ba@enterprisedb.com>
 <bbbeed26-41f9-468c-ac24-ad99f429ccd7@app.fastmail.com>
 <CAA4eK1KPoLUUGzjw9eEQd56LUAf2QQj8BqMcsoSe6vycB5CYUQ@mail.gmail.com>
 <2e86cf53-c806-49d2-9a10-1a1257776d1f@app.fastmail.com>
 <CAA4eK1JJq_ER6Kq_H=jKHR75QPRd8y9_D=RtYw=aPYKMfqLi9A@mail.gmail.com>
 <CAA4eK1KdCb+5sjYu6qCMXXdCX1y_ihr8kFzMozq0=P=auYxgog@mail.gmail.com>
 <CANhcyEV6q1Vhd37i1axUeScLi0UAGVxta1LDa0BV0Eh--TcPMg@mail.gmail.com>
 <CAA4eK1JaQcAZ=n513-4VDW4ujE9T9f007fczNCpVDbg1smhT4g@mail.gmail.com>
Date: Wed, 22 May 2024 12:16:24 -0300
From: "Euler Taveira" <euler@eulerto.com>
To: "Amit Kapila" <amit.kapila16@gmail.com>,
 "Shlok Kyal" <shlok.kyal.oss@gmail.com>
Cc: "Tomas Vondra" <tomas.vondra@enterprisedb.com>,
 "kuroda.hayato@fujitsu.com" <kuroda.hayato@fujitsu.com>,
 "Peter Eisentraut" <peter@eisentraut.org>,
 "Bharath Rupireddy" <bharath.rupireddyforpostgres@gmail.com>,
 "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>,
 "Michael Paquier" <michael@paquier.xyz>,
 "Andres Freund" <andres@anarazel.de>,
 "Ashutosh Bapat" <ashutosh.bapat.oss@gmail.com>,
 =?UTF-8?Q?Fabr=C3=ADzio_de_Royes_Mello?= <fabriziomello@gmail.com>,
 "vignesh C" <vignesh21@gmail.com>
Subject: Re: speed up a logical replica setup
Content-Type: multipart/alternative;
 boundary=130ade43fb1b441b93b475425459e0ae
Archived-At: <https://www.postgresql.org/message-id/5da4b286-e570-4341-8262-92d763f6b1a4%40app.fastmail.com>
Precedence: bulk

--130ade43fb1b441b93b475425459e0ae
Content-Type: text/plain

On Wed, May 22, 2024, at 8:19 AM, Amit Kapila wrote:
> >
> > v2-0001: not changed
> >
> 
> Shouldn't we modify it as per the suggestion given in the email [1]? I
> am wondering if we can entirely get rid of checking the primary
> business and simply rely on recovery_timeout and keep checking
> server_is_in_recovery(). If so, we can modify the test to use
> non-default recovery_timeout (say 180s or something similar if we have
> used it at any other place). As an additional check we can ensure that
> constent_lsn is present on standby.

That's exactly what I want to propose as Tomas convinced me offlist that less is
better when we don't have a useful recovery progress reporting mechanism to make
sure it is still working on the recovery and we should wait.

> > v2-0002: not changed
> >
> 
> We have added more tries to see if the primary_slot_name becomes
> active but I think it is still fragile because it is possible on slow
> machines that the required slot didn't become active even after more
> retries. I have raised the same comment previously [2] and asked an
> additional question but didn't get any response.

Following the same line that simplifies the code, we can: (a) add a loop in
check_subscriber() that waits until walreceiver is available on subscriber or
(b) use a timeout. The main advantage of (a) is that the primary slot is already
available but I'm afraid we need a escape mechanism for the loop (timeout?).

I'll summarize all issues as soon as I finish the review of sync slot support. I
think we should avoid new development if we judge that the item can be
documented as a limitation for this version. Nevertheless, I will share patches
so you can give your opinion on whether it is an open item or new development.


--
Euler Taveira
EDB   https://www.enterprisedb.com/

--130ade43fb1b441b93b475425459e0ae
Content-Type: text/html
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE html><html><head><title></title><style type=3D"text/css">p.Mso=
Normal,p.MsoNoSpacing{margin:0}
p.MsoNormal,p.MsoNoSpacing{margin:0}
p.MsoNormal,p.MsoNoSpacing{margin:0}</style></head><body><div>On Wed, Ma=
y 22, 2024, at 8:19 AM, Amit Kapila wrote:<br></div><blockquote type=3D"=
cite" id=3D"qt" style=3D""><div>&gt;<br></div><div>&gt; v2-0001: not cha=
nged<br></div><div>&gt;<br></div><div><br></div><div>Shouldn't we modify=
 it as per the suggestion given in the email [1]? I<br></div><div>am won=
dering if we can entirely get rid of checking the primary<br></div><div>=
business and simply rely on recovery_timeout and keep checking<br></div>=
<div>server_is_in_recovery(). If so, we can modify the test to use<br></=
div><div>non-default recovery_timeout (say 180s or something similar if =
we have<br></div><div>used it at any other place). As an additional chec=
k we can ensure that<br></div><div>constent_lsn is present on standby.<b=
r></div></blockquote><div><br></div><div>That's exactly what I want to p=
ropose as Tomas convinced me offlist that less is<br></div><div>better w=
hen we don't have a useful recovery progress reporting mechanism to make=
<br></div><div>sure it is still working on the recovery and we should wa=
it.<br></div><div><br></div><blockquote type=3D"cite" id=3D"qt" style=3D=
""><div>&gt; v2-0002: not changed<br></div><div>&gt;<br></div><div><br><=
/div><div>We have added more tries to see if the primary_slot_name becom=
es<br></div><div>active but I think it is still fragile because it is po=
ssible on slow<br></div><div>machines that the required slot didn't beco=
me active even after more<br></div><div>retries. I have raised the same =
comment previously [2] and asked an<br></div><div>additional question bu=
t didn't get any response.<br></div></blockquote><div><br></div><div>Fol=
lowing the same line that simplifies the code, we can: (a) add a loop in=
<br></div><div>check_subscriber() that waits until walreceiver is availa=
ble on subscriber or<br></div><div>(b) use a timeout. The main advantage=
 of (a) is that the primary slot is already<br></div><div>available but =
I'm afraid we need a escape mechanism for the loop (timeout?).<br></div>=
<div><br></div><div>I'll summarize all issues as soon as I finish the re=
view of sync slot support. I<br></div><div>think we should avoid new dev=
elopment if we judge that the item can be<br></div><div>documented as a =
limitation for this version. Nevertheless, I will share patches<br></div=
><div>so you can give your opinion on whether it is an open item or new =
development.<br></div><div><br></div><div><br></div><div id=3D"sig112409=
080"><div class=3D"signature">--<br></div><div class=3D"signature">Euler=
 Taveira<br></div><div class=3D"signature">EDB&nbsp;&nbsp; <a href=3D"ht=
tps://www.enterprisedb.com/">https://www.enterprisedb.com/</a><br></div>=
</div><div><br></div></body></html>
--130ade43fb1b441b93b475425459e0ae--