Feedback-ID: i0c21471d:Fastmail
User-Agent: Cyrus-JMAP/3.9.0-alpha0-1374-gc37f3abe3d-fm-20240102.001-gc37f3abe
MIME-Version: 1.0
Message-Id: <bcd3db13-f56e-47e3-8fa7-7f0c6243c8a6@app.fastmail.com>
In-Reply-To: 
 <CANhcyEUCt-g4JLQU3Q3ofFk_Vt-Tqh3ZdXoLcpT8fjz9LY_-ww@mail.gmail.com>
References: <5ac50071-f2ed-4ace-a8fd-b892cffd33eb@www.fastmail.com>
 <4710d895-689a-4ae6-b8e3-344286dbf970@app.fastmail.com>
 <da078f11-d2de-420e-9611-f3b7d4dadd9a@eisentraut.org>
 <ZUrEclC-uZV-rGd2@paquier.xyz>
 <96e3b739-d63d-4d35-90a3-7869465b7167@eisentraut.org>
 <ZU1nR5KMZtSTEQDf@paquier.xyz>
 <e02a2c17-22e5-4ba6-b788-de696ab74f1e@app.fastmail.com>
 <CALDaNm098Jkbh+ye6zMj9Ro9j1bBe6FfPV80BFbs1=pUuTJ07g@mail.gmail.com>
 <2736443c-58e3-42df-8cc4-8c66106e3da5@app.fastmail.com>
 <CAA4eK1KtGC-v_CYd8yaCwisrQvoZN=+As90p9hBM29DAxA+Pmg@mail.gmail.com>
 <CANhcyEUdYCR9oSt4HsZRqYZZVXsb4x-HoPGqDSxSeA2SYz-YGg@mail.gmail.com>
 <CAA4eK1KMQNMhSPmsRdrCQpuc=Tsc08h6=XkLNZ5F35Q2REom6g@mail.gmail.com>
 <CANhcyEXjOcnOJKdOfK8Z1+bbymuZzwSjrppbt4FKHbrDD_t7Mg@mail.gmail.com>
 <CANhcyEUCt-g4JLQU3Q3ofFk_Vt-Tqh3ZdXoLcpT8fjz9LY_-ww@mail.gmail.com>
Date: Wed, 10 Jan 2024 23:29:14 -0300
From: "Euler Taveira" <euler@eulerto.com>
To: "Shlok Kyal" <shlok.kyal.oss@gmail.com>,
 "Amit Kapila" <amit.kapila16@gmail.com>
Cc: "vignesh C" <vignesh21@gmail.com>,
 "Michael Paquier" <michael@paquier.xyz>,
 "Peter Eisentraut" <peter@eisentraut.org>,
 pgsql-hackers@lists.postgresql.org, "Andres Freund" <andres@anarazel.de>,
 "Ashutosh Bapat" <ashutosh.bapat.oss@gmail.com>
Subject: Re: speed up a logical replica setup
Content-Type: multipart/alternative;
 boundary=e3efbe4fdcd64a3b8b13d48542678c4b
Archived-At: <https://www.postgresql.org/message-id/bcd3db13-f56e-47e3-8fa7-7f0c6243c8a6%40app.fastmail.com>
Precedence: bulk

--e3efbe4fdcd64a3b8b13d48542678c4b
Content-Type: text/plain

On Wed, Jan 10, 2024, at 1:33 AM, Shlok Kyal wrote:
> Here the standby node would be waiting for the 'consistent_lsn' wal
> during recovery but this wal will not be present on standby if no
> physical replication is setup. Hence the command will be waiting
> infinitely for the wal.

Hmm. Some validations are missing.

> To solve this added a timeout of 60s for the recovery process and also
> added a check so that pg_subscriber would give a error when it called
> for node which is not in physical replication.
> Have attached the patch for the same. It is a top-up patch of the
> patch shared by Euler at [1].

If the user has a node that is not a standby and it does not set the GUCs to
start the recovery process from a backup, the initial setup is broken. (That's
the case you described.) A good UI is to detect this scenario earlier.
Unfortunately, there isn't a reliable and cheap way to do it. You need to start
the recovery and check if it is having some progress. (I don't have a strong
opinion about requiring a standby to use this tool. It would reduce the
complexity about checking if the setup has all requirements to run this tool.)


--
Euler Taveira
EDB   https://www.enterprisedb.com/

--e3efbe4fdcd64a3b8b13d48542678c4b
Content-Type: text/html
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE html><html><head><title></title><style type=3D"text/css">p.Mso=
Normal,p.MsoNoSpacing{margin:0}</style></head><body><div>On Wed, Jan 10,=
 2024, at 1:33 AM, Shlok Kyal wrote:<br></div><blockquote type=3D"cite" =
id=3D"qt" style=3D""><div>Here the standby node would be waiting for the=
 'consistent_lsn' wal<br></div><div>during recovery but this wal will no=
t be present on standby if no<br></div><div>physical replication is setu=
p. Hence the command will be waiting<br></div><div>infinitely for the wa=
l.<br></div></blockquote><div><br></div><div>Hmm. Some validations are m=
issing.<br></div><div><br></div><blockquote type=3D"cite" id=3D"qt" styl=
e=3D""><div>To solve this added a timeout of 60s for the recovery proces=
s and also<br></div><div>added a check so that pg_subscriber would give =
a error when it called<br></div><div>for node which is not in physical r=
eplication.<br></div><div>Have attached the patch for the same. It is a =
top-up patch of the<br></div><div>patch shared by Euler at [1].<br></div=
></blockquote><div><br></div><div><div>If the user has a node that is no=
t a standby and it does not set the GUCs to<br></div></div><div>start th=
e recovery process from a backup, the initial setup is broken. (That's<b=
r></div><div>the case you described.) A good UI is to detect this scenar=
io earlier.<br></div><div>Unfortunately, there isn't a reliable and chea=
p way to do it. You need to start<br></div><div>the recovery and check i=
f it is having some progress. (I don't have a strong<br></div><div>opini=
on about requiring a standby to use this tool. It would reduce the<br></=
div><div>complexity about checking if the setup has all requirements to =
run this tool.)<br></div><div><br></div><div><br></div><div id=3D"sig112=
409080"><div class=3D"signature">--<br></div><div class=3D"signature">Eu=
ler Taveira<br></div><div class=3D"signature">EDB&nbsp;&nbsp; <a href=3D=
"https://www.enterprisedb.com/">https://www.enterprisedb.com/</a><br></d=
iv></div><div><br></div></body></html>
--e3efbe4fdcd64a3b8b13d48542678c4b--