Feedback-ID: i0c21471d:Fastmail
User-Agent: Cyrus-JMAP/3.11.0-alpha0-332-gdeb4194079-fm-20240319.002-gdeb41940
MIME-Version: 1.0
Message-Id: <07a0f7d7-8ec9-4649-8bcb-610fd430eebd@app.fastmail.com>
In-Reply-To: 
 <CALDaNm3XnSdX6fuAHrQKxqiSrR_Hyj6ETna2a8LdGJN4oSu4=Q@mail.gmail.com>
References: <9a9c7e65-c47b-4400-a63e-ac6169c0860e@app.fastmail.com>
 <TYCPR01MB12077CD333376B53F9CAE7AC0F5562@TYCPR01MB12077.jpnprd01.prod.outlook.com>
 <e390e35e-508e-4eb8-92e4-e6b066407a41@app.fastmail.com>
 <TYCPR01MB12077121263EFF7680486B635F5212@TYCPR01MB12077.jpnprd01.prod.outlook.com>
 <40595e73-c7e1-463a-b8be-49792e870007@app.fastmail.com>
 <6423dfeb-a729-45d3-b71e-7bf1b3adb0c9@enterprisedb.com>
 <TYCPR01MB12077DE77454D68E959E85E35F5272@TYCPR01MB12077.jpnprd01.prod.outlook.com>
 <e8564096-3c4f-4990-ac7d-ba089739b878@app.fastmail.com>
 <CANhcyEW6-dH28gLbFc5XpDTJ6JPizU+t5g-aKUWJBf5W_Zriqw@mail.gmail.com>
 <2f0b73f3-e7c0-4e95-9712-1993442a6b64@app.fastmail.com>
 <CAA4eK1JtnzqYNQ_S+mj_XsPfgk7szxegKiVQq4-8fY1wUcUnpg@mail.gmail.com>
 <34637e7f-0330-420d-8f45-1d022962d2fe@app.fastmail.com>
 <7a970912-0b77-4942-84f7-2c9ca0bc05a5@eisentraut.org>
 <d546c4bb-92d1-4e2d-898f-48234b12ed25@app.fastmail.com>
 <CALDaNm1Dg5tDRmaabk+ZND4WF17NrNq52WZxCE+90-PGz5trQQ@mail.gmail.com>
 <60b45b8a-3047-4a21-ba2a-ddb15daa638f@eisentraut.org>
 <CALDaNm3XnSdX6fuAHrQKxqiSrR_Hyj6ETna2a8LdGJN4oSu4=Q@mail.gmail.com>
Date: Thu, 21 Mar 2024 11:30:05 -0300
From: "Euler Taveira" <euler@eulerto.com>
To: "vignesh C" <vignesh21@gmail.com>,
 "Peter Eisentraut" <peter@eisentraut.org>
Cc: "Amit Kapila" <amit.kapila16@gmail.com>,
 "Shlok Kyal" <shlok.kyal.oss@gmail.com>,
 "kuroda.hayato@fujitsu.com" <kuroda.hayato@fujitsu.com>,
 "Tomas Vondra" <tomas.vondra@enterprisedb.com>,
 "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>,
 "Michael Paquier" <michael@paquier.xyz>,
 "Andres Freund" <andres@anarazel.de>,
 "Ashutosh Bapat" <ashutosh.bapat.oss@gmail.com>,
 =?UTF-8?Q?Fabr=C3=ADzio_de_Royes_Mello?= <fabriziomello@gmail.com>
Subject: Re: speed up a logical replica setup
Content-Type: multipart/alternative;
 boundary=b645710a94464af9bf986aba8aaa5c2d
Archived-At: <https://www.postgresql.org/message-id/07a0f7d7-8ec9-4649-8bcb-610fd430eebd%40app.fastmail.com>
Precedence: bulk

--b645710a94464af9bf986aba8aaa5c2d
Content-Type: text/plain

On Thu, Mar 21, 2024, at 10:33 AM, vignesh C wrote:
> If we commit this we might not be able to change the way the option
> behaves once the customers starts using it. How about removing these
> options in the first version and adding it in the next version after
> more discussion.

We don't need to redesign this one if we want to add a format string in a next
version. A long time ago, pg_dump started to accept pattern for tables without
breaking or deprecating the -t option. If you have 100 databases and you don't
want to specify the options or use a script to generate it for you, you also
have the option to let pg_createsubscriber generate the object names for you.
Per my experience, it will be a rare case.

> Currently dry-run will do the check and might fail on identifying a
> few failures like after checking subscriber configurations. Then the
> user will have to correct the configuration and re-run then fix the
> next set of failures. Whereas the suggest-config will display all the
> optimal configuration for both the primary and the standby in a single
> shot. This is not a must in the first version, it can be done as a
> subsequent enhancement.

Do you meant publisher, right? Per order, check_subscriber is done before
check_publisher and it checks all settings on the subscriber before exiting. In
v30, I changed the way it provides the required settings. In a previous version,
it fails when it found a wrong setting; the current version, check all settings
from that server before providing a suitable error.

pg_createsubscriber: checking settings on publisher
pg_createsubscriber: primary has replication slot "physical_slot"
pg_createsubscriber: error: publisher requires wal_level >= logical
pg_createsubscriber: error: publisher requires 2 replication slots, but only 0 remain
pg_createsubscriber: hint: Consider increasing max_replication_slots to at least 3.
pg_createsubscriber: error: publisher requires 2 wal sender processes, but only 0 remain
pg_createsubscriber: hint: Consider increasing max_wal_senders to at least 3.

If you have such an error, you will fix them all and rerun using dry run mode
again to verify everything is ok. I don't have a strong preference about it. It
can be changed easily (unifying the check functions or providing a return for
each of the check functions).


--
Euler Taveira
EDB   https://www.enterprisedb.com/

--b645710a94464af9bf986aba8aaa5c2d
Content-Type: text/html
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE html><html><head><title></title><style type=3D"text/css">p.Mso=
Normal,p.MsoNoSpacing{margin:0}</style></head><body><div>On Thu, Mar 21,=
 2024, at 10:33 AM, vignesh C wrote:<br></div><blockquote type=3D"cite" =
id=3D"qt" style=3D""><div>If we commit this we might not be able to chan=
ge the way the option<br></div><div>behaves once the customers starts us=
ing it. How about removing these<br></div><div>options in the first vers=
ion and adding it in the next version after<br></div><div>more discussio=
n.<br></div></blockquote><div><br></div><div>We don't need to redesign t=
his one if we want to add a format string in a next<br></div><div>versio=
n. A long time ago, pg_dump started to accept pattern for tables without=
<br></div><div>breaking or deprecating the -t option. If you have 100 da=
tabases and you don't<br></div><div>want to specify the options or use a=
 script to generate it for you, you also<br></div><div>have the option t=
o let pg_createsubscriber generate the object names for you.<br></div><d=
iv>Per my experience, it will be a rare case.<br></div><div><br></div><b=
lockquote type=3D"cite" id=3D"qt" style=3D""><div>Currently dry-run will=
 do the check and might fail on identifying a<br></div><div>few failures=
 like after checking subscriber configurations. Then the<br></div><div>u=
ser will have to correct the configuration and re-run then fix the<br></=
div><div>next set of failures. Whereas the suggest-config will display a=
ll the<br></div><div>optimal configuration for both the primary and the =
standby in a single<br></div><div>shot. This is not a must in the first =
version, it can be done as a<br></div><div>subsequent enhancement.<br></=
div></blockquote><div><br></div><div>Do you meant publisher, right? Per =
order, check_subscriber is done before<br></div><div>check_publisher and=
 it checks all settings on the subscriber before exiting. In<br></div><d=
iv>v30, I changed the way it provides the required settings. In a previo=
us version,<br></div><div>it fails when it found a wrong setting; the cu=
rrent version, check all settings<br></div><div>from that server before =
providing a suitable error.<br></div><div><br></div><div>pg_createsubscr=
iber: checking settings on publisher<br></div><div>pg_createsubscriber: =
primary has replication slot "physical_slot"<br></div><div>pg_createsubs=
criber: error: publisher requires wal_level &gt;=3D logical<br></div><di=
v>pg_createsubscriber: error: publisher requires 2 replication slots, bu=
t only 0 remain<br></div><div>pg_createsubscriber: hint: Consider increa=
sing max_replication_slots to at least 3.<br></div><div>pg_createsubscri=
ber: error: publisher requires 2 wal sender processes, but only 0 remain=
<br></div><div>pg_createsubscriber: hint: Consider increasing max_wal_se=
nders to at least 3.<br></div><div><br></div><div>If you have such an er=
ror, you will fix them all and rerun using dry run mode<br></div><div>ag=
ain to verify everything is ok. I don't have a strong preference about i=
t. It<br></div><div>can be changed easily (unifying the check functions =
or providing a return for<br></div><div>each of the check functions).<br=
></div><div><br></div><div><br></div><div id=3D"sig112409080"><div class=
=3D"signature">--<br></div><div class=3D"signature">Euler Taveira<br></d=
iv><div class=3D"signature">EDB&nbsp;&nbsp; <a href=3D"https://www.enter=
prisedb.com/">https://www.enterprisedb.com/</a><br></div></div><div><br>=
</div></body></html>
--b645710a94464af9bf986aba8aaa5c2d--