From: =?utf-8?q?=D0=94=D0=B0=D0=B2=D1=8B=D0=B4=D0=BE=D0=B2_=D0=92=D0=B8=D1=82=D0=B0=D0=BB=D0=B8=D0=B9?= <v.davydov@postgrespro.ru>
In-Reply-To: <23d1ca-66169f80-1f-5a90aa00@207285398>
Content-Type: multipart/alternative; boundary="----=_=-_OpenGroupware_org_NGMime-2536206-1713195115.092809-4------"
References: <8fab8-65d74c80-1-2f28e880@39088166> <CAA4eK1KOs3s6syZqUgrd2WvjTz64SGf0ToZcRoPMCKKH+M0YFQ@mail.gmail.com>
 <99df2-65d8d200-1-63c26880@172687752> <CAA4eK1KtjjPxe5tjWeGNf5CfF9qfceVQCCC9XGsKzR-ifQD9uA@mail.gmail.com>
 <ba4b6-65ddc580-9-5c3e3080@134362348> <CAA4eK1J6wOMkDDt3iOs+JqkvQvuwSORosTpqdejgvXqFmxex=A@mail.gmail.com>
 <badcf-65dde500-3-24ba3c80@28790566> <cedbe-65e0c000-1-6db17700@133269862>
 <a375c573-6c9b-4f0f-82c4-eb19cef2a4ca@iki.fi> <f9f1b-65e72c80-7-533dda00@76203995>
 <CAFPTHDbU4EkVmKL66i5BzrmQ6LFftQPx_TZ1euKt9fvVzQDyjA@mail.gmail.com>
 <CAA4eK1+n7M2S1OpoGWDd+YZkDCuURMdVRbvP0eELQUvWgmDneg@mail.gmail.com> <CAFPTHDa=pJSZ_4dV5DPAOapRSgPcyyUTP0WzGY2Rz_D3-gwraw@mail.gmail.com> <CAA4eK1K1fSkeK=kc26G5cq87vQG4=1qs_b+no4+ep654SeBy1w@mail.gmail.com> <23d1ca-66169f80-1f-5a90aa00@207285398>
Date: Mon, 15 Apr 2024 18:31:55 +0300
Cc: "Amit Kapila" <amit.kapila16@gmail.com>, "Ajin Cherian" <itsajin@gmail.com>, "Heikki Linnakangas" <hlinnaka@iki.fi>, pgsql-hackers@lists.postgresql.org
To: =?utf-8?q?=D0=94=D0=B0=D0=B2=D1=8B=D0=B4=D0=BE=D0=B2_=D0=92=D0=B8=D1=82=D0=B0=D0=BB=D0=B8=D0=B9?= <v.davydov@postgrespro.ru>
MIME-Version: 1.0
Message-ID: <26b30e-661d4880-9-62d2df00@231402233>
Subject: =?utf-8?q?Re=3A?= Slow catchup of 2PC (twophase) transactions on replica in 
 LR
User-Agent: SOGoMail 5.10.0
Archived-At: <https://www.postgresql.org/message-id/26b30e-661d4880-9-62d2df00%40231402233>
Precedence: bulk

------=_=-_OpenGroupware_org_NGMime-2536206-1713195115.092809-4------
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Length: 3050


Dear All,

On Wednesday, April 10, 2024 17:16 MSK, =D0=94=D0=B0=D0=B2=D1=8B=D0=B4=D0=
=BE=D0=B2 =D0=92=D0=B8=D1=82=D0=B0=D0=BB=D0=B8=D0=B9 <v.davydov@postgre=
spro.ru> wrote:
=C2=A0Hi Amit, Ajin, All

Thank you for the patch and the responses. I apologize for my delayed a=
nswer due to some=C2=A0curcumstances.
On Wednesday, April 10, 2024 14:18 MSK, Amit Kapila <amit.kapila16@gmai=
l.com> wrote:

Vitaly, does the minimal solution provided by the proposed patch (Allow=
 to alter two=5Fphase option of a subscriber provided no uncommitted pr=
epared transactions are pending on that subscription.) address your use=
 case?In general, the idea behind the patch seems to be suitable for my=
 case. Furthermore, the case of two=5Fphase switch from false to true w=
ith uncommitted pending prepared transactions probably never happens in=
 my case. The switch from false to true means that the replica complete=
s the catchup from the master and switches to the normal mode when it p=
articipates in the multi-node configuration. There should be no uncommi=
tted pending prepared transactions at the moment of the switch to the n=
ormal mode.

I'm going to try this patch. Give me please some time to investigate th=
e patch. I will come with some feedback a little bit later.
I looked at the patch and realized that I can't try it easily in the ne=
ar future because the solution I'm working on is based on PG16 or earli=
er. This patch is not easily applicable to the older releases. I have t=
o port my solution to the master, which is not done yet. I apologize fo=
r that - so much work should be done before applying the patch. BTW, I =
tested the idea with async 2PC commit on my side and it seems to work f=
ine in my case. Anyway, I agree, the idea with altering of subscription=
 seems the best one but much harder to implement.

To summarise my case of a synchronous multimaster where twophase is use=
d to implement global transactions:
 * Replica may have prepared but not committed transactions when I togg=
le subscription twophase from true to false. In this case, all prepared=
 transactions may be aborted on the replica before altering=C2=A0the su=
bscription. * Replica will=C2=A0not have prepared transactions when sub=
scription is toggled from false to true. In this scenario, the replica =
completes the catchup (with twophase=3Doff) and=C2=A0becomes the part o=
f the multi-nodal cluster and is ready to accept new 2PC transactions. =
All the new pending transactions will wait until replica responds. But =
it may work=C2=A0differently=C2=A0for some other solutions. In general,=
 it would be great to allow toggling for all scenarious.Just interested=
, does anyone tried to reproduce the problem with slow catchup of twoph=
ase transactions (pgbench should be used with big number of clients)? I=
 haven't seen any messages from anyone other that me that the problem t=
akes place.=E2=80=8B=E2=80=8B=E2=80=8B=E2=80=8B

Thank you for your help!

With best regards,
Vitaly


=C2=A0

------=_=-_OpenGroupware_org_NGMime-2536206-1713195115.092809-4------
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Content-Length: 3378

<html>Dear All,<br /><br />On Wednesday, April 10, 2024 17:16 MSK, =D0=94=
=D0=B0=D0=B2=D1=8B=D0=B4=D0=BE=D0=B2 =D0=92=D0=B8=D1=82=D0=B0=D0=BB=D0=B8=
=D0=B9 &lt;v.davydov@postgrespro.ru&gt; wrote:<br />&nbsp;<blockquote t=
ype=3D"cite" cite=3D"23d1ca-66169f80-1f-5a90aa00@207285398">Hi Amit, Aj=
in, All<br /><br />Thank you for the patch and the responses. I apologi=
ze for my delayed answer due to some&nbsp;curcumstances.<blockquote><p>=
On Wednesday, April 10, 2024 14:18 MSK, Amit Kapila &lt;amit.kapila16@g=
mail.com&gt; wrote:<br /><br />Vitaly, does the minimal solution provid=
ed by the proposed patch (Allow to alter two=5Fphase option of a subscr=
iber provided no uncommitted prepared transactions are pending on that =
subscription.) address your use case?</p></blockquote>In general, the i=
dea behind the patch seems to be suitable for my case. Furthermore, the=
 case of two=5Fphase switch from false to true with uncommitted pending=
 prepared transactions probably never happens in my case. The switch fr=
om false to true means that the replica completes the catchup from the =
master and switches to the normal mode when it participates in the mult=
i-node configuration. There should be no uncommitted pending prepared t=
ransactions at the moment of the switch to the normal mode.<br /><br />=
I'm going to try this patch. Give me please some time to investigate th=
e patch. I will come with some feedback a little bit later.</blockquote=
><br />I looked at the patch and realized that I can't try it easily in=
 the near future because the solution I'm working on is based on PG16 o=
r earlier. This patch is not easily applicable to the older releases. I=
 have to port my solution to the master, which is not done yet. I apolo=
gize for that - so much work should be done before applying the patch. =
BTW, I tested the idea with async 2PC commit on my side and it seems to=
 work fine in my case. Anyway, I agree, the idea with altering of subsc=
ription seems the best one but much harder to implement.<br /><br />To =
summarise my case of a synchronous multimaster where twophase is used t=
o implement global transactions:<ul><li>Replica may have prepared but n=
ot committed transactions when I toggle subscription twophase from true=
 to false. In this case, all prepared transactions may be aborted on th=
e replica before altering&nbsp;the subscription.</li><li>Replica will&n=
bsp;not have prepared transactions when subscription is toggled from fa=
lse to true. In this scenario, the replica completes the catchup (with =
twophase=3Doff) and&nbsp;becomes the part of the multi-nodal cluster an=
d is ready to accept new 2PC transactions. All the new pending transact=
ions will wait until replica responds. But it may work&nbsp;differently=
&nbsp;for some other solutions. In general, it would be great to allow =
toggling for all scenarious.</li></ul>Just interested, does anyone trie=
d to reproduce the problem with slow catchup of twophase transactions (=
pgbench should be used with big number of clients)? I haven't seen any =
messages from anyone other that me that the problem takes place.=E2=80=8B=
=E2=80=8B=E2=80=8B=E2=80=8B<br /><br />Thank you for your help!<br /><b=
r />With best regards,<br />Vitaly<br /><br /><br /><br /><br /><br /><=
br /><br /><br /><br /><br />&nbsp;</html>

------=_=-_OpenGroupware_org_NGMime-2536206-1713195115.092809-4--------