Feedback-ID: i0c21471d:Fastmail
User-Agent: Cyrus-JMAP/3.7.0-alpha0-758-ge0d20a54e1-fm-20220729.001-ge0d20a54
Mime-Version: 1.0
Message-Id: <94124b46-d856-4cd7-83d7-9f4035ac9831@www.fastmail.com>
In-Reply-To: 
 <CAA4eK1L54hcV6eOM1=k75-0bRR3bK3KoPPj6SopJ1q4xk0F0Sw@mail.gmail.com>
References: 
 <CAB-JLwYOYwL=XTyAXKiH5CtM_Vm8KjKh7aaitCKvmCh4rzr5pQ@mail.gmail.com>
 <CAA4eK1JW+_Pff3TOaBqbOqn-LbMT2kj2bpw-a88ZSTdbTZ-aMg@mail.gmail.com>
 <CAB-JLwas_=FGr8bXKD7K6g0=RDUFuYU71TLLGTsQ-DTRNpq1QQ@mail.gmail.com>
 <CAA4eK1KSBGrLKR63iS38Bkvjq6fBCWzbuD6R15kEWdmd8aj3mw@mail.gmail.com>
 <73b06a32-56ab-4056-86ff-e307f3c316f1@www.fastmail.com>
 <f026292b-c9ee-472e-beaa-d32c5c3a2ced@www.fastmail.com>
 <d192bc58-e9c7-442c-bae8-63f8e8f43fda@www.fastmail.com>
 <20220322010410.m6ccotflls45d7vq@alap3.anarazel.de>
 <aec5dd79-245c-454e-ae28-00ba3c8977ed@www.fastmail.com>
 <29bd8c6e-9c87-4a72-abea-52218a387e2e@www.fastmail.com>
 <fa0d60e1-87af-4b55-9385-c8ff4be1b836@www.fastmail.com>
 <CAHut+Pvugkna7avUQLydg602hymc8qMp=CRT2ZCTGbi8Bkfv+A@mail.gmail.com>
 <CAA4eK1JgS2wDjNT2F-sCSxYVffLHOxP5_=66Fssm01+qoGcX7A@mail.gmail.com>
 <594ad346-da36-4f53-91ad-da366787c67f@www.fastmail.com>
 <CAA4eK1L54hcV6eOM1=k75-0bRR3bK3KoPPj6SopJ1q4xk0F0Sw@mail.gmail.com>
Date: Mon, 08 Aug 2022 19:22:22 -0300
From: "Euler Taveira" <euler@eulerto.com>
To: "Amit Kapila" <amit.kapila16@gmail.com>
Cc: "Peter Smith" <smithpb2250@gmail.com>,
 "Andres Freund" <andres@anarazel.de>, "Marcos Pegoraro" <marcos@f10.com.br>,
 pgsql-hackers <pgsql-hackers@postgresql.org>
Subject: Re: logical replication restrictions
Content-Type: multipart/alternative;
 boundary=02c68161a5df47aaa92213e9247a895a
Archived-At: 
 <https://www.postgresql.org/message-id/94124b46-d856-4cd7-83d7-9f4035ac9831%40www.fastmail.com>
Precedence: bulk

--02c68161a5df47aaa92213e9247a895a
Content-Type: text/plain

On Wed, Aug 3, 2022, at 10:27 AM, Amit Kapila wrote:
> Your explanation makes sense to me. The other point to consider is
> that there can be cases where we may not apply operation for the
> transaction because of empty transactions (we don't yet skip empty
> xacts for prepared transactions). So, won't it be better to apply the
> delay just before we apply the first change for a transaction? Do we
> want to apply the delay during table sync as we sometimes do need to
> enter apply phase while doing table sync?
I thought about the empty transactions but decided to not complicate the code
mainly because skipping transactions is not a code path that will slow down
this feature. As explained in the documentation, there is no harm in delaying a
transaction for more than min_apply_delay; it cannot apply earlier. Having said
that I decided to do nothing. I'm also not sure if it deserves a comment or if
this email is a possible explanation for this decision.

Regarding the table sync that was mention by Melih, I sent a new version (v6)
that fixed this oversight. The current logical replication worker design make
it difficult to apply the delay in the catchup phase; tablesync workers are not
closed as soon as the COPY finishes (which means possibly running out of
workers sooner). After all tablesync workers have reached READY state, the
apply delay is activated. The documentation was correct; the code wasn't.


--
Euler Taveira
EDB   https://www.enterprisedb.com/

--02c68161a5df47aaa92213e9247a895a
Content-Type: text/html
Content-Transfer-Encoding: quoted-printable

<!DOCTYPE html><html><head><title></title><style type=3D"text/css">p.Mso=
Normal,p.MsoNoSpacing{margin:0}</style></head><body><div>On Wed, Aug 3, =
2022, at 10:27 AM, Amit Kapila wrote:<br></div><blockquote type=3D"cite"=
 id=3D"qt" style=3D""><div>Your explanation makes sense to me. The other=
 point to consider is<br></div><div>that there can be cases where we may=
 not apply operation for the<br></div><div>transaction because of empty =
transactions (we don't yet skip empty<br></div><div>xacts for prepared t=
ransactions). So, won't it be better to apply the<br></div><div>delay ju=
st before we apply the first change for a transaction? Do we<br></div><d=
iv>want to apply the delay during table sync as we sometimes do need to<=
br></div><div>enter apply phase while doing table sync?<br></div></block=
quote><div>I thought about the empty transactions but decided to not com=
plicate the code<br></div><div>mainly because skipping transactions is n=
ot a code path that will slow down<br></div><div>this feature. As explai=
ned in the documentation, there is no harm in delaying a<br></div><div>t=
ransaction for more than min_apply_delay; it cannot apply earlier. Havin=
g said<br></div><div>that I decided to do nothing. I'm also not sure if =
it deserves a comment or if<br></div><div>this email is a possible expla=
nation for this decision.<br></div><div><br></div><div>Regarding the tab=
le sync that was mention by Melih, I sent a new version (v6)<br></div><d=
iv>that fixed this oversight. The current logical replication worker des=
ign make<br></div><div>it difficult to apply the delay in the catchup ph=
ase; tablesync workers are not<br></div><div>closed as soon as the COPY =
finishes (which means possibly running out of<br></div><div>workers soon=
er). After all tablesync workers have reached READY state, the<br></div>=
<div>apply delay is activated. The documentation was correct; the code w=
asn't.<br></div><div><br></div><div><br></div><div id=3D"sig112409080"><=
div class=3D"signature">--<br></div><div class=3D"signature">Euler Tavei=
ra<br></div><div class=3D"signature">EDB&nbsp;&nbsp; <a href=3D"https://=
www.enterprisedb.com/">https://www.enterprisedb.com/</a><br></div></div>=
<div><br></div></body></html>
--02c68161a5df47aaa92213e9247a895a--