Date: Wed, 25 Jan 2023 10:17:18 +0900 (JST)
Message-Id: <20230125.101718.700668541820262311.horikyota.ntt@gmail.com>
To: amit.kapila16@gmail.com
Cc: osumi.takamichi@fujitsu.com, smithpb2250@gmail.com,
 vignesh21@gmail.com, kuroda.hayato@fujitsu.com, shveta.malik@gmail.com,
 dilipbalaut@gmail.com, euler@eulerto.com, m.melihmutlu@gmail.com,
 andres@anarazel.de, marcos@f10.com.br, pgsql-hackers@postgresql.org
Subject: Re: Time delayed LR (WAS Re: logical replication restrictions)
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
In-Reply-To: <CAA4eK1JTTCcMrsMAmz9LSEE6gvGHnTK0u1w0kg9hV6rcFFB5hQ@mail.gmail.com>
References: <TYCPR01MB8373A23EFC25B22CB132383AEDCB9@TYCPR01MB8373.jpnprd01.prod.outlook.com>
	<20230124.114535.1107425862791533999.horikyota.ntt@gmail.com>
	<CAA4eK1JTTCcMrsMAmz9LSEE6gvGHnTK0u1w0kg9hV6rcFFB5hQ@mail.gmail.com>
User-Agent: Mew version 6.8 on Emacs 26.1
Mime-Version: 1.0
Content-Type: Text/Plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Archived-At: <https://www.postgresql.org/message-id/20230125.101718.700668541820262311.horikyota.ntt%40gmail.com>
Precedence: bulk

In short, I'd like to propose renaming the parameter in_delayed_apply
of send_feedback to "has_unprocessed_change".

At Tue, 24 Jan 2023 12:27:58 +0530, Amit Kapila <amit.kapila16@gmail.com> wrote in 
> > send_feedback():
> > +        * If the subscriber side apply is delayed (because of time-delayed
> > +        * replication) then do not tell the publisher that the received latest
> > +        * LSN is already applied and flushed, otherwise, it leads to the
> > +        * publisher side making a wrong assumption of logical replication
> > +        * progress. Instead, we just send a feedback message to avoid a publisher
> > +        * timeout during the delay.
> >          */
> > -       if (!have_pending_txes)
> > +       if (!have_pending_txes && !in_delayed_apply)
> >                 flushpos = writepos = recvpos;
> >
> > Honestly I don't like this wart. The reason for this is the function
> > assumes recvpos = applypos but we actually call it while holding
> > unapplied changes, that is, applypos < recvpos.
> >
> > Couldn't we maintain an additional static variable "last_applied"
> > along with last_received?
> >
> 
> It won't be easy to maintain the meaning of last_applied because there
> are cases where we don't apply the change directly. For example, in
> case of streaming xacts, we will just keep writing it to the file,
> now, say, due to some reason, we have to send the feedback, then it
> will not allow you to update the latest write locations. This would
> then become different then what we are doing without the patch.
> Another point to think about is that we also need to keep the variable
> updated for keep-alive ('k') messages even though we don't apply
> anything in that case. Still, other cases to consider are where we
> have mix of streaming and non-streaming transactions.

Yeah.  Even though I named it as "last_applied", its objective is to
have get_flush_position returning the correct have_pending_txes
without a hint from callers, that is, "let g_f_position know if
store_flush_position has been called with the last received data".

Anyway I tried that but didn't find a clean and simple way. However,
while on it, I realized what the code made me confused.

+static void send_feedback(XLogRecPtr recvpos, bool force, bool requestReply,
+						  bool in_delayed_apply);

The name "in_delayed_apply" doesn't donsn't give me an idea of what
the function should do for it. If it is named "has_unprocessed_change",
I think it makes sense that send_feedback should think there may be an
outstanding transaction that is not known to the function.


So, my conclusion here is I'd like to propose changing the parameter
name to "has_unapplied_change".


> >  In this case the condition cited above
> > would be as follows and in_delayed_apply will become unnecessary.
> >
> > +       if (!have_pending_txes && last_received == last_applied)
> >
> > The function is a static function and always called with a variable
> > last_received that has the same scope with the function, as the first

Sorry for the noise, I misread it. Maybe I took the "function-scoped"
variable as file-scoped.. Thus the discussion is false.

> > parameter. Thus we can remove the first parameter then let the
> > function directly look at the both two varaibles instead.
> >
> 
> I think this is true without this patch, so why that has not been
> followed in the first place? One comment, I see in this regard is as
> below:
> 
> /* It's legal to not pass a recvpos */
> if (recvpos < last_recvpos)
> recvpos = last_recvpos;

Sorry. I don't understand this. It is just a part of the ratchet
mechanism for the last received lsn to report.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center