Subject: Re: Time delayed LR (WAS Re: logical replication restrictions)

From: Kyotaro Horiguchi <[email-masked]>
Date: Wed, 25 Jan 2023 10:45:25 +0900 (JST)

Sorry for making you bothered by this.

At Tue, 24 Jan 2023 10:12:40 +0000, "Hayato Kuroda (Fujitsu)" <[email-masked]> wrote in 
> > > Couldn't we maintain an additional static variable "last_applied"
> > > along with last_received?
> > >
> > 
> > It won't be easy to maintain the meaning of last_applied because there
> > are cases where we don't apply the change directly. For example, in
> > case of streaming xacts, we will just keep writing it to the file,
> > now, say, due to some reason, we have to send the feedback, then it
> > will not allow you to update the latest write locations. This would
> > then become different then what we are doing without the patch.
> > Another point to think about is that we also need to keep the variable
> > updated for keep-alive ('k') messages even though we don't apply
> > anything in that case. Still, other cases to consider are where we
> > have mix of streaming and non-streaming transactions.
> 
> I have tried to implement that, but it might be difficult because of a corner
> case related with the initial data sync.
> 
> First of all, I have made last_applied to update when
> 
> * transactions are committed, prepared, or aborted
> * apply worker receives keepalive message.

Yeah, I vagurly thought that it is enough that the update happens just
befor existing send_feecback() calls. But it turned out to introduce
another unprincipledness..

> I thought during the initial data sync, we must not update the last applied
> triggered by keepalive messages, so following lines were added just after
> updating last_received.
> 
> ```
> +                                               if (last_applied < end_lsn && AllTablesyncsReady())
> +                                                       last_applied = end_lsn;
> ```

Maybe, the name "last_applied" made you confused. As I mentioned in
another message, the variable points to the remote LSN of last
"processed" 'w/k' messages.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center