Date: Wed, 25 Jan 2023 10:45:25 +0900 (JST)
Message-Id: <20230125.104525.433115861591453756.horikyota.ntt@gmail.com>
To: kuroda.hayato@fujitsu.com
Cc: amit.kapila16@gmail.com, osumi.takamichi@fujitsu.com,
 smithpb2250@gmail.com, vignesh21@gmail.com, shveta.malik@gmail.com,
 dilipbalaut@gmail.com, euler@eulerto.com, m.melihmutlu@gmail.com,
 andres@anarazel.de, marcos@f10.com.br, pgsql-hackers@postgresql.org
Subject: Re: Time delayed LR (WAS Re: logical replication restrictions)
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
In-Reply-To: <TYAPR01MB5866ECFC52FAED493007A2CEF5C99@TYAPR01MB5866.jpnprd01.prod.outlook.com>
References: <20230124.114535.1107425862791533999.horikyota.ntt@gmail.com>
	<CAA4eK1JTTCcMrsMAmz9LSEE6gvGHnTK0u1w0kg9hV6rcFFB5hQ@mail.gmail.com>
	<TYAPR01MB5866ECFC52FAED493007A2CEF5C99@TYAPR01MB5866.jpnprd01.prod.outlook.com>
User-Agent: Mew version 6.8 on Emacs 26.1
Mime-Version: 1.0
Content-Type: Text/Plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Archived-At: <https://www.postgresql.org/message-id/20230125.104525.433115861591453756.horikyota.ntt%40gmail.com>
Precedence: bulk

Sorry for making you bothered by this.

At Tue, 24 Jan 2023 10:12:40 +0000, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com> wrote in 
> > > Couldn't we maintain an additional static variable "last_applied"
> > > along with last_received?
> > >
> > 
> > It won't be easy to maintain the meaning of last_applied because there
> > are cases where we don't apply the change directly. For example, in
> > case of streaming xacts, we will just keep writing it to the file,
> > now, say, due to some reason, we have to send the feedback, then it
> > will not allow you to update the latest write locations. This would
> > then become different then what we are doing without the patch.
> > Another point to think about is that we also need to keep the variable
> > updated for keep-alive ('k') messages even though we don't apply
> > anything in that case. Still, other cases to consider are where we
> > have mix of streaming and non-streaming transactions.
> 
> I have tried to implement that, but it might be difficult because of a corner
> case related with the initial data sync.
> 
> First of all, I have made last_applied to update when
> 
> * transactions are committed, prepared, or aborted
> * apply worker receives keepalive message.

Yeah, I vagurly thought that it is enough that the update happens just
befor existing send_feecback() calls. But it turned out to introduce
another unprincipledness..

> I thought during the initial data sync, we must not update the last applied
> triggered by keepalive messages, so following lines were added just after
> updating last_received.
> 
> ```
> +                                               if (last_applied < end_lsn && AllTablesyncsReady())
> +                                                       last_applied = end_lsn;
> ```

Maybe, the name "last_applied" made you confused. As I mentioned in
another message, the variable points to the remote LSN of last
"processed" 'w/k' messages.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center