Date: Mon, 23 May 2022 13:33:23 +0900 (JST)
Message-Id: <20220523.133323.444599955292141474.horikyota.ntt@gmail.com>
To: amit.kapila16@gmail.com
Cc: sawada.mshk@gmail.com, bdrouvot@amazon.com,
 pgsql-hackers@lists.postgresql.org, minsoo@amazon.com
Subject: Re: [BUG] Logical replication failure "ERROR: could not map
 filenode "base/13237/442428" to relation OID" with catalog modifying txns
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
In-Reply-To: 
 <CAA4eK1L_e4r4=10TTzoOiWGQ6hH4vxWAwD=0aofz04czCTV0kQ@mail.gmail.com>
References: <20211008.165055.1621145185927268721.horikyota.ntt@gmail.com>
	<CAD21AoD00wV4gt-53ze+ZB8n4bqJrdH8J_UnDHddy8S2A+a25g@mail.gmail.com>
	<CAA4eK1L_e4r4=10TTzoOiWGQ6hH4vxWAwD=0aofz04czCTV0kQ@mail.gmail.com>
User-Agent: Mew version 6.8 on Emacs 26.1
Mime-Version: 1.0
Content-Type: Text/Plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/20220523.133323.444599955292141474.horikyota.ntt%40gmail.com>
Precedence: bulk

At Sat, 21 May 2022 15:35:58 +0530, Amit Kapila <amit.kapila16@gmail.com> wrote in 
> I think if we don't have any better ideas then we should go with
> either this or one of the other proposals in this thread. The other
> idea that occurred to me is whether we can somehow update the snapshot
> we have serialized on disk about this information. On each
> running_xact record when we serialize the snapshot, we also try to
> purge the committed xacts (via SnapBuildPurgeCommittedTxn). So, during
> that we can check if there are committed xacts to be purged and if we
> have previously serialized the snapshot for the prior running xact
> record, if so, we can update it with the list of xacts that have
> catalog changes. If this is feasible then I think we need to somehow
> remember the point where we last serialized the snapshot (maybe by
> using builder->last_serialized_snapshot). Even, if this is feasible we
> may not be able to do this in back-branches because of the disk-format
> change required for this.
> 
> Thoughts?

I didn't look it closer, but it seems to work. I'm not sure how much
spurious invalidations at replication start impacts on performance,
but it is promising if the impact is significant.  That being said I'm
a bit negative for doing that in post-beta1 stage.

I thought for a moment that RUNNING_XACT might be able to contain
invalidation information but it seems too complex to happen with such
a frequency..

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center