Date: Tue, 13 Sep 2022 16:30:06 +0900 (JST)
Message-Id: <20220913.163006.1069510603405463109.horikyota.ntt@gmail.com>
To: dilipbalaut@gmail.com
Cc: andres@anarazel.de, jchampion@timescale.com, y.sokolov@postgrespro.ru,
 rjuju123@gmail.com, pgsql-hackers@lists.postgresql.org
Subject: Re: Error "initial slot snapshot too large" in create replication
 slot
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
In-Reply-To: <20220913.161059.331955937797746838.horikyota.ntt@gmail.com>
References: <20220913.152237.338218493796303493.horikyota.ntt@gmail.com>
	<CAFiTN-tdHJ1ereRR=jXJgcBr2rywwNrU9y6Jf0cGGv8cmTjWJQ@mail.gmail.com>
	<20220913.161059.331955937797746838.horikyota.ntt@gmail.com>
User-Agent: Mew version 6.8 on Emacs 26.1
Mime-Version: 1.0
Content-Type: Text/Plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Archived-At: 
 <https://www.postgresql.org/message-id/20220913.163006.1069510603405463109.horikyota.ntt%40gmail.com>
Precedence: bulk

At Tue, 13 Sep 2022 16:10:59 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in 
> At Tue, 13 Sep 2022 12:08:18 +0530, Dilip Kumar <dilipbalaut@gmail.com> wrote in 
> > On Tue, Sep 13, 2022 at 11:52 AM Kyotaro Horiguchi
> > <horikyota.ntt@gmail.com> wrote:
> > > That function is called after the SnapBuild reaches
> > > SNAPBUILD_CONSISTENT state ,or SnapBuildInitialSnapshot() rejects
> > > other than that state. That is, IIUC the top-sub relationship of all
> > > the currently running transactions is fully known to reorder buffer.
> > > We need a comment about that.
> > 
> > I don't think this assumption is true, any xid started after switching
> > to the SNAPBUILD_FULL_SNAPSHOT and before switching to the
> > SNAPBUILD_CONSISTENT, might still be in progress so we can not
> > identify whether they are subxact or not from reorder buffer.
> 
> Yeah, I misunderstood that the relationship is recorded earlier
> (how?).  Thus it is not reliable in the first place.
> 
> I agree that the best way is oversized xip. 
> 
> 
> By the way, I feel that "is >= than" is redundant or plain wrong..

By the way GetSnapshotData() does this:

>		snapshot->subxip = (TransactionId *)
>			malloc(GetMaxSnapshotSubxidCount() * sizeof(TransactionId));
...
>	if (!snapshot->takenDuringRecovery)
...
>	else
>	{
>		subcount = KnownAssignedXidsGetAndSetXmin(snapshot->subxip, &xmin,
>												  xmax);

It is possible that the subxip is overrun. We need to expand the array
somehow. Or assign the array of the size (GetMaxSnapshotXidCount() +
GetMaxSnapshotSubxidCount()) for takenDuringRecovery snapshots.

(I feel deja vu..)

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center