Message-ID: <755c7bb23ad97a4a5319ebd13cdf8afccf5f05ed.camel@postgrespro.ru>
Subject: Re: Error "initial slot snapshot too large" in create replication
 slot
From: Yura Sokolov <y.sokolov@postgrespro.ru>
To: Dilip Kumar <dilipbalaut@gmail.com>, Kyotaro Horiguchi
	 <horikyota.ntt@gmail.com>
Cc: Julien Rouhaud <rjuju123@gmail.com>, PostgreSQL Hackers
	 <pgsql-hackers@lists.postgresql.org>
Date: Sun, 13 Feb 2022 17:35:38 +0300
In-Reply-To: <CAFiTN-s_PYicXeb6+uZkERqATpZLzpfsg0qP6G_GMPhu=rgxUA@mail.gmail.com>
References: 
	<CAFiTN-ui+vtbUq_-3+SzT4KU_=iAJz7_-en+_tnPz-p34h76mQ@mail.gmail.com>
	 <20220112103855.ib4455yyebaxkuwp@jrouhaud>
	 <CAFiTN-vb_Mv+rRVDWtagvdkow-eWaT_xsGLSDiPfzL41UKiDwQ@mail.gmail.com>
	 <20220131.143158.896330709668373625.horikyota.ntt@gmail.com>
	 <20220131.152011.148738317202375552.horikyota.ntt@gmail.com>
	 <CAFiTN-s_PYicXeb6+uZkERqATpZLzpfsg0qP6G_GMPhu=rgxUA@mail.gmail.com>
Organization: PostgresPro
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.36.5-0ubuntu1 
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Archived-At: <https://www.postgresql.org/message-id/755c7bb23ad97a4a5319ebd13cdf8afccf5f05ed.camel%40postgrespro.ru>
Precedence: bulk

В Пн, 07/02/2022 в 13:52 +0530, Dilip Kumar пишет:
> On Mon, Jan 31, 2022 at 11:50 AM Kyotaro Horiguchi
> <horikyota.ntt@gmail.com> wrote:
> > At Mon, 17 Jan 2022 09:27:14 +0530, Dilip Kumar <dilipbalaut@gmail.com> wrote in
> > 
> > me> Mmm. The size of the array cannot be larger than the numbers the
> > me> *Connt() functions return.  Thus we cannot attach the oversized array
> > me> to ->subxip.  (I don't recall clearly but that would lead to assertion
> > me> failure somewhere..)
> > 
> > Then, I fixed the v3 error and post v4.
> 
> Yeah you are right, SetTransactionSnapshot() has that assertion.
> Anyway after looking again it appears that
> GetMaxSnapshotSubxidCount is the correct size because this is
> PGPROC_MAX_CACHED_SUBXIDS +1, i.e. it considers top transactions as
> well so we don't need to add them separately.
> 
> > SnapBUildInitialSnapshot tries to store XIDS of both top and sub
> > transactions into snapshot->xip array but the array is easily
> > overflowed and CREATE_REPLICATOIN_SLOT command ends with an error.
> > 
> > To fix this, this patch is doing the following things.
> > 
> > - Use subxip array instead of xip array to allow us have larger array
> >   for xids.  So the snapshot is marked as takenDuringRecovery, which
> >   is a kind of abuse but largely reduces the chance of getting
> >   "initial slot snapshot too large" error.
> 
> Right. I think the patch looks fine to me.
> 

Good day.

I've looked to the patch. Personally I'd prefer dynamically resize
xip array. But I think there is issue with upgrade if replica source
is upgraded before destination, right?

Concerning patch, I think more comments should be written about new
usage case for `takenDuringRecovery`. May be this field should be renamed
at all?

And there are checks for `takenDuringRecovery` in `heapgetpage` and
`heapam_scan_sample_next_tuple`. Are this checks affected by the change?
Neither the preceding discussion nor commit message answer me.

-------

regards

Yura Sokolov
Postgres Professional
y.sokolov@postgrespro.ru
funny.falcon@gmail.com