MIME-Version: 1.0
References: <c5e671bc-3400-c2f4-14d3-1ae06c462dad@enterprisedb.com>
 <a101c08e-1fc8-1238-34da-bb984c9806ef@enterprisedb.com> <36f3a200-f185-887c-da08-fdc866436e6c@enterprisedb.com>
 <CAA4eK1LkZ+99rKhp1Nxhg9iGBppf1FgLEFmC5SntNFtc1xJFTg@mail.gmail.com>
 <195be236-7580-72c6-97c7-59f06c4e093c@enterprisedb.com> <4954a4f8-ba9d-d352-571b-7b94372ea5ae@enterprisedb.com>
 <CAA4eK1+dsdB_g=Y9UnBf8GQk1VnHnugbPWp3Ebt1JXtETcQUgQ@mail.gmail.com>
 <418ff5ac-1664-9038-8ac6-0dd308b5df07@enterprisedb.com> <e66ff0de-6f2d-baf9-aee9-ba412868df75@enterprisedb.com>
 <TY3PR01MB9889D457278B254CA87D1325F581A@TY3PR01MB9889.jpnprd01.prod.outlook.com>
 <c6a08e9d-7b08-76d4-197d-fab54039ed91@enterprisedb.com> <OS3PR01MB9882CCE9A9F037309078DC3DF587A@OS3PR01MB9882.jpnprd01.prod.outlook.com>
 <8a2f25d8-23d4-4ab8-2a23-7c8d2d976209@enterprisedb.com> <CAFiTN-vm3-bGfm-uJdzRLERMHozW8xjZHu4rdmtWR-rP-SJYMQ@mail.gmail.com>
 <CAFiTN-vAx-Y+19ROKOcWnGf7ix2VOTUebpzteaGw9XQyCAeK6g@mail.gmail.com>
 <b6f118e6-02b8-759c-db15-f9b25e805d65@enterprisedb.com> <CAFiTN-un8w_owWokm=425=ucSL90fHxJShHUq=Zip=mfWbt1Dw@mail.gmail.com>
 <CAA4eK1LFise9iN+NN=agrk4prR1qD+ebvzNjKAWUog2+hy3HxQ@mail.gmail.com>
 <CAFiTN-sYpyUBabxopJysqH3DAp4OZUCTi6m_qtgt8d32vDcWSA@mail.gmail.com>
 <CAExHW5szMs=Wzy1Ek00rkByQbTk57nK_UvMp+VYwT9Np6LnO5A@mail.gmail.com>
 <CAA4eK1L3rxS9zyvV-SOqOTx1Rqa_9B0UxuHWX3rHfUC7z6dc6w@mail.gmail.com> <CAExHW5uRGwr0hAC5Gr4EbA4Q=Odjk+8nFTmdsfB1Hk-9r-81DA@mail.gmail.com>
In-Reply-To: <CAExHW5uRGwr0hAC5Gr4EbA4Q=Odjk+8nFTmdsfB1Hk-9r-81DA@mail.gmail.com>
From: Amit Kapila <amit.kapila16@gmail.com>
Date: Thu, 14 Dec 2023 14:50:54 +0530
Message-ID: <CAA4eK1+pkuJhGZNA9RHOGDBUF2hhmyUr6gVCOSCTZ3wDuGbzdg@mail.gmail.com>
Subject: Re: logical decoding and replication of sequences, take 2
To: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Cc: Dilip Kumar <dilipbalaut@gmail.com>, Tomas Vondra <tomas.vondra@enterprisedb.com>, 
	"Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, 
	PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, 
	Masahiko Sawada <sawada.mshk@gmail.com>, 
	Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Archived-At: <https://www.postgresql.org/message-id/CAA4eK1%2BpkuJhGZNA9RHOGDBUF2hhmyUr6gVCOSCTZ3wDuGbzdg%40mail.gmail.com>
Precedence: bulk

On Thu, Dec 14, 2023 at 2:45=E2=80=AFPM Ashutosh Bapat
<ashutosh.bapat.oss@gmail.com> wrote:
>
> On Thu, Dec 14, 2023 at 12:37=E2=80=AFPM Dilip Kumar <dilipbalaut@gmail.c=
om> wrote:
> >
> > I think you forgot to attach the patch.
>
> Sorry. Here it is.
>
> On Thu, Dec 14, 2023 at 2:36=E2=80=AFPM Amit Kapila <amit.kapila16@gmail.=
com> wrote:
> >
> > >
> > It looks like the solution works. But this is the only place where we
> > process a change before SNAPSHOT reaches FULL. But this is also the
> > only record which affects a decision to queue/not a following change.
> > So it should be ok. The sequence_hash'es as separate for each
> > transaction and they are cleaned when processing COMMIT record.
> > >
> >
> > But it is possible that even commit or abort also happens before the
> > snapshot reaches full state in which case the hash table will have
> > stale or invalid (for aborts) entries. That will probably be cleaned
> > at a later point by running_xact records.
>
> Why would cleaning wait till running_xact records? Won't txn entry
> itself be removed when processing commit/abort record? At the same the
> sequence hash will be cleaned as well.
>
> > Now, I think in theory, it
> > is possible that the same RelFileLocator can again be allocated before
> > we clean up the existing entry which can probably confuse the system.
>
> How? The transaction allocating the first time would be cleaned before
> it happens the second time. So shouldn't matter.
>

It can only be cleaned if we process it but xact_decode won't allow us
to process it and I don't think it would be a good idea to add another
hack for sequences here. See below code:

xact_decode(LogicalDecodingContext *ctx, XLogRecordBuffer *buf)
{
SnapBuild  *builder =3D ctx->snapshot_builder;
ReorderBuffer *reorder =3D ctx->reorder;
XLogReaderState *r =3D buf->record;
uint8 info =3D XLogRecGetInfo(r) & XLOG_XACT_OPMASK;

/*
* If the snapshot isn't yet fully built, we cannot decode anything, so
* bail out.
*/
if (SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT)
return;


--=20
With Regards,
Amit Kapila.