Message-ID: <b6f118e6-02b8-759c-db15-f9b25e805d65@enterprisedb.com>
Date: Wed, 6 Dec 2023 14:39:42 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.15.1
Subject: Re: logical decoding and replication of sequences, take 2
To: Dilip Kumar <dilipbalaut@gmail.com>
Cc: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>,
 Amit Kapila <amit.kapila16@gmail.com>,
 "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>,
 Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
 Masahiko Sawada <sawada.mshk@gmail.com>,
 Peter Eisentraut <peter.eisentraut@enterprisedb.com>
References: <c5e671bc-3400-c2f4-14d3-1ae06c462dad@enterprisedb.com>
 <a101c08e-1fc8-1238-34da-bb984c9806ef@enterprisedb.com>
 <36f3a200-f185-887c-da08-fdc866436e6c@enterprisedb.com>
 <CAA4eK1LkZ+99rKhp1Nxhg9iGBppf1FgLEFmC5SntNFtc1xJFTg@mail.gmail.com>
 <195be236-7580-72c6-97c7-59f06c4e093c@enterprisedb.com>
 <4954a4f8-ba9d-d352-571b-7b94372ea5ae@enterprisedb.com>
 <CAA4eK1+dsdB_g=Y9UnBf8GQk1VnHnugbPWp3Ebt1JXtETcQUgQ@mail.gmail.com>
 <418ff5ac-1664-9038-8ac6-0dd308b5df07@enterprisedb.com>
 <e66ff0de-6f2d-baf9-aee9-ba412868df75@enterprisedb.com>
 <TY3PR01MB9889D457278B254CA87D1325F581A@TY3PR01MB9889.jpnprd01.prod.outlook.com>
 <c6a08e9d-7b08-76d4-197d-fab54039ed91@enterprisedb.com>
 <OS3PR01MB9882CCE9A9F037309078DC3DF587A@OS3PR01MB9882.jpnprd01.prod.outlook.com>
 <8a2f25d8-23d4-4ab8-2a23-7c8d2d976209@enterprisedb.com>
 <CAFiTN-vm3-bGfm-uJdzRLERMHozW8xjZHu4rdmtWR-rP-SJYMQ@mail.gmail.com>
 <CAFiTN-vAx-Y+19ROKOcWnGf7ix2VOTUebpzteaGw9XQyCAeK6g@mail.gmail.com>
Content-Language: en-US
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
In-Reply-To: <CAFiTN-vAx-Y+19ROKOcWnGf7ix2VOTUebpzteaGw9XQyCAeK6g@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Archived-At: <https://www.postgresql.org/message-id/b6f118e6-02b8-759c-db15-f9b25e805d65%40enterprisedb.com>
Precedence: bulk

On 12/6/23 10:05, Dilip Kumar wrote:
> On Wed, Dec 6, 2023 at 11:12 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>>
>> On Sun, Dec 3, 2023 at 11:22 PM Tomas Vondra
>> <tomas.vondra@enterprisedb.com> wrote:
>>>
> 
> I was also wondering what happens if the sequence changes are
> transactional but somehow the snap builder state changes to
> SNAPBUILD_FULL_SNAPSHOT in between processing of the smgr_decode() and
> the seq_decode() which means RelFileLocator will not be added to the
> hash table and during the seq_decode() we will consider the change as
> non-transactional.  I haven't fully analyzed that what is the real
> problem in this case but have we considered this case? what happens if
> the transaction having both ALTER SEQUENCE and nextval() gets aborted
> but the nextva() has been considered as non-transactional because
> smgr_decode() changes were not processed because snap builder state
> was not yet SNAPBUILD_FULL_SNAPSHOT.
> 

Yes, if something like this happens, that'd be a problem:

1) decoding starts, with

   SnapBuildCurrentState(builder) < SNAPBUILD_FULL_SNAPSHOT

2) transaction that creates a new refilenode gets decoded, but we skip
   it because we don't have the correct snapshot

3) snapshot changes to SNAPBUILD_FULL_SNAPSHOT

4) we decode sequence change from nextval() for the sequence

This would lead to us attempting to apply sequence change for a
relfilenode that's not visible yet (and may even get aborted).

But can this even happen? Can we start decoding in the middle of a
transaction? How come this wouldn't affect e.g. XLOG_HEAP2_NEW_CID,
which is also skipped until SNAPBUILD_FULL_SNAPSHOT. Or logical
messages, where we also call the output plugin in non-transactional cases.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company