Message-ID: <44abc443-3266-0227-4d45-4dacadd4cec9@enterprisedb.com>
Date: Sun, 3 Dec 2023 19:26:16 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.15.1
Subject: Re: logical decoding and replication of sequences, take 2
Content-Language: en-US
From: Tomas Vondra <tomas.vondra@enterprisedb.com>
To: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>,
 Amit Kapila <amit.kapila16@gmail.com>
Cc: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>,
 Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>,
 PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>,
 Masahiko Sawada <sawada.mshk@gmail.com>,
 Peter Eisentraut <peter.eisentraut@enterprisedb.com>,
 Dilip Kumar <dilipbalaut@gmail.com>
References: <c5e671bc-3400-c2f4-14d3-1ae06c462dad@enterprisedb.com>
 <CAA4eK1+fEHLn2sYS+6eZV-S8boejmo1sAUYWL4G5KeFbvmy5kA@mail.gmail.com>
 <TY3PR01MB9889DF7B06484285501A563FF5BDA@TY3PR01MB9889.jpnprd01.prod.outlook.com>
 <a101c08e-1fc8-1238-34da-bb984c9806ef@enterprisedb.com>
 <36f3a200-f185-887c-da08-fdc866436e6c@enterprisedb.com>
 <CAA4eK1LkZ+99rKhp1Nxhg9iGBppf1FgLEFmC5SntNFtc1xJFTg@mail.gmail.com>
 <195be236-7580-72c6-97c7-59f06c4e093c@enterprisedb.com>
 <4954a4f8-ba9d-d352-571b-7b94372ea5ae@enterprisedb.com>
 <CAA4eK1+dsdB_g=Y9UnBf8GQk1VnHnugbPWp3Ebt1JXtETcQUgQ@mail.gmail.com>
 <418ff5ac-1664-9038-8ac6-0dd308b5df07@enterprisedb.com>
 <e66ff0de-6f2d-baf9-aee9-ba412868df75@enterprisedb.com>
 <TY3PR01MB9889D457278B254CA87D1325F581A@TY3PR01MB9889.jpnprd01.prod.outlook.com>
 <c6a08e9d-7b08-76d4-197d-fab54039ed91@enterprisedb.com>
 <OS3PR01MB9882CCE9A9F037309078DC3DF587A@OS3PR01MB9882.jpnprd01.prod.outlook.com>
 <8a2f25d8-23d4-4ab8-2a23-7c8d2d976209@enterprisedb.com>
In-Reply-To: <8a2f25d8-23d4-4ab8-2a23-7c8d2d976209@enterprisedb.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Archived-At: <https://www.postgresql.org/message-id/44abc443-3266-0227-4d45-4dacadd4cec9%40enterprisedb.com>
Precedence: bulk

On 12/3/23 18:52, Tomas Vondra wrote:
> ...
> 
> Some time ago I floated the idea of maybe "queuing" the sequence changes
> and only replay them on the next commit, somehow. But we did ran into
> problems with which snapshot to use, that I didn't know how to solve.
> Maybe we should try again. The idea is we'd queue the non-transactional
> changes somewhere (can't be in the transaction, because we must keep
> them even if it aborts), and then "inject" them into the next commit.
> That'd mean we wouldn't do the separate start/abort for each change.
> 

Another idea is that maybe we could somehow inform ReorderBuffer whether
the output plugin even is interested in sequences. That'd help with
cases where we don't even want/need to replicate sequences, e.g. because
the publication does not specify (publish=sequence).

What happens now in that case is we call ReorderBufferQueueSequence(),
it does the whole dance with starting/aborting the transaction, calls
rb->sequence() which just does "meh" and doesn't do anything. Maybe we
could just short-circuit this by asking the output plugin somehow.

In an extreme case the plugin may not even specify the sequence
callbacks, and we're still doing all of this.


regards

-- 
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company