From: Zhijie Hou <houzhijie22@gmail.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
To: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>
Cc: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Cc: Chao Li <li.evan.chao@gmail.com>
Cc: Fujii Masao <masao.fujii@gmail.com>
Cc: PostgreSQL-development <pgsql-hackers@postgresql.org>
Subject: Re: Add a hook for handling logical decoding messages on subscribers.
Date: Mon, 16 Nov 2026 01:13:15 +0800
Message-ID: <4d9e1d2f-27a2-40ec-92c8-4b5740736cf8@gmail.com> (raw)
In-Reply-To: <CAD21AoDQdj2xouprHesGvSFGVwZYJJp=qZfgUL6m_HLdEMvaoQ@mail.gmail.com>
References: <CALj2ACVj+khUL=8Y3ukqJ6uF8cr0kmY6Q++fxJKYZ4P4bww=AA@mail.gmail.com>
<CAD21AoAP9iYrJh4HkPYBfiQAncyw-ch7uiyXyeQ=yyyxU-V8xQ@mail.gmail.com>
<CALj2ACVuHCZJ2vdrbB22LG77gCjwsw2DYcqNEy8GuHuKe_1Agg@mail.gmail.com>
<CAD21AoD5rwBPQv-_dGZEdsgzL7H1KywpsAd5HD3-DWKGQ5oTeQ@mail.gmail.com>
<b9b5bd27-6416-48b4-8f73-c5a60e2c97e2@gmail.com>
<CAA4eK1J1Jdz6DunCPWJ2KCpGbSERxgv+xydXkTa+19GgG5dPOA@mail.gmail.com>
<CAD21AoAhdXE1+i_bfGx3H=MheEizysk8uRirkZ3Y-v=AJzVvYw@mail.gmail.com>
<CAD21AoC-pOvm54YzNiihFS90kSDRWr=qG6BE-9HW-YW=tqRubw@mail.gmail.com>
<CAD21AoAWtvvHi1F2kSU2wOvNxfSDzQWNXYW4yL4B4Ttoq-Ek7A@mail.gmail.com>
<CAA4eK1K3ypFDLk1R1DEx7YqCRbjNLTtJPP5+FfcX0F63rQ+Ekg@mail.gmail.com>
<avWPil2vQXgXQBn+@bdtpg>
<CAD21AoDQdj2xouprHesGvSFGVwZYJJp=qZfgUL6m_HLdEMvaoQ@mail.gmail.com>
Hi,
On Sat, Nov14, 2026 at 07:35 AM Masahiko Sawada wrote:
>
> Agreed. It's better for callbacks to get both top-level and
> subtransaction's ids.
>
> I've updated the patches (the 0002 patch unchanged).
> - ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
> + ctx->write_xid = xid;
I think it would be better to keep the original code for consistency, as
all the other callbacks set ctx->write_xid to txn->xid (the top-level
transaction's XID). IIUC, this was changed to make the test pass since
test_decoding does not otherwise output the subxid, but we could instead
use pgoutput to test that and keep passing the top-level XID in
ctx->write_xid. Please see the attached top-up patch for this change.
BTW, since we are changing the callback signature anyway, another wild
idea would be to pass ReorderBufferChange directly instead so that it is
fully consistent with the other change callbacks. That said, it would
require more changes in third-party output plugins, so I don't insist on
it if you disagree — just sharing for reference.
--
Best Regards,
Zhijie Hou
Attachments:nocfbot-0001-Top-up-for-v8-0001-Fix-streamed-block-XID-in-stream_.patch (text/plain, 5.2K) download
From 952f55c52d9706be6f34f9c2554f4bd748f845f2 Mon Sep 17 00:00:00 2001
From: Zhijie Hou <houzhijie22@gmail.com>
Date: Mon, 16 Nov 2026 00:41:24 +0800
Subject: [PATCH] Top-up for v8-0001: Fix streamed block XID in
stream_message_cb_wrapper.
Author: Zhijie Hou <houzhijie22@gmail.com>
---
contrib/test_decoding/expected/stream.out | 31 ++++++++++++++++++-----
contrib/test_decoding/sql/stream.sql | 21 ++++++++++-----
src/backend/replication/logical/logical.c | 2 +-
3 files changed, 41 insertions(+), 13 deletions(-)
diff --git a/contrib/test_decoding/expected/stream.out b/contrib/test_decoding/expected/stream.outindex f34a07ba761..d589c7a2b3f 100644--- a/contrib/test_decoding/expected/stream.out+++ b/contrib/test_decoding/expected/stream.out@@ -159,6 +159,13 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
-- Verify that a transactional message emitted in a streamed subtransaction
-- is associated with the subtransaction's XID.
+CREATE PUBLICATION stream_pub;+SELECT 'init' FROM pg_create_logical_replication_slot('regression_pgoutput_slot', 'pgoutput');+ ?column? +----------+ init+(1 row)+
SET debug_logical_replication_streaming = immediate;
BEGIN;
SAVEPOINT sp;
@@ -170,21 +177,33 @@ SELECT 'msg_subxact' FROM pg_logical_emit_message(true, 'test', 'msg');
ROLLBACK TO SAVEPOINT sp;
COMMIT;
+-- Check that the streamed MESSAGE ('M') carries the aborted subtransaction's+-- XID rather than the top-level XID: in pgoutput's wire format, 'M' stores+-- its 4-byte XID at bytes 2..5, while STREAM ABORT ('A') stores the top-level+-- XID at bytes 2..5 and the aborted subtransaction's XID at bytes 6..9.
WITH changes AS (
- SELECT xid, data- FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL,- 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1')+ SELECT data+ FROM pg_logical_slot_get_binary_changes('regression_pgoutput_slot', NULL, NULL,+ 'proto_version', '2', 'publication_names', 'stream_pub',+ 'messages', 'true', 'streaming', 'on')
)
-SELECT m.xid = a.xid AS subxid_matches+SELECT substr(m.data, 2, 4) = substr(a.data, 6, 4) AS subxid_matches
FROM changes m, changes a
-WHERE m.data LIKE 'streaming message:%'- AND a.data = 'aborting streamed (sub)transaction';+WHERE get_byte(m.data, 0) = ascii('M')+ AND get_byte(a.data, 0) = ascii('A');
subxid_matches
----------------
t
(1 row)
RESET debug_logical_replication_streaming;
+SELECT pg_drop_replication_slot('regression_pgoutput_slot');+ pg_drop_replication_slot +--------------------------+ +(1 row)++DROP PUBLICATION stream_pub;
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
pg_drop_replication_slot
diff --git a/contrib/test_decoding/sql/stream.sql b/contrib/test_decoding/sql/stream.sqlindex ec68efed421..6cef3c83b09 100644--- a/contrib/test_decoding/sql/stream.sql+++ b/contrib/test_decoding/sql/stream.sql@@ -84,6 +84,8 @@ SELECT data FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL, 'inc
-- Verify that a transactional message emitted in a streamed subtransaction
-- is associated with the subtransaction's XID.
+CREATE PUBLICATION stream_pub;+SELECT 'init' FROM pg_create_logical_replication_slot('regression_pgoutput_slot', 'pgoutput');
SET debug_logical_replication_streaming = immediate;
BEGIN;
SAVEPOINT sp;
@@ -91,16 +93,23 @@ SELECT 'msg_subxact' FROM pg_logical_emit_message(true, 'test', 'msg');
ROLLBACK TO SAVEPOINT sp;
COMMIT;
+-- Check that the streamed MESSAGE ('M') carries the aborted subtransaction's+-- XID rather than the top-level XID: in pgoutput's wire format, 'M' stores+-- its 4-byte XID at bytes 2..5, while STREAM ABORT ('A') stores the top-level+-- XID at bytes 2..5 and the aborted subtransaction's XID at bytes 6..9.
WITH changes AS (
- SELECT xid, data- FROM pg_logical_slot_get_changes('regression_slot', NULL, NULL,- 'include-xids', '0', 'skip-empty-xacts', '1', 'stream-changes', '1')+ SELECT data+ FROM pg_logical_slot_get_binary_changes('regression_pgoutput_slot', NULL, NULL,+ 'proto_version', '2', 'publication_names', 'stream_pub',+ 'messages', 'true', 'streaming', 'on')
)
-SELECT m.xid = a.xid AS subxid_matches+SELECT substr(m.data, 2, 4) = substr(a.data, 6, 4) AS subxid_matches
FROM changes m, changes a
-WHERE m.data LIKE 'streaming message:%'- AND a.data = 'aborting streamed (sub)transaction';+WHERE get_byte(m.data, 0) = ascii('M')+ AND get_byte(a.data, 0) = ascii('A');
RESET debug_logical_replication_streaming;
+SELECT pg_drop_replication_slot('regression_pgoutput_slot');+DROP PUBLICATION stream_pub;
DROP TABLE stream_test;
SELECT pg_drop_replication_slot('regression_slot');
diff --git a/src/backend/replication/logical/logical.c b/src/backend/replication/logical/logical.cindex 44a27d0b653..95760981b26 100644--- a/src/backend/replication/logical/logical.c+++ b/src/backend/replication/logical/logical.c@@ -1674,7 +1674,7 @@ stream_message_cb_wrapper(ReorderBuffer *cache, ReorderBufferTXN *txn,
/* set output state */
ctx->accept_writes = true;
- ctx->write_xid = xid;+ ctx->write_xid = txn != NULL ? txn->xid : InvalidTransactionId;
ctx->write_location = message_lsn;
ctx->end_xact = false;
--
2.54.0 (Apple Git-157)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-hackers@postgresql.org
Cc: houzhijie22@gmail.com, sawada.mshk@gmail.com, bertranddrouvot.pg@gmail.com, amit.kapila16@gmail.com, bharath.rupireddyforpostgres@gmail.com, li.evan.chao@gmail.com, masao.fujii@gmail.com
Subject: Re: Add a hook for handling logical decoding messages on subscribers.
In-Reply-To: <4d9e1d2f-27a2-40ec-92c8-4b5740736cf8@gmail.com>
References: <CALj2ACVj+khUL=8Y3ukqJ6uF8cr0kmY6Q++fxJKYZ4P4bww=AA@mail.gmail.com> <CAD21AoAP9iYrJh4HkPYBfiQAncyw-ch7uiyXyeQ=yyyxU-V8xQ@mail.gmail.com> <CALj2ACVuHCZJ2vdrbB22LG77gCjwsw2DYcqNEy8GuHuKe_1Agg@mail.gmail.com> <CAD21AoD5rwBPQv-_dGZEdsgzL7H1KywpsAd5HD3-DWKGQ5oTeQ@mail.gmail.com> <b9b5bd27-6416-48b4-8f73-c5a60e2c97e2@gmail.com> <CAA4eK1J1Jdz6DunCPWJ2KCpGbSERxgv+xydXkTa+19GgG5dPOA@mail.gmail.com> <CAD21AoAhdXE1+i_bfGx3H=MheEizysk8uRirkZ3Y-v=AJzVvYw@mail.gmail.com> <CAD21AoC-pOvm54YzNiihFS90kSDRWr=qG6BE-9HW-YW=tqRubw@mail.gmail.com> <CAD21AoAWtvvHi1F2kSU2wOvNxfSDzQWNXYW4yL4B4Ttoq-Ek7A@mail.gmail.com> <CAA4eK1K3ypFDLk1R1DEx7YqCRbjNLTtJPP5+FfcX0F63rQ+Ekg@mail.gmail.com> <avWPil2vQXgXQBn+@bdtpg> <CAD21AoDQdj2xouprHesGvSFGVwZYJJp=qZfgUL6m_HLdEMvaoQ@mail.gmail.com> <4d9e1d2f-27a2-40ec-92c8-4b5740736cf8@gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox