Content-Type: text/plain;
	charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3864.600.51.1.1\))
Subject: Re: Add a hook for handling logical decoding messages on subscribers.
From: Chao Li <li.evan.chao@gmail.com>
In-Reply-To: 
 <CAD21AoBqVO_fwLkvxwzibohpMG-7n+7MqgkD9qRMGTVN6e8x7g@mail.gmail.com>
Date: Tue, 4 Aug 2026 16:05:00 +0800
Cc: Fujii Masao <masao.fujii@gmail.com>,
 Amit Kapila <amit.kapila16@gmail.com>,
 Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>,
 PostgreSQL-development <pgsql-hackers@postgresql.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <875ACE0E-20ED-4ECC-AEBF-733BFA46D97D@gmail.com>
References: 
 <CAD21AoCTNGiddikkUcDKj5QLnsg-51bpr-o6L-GTHWZL4ZFYtQ@mail.gmail.com>
 <CALj2ACXLkU4y+mmk2gHzmvSV4KTishSe=HhZ_yn=NHFBDGrwFQ@mail.gmail.com>
 <CAA4eK1JC9+R4S7L7y3DazOOMpnANVyCpsx84RSYjX6vfxLdK6w@mail.gmail.com>
 <CAD21AoATy+CzYmNYp_fKi4zsGLEGOmZen3OR+hxsCqn2GsQBoQ@mail.gmail.com>
 <CAHGQGwELqH7P8PBOGE2-un8qC_oQpT2jOgbPsAjK6YA5dnbKWQ@mail.gmail.com>
 <CAD21AoBqVO_fwLkvxwzibohpMG-7n+7MqgkD9qRMGTVN6e8x7g@mail.gmail.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Archived-At: 
 <https://www.postgresql.org/message-id/875ACE0E-20ED-4ECC-AEBF-733BFA46D97D%40gmail.com>
Precedence: bulk


> On Aug 4, 2026, at 08:41, Masahiko Sawada <sawada.mshk@gmail.com> =
wrote:
>=20
> On Thu, Jul 9, 2026 at 5:14=E2=80=AFAM Fujii Masao =
<masao.fujii@gmail.com> wrote:
>>=20
>> On Wed, Jun 24, 2026 at 3:02=E2=80=AFAM Masahiko Sawada =
<sawada.mshk@gmail.com> wrote:
>>> Right. I'm implementing a basic DDL replication solution using this
>>> hook as a sample implementation. Other than that, this feature can =
be
>>> used to add additional information for the replicated transaction =
that
>>> is not replicated via logical replication protocol, such as the
>>> executed user or context-specific information, which can then be
>>> dispatched to a CDC system connected to the subscriber.
>>=20
>> +1 for the proposed hook, although I haven't read the patch yet.
>>=20
>> I know of a system that uses logical decoding messages to propagate =
data
>> to external systems. In that system, the application writes the data =
to
>> be propagated as logical decoding messages, and a CDC pipeline =
consumes
>> them via logical decoding and sends them to other systems, for =
example
>> through Kafka.
>>=20
>> In that system, when the remote site is maintained by physical =
replication,
>> the same logical decoding messages can be decoded on the standby
>> at the remote site to deliver them to external systems there. =
However,
>> if the remote site uses logical replication instead, those messages =
are
>> currently neither delivered to nor processed on the subscriber at
>> the remote site. As a result, the CDC pipeline at the remote site =
cannot
>> consume the data they carry.
>>=20
>> The proposed hook might be useful for this use case. An extension =
could
>> process the incoming logical decoding messages on the subscriber and
>> forward them to the local CDC pipeline, or store or re-emit them in a =
form
>> that local consumers can process.
>=20
> Thank you for sharing the concrete use case. This is one of the use
> cases I initially imagined.
>=20
> I've rebased and updated the patch. Please review it.
>=20
> Regards,
>=20
> --=20
> Masahiko Sawada
> Amazon Web Services: https://aws.amazon.com
> <v2-0001-Add-a-hook-for-handling-logical-messages-on-subsc.patch>

I played with this feature today. I have a few comments from a design =
perspective:

1. Would it make sense to pass the receiving subscription=E2=80=99s =
identity explicitly to the hook? Otherwise, a hook that needs to =
distinguish messages received by different subscriptions has to obtain =
it from a global variable such as MySubscription->oid.

2. A hook is invoked by an apply worker, and since a message has no =
target relation, the hook runs as the subscription owner regardless of =
run_as_owner. I think it would be useful to document this explicitly.

3. I=E2=80=99m thinking out loud here. Would it be useful to provide an =
opt-in default hook that re-emits received messages, so that they are =
written to the subscriber-side WAL and can be consumed by a local =
logical-decoding client? I understand that a re-emitted message would =
have a different LSN, just as logically replicated row changes generate =
new local WAL records. For the use case Fujii-san described, this might =
allow an existing decoder to continue working after the remote site =
switches from physical to logical replication.

And a few comment for the code changes:

1 - worker_internal.h
```
+typedef void (*LogicalRepMessageHandle_hook_type) =
(LogicalRepMessageData *msg);
+extern PGDLLIMPORT LogicalRepMessageHandle_hook_type =
LogicalRepMessageHandle_hook;
```

I guess we don=E2=80=99t expect a hook function to mutate the message, =
so maybe make the pointer const.

2 - worker.c
```
+	/*
+	 * Logical messages are relation-agnostic, so they don't map =
cleanly onto
+	 * the tablesync worker of a particular relation, and there =
would be no
+	 * well-defined ordering between a message and the initial copy. =
Leave
+	 * them to the (parallel) apply workers. Logical messages are =
handled only
+	 * the (parallel) apply workers
+	 */
```

The last sentence looks duplicate and incomplete.

3 - proto.c
```
+	/* read message length */
+	len =3D pq_getmsgint(in, 4);
+	msg_data->message_size =3D len;
+
+	/* and data */
+	msg =3D palloc(len + 1);
+	pq_copymsgbytes(in, msg, len);
+
+	msg[len] =3D '\0';
+	msg_data->message =3D msg;
```

logicalrep_read_message() allocates one extra byte for NULL terminator, =
which is unnecessary, as the contract is to use msg_data->message_size =
to decide the message length, and a message contain contains 0 in the =
middle. But I agree it may be useful for debugging and logging, so maybe =
add a comment to explain why using this extra byte.

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/