Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xNtce-00000002nqH-0rlZ for pgsql-hackers@arkaria.postgresql.org; Mon, 02 Nov 2026 15:06:57 +0000 Received: from localhost ([127.0.0.1]:38160 helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.98.2) (envelope-from ) id 1xNtcd-00000004JTX-2mfo for pgsql-hackers@arkaria.postgresql.org; Mon, 02 Nov 2026 15:06:55 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]:36464) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xNtcc-00000004JTO-2Q6e for pgsql-hackers@lists.postgresql.org; Mon, 02 Nov 2026 15:06:55 +0000 Received: from oss.nttdata.com ([49.212.34.109]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.98.2) (envelope-from ) id 1xNtcY-00000002FT5-2GMx for pgsql-hackers@postgresql.org; Mon, 02 Nov 2026 15:06:54 +0000 Received: from oss.nttdata.com (oss.nttdata.com [49.212.34.109]) by oss.nttdata.com (Postfix) with ESMTPSA id 0EB9120202A5; Tue, 3 Nov 2026 00:06:43 +0900 (JST) DKIM-Filter: OpenDKIM Filter v2.11.0 oss.nttdata.com 0EB9120202A5 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oss.nttdata.com; s=s2026q4; t=1793632003; bh=zG4Q6k//uPbms4vBdu2b6hLyBjW6baPEq3lhVPu2Cdw=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=G37Qc47Hu5pDCoWFlzt2LYU+YcvPzhRvlTPRxu8rXYzBitZB8T60V70n4aBVieJ6/ /0SW/LcBjtD3ftbawo0019SFOLCj52DmzYElh7yb9zglHNNO02lUwEdYTukk6XfvbP J6WENI/OtTo3HDoPpBlnzFZb1kDhSyJWg79qGITH/L8sROM/Osv5WMMrtceGWya5wJ Djgzyss2EtIp87owErOVyVWFw1yvn5m1RoKfcDaGB1URx/RF59DyAygVlF7Kvxz93d Z5gGq+S7CjY55akiJ4NtlCr8gDrmCnRQzPuIw0OHouS61JN9IrXGUYLwiVgr2c3Cck sSpjBfuCYhExg== MIME-Version: 1.0 Date: Tue, 03 Nov 2026 00:06:42 +0900 From: torikoshia To: Masahiko Sawada Cc: Bharath Rupireddy , Amit Kapila , Chao Li , Fujii Masao , PostgreSQL-development Subject: Re: Add a hook for handling logical decoding messages on subscribers. In-Reply-To: References: <875ACE0E-20ED-4ECC-AEBF-733BFA46D97D@gmail.com> Message-ID: <281df141d2095f0c09d12fbb21cfe8e5@oss.nttdata.com> X-Sender: torikoshia@oss.nttdata.com Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Virus-Scanned: clamav-milter 1.4.6 at oss.nttdata.com X-Virus-Status: Clean List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On 2026-10-31 07:34, Masahiko Sawada wrote: > On Fri, Oct 16, 2026 at 3:16 PM Bharath Rupireddy > wrote: >> >> Hi, >> >> On Fri, Oct 16, 2026 at 2:45 PM Masahiko Sawada >> wrote: >> > >> > > > 4. Based on above two points, I think we can fill the hole with something like: >> > >> > > > CREATE SUBSCRIPTION ... WITH (messages = true, message_handler_role = >> > > > 'msg_handler') >> > >> > > > and then run the hook under SwitchToUntrustedUser(that_role). That >> > >> > > I like this idea more than all the other approaches. >> > >> > My concern is that since the handler is a C function, this doesn't >> > help much as a security boundary. The handler can get back to the >> > subscription owner by calling SetUserIdAndSecContext() with >> > MySubscription->owner or even switch to the superuser. That function >> > does no permission check, and since the security context is its second >> > argument, the same call clears SECURITY_RESTRICTED_OPERATION as well. >> > As I mentioned in the reply to Amit, the option would help in cases >> > where the handler doesn't switch to the target role, or forgets to, >> > and keeps working as the subscription owner unexpectedly. But I'm not >> > sure we need this option just for that case. >> >> Agreed. In a way, we already trust every hook as far as switching to >> whatever role with SetUserIdAndSecContext() is concerned. So >> documenting the risk clearly here works for me. >> >> > > > I think we should have a similar ERROR for parallel apply worker. >> > >> > > Also, error out for the sequence sync worker? >> > >> > I agree to add a check for parallel apply workers, but I don't think >> > we need it for the sequence sync workers as they neither enter the >> > apply loop nor receive logical replication changes in the first place. >> >> Right, we don't need to error out for seq sync workers. However, do we >> need to error out for the table sync worker, since we don't expect a >> logical message there? > > It seems to me worth adding a check for tablesync workers since both > the apply workers and the tablesync workers call > walrcv_startstreaming() and it would prevent mistakes by future > changes. > > I've attached the updated patch. Thanks for the proposal and patch! As a precaution, I checked whether this patch could cause data loss similar to the issue I previously reported with Debezium [1], where logical messages report their end LSN while ordinary changes report their start LSN. Based on my code review and some tests, I haven't found a similar issue with this patch. The issue occurred with transactional messages. Debezium tracks processing progress at the event level, including individual DML operations and logical messages. In PostgreSQL's built-in logical replication, however, the handler's changes are committed as part of the replicated transaction, and replication progress is tracked at the transaction boundary. Therefore, the difference between start and end LSNs does not appear to cause the same problem. I also wondered whether there could be a timing window in which consecutive non-transactional logical messages might lead to data loss, so I checked this case as well. It seems that the following condition in logicalmsg_decode() handles this case correctly, so I don't see a problem here either. else if (!message->transactional && (SnapBuildCurrentState(builder) != SNAPBUILD_CONSISTENT || SnapBuildXactNeedsSkip(builder, buf->origptr))) return; [1] https://www.postgresql.org/message-id/d99c688994ab3a998afe26e61fe4f69f%40oss.nttdata.com -- Thanks, -- Atsushi Torikoshi Seconded from NTT DATA CORPORATION to SRA OSS K.K.