Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qFHa7-0005Iv-PK for pgsql-general@arkaria.postgresql.org; Fri, 30 Jun 2023 17:06:52 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1qFHa6-0006J0-Lo for pgsql-general@arkaria.postgresql.org; Fri, 30 Jun 2023 17:06:50 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qFHa5-0006Ij-Ut for pgsql-general@lists.postgresql.org; Fri, 30 Jun 2023 17:06:50 +0000 Received: from sonic308-20.consmr.mail.sg3.yahoo.com ([106.10.241.210]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1qFHa2-001JU3-UI for pgsql-general@lists.postgresql.org; Fri, 30 Jun 2023 17:06:48 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1688144802; bh=ErXujM7XxQDcAbLXsRIlYMEqg+IiEk4v85W1uMJ8dBk=; h=Date:From:To:Subject:References:From:Subject:Reply-To; b=X15DeIb+1j8od2Uyu7Ypfpd5a+43lPUmWXnFXwcPyNf10rSeMgZqipVgcFsazOyKDQxcdneHCoHWBoeD+cXSBCmmfkZ8rLCg1nAaTpLr63XKI+r9lBa0kUl018yxWL9nugyPTMl3awEaHt3Il6y//5op5NS6x2GR9wAjgNrf41t8fWFvvLWUsx2dR+Fug4dbyYSh3/ytuwXRs5jxrzMhYNQKWF1Uzlw3P8ndTMaoIThlTvwaDzt4Ifa/ZdiYgDH4ZjzdT7pQftq2JlBwSbpf1CE8TlrM6JXYtk+Feg4+MLevpvP5bOgXIR8NFJEeCO7ILSbMyTHq5ZYu0z5tlKvSQg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1688144802; bh=Mx/3AFF4DvCVhEuM8Ae6+UpPnbCBzImfFWsP9m5sl37=; h=X-Sonic-MF:Date:From:To:Subject:From:Subject; b=j956wqTe+3nXGu9+Qfn6mCuHTS7Ahu9V47KvMKVDRXlUNY1BKsrmdwTWmXd993I9vBtHFJqU2Ed2epxMHPRAPrpO0Tm8U5v1cT4zH4akD1DoYc62X85p+DILW3u1cmod7n899EzRkIPcE3ffbarlWZyWBPoq+mAYqBFLqd0LtqpN1cSG9txCcwOLr0mvBOo3dx5iDR+pfT1tQJA1cAhU5J4gpg4jB/yacG7EjVYThbAU5qFmTgOSgYdgVheXGefZwIaen7VWQXjfnspbFAybeEdxiY6Y/YEKNgpD7+BHhABLz9D2nJ63U28ShGSaNgjMKxtf3DUukq38DKx9yENBPQ== X-YMail-OSG: 4NXCNYwVM1mjYenHxy1cT3eONhG2XRhI_GxI0cbxlq4E1NCzx0QV04ADP1Pot7D lMObm.l_g6pifc.QMY4xrQbGmRWIv7s_ofBZ23kYAiwhrIfo2QARpkOX6U8KBOrPmKYVcE7Syy4q 0SS1YteQp4EGKdVzeGyP6oeCExoy6s_Zfv64NYHWUO.Yn4hV4NMgi3bZWvfF.vHIpCaKmrAV4pt0 Abe48lRdkSLu7qg9MFbggsmWqyTDE_BHz8FZ3C67lfImRkwHPUWBQ1yU2Q_akiM_lu2D.2hrmGM6 jbzXJM00uOF3SuU6qZGxFdPCehf3_X9gD10oT0B04qIdqxfv1qyTsRlS7CwMxgFs5u1YLeqtDwid EkjhRGJklgfqgPBItAgo8N.mQc1M6WIg51qyngMLxI99jUFumEd20.HQ10NPikBFIX7wBwWUZsg6 yfuzLrjbV6i7XPFc6syWlZPcppXLVBy9SCkAhlb5wHXWvZvQ8sWM9kl38xIzoVHh74uXg_yg6vsF REwqEMmX1vCqO6roH3k1K6J4ttoeEIt79vFn8OpMnGG5QNUTo.ibsHGQRaEy3T6q9ZZs4sxZzPNi jpEgl.WBv_inRMRwuz2X_MKn.DwZENSk2jFNVJk1w9qCcIZabpOuRtav1cD2C31x0Y1vVFUSeZU1 4dySAiSiM_cH.nwaoDSKDlNshxGVPhgK9176HOqrkVEV3Zvyq0sTKnOg0X1iN4i_VIAfM6sCVwPR feqpEuMbgkjQ3g_O47Md6mbCVbgBusCBiMlOtNqtWaIQsphahuq6dkVGzK._S4_dDRvlKM51xc63 jVt.G0WmBAW5dc5ccjjKQR8Tzh_fbcQV5bZj4SzZIRIKkKdF4UdQgkAXPxTfu3FqVq73C_6cfj3n V_spX34T7F2diQXMrr5Egs.ykA3js_hutlRTpjy5bWoxer3gpg8ysJWHzdtyTt5QLQ3AmAe04VgJ ofCajq.sGMJavQ7E9G45LhNh60EzmCLKv2L4apoVbswS6CW8Wiurrypsv6MSLb5rC9NrNdTrc5l5 hEh3RMg9aRveMSHOGZ7uZNXkesGGk2dn2CzP7a7818iK.IAKKUxBsnCZf1RRuy7uTeQyX6GxVF4x edW4w5xwbxYbnxGjuGP.wbvTIOfZGy67M02Z_56QM1Cb6xoRRb7Y34lu77vIH7ZWd.eeRNF3O3b4 bZ8np9QULWn4DpGSmp64lyXVv3gNpzHppLknhrg0YdLXazfj4sCuOuqnTw6B0vvAOlY2musOxbXK oe8CzUjp0ROr33SuPMUpcjtCZQIOdiJ1RESbYHJI88MuZNvjDVPsjcc9jUT0kcdzYkZgTezfD8K5 ee2.J8wpf1HOEWavVnDBf5xy6MR8S38kU_2QmynR29pFrNaAVoB8zuxS8n7anqt9Fy.lRK41vxOo 4LFKuv_Vr9H9hI3yXMV4xnf8IimUVsabeVVoqS3MoBqSQPzyOxPMipMslX5872aYYdCyFt371F8e Hajh86Ire_o_MWqhUwZY5dBegxtGaYtiek.xM.G2EKKdrSaz7I5k1Nrc1NUE.5PaSKPPNAYDRY7a ZFw_C5acw_5ag62b47X1Tk4opNbQ5C_G7U27juRYBjsr7FsIlh6MMynL9cDU70LXxsvzRr5EnFJu OvnM6n884OHrFo1KSekMhl2LbkqD6CMqqCTIQKrJfnczss47Auaehe4fEQefXEa6jxLcvGnYv.7z zgWBt_fnZXeAnCPTqo1w1RyKb5Ev0pJGcQaO1sv671o8jlk9W3Bbp39K3Pq_Nomyhu6S.Xb9lW51 lSKKjWLsDY8Q3eHn11Knft3XeMhaJ1WAlMvrWXCZYYrG_mQdXuZd5yVs7dB2UVaW95v7BFyZ2Eba HHGNBwGPrT3OTyphsgx7dxd11EAaanGLiCSkGWU08lDd3Sz9PeXbbFUfVzxstrLVMfXwtEbjyx6Q _.NxZR8PxssmQ4mcSyXjHgvSCZGT_nrA_Jos_fF.v1uQwj5cDvtefDM.MlNAt7MVWn0tcW8dC1QQ c0W1ddzsPm8lGIDCoaVsjAtjxdA0bs23sratD5kfy2XnXsMtvlrbY2D5Yub_AKOrdnttVaJ8l.mI pFAQkNYyC1ZmtgWmM9A68K9k3MgjmrXtro5skyzXa1YYMtAnFSvtMKzec6PrBZ_LCpn0nMsJo93x SMNrls7SmydVLcfQpYyeEZdiX8q79XJNee01xqMWVVGjOEWRJ7vlBzpE- X-Sonic-MF: X-Sonic-ID: b84fd3b7-1507-4469-ab38-ca8f149c9352 Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.sg3.yahoo.com with HTTP; Fri, 30 Jun 2023 17:06:42 +0000 Date: Fri, 30 Jun 2023 17:06:38 +0000 (UTC) From: Chandy G To: "pgsql-general@lists.postgresql.org" Message-ID: <676545357.616656.1688144798077@mail.yahoo.com> Subject: Toasted column values during replication MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_616655_2111419987.1688144798076" References: <676545357.616656.1688144798077.ref@mail.yahoo.com> X-Mailer: WebService/1.1.21612 YMailNorrin Content-Length: 5950 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk ------=_Part_616655_2111419987.1688144798076 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi PG Users,=C2=A0 We are running into the postgres toasted values showing = up in replication slots. We are using debezium and have gone through the fo= llowing 1. Various options for handling toasted values during replication at the ap= plication level.2. Postgres option to turn on replica identitiy to full. While these are solutions to the problem. Wanted to understand if there is = any scope for enhancement / improvements in postgres itself to handle such = scenarios. For instance : =C2=A0Can the postgres internal component (wal-sender / *) i= nterpret this special toasted-col-value sitting in the wal-file -> replace = it with the actual value and send it to the replication slot. I presume, to= day the wal-sender=C2=A0 does not have this support and it just sends whate= ver data that the wal log has.=C2=A0 So if the wal-sender - can intercept s= uch toasted values and translate it to actual values before being sent - th= at would solve the problem without every consumer having to implement a sol= ution to handle toasted value. This would also help prevent the unwarranted= increase of the wal-log if replica-identity to full is set as a w/a. Any thoughts? Thanks. ------=_Part_616655_2111419987.1688144798076 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Hi PG Users,
  We are running into the postgres toasted values showing= up in replication slots. We are using debezium and have gone through the f= ollowing

1. Various options for handling toasted values d= uring replication at the application level.
2. Postgres option to turn on replica identitiy to full.
=

While these are solutions to the problem. Wanted to understand= if there is any scope for enhancement / improvements in postgres itself to= handle such scenarios.

For instance : =  Can the postgres internal component (wal-sender / *) int= erpret this special toasted-col-value sitting in the wal-file -> replace= it with the actual value and send it to the replication slot. I presume, t= oday the wal-sender  does not have this support and it just sends what= ever data that the wal log has.  So if the wal-sender - can intercept = such toasted values and translate it to actual values before being sent - t= hat would solve the problem without every consumer having to implement a so= lution to handle toasted value. This would also help prevent the unwarrante= d increase of the wal-log if replica-identity to full is set as a w/a.

=
Any thoughts?

Thanks.
------=_Part_616655_2111419987.1688144798076--