From: "Takamichi Osumi (Fujitsu)" <osumi.takamichi@fujitsu.com>
To: 'Melih Mutlu' <m.melihmutlu@gmail.com>, "Hayato Kuroda (Fujitsu)"
	<kuroda.hayato@fujitsu.com>
CC: "shiy.fnst@fujitsu.com" <shiy.fnst@fujitsu.com>, Euler Taveira
	<euler@eulerto.com>, Amit Kapila <amit.kapila16@gmail.com>, pgsql-hackers
	<pgsql-hackers@postgresql.org>, Bharath Rupireddy
	<bharath.rupireddyforpostgres@gmail.com>
Subject: RE: Allow logical replication to copy tables in binary format
Thread-Topic: Allow logical replication to copy tables in binary format
Thread-Index:
 AQHYrMp6CpcsVgwRoEW672SquV0Nla2o82YAgACXPICAACfDAIAABYCAgAA6YICABkYQAIAyEoUAgBqYN4CADXil8IA0jgsAgFrx2ACAAB4wAIABEh+QgABYt4CACVt1gIATF2sAgAAdRQCAGoEHgIAGJeOAgABMzoCACifvoA==
Date: Sun, 26 Feb 2023 22:58:26 +0000
Message-ID:
 <TYCPR01MB83731AED8ACAC59A2DD18D94EDAE9@TYCPR01MB8373.jpnprd01.prod.outlook.com>
References:
 <CAGPVpCQvAziCLknEnygY0v1-KBtg+Om-9JHJYZOnNPKFJPompw@mail.gmail.com>
 <2ebc7ea8-3c46-474e-aea7-5d73ff6165fb@www.fastmail.com>
 <CAA4eK1KxYj=3rYHbPMQvNSFytHdyS5=q0D3A1T=uerC7HMFCtg@mail.gmail.com>
 <7cf14e66-8cea-406f-9607-607a397292f6@www.fastmail.com>
 <CAGPVpCR15YfE=4hthY6S2OC5G0nj0QM4oE2bV769+FANM_UvwQ@mail.gmail.com>
 <f1d58324-8df4-4bb5-a546-8c741c2e6fa8@www.fastmail.com>
 <CAGPVpCRjdFHa71Qqd4Xj22UTa5=-NBUPgtT==X0w7TfWUZrpUg@mail.gmail.com>
 <TYCPR01MB837354C136F914D2AE174AE1ED489@TYCPR01MB8373.jpnprd01.prod.outlook.com>
 <CAGPVpCR_g8c1xXfR4kkSXiSt314C0it0seRzfqgTd+oF7N_5jw@mail.gmail.com>
 <TYCPR01MB8373B593010467315C2BA8EBED229@TYCPR01MB8373.jpnprd01.prod.outlook.com>
 <CAGPVpCRaxRA-SqWYsWvqWPB_CfVyasvCwC2oqg9oQgZfP0QBtQ@mail.gmail.com>
 <OSZPR01MB63105595F660457538F9024EFDFC9@OSZPR01MB6310.jpnprd01.prod.outlook.com>
 <CAGPVpCR2ffpiNYea-cU4sXaijCV4GHiUBf=yupKbVFM0=zWtVw@mail.gmail.com>
 <TYCPR01MB83731E3ACC68CCEBD2E48541EDFD9@TYCPR01MB8373.jpnprd01.prod.outlook.com>
 <CAGPVpCS0OzZ1tx3qT1Lc83HVK2JCc=TQ1G73tbyHYDKyo+mzFQ@mail.gmail.com>
 <CALj2ACU47pbGUUxyxVbGg3RAb8JsZhUuK+KogPyiWxYZbGVrxg@mail.gmail.com>
 <CAGPVpCSS_S8ZHiV+=3RvsDro16gJhVckzQDUrZ08b3LSPWTmsA@mail.gmail.com>
 <CALj2ACW5Oa7_v25iZb326UXvtM_tjQfw0Tc3hPJ8zN4FZqc9cw@mail.gmail.com>
 <CAGPVpCQYi9AYQSS=RmGgVNjz5ZEnLB8mACwd9aioVhLmbgiAMA@mail.gmail.com>
 <TYAPR01MB5866F36CF5D12DFEE90B0B8BF5A49@TYAPR01MB5866.jpnprd01.prod.outlook.com>
 <CAGPVpCTULGLwNGH7mMytR8dP1zg5f3wWq4=V=z=nT=-t8=2Vjg@mail.gmail.com>
In-Reply-To:
 <CAGPVpCTULGLwNGH7mMytR8dP1zg5f3wWq4=V=z=nT=-t8=2Vjg@mail.gmail.com>
Accept-Language: ja-JP, en-US
Content-Language: ja-JP
msip_labels: MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_Enabled=true;
 MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_SetDate=2023-02-26T22:58:25Z;
 MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_Method=Standard;
 MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_Name=FUJITSU-RESTRICTED?;
 MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_SiteId=a19f121d-81e1-4858-a9d8-736e267fd4c7;
 MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_ActionId=7e2b8f2b-804a-461e-bdc1-149399b39c88;
 MSIP_Label_a7295cc1-d279-42ac-ab4d-3b0f4fece050_ContentBits=0
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=fujitsu.com;
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: TYCPR01MB8373:EE_|TYWPR01MB11893:EE_
x-ms-office365-filtering-correlation-id: 379de7e4-c56c-45f7-82c3-08db184cf87b
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 iZi/bqmgEyWMN+3xngXncboCzjpNiyrdzng5pcrSku9V9u/3uiefTnIv8UwzlwZTDSfR3zWQMYmbtb3gvGzchPkhamY1la7WhwRUonbYviGAdA86FKcefA8t+MJn6WsgJU2SpmPI8O/XbaEBT0FDl9vWQpMYKWANty8cItzi9GDMwZzQZfYMJMnkWgOO2A4fNVt9/Tzk3Ao1QPWykHeLCmyYAvBqrwxVN6Cgnu+vwEL06QsysEYFFO21+nGYuEd4jBeItH+ZAnmhbf2uRp7RKfkzG7sH824sxHl/Ut/g8e9mfOZllUgzBIlXJK93LN9nnfw9diwvIWugvWPyudjQ3chqNmJ/z70yPt/Hcqj8NEL43tlLeac/KUnSVdUgm1Dge97eNyMS84bm51Aw/4iuzLjZAIkglAr78q01pI4jei8jF92bbCGiJLnNzzbkkDCjcV73dAk3SyRlmxTtmUwjKLB035GFxswyROTt9C2jtUdty4H7xNNWg+J92zYeKgbhB+9g1/3F4lfmmThvIxul6lT8EgyPXs/4LEBIxsOVz9QkFaZQcRTvg3JiWy1Kp1/ETeoi9GK+7J96TcwRm7kPeO5WbarHcnU5TI/liHx+b15doeAw3p+sUJn5D1285M7c32IXOeO+hpQ9t679IS3TeLW4MvhrZ+ySflemdLCoAqof92g2DFtukP8jeOWRQb08
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TYCPR01MB8373.jpnprd01.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230025)(4636009)(376002)(396003)(136003)(346002)(366004)(39860400002)(1590799015)(451199018)(1580799012)(84970400001)(316002)(110136005)(54906003)(6636002)(85182001)(33656002)(86362001)(55016003)(38070700005)(38100700002)(82960400001)(122000001)(6506007)(53546011)(83380400001)(9686003)(186003)(26005)(5660300002)(7696005)(71200400001)(2906002)(8936002)(52536014)(478600001)(41300700001)(4326008)(8676002)(66946007)(66556008)(66476007)(76116006)(66446008)(64756008);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?iso-2022-jp?B?V2phVkdoSDJvVWRjL24yK1RCSWdSd0dQSHpJMGVIQzhndmRmbU5nZ2pa?=
 =?iso-2022-jp?B?Z3p6TG5GNjYrTTJvbXcwNG41VHpSZ2ozcE9vZE52Q0VSaFpPSzdNWWtl?=
 =?iso-2022-jp?B?aHdBeldWS1g1dHV5a0xJWnJScCsxM0JHWGkxTzZwMUtYbHJBQmE1Nys2?=
 =?iso-2022-jp?B?bWFTT25vYmc0bERoOERJZkxVcjNPNXRpZytWRkUxb3RFM1pJcFg3NFhG?=
 =?iso-2022-jp?B?MHFvOWxmS25MZW4xTEQxdXpNQncyZTFLTEhDazlIV1owSGcremVDajh4?=
 =?iso-2022-jp?B?ZnJsMlZGbXRwQ1YvYTEwRjRJMU9xdk1JU0dSWkpBUFZ2dGxsVW1HbnFR?=
 =?iso-2022-jp?B?UG5SVWw1L1hIVkVQWmovaUZuN1UwYmxBU05jOWs1a3NWczV5THZNcGVX?=
 =?iso-2022-jp?B?UGMrdnFyMmdwR1pnZjBYd1hIY21Ea3ZvSzZSak5HUXc4R2lmZkZDZS96?=
 =?iso-2022-jp?B?ZFIzM2FSRENEK1ZGd1RneGhVend4R2ludHAxY0tUTjRUemFDb0p3WVpK?=
 =?iso-2022-jp?B?akRvSnBrL3dQb1A5eDg3Yk5xSXpHZHRHWWpjVWEwOXJHMkg2eElVd0h1?=
 =?iso-2022-jp?B?QjZyeWJlekNOY0ZsUHg3eVlQU1lMc2hEQS9HVk5FTm9HSlA2MUN2MGdF?=
 =?iso-2022-jp?B?K214SXJTb2N6L2xLcldSUDFKNHhIN1J2VWphTUVnYkpuaEwvVVFGaXFI?=
 =?iso-2022-jp?B?NXBXUVVwQWZKMUoyWm9OZTlzTm1RVUpXT2tuVmMwSXZNQXFQZmVzc0tV?=
 =?iso-2022-jp?B?SUxQQXpQRkIwSGRpV0E5MFR6ZE1JbmlkamRXZ3JUdHd3c3VGQTUxcFBq?=
 =?iso-2022-jp?B?eVI4alVnVnJDOUQrZ2VmS1VqNU1IM3VxVy9Bb3loa2pDWHh3WU5Fb1JW?=
 =?iso-2022-jp?B?ajhKa3YwcEJKWmZsUzcyb3lIY3loem5vQ24vWFZRQWR6RVZCajh5NkRu?=
 =?iso-2022-jp?B?ZU9xbGYxME4rZkRsUWVuSG5qT05nRHprL0VlMEsxOWgyRzYzNHpuOWE3?=
 =?iso-2022-jp?B?dnlBbTRmQkxtM1FmMlJ6NWVOS2VKcE9PcFlqRGx4OVNmOHBMTjVOMmNy?=
 =?iso-2022-jp?B?L3k1S0d1cjdXQXJiOFArSXB3WjNObHlOUkE5dXozSUNkOVVGNU13dm9G?=
 =?iso-2022-jp?B?WWF0VENyT29ycGo3YkVNTlplcTFWOUk0VmpEK21ZTnptbnZiby9NUmhy?=
 =?iso-2022-jp?B?NnlHYSt3UU00emVPaEsxYVBhY0NLaWdxbWtzVlFHeXRpMkYrNFZWZnpy?=
 =?iso-2022-jp?B?SXhVZ1owdWxNWW5JSE0wZU5QeXhSNGx4VkloR3hRK0ZjM2gxT1BLcEJW?=
 =?iso-2022-jp?B?VmFEdkw3aGhFdVRiMWNYcmxPTmszL2dJRmNzS1phQUlkYXQyZ0lyUW5C?=
 =?iso-2022-jp?B?TUpya0hNYXFOc1pMaTRsQVJPdUJ0MWNwemxhOWFtYmxoT1pBeGVNNUpV?=
 =?iso-2022-jp?B?WkZvWWwrOGprZFNJTzlHL0REam56RnFPdFhPaWROYUpjUkdhRWtYRDdZ?=
 =?iso-2022-jp?B?UEV6SGFaVG5zeWR0Y3dJRERYNFFXWUJZRUdRcUg1S1R5eHRIOHdib0sz?=
 =?iso-2022-jp?B?QmdkQW45cVh2MVRhWDl4aEF1dG0rbkdzN1dVZk5GQWhic1hqV1BBZWl6?=
 =?iso-2022-jp?B?ZS9ZaVk2L2dzQXhoT1NuQzJBeXc0Q0F0RHEwOTRLekt3NnBwTURQS3py?=
 =?iso-2022-jp?B?d0Vrazcwb2NYWXlxMytCNUVMRUo5NzVUL1dEdUNRSzhCYUFINUJkOUJ3?=
 =?iso-2022-jp?B?dUE2SkNDY2IyMUlCbDlGZGxCWGV1WEZzd2FtazVpeTdSbFZsdFg5RTNh?=
 =?iso-2022-jp?B?YStvM3FValAzbFRtbDc1dldkd0hoY0VPUEhTUTUzT2ZKemZIL0pHVGdJ?=
 =?iso-2022-jp?B?RkFtRjBaanV1aG1KYzI0M2paUzhRcHZVeWtTME1kNjUvbURWcnhyVW1E?=
 =?iso-2022-jp?B?NHpEU0RJZEttTWx5VDlLUStENnBLYXlXZlBhcmdTYlhGbVhMQzJqeHh0?=
 =?iso-2022-jp?B?cnd6bHp3aHl2N3ZkcEw1K09aRC95ZllwakVPYXA2cWlmK21FNm1BRmRO?=
 =?iso-2022-jp?B?dDR4N29SZ2tQcUdTd3N5NEhhNWdBWmh5MDJoS2lVa1VrbytUREZ4a0Q0?=
 =?iso-2022-jp?B?ek5leGJPbnlValFWbFFCTGFtbDVMYVI3YW5xbUtEOE5MbnFQUG9lWDlu?=
 =?iso-2022-jp?B?WVpEcDJnRVB6aEJ1MUhRUmNuOVNscjZmRlVjNUw3KzBHK1JOZlNZc0Vr?=
 =?iso-2022-jp?B?Z3ZNdUYzYmVUSVpaZ3JjTTVkT1oyQUJQa0FzS2hheE9nbEd1T2ZJa1JD?=
 =?iso-2022-jp?B?VFJDeTRjUFZJUDFockZNcHp1bDdYZXZ0SkE9PQ==?=
Content-Type: text/plain; charset="iso-2022-jp"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0:
	5nIXnhkjCEjznl83udzd9couN6aE+0fBH8xPB5ebVUinIyUYLIGmrwekB9M729f7II2AgSZmElC3DPnj15wv2LZ3eY/0I70xlc1Z1zVidQdol/C7n9lO6WsmR0ft+7GmyM2SpySAJMY1ncQuwfqujIGJ9fGqYRQXI2ni6UiIkhAHGq28kNOkmkQXdgsZYtx4TINQZO2aKEpVQxxVnwyAl6ngUzFq1omH7T9DlnMyiJMvhxeExRXOXsrAhWAgNNxiS39OQu0FzFIms1+F8aKcOBK8/RBsKbzK+NqnJ8qdKUewOdum2MCQSPguSQhwH+++L7uXXPxzK+2M+BoP2GDlXn/Kvn8jHfIyv7yYoQ2BNd7UUOEqXO8uDEeSlneg646OvDONT0ACvVbpjXxPhC0CKnaz22uCiTWgrUvcZ2AqqVLw5KfqdFCg7cJg8iIuAZmL+Eo6/dATnJJIiOoA21Q/Vm0U0Fv6ETbyIF/Y/eL46WRs4AATcQYdhl3mC6Yp6LuWXZlR0mrI1Ltr7FXeYVktDEycYpb/m6hlyPYfeY2FHHcoetpMbCF/T/yfEQ2bctXrDtyBoMKnlVcnlOrGQHgpKGSV64aN778VOze0nyJyGjrdrzoBGyW5qCyglB+WmFsK6O/fC11IXkjvIyKFBOy3xs6hqXrppG44wc6AkL5AbMVADSNi3k1eEBythWZk9CNwJliY/CK6UA2G4h6ExHKgLwOSvuw0yAwRowWzIslYjCfKaNqaJaZnUkP1mUFmL4a6/8zRqnT49hN+3q9EtxCW5s5ipvB4E9GkQWhsUVsq7BqphGOCiz6q2gdv4Qydv9icIcdJbLIlt8RPsDo4pohJeg==
X-OriginatorOrg: fujitsu.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: TYCPR01MB8373.jpnprd01.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 379de7e4-c56c-45f7-82c3-08db184cf87b
X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Feb 2023 22:58:26.8013
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: a19f121d-81e1-4858-a9d8-736e267fd4c7
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Rl0T9UN3Dp5GpSL/T2ZoNV/V14cd9Y663+f3zKw9JxwFlMEnOtVYvZNgMJTM82yN3ZwJjym2TXjnxgt3uuS/p/mY989/d4cz/gQPWQtxfZ8=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: TYWPR01MB11893
List-Id: <pgsql-hackers.lists.postgresql.org>
List-Help: <https://lists.postgresql.org/manage/>
List-Subscribe: <https://lists.postgresql.org/manage/>
List-Post: <mailto:pgsql-hackers@lists.postgresql.org>
List-Owner: <mailto:pgsql-hackers-owner@lists.postgresql.org>
List-Archive: <https://www.postgresql.org/list/pgsql-hackers>
Archived-At: <https://www.postgresql.org/message-id/TYCPR01MB83731AED8ACAC59A2DD18D94EDAE9%40TYCPR01MB8373.jpnprd01.prod.outlook.com>
Precedence: bulk

On Monday, February 20, 2023 8:47 PM Melih Mutlu <m.melihmutlu@gmail.com> w=
rote:
> Thanks for letting me know.=20
> Attached the fixed version of the patch.
Hi, Melih


Thanks for updating the patch. Minor comments on v9.

(1) commit message

"The patch introduces a new parameter, copy_format, to CREATE SUBSCRIPTION =
to allow to choose
the format used in initial table synchronization."

This patch introduces the new parameter not only to CREATE SUBSCRIPTION and=
 ALTER SUBSCRIPTION, then this description should be more general, somethin=
g like below.

"The patch introduces a new parameter, copy_format, as subscription option =
to
allow user to choose the format of initial table synchronization."

(2) copy_table

We don't need to check the publisher's version below.

+
+       /* If the publisher is v16 or later, specify the format to copy dat=
a. */
+       if (walrcv_server_version(LogRepWorkerWalRcvConn) >=3D 160000)
+       {
+               char *format =3D MySubscription->copyformat =3D=3D LOGICALR=
EP_COPY_AS_BINARY ? "binary" : "text";
+               appendStringInfo(&cmd, "  WITH (FORMAT %s)", format);
+               options =3D lappend(options, makeDefElem("format", (Node *)=
 makeString(format), -1));
+       }
+

We don't have this kind of check for "binary" option and it seems this is u=
ser's responsibility to avoid any errors during replication. If we want to =
add this kind of check, then we can add checks for both "binary" and "copy_=
format" option together as an independent patch.

(3) subscription.sql/out

The other existing other subscription options check the invalid input for n=
ewly created option (e.g. "foo" for disable_on_error,  streaming mode). So,=
 I think we can add this type of test for this feature.


Best Regards,
	Takamichi Osumi