Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rFKLO-009Mgw-4Y for pgsql-hackers@arkaria.postgresql.org; Mon, 18 Dec 2023 20:36:06 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1rFKLK-00CYMZ-Rc for pgsql-hackers@arkaria.postgresql.org; Mon, 18 Dec 2023 20:36:02 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rFKLK-00CYMR-GP for pgsql-hackers@lists.postgresql.org; Mon, 18 Dec 2023 20:36:02 +0000 Received: from mail.verite.pro ([185.16.44.216]) by magus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rFKLH-00CgEQ-5f for pgsql-hackers@postgresql.org; Mon, 18 Dec 2023 20:36:01 +0000 Received: by mail.verite.pro (Postfix, from userid 1000) id E31588C03EA; Mon, 18 Dec 2023 21:35:56 +0100 (CET) MIME-Version: 1.0 From: "Daniel Verite" Subject: Fixing backslash dot for COPY FROM...CSV To: pgsql-hackers@postgresql.org Date: Mon, 18 Dec 2023 21:35:53 +0100 Message-Id: X-Mailer: Manitou v1.7.3 Content-type: multipart/mixed; boundary="----------=_1702931756-31220-6" List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format... ------------=_1702931756-31220-6 Content-Type: text/plain; charset="iso-8859-15" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi, PFA a patch that attempts to fix the bug that \. on a line by itself is handled incorrectly by COPY FROM ... CSV. This issue has been discussed several times previously, for instance in [1] and [2], and mentioned in the doc for \copy in commit 42d3125. There's one case that works today: when the line is part of a multi-line quoted section, and the data is read from a file, not from the client. In other situations, an error is raised or the data is cut at the point of \. without an error. The patch addresses that issue in the server and in psql, except for the case of inlined data, where \. cannot be=20 both valid data and an EOF marker at the same time, so it keeps treating it as an EOF marker.=20 [1] https://www.postgresql.org/message-id/10e3eff6-eb04-4b3f-aeb4-b920192b977a@= manitou-mail.org [2] https://www.postgresql.org/message-id/8aeab305-5e94-4fa5-82bf-6da6baee6e05%= 40app.fastmail.com Best regards, --=20 Daniel V=E9rit=E9 https://postgresql.verite.pro/ Twitter: @DanielVerite ------------=_1702931756-31220-6 Content-Type: text-x-patch; name="v1-0001-Support-backslash-dot-on-a-line-by-itself-as-vali.patch" Content-Disposition: attachment; filename="v1-0001-Support-backslash-dot-on-a-line-by-itself-as-vali.patch" Content-Transfer-Encoding: base64 RnJvbSA4NjhiMWUwNjVjZjcxNGYzMTViYWI2NTEyOWZkMDVhMWQ2MGZjODFi IE1vbiBTZXAgMTcgMDA6MDA6MDAgMjAwMQpGcm9tOiA9P1VURi04P3E/RGFu aWVsPTIwVj1DMz1BOXJpdD1DMz1BOT89IDxkYW5pZWxAbWFuaXRvdS1tYWls Lm9yZz4KRGF0ZTogTW9uLCAxOCBEZWMgMjAyMyAyMDo0NzowMiArMDEwMApT dWJqZWN0OiBbUEFUQ0ggdjFdIFN1cHBvcnQgYmFja3NsYXNoLWRvdCBvbiBh IGxpbmUgYnkgaXRzZWxmIGFzIHZhbGlkIGRhdGEgaW4KIENPUFkgRlJPTS4u LkNTVgoKLS0tCiBkb2Mvc3JjL3NnbWwvcmVmL3BzcWwtcmVmLnNnbWwgICAg ICAgfCAgNCAtLS0tCiBzcmMvYmFja2VuZC9jb21tYW5kcy9jb3B5ZnJvbXBh cnNlLmMgfCAxMyArKy0tLS0tLS0tLQogc3JjL2Jpbi9wc3FsL2NvcHkuYyAg ICAgICAgICAgICAgICAgIHwgMzIgKysrKysrKysrKysrKysrKysrKystLS0t LS0tLQogc3JjL3Rlc3QvcmVncmVzcy9leHBlY3RlZC9jb3B5Lm91dCAgIHwg MTUgKysrKysrKysrKysrKwogc3JjL3Rlc3QvcmVncmVzcy9zcWwvY29weS5z cWwgICAgICAgIHwgMTEgKysrKysrKysrKwogNSBmaWxlcyBjaGFuZ2VkLCA1 MSBpbnNlcnRpb25zKCspLCAyNCBkZWxldGlvbnMoLSkKCmRpZmYgLS1naXQg YS9kb2Mvc3JjL3NnbWwvcmVmL3BzcWwtcmVmLnNnbWwgYi9kb2Mvc3JjL3Nn bWwvcmVmL3BzcWwtcmVmLnNnbWwKaW5kZXggY2M3ZDc5NzE1OS4uZDk0ZTNj YWNmYyAxMDA2NDQKLS0tIGEvZG9jL3NyYy9zZ21sL3JlZi9wc3FsLXJlZi5z Z21sCisrKyBiL2RvYy9zcmMvc2dtbC9yZWYvcHNxbC1yZWYuc2dtbApAQCAt MTExOSwxMCArMTExOSw2IEBAIElOU0VSVCBJTlRPIHRibDEgVkFMVUVTICgk MSwgJDIpIFxiaW5kICdmaXJzdCB2YWx1ZScgJ3NlY29uZCB2YWx1ZScgXGcK ICAgICAgICAgZGVzdGluYXRpb24sIGJlY2F1c2UgYWxsIGRhdGEgbXVzdCBw YXNzIHRocm91Z2ggdGhlIGNsaWVudC9zZXJ2ZXIKICAgICAgICAgY29ubmVj dGlvbi4gIEZvciBsYXJnZSBhbW91bnRzIG9mIGRhdGEgdGhlIDxhY3Jvbnlt PlNRTDwvYWNyb255bT4KICAgICAgICAgY29tbWFuZCBtaWdodCBiZSBwcmVm ZXJhYmxlLgotICAgICAgICBBbHNvLCBiZWNhdXNlIG9mIHRoaXMgcGFzcy10 aHJvdWdoIG1ldGhvZCwgPGxpdGVyYWw+XGNvcHkKLSAgICAgICAgLi4uIGZy b208L2xpdGVyYWw+IGluIDxhY3JvbnltPkNTVjwvYWNyb255bT4gbW9kZSB3 aWxsIGVycm9uZW91c2x5Ci0gICAgICAgIHRyZWF0IGEgPGxpdGVyYWw+XC48 L2xpdGVyYWw+IGRhdGEgdmFsdWUgYWxvbmUgb24gYSBsaW5lIGFzIGFuCi0g ICAgICAgIGVuZC1vZi1pbnB1dCBtYXJrZXIuCiAgICAgICAgIDwvcGFyYT4K ICAgICAgICAgPC90aXA+CiAKZGlmZiAtLWdpdCBhL3NyYy9iYWNrZW5kL2Nv bW1hbmRzL2NvcHlmcm9tcGFyc2UuYyBiL3NyYy9iYWNrZW5kL2NvbW1hbmRz L2NvcHlmcm9tcGFyc2UuYwppbmRleCBmNTUzNzM0NTgyLi5iNGMyOTFiMjVi IDEwMDY0NAotLS0gYS9zcmMvYmFja2VuZC9jb21tYW5kcy9jb3B5ZnJvbXBh cnNlLmMKKysrIGIvc3JjL2JhY2tlbmQvY29tbWFuZHMvY29weWZyb21wYXJz ZS5jCkBAIC0xMTM3LDcgKzExMzcsNiBAQCBDb3B5UmVhZExpbmVUZXh0KENv cHlGcm9tU3RhdGUgY3N0YXRlKQogCWJvb2wJCXJlc3VsdCA9IGZhbHNlOwog CiAJLyogQ1NWIHZhcmlhYmxlcyAqLwotCWJvb2wJCWZpcnN0X2NoYXJfaW5f bGluZSA9IHRydWU7CiAJYm9vbAkJaW5fcXVvdGUgPSBmYWxzZSwKIAkJCQls YXN0X3dhc19lc2MgPSBmYWxzZTsKIAljaGFyCQlxdW90ZWMgPSAnXDAnOwpA QCAtMTMzNSwxMCArMTMzNCw5IEBAIENvcHlSZWFkTGluZVRleHQoQ29weUZy b21TdGF0ZSBjc3RhdGUpCiAJCX0KIAogCQkvKgotCQkgKiBJbiBDU1YgbW9k ZSwgd2Ugb25seSByZWNvZ25pemUgXC4gYWxvbmUgb24gYSBsaW5lLiAgVGhp cyBpcyBiZWNhdXNlCi0JCSAqIFwuIGlzIGEgdmFsaWQgQ1NWIGRhdGEgdmFs dWUuCisJCSAqIEluIENTViBtb2RlLCBcLiBpcyBhIHZhbGlkIENTViBkYXRh IHZhbHVlIGFueXdoZXJlIGluIHRoZSBsaW5lLgogCQkgKi8KLQkJaWYgKGMg PT0gJ1xcJyAmJiAoIWNzdGF0ZS0+b3B0cy5jc3ZfbW9kZSB8fCBmaXJzdF9j aGFyX2luX2xpbmUpKQorCQlpZiAoYyA9PSAnXFwnICYmICFjc3RhdGUtPm9w dHMuY3N2X21vZGUpCiAJCXsKIAkJCWNoYXIJCWMyOwogCkBAIC0xNDQyLDE0 ICsxNDQwLDcgQEAgQ29weVJlYWRMaW5lVGV4dChDb3B5RnJvbVN0YXRlIGNz dGF0ZSkKIAkJCX0KIAkJfQogCi0JCS8qCi0JCSAqIFRoaXMgbGFiZWwgaXMg Zm9yIENTViBjYXNlcyB3aGVyZSBcLiBhcHBlYXJzIGF0IHRoZSBzdGFydCBv ZiBhCi0JCSAqIGxpbmUsIGJ1dCB0aGVyZSBpcyBtb3JlIHRleHQgYWZ0ZXIg aXQsIG1lYW5pbmcgaXQgd2FzIGEgZGF0YSB2YWx1ZS4KLQkJICogV2UgYXJl IG1vcmUgc3RyaWN0IGZvciBcLiBpbiBDU1YgbW9kZSBiZWNhdXNlIFwuIGNv dWxkIGJlIGEgZGF0YQotCQkgKiB2YWx1ZSwgd2hpbGUgaW4gbm9uLUNTViBt b2RlLCBcLiBjYW5ub3QgYmUgYSBkYXRhIHZhbHVlLgotCQkgKi8KIG5vdF9l bmRfb2ZfY29weToKLQkJZmlyc3RfY2hhcl9pbl9saW5lID0gZmFsc2U7CiAJ fQkJCQkJCQkvKiBlbmQgb2Ygb3V0ZXIgbG9vcCAqLwogCiAJLyoKZGlmZiAt LWdpdCBhL3NyYy9iaW4vcHNxbC9jb3B5LmMgYi9zcmMvYmluL3BzcWwvY29w eS5jCmluZGV4IGRiYmJkYjg4OTguLmYzMWE3YWNiYjYgMTAwNjQ0Ci0tLSBh L3NyYy9iaW4vcHNxbC9jb3B5LmMKKysrIGIvc3JjL2Jpbi9wc3FsL2NvcHku YwpAQCAtNjIwLDIwICs2MjAsMzQgQEAgaGFuZGxlQ29weUluKFBHY29ubiAq Y29ubiwgRklMRSAqY29weXN0cmVhbSwgYm9vbCBpc2JpbmFyeSwgUEdyZXN1 bHQgKipyZXMpCiAJCQkJLyogY3VycmVudCBsaW5lIGlzIGRvbmU/ICovCiAJ CQkJaWYgKGJ1ZltidWZsZW4gLSAxXSA9PSAnXG4nKQogCQkJCXsKLQkJCQkJ LyogY2hlY2sgZm9yIEVPRiBtYXJrZXIsIGJ1dCBub3Qgb24gYSBwYXJ0aWFs IGxpbmUgKi8KLQkJCQkJaWYgKGF0X2xpbmVfYmVnaW4pCisJCQkJCS8qCisJ CQkJCSAqIFdoZW4gYXQgdGhlIGJlZ2lubmluZyBvZiB0aGUgbGluZSwgY2hl Y2sgZm9yIEVPRiBtYXJrZXIuCisJCQkJCSAqIElmIHRoZSBtYXJrZXIgaXMg Zm91bmQgYW5kIHRoZSBkYXRhIGlzIGlubGluZWQsCisJCQkJCSAqIHdlIG11 c3Qgc3RvcCBhdCB0aGlzIHBvaW50LiAgSWYgbm90LCB0aGUgXC4gbGluZSBj YW4gYmUKKwkJCQkJICogc2VudCB0byB0aGUgc2VydmVyLCBhbmQgd2UgbGV0 IGl0IGRlY2lkZSB3aGV0aGVyCisJCQkJCSAqIGl0J3MgYW4gRU9GIG9yIG5v dCBkZXBlbmRpbmcgb24gdGhlIGZvcm1hdDogaW4KKwkJCQkJICogYmFzaWMg VEVYVCwgXC4gaXMgZ29pbmcgdG8gYmUgaW50ZXJwcmV0ZWQgYXMgYW4gRU9G LCBpbgorCQkJCQkgKiBDU1YsIGl0IHdpbGwgbm90LgorCQkJCQkgKi8KKwkJ CQkJaWYgKGF0X2xpbmVfYmVnaW4gJiYgY29weXN0cmVhbSA9PSBwc2V0LmN1 cl9jbWRfc291cmNlKQogCQkJCQl7Ci0JCQkJCQkvKgotCQkJCQkJICogVGhp cyBjb2RlIGVycm9uZW91c2x5IGFzc3VtZXMgJ1wuJyBvbiBhIGxpbmUgYWxv bmUKLQkJCQkJCSAqIGluc2lkZSBhIHF1b3RlZCBDU1Ygc3RyaW5nIHRlcm1p bmF0ZXMgdGhlIFxjb3B5LgotCQkJCQkJICogaHR0cHM6Ly93d3cucG9zdGdy ZXNxbC5vcmcvbWVzc2FnZS1pZC9FMVRkTlZRLTAwMDFqdS1HT0B3cmlnbGV5 cy5wb3N0Z3Jlc3FsLm9yZwotCQkJCQkJICoKLQkJCQkJCSAqIGh0dHBzOi8v d3d3LnBvc3RncmVzcWwub3JnL21lc3NhZ2UtaWQvYmZjZDU3ZTQtOGYyMy00 YzNlLWE1ZGItMjU3MWQwOTIwOGUyQGJldGEuZmFzdG1haWwuY29tCi0JCQkJ CQkgKi8KIAkJCQkJCWlmICgobGluZWxlbiA9PSAzICYmIG1lbWNtcChmZ3Jl c3VsdCwgIlxcLlxuIiwgMykgPT0gMCkgfHwKIAkJCQkJCQkobGluZWxlbiA9 PSA0ICYmIG1lbWNtcChmZ3Jlc3VsdCwgIlxcLlxyXG4iLCA0KSA9PSAwKSkK IAkJCQkJCXsKIAkJCQkJCQljb3B5ZG9uZSA9IHRydWU7CisJCQkJCQkJLyoK KwkJCQkJCQkgKiBSZW1vdmUgdGhlIEVPRiBtYXJrZXIgZnJvbSB0aGUgZGF0 YSBzZW50LCB1bmxlc3MKKwkJCQkJCQkgKiB1c2luZyB0aGUgb2Jzb2xldGUg djIgcHJvdG9jb2wgdGhhdCBuZWVkcyBpdC4KKwkJCQkJCQkgKiBJbiB0aGUg Y2FzZSBvZiBDU1YsIHRoZSBFT0YgbWFya2VyIG11c3QgYmUKKwkJCQkJCQkg KiByZW1vdmVkLCBvdGhlcndpc2UgaXQgd291bGQgYmUgaW50ZXJwcmV0ZWQg YnkKKwkJCQkJCQkgKiB0aGUgc2VydmVyIGFzIHZhbGlkIGRhdGEuCisJCQkJ CQkJICovCisJCQkJCQkJaWYgKGNvcHlzdHJlYW0gPT0gcHNldC5jdXJfY21k X3NvdXJjZSAmJgorCQkJCQkJCQlQUXByb3RvY29sVmVyc2lvbihjb25uKSA+ PSAzKQorCQkJCQkJCXsKKwkJCQkJCQkJKmZncmVzdWx0PSdcMCc7CisJCQkJ CQkJCWJ1ZmxlbiAtPSBsaW5lbGVuOworCQkJCQkJCX0KIAkJCQkJCX0KIAkJ CQkJfQogCmRpZmYgLS1naXQgYS9zcmMvdGVzdC9yZWdyZXNzL2V4cGVjdGVk L2NvcHkub3V0IGIvc3JjL3Rlc3QvcmVncmVzcy9leHBlY3RlZC9jb3B5Lm91 dAppbmRleCBiNDgzNjVlYzk4Li5mYzFlNmNmZDdiIDEwMDY0NAotLS0gYS9z cmMvdGVzdC9yZWdyZXNzL2V4cGVjdGVkL2NvcHkub3V0CisrKyBiL3NyYy90 ZXN0L3JlZ3Jlc3MvZXhwZWN0ZWQvY29weS5vdXQKQEAgLTMyLDYgKzMyLDIx IEBAIHNlbGVjdCAqIGZyb20gY29weXRlc3QgZXhjZXB0IHNlbGVjdCAqIGZy b20gY29weXRlc3QyOwogLS0tLS0tLSstLS0tLS0rLS0tLS0tLS0KICgwIHJv d3MpCiAKKy0tLSB0ZXN0IHVucXVvdGVkIC5cIGFzIGRhdGEgaW5zaWRlIENT VgordHJ1bmNhdGUgY29weXRlc3QyOworaW5zZXJ0IGludG8gY29weXRlc3Qy KHRlc3QpIHZhbHVlcygnbGluZTEnKSwgKCdcLicpLCAoJ2xpbmUyJyk7Citj b3B5IChzZWxlY3QgdGVzdCBmcm9tIGNvcHl0ZXN0MiBvcmRlciBieSB0ZXN0 IGNvbGxhdGUgIkMiKSB0byA6J2ZpbGVuYW1lJyBjc3Y7CistLSBnZXQgdGhl IGRhdGEgYmFjayBpbiB3aXRoIGNvcHkKK3RydW5jYXRlIGNvcHl0ZXN0MjsK K2NvcHkgY29weXRlc3QyKHRlc3QpIGZyb20gOidmaWxlbmFtZScgY3N2Owor c2VsZWN0IHRlc3QgZnJvbSBjb3B5dGVzdDIgb3JkZXIgYnkgdGVzdCBjb2xs YXRlICJDIjsKKyB0ZXN0ICAKKy0tLS0tLS0KKyBcLgorIGxpbmUxCisgbGlu ZTIKKygzIHJvd3MpCisKIC0tIHRlc3QgaGVhZGVyIGxpbmUgZmVhdHVyZQog Y3JlYXRlIHRlbXAgdGFibGUgY29weXRlc3QzICgKIAljMSBpbnQsCmRpZmYg LS1naXQgYS9zcmMvdGVzdC9yZWdyZXNzL3NxbC9jb3B5LnNxbCBiL3NyYy90 ZXN0L3JlZ3Jlc3Mvc3FsL2NvcHkuc3FsCmluZGV4IDQzZDJlOTA2ZGQuLjI0 ZWNiMDQ4NWEgMTAwNjQ0Ci0tLSBhL3NyYy90ZXN0L3JlZ3Jlc3Mvc3FsL2Nv cHkuc3FsCisrKyBiL3NyYy90ZXN0L3JlZ3Jlc3Mvc3FsL2NvcHkuc3FsCkBA IC0zOCw2ICszOCwxNyBAQCBjb3B5IGNvcHl0ZXN0MiBmcm9tIDonZmlsZW5h bWUnIGNzdiBxdW90ZSAnJycnIGVzY2FwZSBFJ1xcJzsKIAogc2VsZWN0ICog ZnJvbSBjb3B5dGVzdCBleGNlcHQgc2VsZWN0ICogZnJvbSBjb3B5dGVzdDI7 CiAKKy0tLSB0ZXN0IHVucXVvdGVkIC5cIGFzIGRhdGEgaW5zaWRlIENTVgor Cit0cnVuY2F0ZSBjb3B5dGVzdDI7CisKK2luc2VydCBpbnRvIGNvcHl0ZXN0 Mih0ZXN0KSB2YWx1ZXMoJ2xpbmUxJyksICgnXC4nKSwgKCdsaW5lMicpOwor Y29weSAoc2VsZWN0IHRlc3QgZnJvbSBjb3B5dGVzdDIgb3JkZXIgYnkgdGVz dCBjb2xsYXRlICJDIikgdG8gOidmaWxlbmFtZScgY3N2OworLS0gZ2V0IHRo ZSBkYXRhIGJhY2sgaW4gd2l0aCBjb3B5Cit0cnVuY2F0ZSBjb3B5dGVzdDI7 Citjb3B5IGNvcHl0ZXN0Mih0ZXN0KSBmcm9tIDonZmlsZW5hbWUnIGNzdjsK K3NlbGVjdCB0ZXN0IGZyb20gY29weXRlc3QyIG9yZGVyIGJ5IHRlc3QgY29s bGF0ZSAiQyI7CisKIAogLS0gdGVzdCBoZWFkZXIgbGluZSBmZWF0dXJlCiAK LS0gCjIuMzQuMQoK ------------=_1702931756-31220-6--