Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1t1Qp9-00GHNm-QL for pgsql-hackers@arkaria.postgresql.org; Thu, 17 Oct 2024 13:45:55 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1t1Qp6-004xYf-Pl for pgsql-hackers@arkaria.postgresql.org; Thu, 17 Oct 2024 13:45:53 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1t1Qp6-004xYV-Ed for pgsql-hackers@lists.postgresql.org; Thu, 17 Oct 2024 13:45:52 +0000 Received: from oss.nttdata.com ([49.212.34.109]) by magus.postgresql.org with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1t1Qp3-001Xnl-SF for pgsql-hackers@postgresql.org; Thu, 17 Oct 2024 13:45:51 +0000 Received: from oss.nttdata.com (localhost [127.0.0.1]) by oss.nttdata.com (Postfix) with ESMTPA id D54426064A for ; Thu, 17 Oct 2024 22:45:46 +0900 (JST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at oss.nttdata.com MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=_b654ed99fd001abfec280c45cd08d8c4" Content-Transfer-Encoding: 8bit Date: Thu, 17 Oct 2024 22:45:46 +0900 From: torikoshia To: Pgsql Hackers Subject: Add reject_limit option to file_fdw User-Agent: Roundcube Webmail/1.4.11 Message-ID: X-Sender: torikoshia@oss.nttdata.com List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --=_b654ed99fd001abfec280c45cd08d8c4 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8; format=flowed Hi, 4ac2a9bec introduced reject_limit option to the COPY command, and I was wondering if it might be beneficial to add the same option to file_fdw. Although there may be fewer practical use cases compared to COPY, it could still be useful in situations where the file being read via file_fdw is subject to modifications and there is a need to tolerate a limited number of errors. What do you think? I've attached a patch. Based on the synopsis of the CREATE/ALTER FOREIGN TABLE commands, the value for the foreign table's option must be single-quoted. I’m not entirely sure if this is the correct approach, but in order to accommodate this, the patch modifies the value of reject_limit option to accept not only numeric values but also strings. -- Regards, -- Atsushi Torikoshi NTT DATA Group Corporation --=_b654ed99fd001abfec280c45cd08d8c4 Content-Transfer-Encoding: base64 Content-Type: text/x-diff; name=v1-0001-file_fdw-Add-reject_limit-option-to-file_fdw.patch Content-Disposition: attachment; filename=v1-0001-file_fdw-Add-reject_limit-option-to-file_fdw.patch; size=4594 RnJvbSA4ZDk2OTQ2NTI1N2ViYzUxMWEwZDFiMTUyMGYyOTA5NTEwM2FlNGFmIE1vbiBTZXAgMTcg MDA6MDA6MDAgMjAwMQpGcm9tOiBBdHN1c2hpIFRvcmlrb3NoaSA8dG9yaWtvc2hpYUBvc3MubnR0 ZGF0YS5jb20+CkRhdGU6IFRodSwgMTcgT2N0IDIwMjQgMjI6MTk6MDYgKzA5MDAKU3ViamVjdDog W1BBVENIIHYxXSBmaWxlX2ZkdzogQWRkIHJlamVjdF9saW1pdCBvcHRpb24gdG8gZmlsZV9mZHcK Ci0tLQogY29udHJpYi9maWxlX2Zkdy9leHBlY3RlZC9maWxlX2Zkdy5vdXQgfCAxMCArKysrKysr KystCiBjb250cmliL2ZpbGVfZmR3L2ZpbGVfZmR3LmMgICAgICAgICAgICB8ICA4ICsrKysrKysr CiBjb250cmliL2ZpbGVfZmR3L3NxbC9maWxlX2Zkdy5zcWwgICAgICB8ICA0ICsrKy0KIGRvYy9z cmMvc2dtbC9maWxlLWZkdy5zZ21sICAgICAgICAgICAgIHwgMTIgKysrKysrKysrKysrCiBzcmMv YmFja2VuZC9jb21tYW5kcy9jb3B5LmMgICAgICAgICAgICB8IDEyICsrKysrKysrKysrLQogNSBm aWxlcyBjaGFuZ2VkLCA0MyBpbnNlcnRpb25zKCspLCAzIGRlbGV0aW9ucygtKQoKZGlmZiAtLWdp dCBhL2NvbnRyaWIvZmlsZV9mZHcvZXhwZWN0ZWQvZmlsZV9mZHcub3V0IGIvY29udHJpYi9maWxl X2Zkdy9leHBlY3RlZC9maWxlX2Zkdy5vdXQKaW5kZXggNTkzZmRjNzgyZS4uMWFjY2MzNmQ2OCAx MDA2NDQKLS0tIGEvY29udHJpYi9maWxlX2Zkdy9leHBlY3RlZC9maWxlX2Zkdy5vdXQKKysrIGIv Y29udHJpYi9maWxlX2Zkdy9leHBlY3RlZC9maWxlX2Zkdy5vdXQKQEAgLTIwNiw3ICsyMDYsNyBA QCBTRUxFQ1QgKiBGUk9NIGFnZ19jc3YgYyBKT0lOIGFnZ190ZXh0IHQgT04gKHQuYSA9IGMuYSkg T1JERVIgQlkgYy5hOwogU0VMRUNUICogRlJPTSBhZ2dfYmFkOyAgICAgICAgICAgICAgIC0tIEVS Uk9SCiBFUlJPUjogIGludmFsaWQgaW5wdXQgc3ludGF4IGZvciB0eXBlIHJlYWw6ICJhYWEiCiBD T05URVhUOiAgQ09QWSBhZ2dfYmFkLCBsaW5lIDMsIGNvbHVtbiBiOiAiYWFhIgotLS0gb25fZXJy b3IgYW5kIGxvZ192ZXJib3NpdHkgdGVzdHMKKy0tIG9uX2Vycm9yLCBsb2dfdmVyYm9zaXR5IGFu ZCByZWplY3RfbGltaXQgdGVzdHMKIEFMVEVSIEZPUkVJR04gVEFCTEUgYWdnX2JhZCBPUFRJT05T IChBREQgb25fZXJyb3IgJ2lnbm9yZScpOwogU0VMRUNUICogRlJPTSBhZ2dfYmFkOwogTk9USUNF OiAgMSByb3cgd2FzIHNraXBwZWQgZHVlIHRvIGRhdGEgdHlwZSBpbmNvbXBhdGliaWxpdHkKQEAg LTIyNCw2ICsyMjQsMTQgQEAgU0VMRUNUICogRlJPTSBhZ2dfYmFkOwogICA0MiB8IDMyNC43OAog KDIgcm93cykKIAorQUxURVIgRk9SRUlHTiBUQUJMRSBhZ2dfYmFkIE9QVElPTlMgKHJlamVjdF9s aW1pdCAnMScpOworU0VMRUNUICogRlJPTSBhZ2dfYmFkOworICBhICB8ICAgYiAgICAKKy0tLS0t Ky0tLS0tLS0tCisgMTAwIHwgOTkuMDk3CisgIDQyIHwgMzI0Ljc4CisoMiByb3dzKQorCiBBTkFM WVpFIGFnZ19iYWQ7CiAtLSBtaXNjIHF1ZXJ5IHRlc3RzCiBcdCBvbgpkaWZmIC0tZ2l0IGEvY29u dHJpYi9maWxlX2Zkdy9maWxlX2Zkdy5jIGIvY29udHJpYi9maWxlX2Zkdy9maWxlX2Zkdy5jCmlu ZGV4IDA0MzIwNGMzZTcuLjJhZTgzYmEwOWIgMTAwNjQ0Ci0tLSBhL2NvbnRyaWIvZmlsZV9mZHcv ZmlsZV9mZHcuYworKysgYi9jb250cmliL2ZpbGVfZmR3L2ZpbGVfZmR3LmMKQEAgLTc3LDYgKzc3 LDcgQEAgc3RhdGljIGNvbnN0IHN0cnVjdCBGaWxlRmR3T3B0aW9uIHZhbGlkX29wdGlvbnNbXSA9 IHsKIAl7ImVuY29kaW5nIiwgRm9yZWlnblRhYmxlUmVsYXRpb25JZH0sCiAJeyJvbl9lcnJvciIs IEZvcmVpZ25UYWJsZVJlbGF0aW9uSWR9LAogCXsibG9nX3ZlcmJvc2l0eSIsIEZvcmVpZ25UYWJs ZVJlbGF0aW9uSWR9LAorCXsicmVqZWN0X2xpbWl0IiwgRm9yZWlnblRhYmxlUmVsYXRpb25JZH0s CiAJeyJmb3JjZV9ub3RfbnVsbCIsIEF0dHJpYnV0ZVJlbGF0aW9uSWR9LAogCXsiZm9yY2VfbnVs bCIsIEF0dHJpYnV0ZVJlbGF0aW9uSWR9LAogCkBAIC03ODgsNiArNzg5LDEzIEBAIHJldHJ5Ogog CQkJICovCiAJCQlSZXNldFBlclR1cGxlRXhwckNvbnRleHQoZXN0YXRlKTsKIAorCQkJaWYgKGNz dGF0ZS0+b3B0cy5yZWplY3RfbGltaXQgPiAwICYmIFwKKwkJCQljc3RhdGUtPm51bV9lcnJvcnMg PiBjc3RhdGUtPm9wdHMucmVqZWN0X2xpbWl0KQorCQkJCWVyZXBvcnQoRVJST1IsCisJCQkJCQko ZXJyY29kZShFUlJDT0RFX0lOVkFMSURfVEVYVF9SRVBSRVNFTlRBVElPTiksCisJCQkJCQkgZXJy bXNnKCJza2lwcGVkIG1vcmUgdGhhbiBSRUpFQ1RfTElNSVQgKCVsbGQpIHJvd3MgZHVlIHRvIGRh dGEgdHlwZSBpbmNvbXBhdGliaWxpdHkiLAorCQkJCQkJCQkobG9uZyBsb25nKSBjc3RhdGUtPm9w dHMucmVqZWN0X2xpbWl0KSkpOworCiAJCQkvKiBSZXBlYXQgTmV4dENvcHlGcm9tKCkgdW50aWwg bm8gc29mdCBlcnJvciBvY2N1cnMgKi8KIAkJCWdvdG8gcmV0cnk7CiAJCX0KZGlmZiAtLWdpdCBh L2NvbnRyaWIvZmlsZV9mZHcvc3FsL2ZpbGVfZmR3LnNxbCBiL2NvbnRyaWIvZmlsZV9mZHcvc3Fs L2ZpbGVfZmR3LnNxbAppbmRleCBlZGQ3N2M1Y2QyLi5lZTY2ZDdkODMxIDEwMDY0NAotLS0gYS9j b250cmliL2ZpbGVfZmR3L3NxbC9maWxlX2Zkdy5zcWwKKysrIGIvY29udHJpYi9maWxlX2Zkdy9z cWwvZmlsZV9mZHcuc3FsCkBAIC0xNTAsMTEgKzE1MCwxMyBAQCBTRUxFQ1QgKiBGUk9NIGFnZ19j c3YgYyBKT0lOIGFnZ190ZXh0IHQgT04gKHQuYSA9IGMuYSkgT1JERVIgQlkgYy5hOwogLS0gZXJy b3IgY29udGV4dCByZXBvcnQgdGVzdHMKIFNFTEVDVCAqIEZST00gYWdnX2JhZDsgICAgICAgICAg ICAgICAtLSBFUlJPUgogCi0tLSBvbl9lcnJvciBhbmQgbG9nX3ZlcmJvc2l0eSB0ZXN0cworLS0g b25fZXJyb3IsIGxvZ192ZXJib3NpdHkgYW5kIHJlamVjdF9saW1pdCB0ZXN0cwogQUxURVIgRk9S RUlHTiBUQUJMRSBhZ2dfYmFkIE9QVElPTlMgKEFERCBvbl9lcnJvciAnaWdub3JlJyk7CiBTRUxF Q1QgKiBGUk9NIGFnZ19iYWQ7CiBBTFRFUiBGT1JFSUdOIFRBQkxFIGFnZ19iYWQgT1BUSU9OUyAo QUREIGxvZ192ZXJib3NpdHkgJ3NpbGVudCcpOwogU0VMRUNUICogRlJPTSBhZ2dfYmFkOworQUxU RVIgRk9SRUlHTiBUQUJMRSBhZ2dfYmFkIE9QVElPTlMgKHJlamVjdF9saW1pdCAnMScpOworU0VM RUNUICogRlJPTSBhZ2dfYmFkOwogQU5BTFlaRSBhZ2dfYmFkOwogCiAtLSBtaXNjIHF1ZXJ5IHRl c3RzCmRpZmYgLS1naXQgYS9kb2Mvc3JjL3NnbWwvZmlsZS1mZHcuc2dtbCBiL2RvYy9zcmMvc2dt bC9maWxlLWZkdy5zZ21sCmluZGV4IGJiMzU3OWIwNzcuLmIwNTA4NWZkN2QgMTAwNjQ0Ci0tLSBh L2RvYy9zcmMvc2dtbC9maWxlLWZkdy5zZ21sCisrKyBiL2RvYy9zcmMvc2dtbC9maWxlLWZkdy5z Z21sCkBAIC0xNDksNiArMTQ5LDE4IEBACiAgICA8L2xpc3RpdGVtPgogICA8L3Zhcmxpc3RlbnRy eT4KIAorICA8dmFybGlzdGVudHJ5PgorICAgPHRlcm0+PGxpdGVyYWw+cmVqZWN0X2xpbWl0PC9s aXRlcmFsPjwvdGVybT4KKworICAgPGxpc3RpdGVtPgorICAgIDxwYXJhPgorICAgICBTcGVjaWZp ZXMgdGhlIG1heGltdW0gbnVtYmVyIG9mIGVycm9ycyB0b2xlcmF0ZWQgd2hpbGUgY29udmVydGlu ZyBhIGNvbHVtbidzCisgICAgIGlucHV0IHZhbHVlIHRvIGl0cyBkYXRhIHR5cGUsIHRoZSBzYW1l IGFzIDxjb21tYW5kPkNPUFk8L2NvbW1hbmQ+J3MKKyAgICA8bGl0ZXJhbD5SRUpFQ1RfTElNSVQ8 L2xpdGVyYWw+IG9wdGlvbi4KKyAgICA8L3BhcmE+CisgICA8L2xpc3RpdGVtPgorICA8L3Zhcmxp c3RlbnRyeT4KKwogIDwvdmFyaWFibGVsaXN0PgogCiAgPHBhcmE+CmRpZmYgLS1naXQgYS9zcmMv YmFja2VuZC9jb21tYW5kcy9jb3B5LmMgYi9zcmMvYmFja2VuZC9jb21tYW5kcy9jb3B5LmMKaW5k ZXggMzQ4NWJhODY2My4uYzZjOGJjMTBiMSAxMDA2NDQKLS0tIGEvc3JjL2JhY2tlbmQvY29tbWFu ZHMvY29weS5jCisrKyBiL3NyYy9iYWNrZW5kL2NvbW1hbmRzL2NvcHkuYwpAQCAtNDI0LDcgKzQy NCwxNyBAQCBkZWZHZXRDb3B5T25FcnJvckNob2ljZShEZWZFbGVtICpkZWYsIFBhcnNlU3RhdGUg KnBzdGF0ZSwgYm9vbCBpc19mcm9tKQogc3RhdGljIGludDY0CiBkZWZHZXRDb3B5UmVqZWN0TGlt aXRPcHRpb24oRGVmRWxlbSAqZGVmKQogewotCWludDY0CQlyZWplY3RfbGltaXQgPSBkZWZHZXRJ bnQ2NChkZWYpOworCWludDY0CQlyZWplY3RfbGltaXQ7CisKKwlpZiAoZGVmLT5hcmcgPT0gTlVM TCkKKwkJZXJlcG9ydChFUlJPUiwKKwkJCQkoZXJyY29kZShFUlJDT0RFX1NZTlRBWF9FUlJPUiks CisJCQkJIGVycm1zZygiJXMgcmVxdWlyZXMgYSBudW1lcmljIHZhbHVlIiwKKwkJCQkJCWRlZi0+ ZGVmbmFtZSkpKTsKKwllbHNlIGlmIChub2RlVGFnKGRlZi0+YXJnKSA9PSBUX1N0cmluZykKKwkJ cmVqZWN0X2xpbWl0ID0gcGdfc3RydG9pbnQ2NChzdHJWYWwoZGVmLT5hcmcpKTsKKwllbHNlCisJ CXJlamVjdF9saW1pdCA9IGRlZkdldEludDY0KGRlZik7CiAKIAlpZiAocmVqZWN0X2xpbWl0IDw9 IDApCiAJCWVyZXBvcnQoRVJST1IsCi0tIAoyLjM5LjIKCg== --=_b654ed99fd001abfec280c45cd08d8c4--