Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rP5F6-009CSV-Bi for pgsql-bugs@arkaria.postgresql.org; Sun, 14 Jan 2024 18:29:57 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1rP5E6-00CRFX-Of for pgsql-bugs@arkaria.postgresql.org; Sun, 14 Jan 2024 18:28:54 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rP5E6-00CRFP-AJ for pgsql-bugs@lists.postgresql.org; Sun, 14 Jan 2024 18:28:54 +0000 Received: from mail.postgrespro.ru ([93.174.131.139]) by magus.postgresql.org with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rP5E3-001lG3-49 for pgsql-bugs@lists.postgresql.org; Sun, 14 Jan 2024 18:28:53 +0000 Received: from [192.168.1.120] (unknown [46.164.243.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: d.koval@postgrespro.ru) by mail.postgrespro.ru (Postfix/587) with ESMTPSA id 405E2E2103C for ; Sun, 14 Jan 2024 21:28:49 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=postgrespro.ru; s=mx2023; t=1705256929; bh=WYIikc7PKstJEK3X6hBLDpFbEYnMZxWPV9PkbRlf+VU=; h=Message-ID:Date:User-Agent:Subject:To:References:From:In-Reply-To: From; b=hIZ5TGF1KX9LqXqmVZQvCIhWhNVV4GmiKqEVeulE15LnCOvYHbwn+9MVK0mP28UDp +NYlixuA44fbSnwcYWZJsJ68coFJn0HkCL4s+y41Y0kRGL/gSwWw1dFS6Set4s0v8Z yvzXagGRkeHbAFmSqSPPH4Q9Vpc1PqOPUGGwXZg91oIZUQV3FdqvWqolYC1IOIZQjX rOJLYj/44UPiLVIE/FBvUmrPNMMCP1JNC9v8km3hSC+FBoRGXriLIV+alWWHq9gZbb 5cv/XsIPvtG9fCsQGE8lwHgJCSc/chcNQZzWoxGQAEiQwjxrWg1RmgtoIqiALUPXBy sc8Cr5+lidN+A== Content-Type: multipart/mixed; boundary="------------a7BCL8E0aryxjwaxPO1ltgwQ" Message-ID: <5588db01-45be-48ff-b158-85f789bc502c@postgrespro.ru> Date: Sun, 14 Jan 2024 21:28:50 +0300 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: BUG #18274: Error 'invalid XML content' Content-Language: en-US To: pgsql-bugs@lists.postgresql.org References: <18274-98d16bc03520665f@postgresql.org> <1616257.1704651529@sss.pgh.pa.us> From: Dmitry Koval In-Reply-To: <1616257.1704651529@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format. --------------a7BCL8E0aryxjwaxPO1ltgwQ Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hi! Attached a patch that adds the use of XML_PARSE_HUGE flag for libxml2 functions and replaces some functions (that do not support this flag) with their equivalents. Using libxml2 library functions with support of XML_PARSE_HUGE flag increases maximum size allowed for a single text node from 10.000.000 to 1.000.000.000 (see XML_MAX_TEXT_LENGTH macro, libxml2/include/libxml/parserInternals.h) which in most cases solves the problem with insufficient memory. What do you think about the patch? Maybe it would be a good idea to add a GUC-variable for using of the XML_PARSE_HUGE flag? (The current behavior without XML_PARSE_HUGE flag is default). -- With best regards, Dmitry Koval Postgres Professional: http://postgrespro.com --------------a7BCL8E0aryxjwaxPO1ltgwQ Content-Type: text/plain; charset=UTF-8; name="v1-0001-Added-support-of-XML_PARSE_HUGE-flag-for-XML-docu.patch" Content-Disposition: attachment; filename*0="v1-0001-Added-support-of-XML_PARSE_HUGE-flag-for-XML-docu.pa"; filename*1="tch" Content-Transfer-Encoding: base64 RnJvbSAzN2MxZTg2NThjY2RkZGJlYTkxNTIxZDQ4ODUzOTljNTIwZjJhNjhiIE1vbiBTZXAg MTcgMDA6MDA6MDAgMjAwMQpGcm9tOiBLb3ZhbCBEbWl0cnkgPGQua292YWxAcG9zdGdyZXNw cm8ucnU+CkRhdGU6IFN1biwgMTQgSmFuIDIwMjQgMjE6MDI6NTUgKzAzMDAKU3ViamVjdDog W1BBVENIIHYxXSBBZGRlZCBzdXBwb3J0IG9mIFhNTF9QQVJTRV9IVUdFIGZsYWcgZm9yIFhN TCBkb2N1bW5ldHMKClBvc3RncmVTUUwgdXNlcyBmdW5jdGlvbnMgb2YgbGlieG1sMiBsaWJy YXJ5IHdpdGhvdXQgWE1MX1BBUlNFX0hVR0UgZmxhZy4KQnV0IGluIHByYWN0aWNlLCB0aGUg MTAgTUIgbGltaXQgaXMgdG9vIHNtYWxsLgpVc2luZyBsaWJ4bWwyIGxpYnJhcnkgZnVuY3Rp b25zIHdpdGggc3VwcG9ydCBvZiBYTUxfUEFSU0VfSFVHRSBmbGFnCmluY3JlYXNlcyBtYXhp bXVtIHNpemUgYWxsb3dlZCBmb3IgYSBzaW5nbGUgdGV4dCBub2RlIGZyb20gMTAuMDAwLjAw MCB0bwoxLjAwMC4wMDAuMDAwIChzZWUgWE1MX01BWF9URVhUX0xFTkdUSCBtYWNybykgd2hp Y2ggaW4gbW9zdCBjYXNlcyBzb2x2ZXMKdGhlIHByb2JsZW0gd2l0aCBpbnN1ZmZpY2llbnQg bWVtb3J5LgotLS0KIGNvbnRyaWIveG1sMi94cGF0aC5jICAgICAgICB8ICA3ICsrKystLS0K IGNvbnRyaWIveG1sMi94c2x0X3Byb2MuYyAgICB8IDEwICsrKysrKy0tLS0KIHNyYy9iYWNr ZW5kL3V0aWxzL2FkdC94bWwuYyB8IDI5ICsrKysrKysrKysrKysrKysrKysrKystLS0tLS0t CiAzIGZpbGVzIGNoYW5nZWQsIDMyIGluc2VydGlvbnMoKyksIDE0IGRlbGV0aW9ucygtKQoK ZGlmZiAtLWdpdCBhL2NvbnRyaWIveG1sMi94cGF0aC5jIGIvY29udHJpYi94bWwyL3hwYXRo LmMKaW5kZXggYTY5MmRjNmJlOC4uY2EzNGEwM2I2NCAxMDA2NDQKLS0tIGEvY29udHJpYi94 bWwyL3hwYXRoLmMKKysrIGIvY29udHJpYi94bWwyL3hwYXRoLmMKQEAgLTM4MCw4ICszODAs OCBAQCBwZ3htbF94cGF0aCh0ZXh0ICpkb2N1bWVudCwgeG1sQ2hhciAqeHBhdGgsIHhwYXRo X3dvcmtzcGFjZSAqd29ya3NwYWNlKQogCiAJUEdfVFJZKCk7CiAJewotCQl3b3Jrc3BhY2Ut PmRvY3RyZWUgPSB4bWxQYXJzZU1lbW9yeSgoY2hhciAqKSBWQVJEQVRBX0FOWShkb2N1bWVu dCksCi0JCQkJCQkJCQkJCWRvY3NpemUpOworCQl3b3Jrc3BhY2UtPmRvY3RyZWUgPSB4bWxS ZWFkTWVtb3J5KChjaGFyICopIFZBUkRBVEFfQU5ZKGRvY3VtZW50KSwKKwkJCQkJCQkJCQkg ICBkb2NzaXplLCBOVUxMLCBOVUxMLCBYTUxfUEFSU0VfSFVHRSk7CiAJCWlmICh3b3Jrc3Bh Y2UtPmRvY3RyZWUgIT0gTlVMTCkKIAkJewogCQkJd29ya3NwYWNlLT5jdHh0ID0geG1sWFBh dGhOZXdDb250ZXh0KHdvcmtzcGFjZS0+ZG9jdHJlZSk7CkBAIC02MjQsNyArNjI0LDggQEAg eHBhdGhfdGFibGUoUEdfRlVOQ1RJT05fQVJHUykKIAogCQkJLyogUGFyc2UgdGhlIGRvY3Vt ZW50ICovCiAJCQlpZiAoeG1sZG9jKQotCQkJCWRvY3RyZWUgPSB4bWxQYXJzZU1lbW9yeSh4 bWxkb2MsIHN0cmxlbih4bWxkb2MpKTsKKwkJCQlkb2N0cmVlID0geG1sUmVhZE1lbW9yeSh4 bWxkb2MsIHN0cmxlbih4bWxkb2MpLAorCQkJCQkJCQkJCU5VTEwsIE5VTEwsIFhNTF9QQVJT RV9IVUdFKTsKIAkJCWVsc2UJCQkJLyogdHJlYXQgTlVMTCBhcyBub3Qgd2VsbC1mb3JtZWQg Ki8KIAkJCQlkb2N0cmVlID0gTlVMTDsKIApkaWZmIC0tZ2l0IGEvY29udHJpYi94bWwyL3hz bHRfcHJvYy5jIGIvY29udHJpYi94bWwyL3hzbHRfcHJvYy5jCmluZGV4IDIxODliY2E4NmYu LjliMmM2OWU2ZTcgMTAwNjQ0Ci0tLSBhL2NvbnRyaWIveG1sMi94c2x0X3Byb2MuYworKysg Yi9jb250cmliL3htbDIveHNsdF9wcm9jLmMKQEAgLTg1LDE2ICs4NSwxOCBAQCB4c2x0X3By b2Nlc3MoUEdfRlVOQ1RJT05fQVJHUykKIAkJYm9vbAkJeHNsdF9zZWNfcHJlZnNfZXJyb3I7 CiAKIAkJLyogUGFyc2UgZG9jdW1lbnQgKi8KLQkJZG9jdHJlZSA9IHhtbFBhcnNlTWVtb3J5 KChjaGFyICopIFZBUkRBVEFfQU5ZKGRvY3QpLAotCQkJCQkJCQkgVkFSU0laRV9BTllfRVhI RFIoZG9jdCkpOworCQlkb2N0cmVlID0geG1sUmVhZE1lbW9yeSgoY2hhciAqKSBWQVJEQVRB X0FOWShkb2N0KSwKKwkJCQkJCQkJVkFSU0laRV9BTllfRVhIRFIoZG9jdCksIE5VTEwsIE5V TEwsCisJCQkJCQkJCVhNTF9QQVJTRV9IVUdFKTsKIAogCQlpZiAoZG9jdHJlZSA9PSBOVUxM KQogCQkJeG1sX2VyZXBvcnQoeG1sZXJyY3h0LCBFUlJPUiwgRVJSQ09ERV9FWFRFUk5BTF9S T1VUSU5FX0VYQ0VQVElPTiwKIAkJCQkJCSJlcnJvciBwYXJzaW5nIFhNTCBkb2N1bWVudCIp OwogCiAJCS8qIFNhbWUgZm9yIHN0eWxlc2hlZXQgKi8KLQkJc3Nkb2MgPSB4bWxQYXJzZU1l bW9yeSgoY2hhciAqKSBWQVJEQVRBX0FOWShzc2hlZXQpLAotCQkJCQkJCSAgIFZBUlNJWkVf QU5ZX0VYSERSKHNzaGVldCkpOworCQlzc2RvYyA9IHhtbFJlYWRNZW1vcnkoKGNoYXIgKikg VkFSREFUQV9BTlkoc3NoZWV0KSwKKwkJCQkJCQkgIFZBUlNJWkVfQU5ZX0VYSERSKHNzaGVl dCksIE5VTEwsIE5VTEwsCisJCQkJCQkJICBYTUxfUEFSU0VfSFVHRSk7CiAKIAkJaWYgKHNz ZG9jID09IE5VTEwpCiAJCQl4bWxfZXJlcG9ydCh4bWxlcnJjeHQsIEVSUk9SLCBFUlJDT0RF X0VYVEVSTkFMX1JPVVRJTkVfRVhDRVBUSU9OLApkaWZmIC0tZ2l0IGEvc3JjL2JhY2tlbmQv dXRpbHMvYWR0L3htbC5jIGIvc3JjL2JhY2tlbmQvdXRpbHMvYWR0L3htbC5jCmluZGV4IDE5 MWRkMmQxZTIuLmQ2ZjdkOTVjM2MgMTAwNjQ0Ci0tLSBhL3NyYy9iYWNrZW5kL3V0aWxzL2Fk dC94bWwuYworKysgYi9zcmMvYmFja2VuZC91dGlscy9hZHQveG1sLmMKQEAgLTE3OTUsNyAr MTc5NSw3IEBAIHhtbF9wYXJzZSh0ZXh0ICpkYXRhLCBYbWxPcHRpb25UeXBlIHhtbG9wdGlv bl9hcmcsCiAJCQlkb2MgPSB4bWxDdHh0UmVhZERvYyhjdHh0LCB1dGY4c3RyaW5nLAogCQkJ CQkJCQkgTlVMTCwKIAkJCQkJCQkJICJVVEYtOCIsCi0JCQkJCQkJCSBYTUxfUEFSU0VfTk9F TlQgfCBYTUxfUEFSU0VfRFREQVRUUgorCQkJCQkJCQkgWE1MX1BBUlNFX05PRU5UIHwgWE1M X1BBUlNFX0RUREFUVFIgfCBYTUxfUEFSU0VfSFVHRQogCQkJCQkJCQkgfCAocHJlc2VydmVf d2hpdGVzcGFjZSA/IDAgOiBYTUxfUEFSU0VfTk9CTEFOS1MpKTsKIAkJCWlmIChkb2MgPT0g TlVMTCB8fCB4bWxlcnJjeHQtPmVycl9vY2N1cnJlZCkKIAkJCXsKQEAgLTE4MjgsMTAgKzE4 MjgsMjUgQEAgeG1sX3BhcnNlKHRleHQgKmRhdGEsIFhtbE9wdGlvblR5cGUgeG1sb3B0aW9u X2FyZywKIAkJCS8qIGFsbG93IGVtcHR5IGNvbnRlbnQgKi8KIAkJCWlmICgqKHV0ZjhzdHJp bmcgKyBjb3VudCkpCiAJCQl7Ci0JCQkJcmVzX2NvZGUgPSB4bWxQYXJzZUJhbGFuY2VkQ2h1 bmtNZW1vcnkoZG9jLCBOVUxMLCBOVUxMLCAwLAotCQkJCQkJCQkJCQkJCSAgIHV0ZjhzdHJp bmcgKyBjb3VudCwKLQkJCQkJCQkJCQkJCQkgICBwYXJzZWRfbm9kZXMpOwotCQkJCWlmIChy ZXNfY29kZSAhPSAwIHx8IHhtbGVycmN4dC0+ZXJyX29jY3VycmVkKQorCQkJCWNvbnN0IGNo YXIgKmRhdGE7CisJCQkJeG1sTm9kZVB0cglyb290OworCQkJCXhtbFBhcnNlckVycm9ycyB4 bWxfZXJyb3I7CisJCQkJeG1sTm9kZVB0cglsc3Q7CisKKwkJCQlkYXRhID0gKGNvbnN0IGNo YXIgKikgKHV0ZjhzdHJpbmcgKyBjb3VudCk7CisKKwkJCQkvKiBDcmVhdGUgZmFrZSByb290 IG5vZGUuICovCisJCQkJcm9vdCA9IHhtbE5ld05vZGUoTlVMTCwgKGNvbnN0IHhtbENoYXIg KikgImNvbnRlbnQtcm9vdCIpOworCQkJCWlmIChyb290ID09IE5VTEwgfHwgeG1sZXJyY3h0 LT5lcnJfb2NjdXJyZWQpCisJCQkJCXhtbF9lcmVwb3J0KHhtbGVycmN4dCwgRVJST1IsIEVS UkNPREVfT1VUX09GX01FTU9SWSwKKwkJCQkJCQkJImNvdWxkIG5vdCBhbGxvY2F0ZSB4bWwg bm9kZSIpOworCQkJCXhtbERvY1NldFJvb3RFbGVtZW50KGRvYywgcm9vdCk7CisKKwkJCQkv KiBUcnkgdG8gcGFyc2Ugc3RyaW5nIHdpdGggdXNpbmcgcm9vdCBub2RlIGNvbnRleHQuICov CisJCQkJeG1sX2Vycm9yID0geG1sUGFyc2VJbk5vZGVDb250ZXh0KHJvb3QsIGRhdGEsIHN0 cmxlbihkYXRhKSwKKwkJCQkJCQkJCQkJCSAgWE1MX1BBUlNFX0hVR0UsCisJCQkJCQkJCQkJ CQkgIHBhcnNlZF9ub2RlcyA/IHBhcnNlZF9ub2RlcyA6ICZsc3QpOworCQkJCWlmICh4bWxf ZXJyb3IgIT0gWE1MX0VSUl9PSyB8fCB4bWxlcnJjeHQtPmVycl9vY2N1cnJlZCkKIAkJCQl7 CiAJCQkJCXhtbF9lcnJzYXZlKGVzY29udGV4dCwgeG1sZXJyY3h0LAogCQkJCQkJCQlFUlJD T0RFX0lOVkFMSURfWE1MX0NPTlRFTlQsCkBAIC00MzQ0LDcgKzQzNTksNyBAQCB4cGF0aF9p bnRlcm5hbCh0ZXh0ICp4cGF0aF9leHByX3RleHQsIHhtbHR5cGUgKmRhdGEsIEFycmF5VHlw ZSAqbmFtZXNwYWNlcywKIAkJCXhtbF9lcmVwb3J0KHhtbGVycmN4dCwgRVJST1IsIEVSUkNP REVfT1VUX09GX01FTU9SWSwKIAkJCQkJCSJjb3VsZCBub3QgYWxsb2NhdGUgcGFyc2VyIGNv bnRleHQiKTsKIAkJZG9jID0geG1sQ3R4dFJlYWRNZW1vcnkoY3R4dCwgKGNoYXIgKikgc3Ry aW5nICsgeG1sZGVjbF9sZW4sCi0JCQkJCQkJCWxlbiAtIHhtbGRlY2xfbGVuLCBOVUxMLCBO VUxMLCAwKTsKKwkJCQkJCQkJbGVuIC0geG1sZGVjbF9sZW4sIE5VTEwsIE5VTEwsIFhNTF9Q QVJTRV9IVUdFKTsKIAkJaWYgKGRvYyA9PSBOVUxMIHx8IHhtbGVycmN4dC0+ZXJyX29jY3Vy cmVkKQogCQkJeG1sX2VyZXBvcnQoeG1sZXJyY3h0LCBFUlJPUiwgRVJSQ09ERV9JTlZBTElE X1hNTF9ET0NVTUVOVCwKIAkJCQkJCSJjb3VsZCBub3QgcGFyc2UgWE1MIGRvY3VtZW50Iik7 CkBAIC00Njc1LDcgKzQ2OTAsNyBAQCBYbWxUYWJsZVNldERvY3VtZW50KFRhYmxlRnVuY1Nj YW5TdGF0ZSAqc3RhdGUsIERhdHVtIHZhbHVlKQogCiAJUEdfVFJZKCk7CiAJewotCQlkb2Mg PSB4bWxDdHh0UmVhZE1lbW9yeSh4dEN4dC0+Y3R4dCwgKGNoYXIgKikgeHN0ciwgbGVuZ3Ro LCBOVUxMLCBOVUxMLCAwKTsKKwkJZG9jID0geG1sQ3R4dFJlYWRNZW1vcnkoeHRDeHQtPmN0 eHQsIChjaGFyICopIHhzdHIsIGxlbmd0aCwgTlVMTCwgTlVMTCwgWE1MX1BBUlNFX0hVR0Up OwogCQlpZiAoZG9jID09IE5VTEwgfHwgeHRDeHQtPnhtbGVycmN4dC0+ZXJyX29jY3VycmVk KQogCQkJeG1sX2VyZXBvcnQoeHRDeHQtPnhtbGVycmN4dCwgRVJST1IsIEVSUkNPREVfSU5W QUxJRF9YTUxfRE9DVU1FTlQsCiAJCQkJCQkiY291bGQgbm90IHBhcnNlIFhNTCBkb2N1bWVu dCIpOwotLSAKMi40MC4xLndpbmRvd3MuMQoK --------------a7BCL8E0aryxjwaxPO1ltgwQ--