Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mFYGz-0006UW-Vx for pgsql-hackers@arkaria.postgresql.org; Mon, 16 Aug 2021 08:47:10 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1mFYGT-0000ml-DS for pgsql-hackers@arkaria.postgresql.org; Mon, 16 Aug 2021 08:46:37 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mFHbW-0005C2-84 for pgsql-hackers@lists.postgresql.org; Sun, 15 Aug 2021 14:59:14 +0000 Received: from starship.nidsa.net ([212.227.110.112]) by magus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mFHbU-0007AQ-98 for pgsql-hackers@postgresql.org; Sun, 15 Aug 2021 14:59:13 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=nidsa.net; s=k1; h=MIME-Version:Content-Type:Message-ID:Date:Subject:To:From:Sender: Reply-To:Cc:Content-Transfer-Encoding:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: In-Reply-To:References:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=jbqhNhHynGA6akZ1nzIsAGErOt3QYchSSSA9XFeBY8k=; b=AF96raOd0hB3OyvTgOb3a691Zs hKHIiw7Y1GQHxGUnW9xWpDAEOAR5kPgxfarocBR2PvUe6FfbkJ3fM7cc/Xozl75hKri0Qi5VWlwUg 5uN1Rbv0q+dkz3cuA+Bv6iKd2NpwO6VLHY0819Opy7z1f+9ykSK39ZFxTn6jILUzfTiY=; Received: from [10.13.1.101] (port=8914 helo=starship.nidsa.net) by starship.nidsa.net with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 (Exim 4.94.2) (envelope-from ) id 1mFHWx-000DO9-2O for pgsql-hackers@postgresql.org; Sun, 15 Aug 2021 16:59:06 +0200 Received: from W2012-02.nidsa.loc (10.13.1.101) by W2012-02.nidsa.loc (10.13.1.101) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Sun, 15 Aug 2021 16:58:59 +0200 Received: from W2012-02.nidsa.loc ([::1]) by W2012-02.nidsa.loc ([::1]) with mapi id 15.00.1497.023; Sun, 15 Aug 2021 16:58:59 +0200 X-SASI-Hits: BODYTEXTP_SIZE_3000_LESS 0.000000, BODY_SIZE_3000_3999 0.000000, BODY_SIZE_5000_LESS 0.000000, BODY_SIZE_7000_LESS 0.000000, HTML_00_01 0.050000, HTML_00_10 0.050000, INVOICE_ATTACHMENT 0.100000, NO_CTA_URI_FOUND 0.000000, NO_URI_FOUND 0.000000, NO_URI_HTTPS 0.000000, OUTBOUND 0.000000, OUTBOUND_SOPHOS 0.000000, SENDER_NO_AUTH 0.000000, SUPERLONG_LINE 0.050000, WEBMAIL_SOURCE 0.000000, WEBMAIL_XOIP 0.000000, WEBMAIL_X_IP_HDR 0.000000, __ATTACHMENT_NOT_IMG 0.000000, __ATTACHMENT_PHRASE 0.000000, __ATTACHMENT_SIZE_0_10K 0.000000, __BODY_NO_MAILTO 0.000000, __CT 0.000000, __CTYPE_HAS_BOUNDARY 0.000000, __CTYPE_MULTIPART 0.000000, __CTYPE_MULTIPART_MIXED 0.000000, __DQ_NEG_HEUR 0.000000, __DQ_NEG_IP 0.000000, __FROM_DOMAIN_NOT_IN_BODY 0.000000, __FUR_RDNS_SOPHOS 0.000000, __FW_1LN_BOT_MSGID 0.000000, __HAS_ATTACHMENT 0.000000, __HAS_ATTACHMENT1 0.000000, __HAS_ATTACHMENT2 0.000000, __HAS_FROM 0.000000, __HAS_MSGID 0.000000, __HAS_XOIP 0.000000, __INVOICE_MULTILINGUAL 0.000000, __MIME_TEXT_P 0.000000, __MIME_TEXT_P1 0.000000, __MIME_TEXT_P2 0.000000, __MIME_VERSION 0.000000, __NO_HTML_TAG_RAW 0.000000, __OUTBOUND_SOPHOS_FUR 0.000000, __OUTBOUND_SOPHOS_FUR_IP 0.000000, __OUTBOUND_SOPHOS_FUR_RDNS 0.000000, __PHISH_PHRASE10_D 0.000000, __SANE_MSGID 0.000000, __TO_MALFORMED_2 0.000000, __TO_NAME 0.000000, __TO_NO_NAME 0.000000, __URI_NO_MAILTO 0.000000 X-SASI-Probability: 8% X-SASI-RCODE: 200 X-SASI-Version: Antispam-Engine: 4.1.4, AntispamData: 2021.8.15.143016 From: Hans Buschmann To: "pgsql-hackers@postgresql.org" Subject: PG14: Avoid checking output-buffer-length for every encoded byte during pg_hex_encode Thread-Topic: PG14: Avoid checking output-buffer-length for every encoded byte during pg_hex_encode Thread-Index: AQHXkeXisWtiOvbVw02XRaWte1fJUg== Date: Sun, 15 Aug 2021 14:58:59 +0000 Message-ID: <1629039545467.80333@nidsa.net> Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: yes X-MS-TNEF-Correlator: x-ms-exchange-transport-fromentityheader: Hosted x-originating-ip: [185.101.185.196] Content-Type: multipart/mixed; boundary="_002_162903954546780333nidsanet_" MIME-Version: 1.0 List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk --_002_162903954546780333nidsanet_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable During some development on encoding-related parts of postgres I stumbled ov= er the new length-checking-code in common/hex.c/pg_hex_encode.=0A= =0A= Differently when comparing it to all versions before the output-buffer-leng= th is checked during encoding of every individual source byte.=0A= =0A= This may impose quite a regression when using pg_dump on databases with man= y/big bytea or lo columns.=0A= =0A= Because all criteria to check buffer-length are known in advance of encodin= g (srclen and destlen) I propose doing the check only once before starting = the while-loop.=0A= =0A= Please find the attached patch for pg14 and master, older versions did not = have this behavior.=0A= =0A= Tested on pg14-beta3, but applies also on master.=0A= =0A= PS: This is my very first patch, I am in no way an experienced C-developer = and don't have a smoothly running development environment or experience yet= . (originally mostly dealing with postgres on Windows).=0A= =0A= If it seems useful somebody could enter it as an open item / resolved item = for pg14 after beta 3.=0A= =0A= Thanks for looking!=0A= =0A= Hans Buschmann=0A= =0A= --_002_162903954546780333nidsanet_ Content-Type: application/octet-stream; name="hex_encode_length_check_outside_loop.patch" Content-Description: hex_encode_length_check_outside_loop.patch Content-Disposition: attachment; filename="hex_encode_length_check_outside_loop.patch"; size=1059; creation-date="Sun, 15 Aug 2021 14:57:37 GMT"; modification-date="Sun, 15 Aug 2021 14:57:37 GMT" Content-Transfer-Encoding: base64 ZGlmZiAtLWdpdCBhL3Bvc3RncmVzcWwtMTRiZXRhMy9zcmMvY29tbW9uL2hleC5jIGIvcG9zdGdy ZXNxbC0xNGJldGEzX2hiL3NyYy9jb21tb24vaGV4LmMKaW5kZXggM2IxYmM4Zi4uZmEyMzU2NCAx MDA2NDQKLS0tIGEvcG9zdGdyZXNxbC0xNGJldGEzL3NyYy9jb21tb24vaGV4LmMKKysrIGIvcG9z dGdyZXNxbC0xNGJldGEzX2hiL3NyYy9jb21tb24vaGV4LmMKQEAgLTc4LDIyICs3OCwyMiBAQCBw Z19oZXhfZW5jb2RlKGNvbnN0IGNoYXIgKnNyYywgc2l6ZV90IHNyY2xlbiwgY2hhciAqZHN0LCBz aXplX3QgZHN0bGVuKQogCiAJcCA9IGRzdDsKIAotCXdoaWxlIChzcmMgPCBlbmQpCisJLyoKKwkg KiBMZWF2ZSBpZiB0aGVyZSBpcyBhbiBvdmVyZmxvdyBpbiB0aGUgYXJlYSBhbGxvY2F0ZWQgZm9y IHRoZSBlbmNvZGVkCisJICogc3RyaW5nLgorCSAqLworCWlmIChkc3RsZW4gPCBzcmNsZW4gPDwg MSkKIAl7Ci0JCS8qCi0JCSAqIExlYXZlIGlmIHRoZXJlIGlzIGFuIG92ZXJmbG93IGluIHRoZSBh cmVhIGFsbG9jYXRlZCBmb3IgdGhlIGVuY29kZWQKLQkJICogc3RyaW5nLgotCQkgKi8KLQkJaWYg KChwIC0gZHN0ICsgMikgPiBkc3RsZW4pCi0JCXsKICNpZmRlZiBGUk9OVEVORAotCQkJcGdfbG9n X2ZhdGFsKCJvdmVyZmxvdyBvZiBkZXN0aW5hdGlvbiBidWZmZXIgaW4gaGV4IGVuY29kaW5nIik7 Ci0JCQlleGl0KEVYSVRfRkFJTFVSRSk7CisJCXBnX2xvZ19mYXRhbCgib3ZlcmZsb3cgb2YgZGVz dGluYXRpb24gYnVmZmVyIGluIGhleCBlbmNvZGluZyIpOworCQlleGl0KEVYSVRfRkFJTFVSRSk7 CiAjZWxzZQotCQkJZWxvZyhFUlJPUiwgIm92ZXJmbG93IG9mIGRlc3RpbmF0aW9uIGJ1ZmZlciBp biBoZXggZW5jb2RpbmciKTsKKwkJZWxvZyhFUlJPUiwgIm92ZXJmbG93IG9mIGRlc3RpbmF0aW9u IGJ1ZmZlciBpbiBoZXggZW5jb2RpbmciKTsKICNlbmRpZgotCQl9CisJfQogCisJd2hpbGUgKHNy YyA8IGVuZCkKKwl7CiAJCSpwKysgPSBoZXh0YmxbKCpzcmMgPj4gNCkgJiAweEZdOwogCQkqcCsr ID0gaGV4dGJsWypzcmMgJiAweEZdOwogCQlzcmMrKzsK --_002_162903954546780333nidsanet_--