[Hans Buschmann <buschmann@nidsa.net>]

During some development on encoding-related parts of postgres I stumbled over the new length-checking-code in common/hex.c/pg_hex_encode.

Differently when comparing it to all versions before the output-buffer-length is checked during encoding of every individual source byte.

This may impose quite a regression when using pg_dump on databases with many/big bytea or lo columns.

Because all criteria to check buffer-length are known in advance of encoding (srclen and destlen) I propose doing the check only once before starting the while-loop.

Please find the attached patch for pg14 and master, older versions did not have this behavior.

Tested on pg14-beta3, but applies also on master.

PS: This is my very first patch, I am in no way an experienced C-developer and don't have a smoothly running development environment or experience yet. (originally mostly dealing with postgres on Windows).

If it seems useful somebody could enter it as an open item / resolved item for pg14 after beta 3.

Thanks for looking!

Hans Buschmann

=

Attachments:

hex_encode_length_check_outside_loop.patch (application/octet-stream, 1.0K) download
diff --git a/postgresql-14beta3/src/common/hex.c b/postgresql-14beta3_hb/src/common/hex.c
index 3b1bc8f..fa23564 100644
--- a/postgresql-14beta3/src/common/hex.c
+++ b/postgresql-14beta3_hb/src/common/hex.c
@@ -78,22 +78,22 @@ pg_hex_encode(const char *src, size_t srclen, char *dst, size_t dstlen)
 
 	p = dst;
 
-	while (src < end)
+	/*
+	 * Leave if there is an overflow in the area allocated for the encoded
+	 * string.
+	 */
+	if (dstlen < srclen << 1)
 	{
-		/*
-		 * Leave if there is an overflow in the area allocated for the encoded
-		 * string.
-		 */
-		if ((p - dst + 2) > dstlen)
-		{
 #ifdef FRONTEND
-			pg_log_fatal("overflow of destination buffer in hex encoding");
-			exit(EXIT_FAILURE);
+		pg_log_fatal("overflow of destination buffer in hex encoding");
+		exit(EXIT_FAILURE);
 #else
-			elog(ERROR, "overflow of destination buffer in hex encoding");
+		elog(ERROR, "overflow of destination buffer in hex encoding");
 #endif
-		}
+	}
 
+	while (src < end)
+	{
 		*p++ = hextbl[(*src >> 4) & 0xF];
 		*p++ = hextbl[*src & 0xF];
 		src++;