postgres.git / summary / log / commit / refs

commit    c444472af5c202067a9ecb0ff8df7370fb1ea8f4
Author:   Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date:     Fri Feb 05 09:14:56 2021 +0000
First released in: 14.0, 9.5.25, 9.6.21, 10.16, 11.11, 12.6, 13.2

    Fix backslash-escaping multibyte chars in COPY FROM.
    
    If a multi-byte character is escaped with a backslash in TEXT mode input,
    and the encoding is one of the client-only encodings where the bytes after
    the first one can have an ASCII byte "embedded" in the char, we didn't
    skip the character correctly. After a backslash, we only skipped the first
    byte of the next character, so if it was a multi-byte character, we would
    try to process its second byte as if it was a separate character. If it
    was one of the characters with special meaning, like '\n', '\r', or
    another '\\', that would cause trouble.
    
    One such exmple is the byte sequence '\x5ca45c2e666f6f' in Big5 encoding.
    That's supposed to be [backslash][two-byte character][.][f][o][o], but
    because the second byte of the two-byte character is 0x5c, we incorrectly
    treat it as another backslash. And because the next character is a dot, we
    parse it as end-of-copy marker, and throw an "end-of-copy marker corrupt"
    error.
    
    Backpatch to all supported versions.
    
    Reviewed-by: John Naylor, Kyotaro Horiguchi
    Discussion: https://www.postgresql.org/message-id/a897f84f-8dca-8798-3139-07da5bb38728%40iki.fi


 src/backend/commands/copyfromparse.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/src/backend/commands/copyfromparse.c b/src/backend/commands/copyfromparse.c
index b843d315b17..315b16fd7af 100644
--- a/src/backend/commands/copyfromparse.c
+++ b/src/backend/commands/copyfromparse.c
@@ -1084,7 +1084,7 @@ CopyReadLineText(CopyFromState cstate)
 				break;
 			}
 			else if (!cstate->opts.csv_mode)
-
+			{
 				/*
 				 * If we are here, it means we found a backslash followed by
 				 * something other than a period.  In non-CSV mode, anything
@@ -1095,8 +1095,16 @@ CopyReadLineText(CopyFromState cstate)
 				 * backslashes are not special, so we want to process the
 				 * character after the backslash just like a normal character,
 				 * so we don't increment in those cases.
+				 *
+				 * Set 'c' to skip whole character correctly in multi-byte
+				 * encodings.  If we don't have the whole character in the
+				 * buffer yet, we might loop back to process it, after all,
+				 * but that's OK because multi-byte characters cannot have any
+				 * special meaning.
 				 */
 				raw_buf_ptr++;
+				c = c2;
+			}
 		}
 
 		/*


[parent: 5e7fa189ee92]