From: Heikki Linnakangas Date: Wed, 3 Feb 2021 14:00:32 +0200 Subject: [PATCH 1/1] Fix a corner-case in COPY FROM backslash processing. If a multi-byte character is escaped with a backslash in TEXT mode input, we didn't always treat the escape correctly. If: - a multi-byte character is escaped with a backslash, and - the second byte of the character is 0x5C, i.e. the ASCII code of a backslash (\), and - the next character is a dot (.), then copyreadline we would incorrectly interpret the sequence as an end-of-copy marker (\.). this can only happen in encodings that can "embed" ascii characters as the second byte. one example of such sequence is '\x5ca45c2e666f6f' in Big5 encoding. If you put that in a file, and load it with COPY FROM, you'd incorrectly get an "end-of-copy marker corrupt" error. --- src/backend/commands/copyfromparse.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/src/backend/commands/copyfromparse.c b/src/backend/commands/copyfromparse.c index 798e18e013..c20ec482db 100644 --- a/src/backend/commands/copyfromparse.c +++ b/src/backend/commands/copyfromparse.c @@ -1084,7 +1084,7 @@ CopyReadLineText(CopyFromState cstate) break; } else if (!cstate->opts.csv_mode) - + { /* * If we are here, it means we found a backslash followed by * something other than a period. In non-CSV mode, anything @@ -1095,8 +1095,17 @@ CopyReadLineText(CopyFromState cstate) * backslashes are not special, so we want to process the * character after the backslash just like a normal character, * so we don't increment in those cases. + * + * In principle, we would need to use pg_encoding_mblen() to + * skip over the character in some encodings, like at the end + * of the loop. But if it's a multi-byte character, it cannot + * have any special meaning and skipping isn't necessary for + * correctness. We can fall through to process it normally + * instead. */ - raw_buf_ptr++; + if (!IS_HIGHBIT_SET(c2)) + raw_buf_ptr++; + } } /* -- 2.30.0 --------------97F3138F3612F1A4F9200D93--