public inbox for [email protected]
help / color / mirror / Atom feedFrom: Joel Jacobson <[email protected]>
To: [email protected]
Subject: [BUG] psql: Make \copy from 'text' and 'csv' formats fail on NUL bytes
Date: Sun, 10 Nov 2024 22:26:20 +0100
Message-ID: <[email protected]> (raw)
Hi hackers,
When using \copy from in psql to import files containing NUL bytes (\0) in
'text' or 'csv' format, the NUL bytes were not detected and did not result in
an error, leading to silent data corruption.
This behavior is inconsistent with server-side COPY FROM, which reports an error
upon encountering NUL bytes in the 'text' and 'csv' formats.
Fix by adjusting handleCopyIn() to use the binary code path also when the copy
source is a file (i.e., copystream != pset.cur_cmd_source), even in textual
copies. This ensures that NUL bytes are detected and reported in the same way
as server-side COPY.
Example:
% printf 'test_col\nfoo\n\x00\nbar\nbaz\x00aar\nbizarre\n' > nul_bytes.data
% cat -v nul_bytes.data
test_col
foo
^@
bar
baz^@aar
bizarre
% psql
create table nul_bytes_test (test_col text);
--
-- HEAD
--
\copy nul_bytes_test (test_col) from 'nul_bytes.data' (format csv, header match);
COPY 3
select * from nul_bytes_test;
test_col
------------
foo
bar
bazbizarre
(3 rows)
\copy nul_bytes_test (test_col) from 'nul_bytes.data' (format text, header match);
select * from nul_bytes_test;
test_col
------------
foo
bar
bazbizarre
(3 rows)
--
-- 0001-psql-Make-copy-from-text-and-csv-formats-fail-on-NUL.patch
--
\copy nul_bytes_test (test_col) from 'nul_bytes.data' (format csv, header match);
ERROR: invalid byte sequence for encoding "UTF8": 0x00
CONTEXT: COPY nul_bytes_test, line 3
\copy nul_bytes_test (test_col) from 'nul_bytes.data' (format text, header match);
ERROR: invalid byte sequence for encoding "UTF8": 0x00
CONTEXT: COPY nul_bytes_test, line 3
/Joel
Attachments:
[application/octet-stream] 0001-psql-Make-copy-from-text-and-csv-formats-fail-on-NUL.patch (1.3K, ../[email protected]/2-0001-psql-Make-copy-from-text-and-csv-formats-fail-on-NUL.patch)
download | inline diff:
From b6824faf03d4d4bdb909b33230fab0011cbcfc69 Mon Sep 17 00:00:00 2001
From: Joel Jacobson <[email protected]>
Date: Sun, 10 Nov 2024 22:18:48 +0100
Subject: [PATCH] psql: Make \copy from 'text' and 'csv' formats fail on NUL
bytes
When using \copy from in psql to import files containing NUL bytes (\0) in
'text' or 'csv' format, the NUL bytes were not detected and did not result in
an error, leading to silent data corruption.
This behavior is inconsistent with server-side COPY FROM, which reports an error
upon encountering NUL bytes in the 'text' and 'csv' formats.
Fix by adjusting handleCopyIn() to use the binary code path also when the copy
source is a file (i.e., copystream != pset.cur_cmd_source), even in textual
copies. This ensures that NUL bytes are detected and reported in the same way
as server-side COPY.
---
src/bin/psql/copy.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/bin/psql/copy.c b/src/bin/psql/copy.c
index e020e4d665d..306f5161e1d 100644
--- a/src/bin/psql/copy.c
+++ b/src/bin/psql/copy.c
@@ -544,7 +544,7 @@ handleCopyIn(PGconn *conn, FILE *copystream, bool isbinary, PGresult **res)
OK = true;
- if (isbinary)
+ if (isbinary || copystream != pset.cur_cmd_source)
{
/* interactive input probably silly, but give one prompt anyway */
if (showprompt)
--
2.45.1
view thread (2+ messages)
reply
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: [email protected]
Cc: [email protected], [email protected]
Subject: Re: [BUG] psql: Make \copy from 'text' and 'csv' formats fail on NUL bytes
In-Reply-To: <[email protected]>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox