Date: Sun, 13 Oct 2024 18:52:04 +0900 (JST)
Message-Id: <20241013.185204.1992992241505690866.ishii@postgresql.org>
To: joel@compiler.org
Cc: pgsql-hackers@lists.postgresql.org
Subject: Re: New "raw" COPY format
From: Tatsuo Ishii <ishii@postgresql.org>
In-Reply-To: <c12516b1-77dc-4ad3-94a7-88527360aee0@app.fastmail.com>
References: <c12516b1-77dc-4ad3-94a7-88527360aee0@app.fastmail.com>
Mime-Version: 1.0
Content-Type: Text/Plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Archived-At: <https://www.postgresql.org/message-id/20241013.185204.1992992241505690866.ishii%40postgresql.org>
Precedence: bulk

> Hi hackers,
> 
> This thread is about implementing a new "raw" COPY format.
> 
> This idea came up in a different thread [1], moved here.
> 
> [1] https://postgr.es/m/47b5c6a7-5c0e-40aa-8ea2-c7b95ccf296f%40app.fastmail.com
> 
> The main use-case for the raw format, is when needing to import arbitrary
> unstructured text files, such as log files, into a single text column
> of a table.

After copy imported the "unstructured text file" in "row" COPY format,
what the column type is?  text? or bytea? If it's text, how do you
handle encoding conversion if the "unstructured text file" is encoded
in server side unsafe encoding such as SJIS?

>  All characters are taken literally.
>  There is no special handling for quotes, backslashes, or escape sequences.

If SJIS text is imported "literally" (i.e. no encoding conversion), it
should be rejected.

Best reagards,
--
Tatsuo Ishii
SRA OSS K.K.
English: http://www.sraoss.co.jp/index_en/
Japanese:http://www.sraoss.co.jp