From: Tom Lane <tgl@sss.pgh.pa.us>
To: Rick Parrish <ai5jt@unitrunker.net>
cc: bugs <pgsql-bugs@lists.postgresql.org>
Subject: Re: psql include text file with bom
In-reply-to: <81ca2b25-6b3a-499a-9a09-2dd21253c2cb@unitrunker.net>
References: <81ca2b25-6b3a-499a-9a09-2dd21253c2cb@unitrunker.net>
Comments: In-reply-to Rick Parrish <ai5jt@unitrunker.net>
	message dated "Sat, 25 Nov 2023 23:46:37 -0600"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <3852218.1701096569.1@sss.pgh.pa.us>
Date: Mon, 27 Nov 2023 09:49:29 -0500
Message-ID: <3852219.1701096569@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/3852219.1701096569%40sss.pgh.pa.us>
Precedence: bulk

Rick Parrish <ai5jt@unitrunker.net> writes:
> I think it would be reasonable for the include command to skip over any 
> BOM found in the first two or three bytes of a file.

This has been proposed before, and rejected before.  psql has no
inherent knowledge of what encoding an input file is in, and therefore
no justification to assume that a bit-pattern it sees there is a BOM.
In non-UTF8 encodings it could very easily be valid data.

(For that matter, it's also valid data in UTF8: it's the same bit
pattern as U+FEFF ZERO WIDTH NO-BREAK SPACE.  Programs that emit
one into UTF8 streams, and expect it not to be taken as data,
are frankly broken.)

			regards, tom lane