Received: from malur.postgresql.org ([217.196.149.56])
	by arkaria.postgresql.org with esmtps  (TLS1.3) tls
 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
	(Exim 4.94.2)
	(envelope-from <pgsql-bugs-owner+archive@lists.postgresql.org>)
	id 1r7XhV-005lZz-BM
	for pgsql-bugs@arkaria.postgresql.org; Mon, 27 Nov 2023 09:14:45 +0000
Received: from localhost ([127.0.0.1] helo=malur.postgresql.org)
	by malur.postgresql.org with esmtp (Exim 4.94.2)
	(envelope-from <pgsql-bugs-owner+archive@lists.postgresql.org>)
	id 1r7XhT-00BuuD-UC
	for pgsql-bugs@arkaria.postgresql.org; Mon, 27 Nov 2023 09:14:43 +0000
Received: from makus.postgresql.org ([2001:4800:3e1:1::229])
	by malur.postgresql.org with esmtps  (TLS1.3) tls
 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
	(Exim 4.94.2)
	(envelope-from <ai5jt@unitrunker.net>)
	id 1r77yl-006w43-Es
	for pgsql-bugs@lists.postgresql.org; Sun, 26 Nov 2023 05:46:51 +0000
Received: from relay9-d.mail.gandi.net ([217.70.183.199])
	by makus.postgresql.org with esmtps  (TLS1.2) tls
 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
	(Exim 4.94.2)
	(envelope-from <ai5jt@unitrunker.net>)
	id 1r77yd-007Orx-Nm
	for pgsql-bugs@lists.postgresql.org; Sun, 26 Nov 2023 05:46:49 +0000
Received: by mail.gandi.net (Postfix) with ESMTPSA id CF629FF805
	for <pgsql-bugs@lists.postgresql.org>; Sun, 26 Nov 2023 05:46:39 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=unitrunker.net;
	s=gm1; t=1700977600;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:mime-version:mime-version:content-type:content-type;
	bh=XQEgxLNxGK8uREg5ktJ2ZBOfqyKTbhuXBL9JSEIc8Io=;
	b=ByGTfl5NhIUvoKgmVp+lJcxjFg+WG46BiG7x85E5oJEC/yut0jzLTsBozMmITGKyPDy0G9
	/vXr4v39TYk9QywWw7e0xKdhzJh2Et67FbXzahNNTw6mPnwndkiqmwFDK0q+b/JXbkzhOr
	hTau9G+SIZ0c0O/b/Zd/+mPaK04lpMTRiWOhzB64Rnbln0g0UGSFO02BETUGuxSuy1ijA+
	AG3fnQwd7t/SDnXobu2DyHOaFqooG9ro4nV9DEjuALUl5nHyMkBuK08oGVSR2fqQezdBmA
	35z1B7ZKI1P2HCLwHXmQi+BvdTjebF6OTm4PMHwUpHKUQ8dfzWItwakfytQYKA==
Content-Type: multipart/alternative;
 boundary="------------Qzyr0r0V0omGbnZ0PL3bJ9oa"
Message-ID: <81ca2b25-6b3a-499a-9a09-2dd21253c2cb@unitrunker.net>
Date: Sat, 25 Nov 2023 23:46:37 -0600
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: bugs <pgsql-bugs@lists.postgresql.org>
From: Rick Parrish <ai5jt@unitrunker.net>
Subject: psql include text file with bom
X-GND-Sasl: ai5jt@unitrunker.net
List-Id: <pgsql-bugs.lists.postgresql.org>
List-Help: <https://lists.postgresql.org/manage/>
List-Subscribe: <https://lists.postgresql.org/manage/>
List-Post: <mailto:pgsql-bugs@lists.postgresql.org>
List-Owner: <mailto:pgsql-bugs-owner@lists.postgresql.org>
List-Archive: <https://www.postgresql.org/list/pgsql-bugs>
Archived-At: 
 <https://www.postgresql.org/message-id/81ca2b25-6b3a-499a-9a09-2dd21253c2cb%40unitrunker.net>
Precedence: bulk

This is a multi-part message in MIME format.
--------------Qzyr0r0V0omGbnZ0PL3bJ9oa
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

*Summary:*

psql "include" or "\i" command chokes on UTF8 text files prefixed with BOM.

*Steps to reproduce:*

 1. create a UTF8 file with three byte BOM 'EF BB BF'.
 2. include the file from psql via the "include" or \i command.

Example output for file named "test.sql" below:

redacted-# \i test.sql
psql:test.sql:1: ERROR:  syntax error at or near ""
LINE 1: 

*Background*

https://en.wikipedia.org/wiki/Byte_order_mark

Some text editors save text to a file prefixed by a BOM or byte marker. 
This includes Visual Studio, VSCode and others.

I think it would be reasonable for the include command to skip over any 
BOM found in the first two or three bytes of a file.
--------------Qzyr0r0V0omGbnZ0PL3bJ9oa
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit

<!DOCTYPE html>
<html>
  <head>

    <meta http-equiv="content-type" content="text/html; charset=UTF-8">
  </head>
  <body>
    <p><b>Summary:</b></p>
    <p>psql "include" or "\i" command chokes on UTF8 text files prefixed
      with BOM.</p>
    <p><b>Steps to reproduce:</b></p>
    <ol>
      <li>create a UTF8 file with three byte BOM 'EF BB BF'.</li>
      <li>include the file from psql via the "include" or \i command.</li>
    </ol>
    <p>Example output for file named "test.sql" below:<br>
    </p>
    <p>redacted-# \i test.sql<br>
      psql:test.sql:1: ERROR:  syntax error at or near ""<br>
      LINE 1: </p>
    <p><b>Background</b></p>
    <p><a class="moz-txt-link-freetext" href="https://en.wikipedia.org/wiki/Byte_order_mark">https://en.wikipedia.org/wiki/Byte_order_mark</a></p>
    <p>Some text editors save text to a file prefixed by a BOM or byte
      marker. This includes Visual Studio, VSCode and others.</p>
    I think it would be reasonable for the include command to skip over
    any BOM found in the first two or three bytes of a file.
  </body>
</html>

--------------Qzyr0r0V0omGbnZ0PL3bJ9oa--