Date: Sat, 8 Sep 2018 16:36:19 -0400
From: 'Bruce Momjian' <bruce@momjian.us>
To: =?iso-8859-1?Q?Andr=E9_H=E4nsel?= <andre@webkr.de>
Cc: orchideric@yahoo.com, pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #15373: null / utf-8
Message-ID: <20180908203619.GA8672@momjian.us>
References: <153642745785.23144.1699125571356845227@wrigleys.postgresql.org>
 <20180908173648.GJ28811@momjian.us>
 <12e601d447ac$345994a0$9d0cbde0$@webkr.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <12e601d447ac$345994a0$9d0cbde0$@webkr.de>
User-Agent: Mutt/1.5.23 (2014-03-12)
Precedence: bulk

On Sat, Sep  8, 2018 at 09:43:18PM +0200, André Hänsel wrote:
> Bruce Momjian wrote:
>  
> > On Sat, Sep  8, 2018 at 05:24:17PM +0000, PG Bug reporting form wrote:
> >> doesn’t support converting NULL bytes to UTF-8 and therefore returns an
> >> error
> 
> > You need to encode binary values containing nulls as bytea strings.
> 
> Actually this reminds me to log a documentation bug, or rather improvement.
> 
> It is already slightly confusing that Postgres doesn't support null bytes
> in text columns at all and in bytea columns cannot take them in as a normal
> string literal. I'm assuming this is traditional and difficult to change.
> 
> Now, there are two formats in which bytea column content can be specified,
> the "hex" and the deprecated "escape" format. 
> 
> This "escape" format is called that way because it allows escape sequences
> which are un-escaped before the data is written to the column. This step is
> comparable to the parsing of JSON for jsonb columns or the parsing of array
> syntax for any array type.
> 
> However, during the parsing of an SQL there is a second, completely
> different, un-escaping step going on, the un-escaping of C-style escapes in
> string literals starting with an "E", *if* you start the string literal
> with an "E".
> 
> For some reason the example given in the documentation for the "hex"
> format uses such an "escape string literal":
> 
> SELECT E'\\xDEADBEEF';
> 
> I found this very confusing. Can this example be changed to a normal
> string literal, like this?
> 
> SELECT '\xDEADBEEF';

You know, I 100% agree with you.  We used the E'' syntax so we would
produce the same results whether standard_conforming_strings was true or
false.  However, we changed the standard_conforming_strings default to
true in Postgres 9.1 on 2011-09-12, and that release has been
end-of-life for a year.

I think it is time to clarify our documentation examples by assuming
that standard_conforming_strings is true.  I will work on a patch. 
Thanks.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +