pg.ddx.io  pgsql-bugs@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Hex escapes in strings only support ASCII range
3+ messages / 2 participants
[nested] [flat]

* Hex escapes in strings only support ASCII range
@ 2021-03-08 11:42  Vilem Benjamin Liepelt <vileml@sqreamtech.com>
  0 siblings, 1 reply; 3+ messages in thread

From: Vilem Benjamin Liepelt @ 2021-03-08 11:42 UTC (permalink / raw)
  To: pgsql-bugs@lists.postgresql.org

Escaped strings [1] fail on hex values greater than \x7f with a factually incorrect error message:

$ psql --command "select E'\x80';"
ERROR:  invalid byte sequence for encoding "UTF8": 0x80

$ psql --version
psql (PostgreSQL) 13.1

I was able to reproduce this bug on psql 10.

Workaround: 

$ psql --command "select E'\u0080';"
 ?column? 
----------
 \u0080
(1 row)

1: https://www.postgresql.org/docs/13/sql-syntax-lexical.html#id-1.5.3.5.9.5.3=

^ permalink  raw  reply  [nested|flat] 3+ messages in thread

* Re: Hex escapes in strings only support ASCII range
@ 2021-03-08 14:25  Sven Klemm <sven@timescale.com>
  parent: Vilem Benjamin Liepelt <vileml@sqreamtech.com>
  0 siblings, 1 reply; 3+ messages in thread

From: Sven Klemm @ 2021-03-08 14:25 UTC (permalink / raw)
  To: Vilem Benjamin Liepelt <vileml@sqreamtech.com>; +Cc: pgsql-bugs@lists.postgresql.org

On Mon, Mar 8, 2021 at 2:43 PM Vilem Benjamin Liepelt
<vileml@sqreamtech.com> wrote:
>
> Escaped strings [1] fail on hex values greater than \x7f with a factually incorrect error message:
>
> $ psql --command "select E'\x80';"
> ERROR:  invalid byte sequence for encoding "UTF8": 0x80
>
> $ psql --version
> psql (PostgreSQL) 13.1
>
> I was able to reproduce this bug on psql 10.
>
> Workaround:
>
> $ psql --command "select E'\u0080';"
>  ?column?
> ----------
>  \u0080
> (1 row)

0x80 is not equivalent to \u0080 in UTF8. You may use hex values
greater than 0x7F but you still have to produce valid byte sequences
for your target encoding or switch to an encoding that does not
validate those sequences.

postgres=# select E'\xc2\x80';

 ?column?
----------
 \u0080
(1 row)


--
Regards, Sven Klemm





^ permalink  raw  reply  [nested|flat] 3+ messages in thread

* Re: Hex escapes in strings only support ASCII range
@ 2021-03-08 14:53  Vilem Benjamin Liepelt <vileml@sqreamtech.com>
  parent: Sven Klemm <sven@timescale.com>
  0 siblings, 0 replies; 3+ messages in thread

From: Vilem Benjamin Liepelt @ 2021-03-08 14:53 UTC (permalink / raw)
  To: Sven Klemm <sven@timescale.com>; +Cc: pgsql-bugs@lists.postgresql.org

Thank you for the clarification, I misunderstood the documentation. Your explanation makes a lot of sense, my bad for opening a bug about this.

> On 8 Mar 2021, at 15:25, Sven Klemm <sven@timescale.com> wrote:
> 
> On Mon, Mar 8, 2021 at 2:43 PM Vilem Benjamin Liepelt
> <vileml@sqreamtech.com> wrote:
>> 
>> Escaped strings [1] fail on hex values greater than \x7f with a factually incorrect error message:
>> 
>> $ psql --command "select E'\x80';"
>> ERROR:  invalid byte sequence for encoding "UTF8": 0x80
>> 
>> $ psql --version
>> psql (PostgreSQL) 13.1
>> 
>> I was able to reproduce this bug on psql 10.
>> 
>> Workaround:
>> 
>> $ psql --command "select E'\u0080';"
>> ?column?
>> ----------
>> \u0080
>> (1 row)
> 
> 0x80 is not equivalent to \u0080 in UTF8. You may use hex values
> greater than 0x7F but you still have to produce valid byte sequences
> for your target encoding or switch to an encoding that does not
> validate those sequences.
> 
> postgres=# select E'\xc2\x80';
> 
> ?column?
> ----------
> \u0080
> (1 row)
> 
> 
> --
> Regards, Sven Klemm






^ permalink  raw  reply  [nested|flat] 3+ messages in thread


end of thread, other threads:[~2021-03-08 14:53 UTC | newest]

Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2021-03-08 11:42 Hex escapes in strings only support ASCII range Vilem Benjamin Liepelt <vileml@sqreamtech.com>
2021-03-08 14:25 ` Sven Klemm <sven@timescale.com>
2021-03-08 14:53   ` Vilem Benjamin Liepelt <vileml@sqreamtech.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox