From: Tom Lane <tgl@sss.pgh.pa.us>
To: Richard Guo <guofenglinux@gmail.com>
Cc: PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Subject: Re: Assert failure with ICU support
Date: Wed, 19 Apr 2023 11:42:20 -0400
Message-ID: <2983095.1681918940@sss.pgh.pa.us> (raw)
In-Reply-To: <CAMbWs49Q6UoKGeT8pBkMtJGJd+16CBFZaaWUk9Du+2ERE5g_YA@mail.gmail.com>
References: <CAMbWs49Q6UoKGeT8pBkMtJGJd+16CBFZaaWUk9Du+2ERE5g_YA@mail.gmail.com>
Richard Guo <guofenglinux@gmail.com> writes:
> I happened to run into an assert failure by chance with ICU support.
> Here is the query:
> SELECT '1' SIMILAR TO '\൧';
> The failure happens in lexescape(),
> default:
> assert(iscalpha(c));
> FAILW(REG_EESCAPE); /* unknown alphabetic escape */
> break;
> Without ICU support, the same query just gives an error.
Interesting.
> FWIW, I googled a bit and '൧' seems to be number 1 in Malayalam.
The code in lexescape() is assuming that if "c" passes
iscalnum(), then either it's '0'-'9' or it passes iscalpha().
This is clearly wrong in Unicode-land, which has non-ASCII digits.
I imagine you can find libc locales where this fails, not only ICU.
I think the question here is what we want to do with such cases:
throw a regex syntax error, or just return the character as-is?
The fine manual says that if the character after '\' is
alphanumeric, it's an escape, and otherwise the character is
quoted literally. But how shall we interpret "alphanumeric"?
I'm kind of inclined to the idea that anything that's not ASCII
should be considered to be literally quoted by '\', rather than
being an erroneous regex escape. Maybe I'm too English-centric.
But I don't like the idea that what is a valid regex should vary
depending on locale.
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: tgl@sss.pgh.pa.us, guofenglinux@gmail.com, pgsql-bugs@lists.postgresql.org
Subject: Re: Assert failure with ICU support
In-Reply-To: <2983095.1681918940@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox