From: Tom Lane <tgl@sss.pgh.pa.us>
To: h8mastre@gmail.com
cc: pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #15476: Problem on show_trgm with 4 byte UTF-8 characters
In-reply-to: <15476-4314f480acf0f114@postgresql.org>
References: <15476-4314f480acf0f114@postgresql.org>
Comments: In-reply-to =?utf-8?q?PG_Bug_reporting_form?=
 <noreply@postgresql.org>
	message dated "Thu, 01 Nov 2018 02:39:20 -0000"
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-ID: <2100.1541220270.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Sat, 03 Nov 2018 00:44:30 -0400
Message-ID: <2101.1541220270@sss.pgh.pa.us>
Precedence: bulk

=3D?utf-8?q?PG_Bug_reporting_form?=3D <noreply@postgresql.org> writes:
> On Encoding=3DUTF-8 database, try:
> SELECT show_trgm('123');
> =E2=86=92 OK
> SELECT show_trgm('=E6=97=A5=E6=9C=AC=E8=AA=9E');
> =E2=86=92 probably OK.
> SELECT show_trgm('=F0=9F=94=8D');
> ERROR:  invalid multibyte character for locale
> HINT:  The server's LC_CTYPE locale is probably incompatible with the
> database encoding.
> SQL state: 22021

I failed to reproduce this on a Linux machine.  It looks to me like the
problem is that Windows' MultiByteToWideChar doesn't think that UTF8
character is valid.

> Please check: t_isdigit, t_isspace, t_isalpha, and t_isprint.
> https://github.com/postgres/postgres/blob/322548a8abe225f2cfd6a48e07b99e=
2711d28ef7/src/backend/tsearch/ts_locale.c#L35
> char2wchar 4th parameter should take number of input bytes. However they
> pass character count.
> int clen =3D pg_mblen(ptr);
> ...
> char2wchar(character, 2, ptr, clen, mylocale);

Huh?  pg_mblen returns the number of bytes in a multibyte character,
so this looks fine to me.

			regards, tom lane