From: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
To: Jeff Davis <pgsql@j-davis.com>
To: pgsql-bugs@postgresql.org
Subject: Re: unaccent fails when datlocprovider=i and datctype=C
Date: Fri, 10 Mar 2023 09:40:19 +0100
Message-ID: <314313bd-9cec-75d2-97fe-9172ef0a593b@enterprisedb.com> (raw)
In-Reply-To: <79e4354d9eccfdb00483146a6b9f6295202e7890.camel@j-davis.com>
References: <79e4354d9eccfdb00483146a6b9f6295202e7890.camel@j-davis.com>
On 08.03.23 06:49, Jeff Davis wrote:
> $ initdb -D data -N --locale-provider=icu --icu-locale=en --locale=C
Is it even worth supporting that? What is the point of this kind of setup?
> =# create extension unaccent;
> ERROR: invalid multibyte character for locale
> HINT: The server's LC_CTYPE locale is probably incompatible with the
> database encoding.
> CONTEXT: line 1 of configuration file
> ".../share/tsearch_data/unaccent.rules": "ยก !
> "
>
> Cause: t_isspace() implementation is incomplete (notice "TODO"
> comments):
>
> Oid collation = DEFAULT_COLLATION_OID; /* TODO */
> pg_locale_t mylocale = 0; /* TODO */
>
> if (clen == 1 || lc_ctype_is_c(collation))
> return isspace(TOUCHAR(ptr));
>
> char2wchar(character, WC_BUF_LEN, ptr, clen, mylocale);
>
> return iswspace((wint_t) character[0]);
>
> If using datlocprovider=c, then the earlier branch goes straight to
> isspace(). But if datlocprovider=i, then
> lc_ctype_is_c(DEFAULT_COLLATION_OID) returns false, and it goes into
> char2wchar(). char2wchar() is essentially a wrapper around mbstowcs(),
> which does not work on multibyte input when LC_CTYPE=C.
>
> Quick fix (attached): check whether datctype is C rather than the
> default collation.
This seems right. It's unfortunate that we would now have the
possibilty that
lc_ctype_is_c(DEFAULT_COLLATION_OID) != database_ctype_is_c
but that seems to be the nature of things. Maybe a comment somewhere?
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: peter.eisentraut@enterprisedb.com, pgsql@j-davis.com
Subject: Re: unaccent fails when datlocprovider=i and datctype=C
In-Reply-To: <314313bd-9cec-75d2-97fe-9172ef0a593b@enterprisedb.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox