Feedback-ID: ie0a040ee:Fastmail
Message-ID: <fc4596b3-a5c4-47f1-b1d5-213cec127185@eisentraut.org>
Date: Fri, 3 May 2024 20:58:30 +0200
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Support LIKE with nondeterministic collations
To: Daniel Verite <daniel@manitou-mail.org>
Cc: Robert Haas <robertmhaas@gmail.com>,
 Pgsql-Hackers <pgsql-hackers@postgresql.org>
References: <c17bffb0-48f4-4114-8e8e-85d5fb4f2147@manitou-mail.org>
Content-Language: en-US
From: Peter Eisentraut <peter@eisentraut.org>
In-Reply-To: <c17bffb0-48f4-4114-8e8e-85d5fb4f2147@manitou-mail.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Archived-At: <https://www.postgresql.org/message-id/fc4596b3-a5c4-47f1-b1d5-213cec127185%40eisentraut.org>
Precedence: bulk

On 03.05.24 17:47, Daniel Verite wrote:
> 	Peter Eisentraut wrote:
> 
>>   However, off the top of my head, this definition has three flaws: (1)
>> It would make the single-character wildcard effectively an
>> any-number-of-characters wildcard, but only in some circumstances, which
>> could be confusing, (2) it would be difficult to compute, because you'd
>> have to check equality against all possible single-character strings,
>> and (3) it is not what the SQL standard says.
> 
> For #1 we're currently using the definition of a "character" as
> being any single point of code,

That is the definition that is used throughout SQL and PostgreSQL.  We 
can't change that without redefining everything.  To pick just one 
example, the various trim function also behave in seemingly inconsistent 
ways when you apply then to strings in different normalization forms. 
The better fix there is to enforce the normalization form somehow.

> Intuitively I think that our interpretation of "character" here should
> be whatever sequence of code points are between character
> boundaries [1], and that the equality of such characters would be the
> equality of their sequences of code points, with the string equality
> check of the collation, whatever the length of these sequences.
> 
> [1]:
> https://unicode-org.github.io/icu/userguide/boundaryanalysis/#character-boundary

Even that page says, what we are calling character here is really called 
a grapheme cluster.

In a different world, pattern matching, character trimming, etc. would 
work by grapheme, but it does not.