Content-Type: text/plain; charset="iso-8859-15"
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
From: "Daniel Verite" <daniel@manitou-mail.org>
Subject: Re: Support LIKE with nondeterministic collations
To: "Peter Eisentraut" <peter@eisentraut.org>
Cc: Robert Haas <robertmhaas@gmail.com>,
 Pgsql-Hackers <pgsql-hackers@postgresql.org>
In-Reply-To: <68263a89-b6af-4705-ac08-9a57cdd63bd0@eisentraut.org>
Date: Fri, 03 May 2024 17:47:48 +0200
Message-Id: <c17bffb0-48f4-4114-8e8e-85d5fb4f2147@manitou-mail.org>
Archived-At: <https://www.postgresql.org/message-id/c17bffb0-48f4-4114-8e8e-85d5fb4f2147%40manitou-mail.org>
Precedence: bulk

	Peter Eisentraut wrote:

>  However, off the top of my head, this definition has three flaws: (1)=20
> It would make the single-character wildcard effectively an=20
> any-number-of-characters wildcard, but only in some circumstances, which=
=20
> could be confusing, (2) it would be difficult to compute, because you'd=20
> have to check equality against all possible single-character strings,=20
> and (3) it is not what the SQL standard says.

For #1 we're currently using the definition of a "character" as=20
being any single point of code, but this definition fits poorly
with non-deterministic collation rules.

The simplest illustration I can think of is the canonical
equivalence match between the NFD and NFC forms of an
accented character.

postgres=3D# CREATE COLLATION nd (
  provider =3D 'icu',
  locale =3D 'und',
  deterministic =3D false
);=09=09=20=20=20=20=20=20=20

-- match NFD form with NFC form of eacute

postgres=3D# select U&'e\0301' like '=E9' collate nd;
 ?column?=20
----------
 t

postgres=3D# select U&'e\0301' like '_' collate nd;
 ?column?=20
----------
 f
(1 row)

I understand why the algorithm produces these opposite results.
But at the semantic level, when asked if the left-hand string matches
a specific character, it says yes, and when asked if it matches any
character, it says no.
To me it goes beyond counter-intuitive, it's not reasonable enough to
be called correct.

What could we do about it?
Intuitively I think that our interpretation of "character" here should
be whatever sequence of code points are between character
boundaries [1], and that the equality of such characters would be the
equality of their sequences of code points, with the string equality
check of the collation, whatever the length of these sequences.

[1]:
https://unicode-org.github.io/icu/userguide/boundaryanalysis/#character-bou=
ndary


Best regards,
--=20
Daniel V=E9rit=E9
https://postgresql.verite.pro/
Twitter: @DanielVerite