Message-ID: <4e3281d1ba5950dbe3b5d3c8d511da4c5751217a.camel@j-davis.com>
Subject: Re: Order changes in PG16 since ICU introduction
From: Jeff Davis <pgsql@j-davis.com>
To: Peter Eisentraut <peter.eisentraut@enterprisedb.com>, "Jonathan S. Katz"
	 <jkatz@postgresql.org>, Robert Haas <robertmhaas@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Regina Obe <lr@pcorp.us>, Sandro Santilli
	 <strk@kbt.io>, pgsql-hackers@lists.postgresql.org
Date: Mon, 22 May 2023 12:03:36 -0700
In-Reply-To: <74706bb8-4fb0-fbab-7e88-648ae04f2588@enterprisedb.com>
References: <000b01d97465$c34bbd60$49e33820$@pcorp.us>
	 <533d93c5-f604-8e5f-4a48-975c08c53d59@enterprisedb.com>
	 <3365333.1682098091@sss.pgh.pa.us>
	 <002d01d97477$f0395640$d0ac02c0$@pcorp.us>
	 <3367226.1682099185@sss.pgh.pa.us>
	 <003b01d9747a$910471c0$b30d5540$@pcorp.us>
	 <3368986.1682099982@sss.pgh.pa.us>
	 <000001d9747d$066b7560$13426020$@pcorp.us>
	 <3372735.1682101425@sss.pgh.pa.us>
	 <f83f089ee1e9acd5dbbbf3353294d24e1f196e95.camel@j-davis.com>
	 <CA+TgmobxgvonJT8vO_v0k9_rgDmn0CKvTe48Z=w95rrBqC95KA@mail.gmail.com>
	 <7ee07b7d5aa4fb0b47e5657640568fdd25ba168f.camel@j-davis.com>
	 <CA+TgmoaPGx+R6mOcYQwDfqGoZxd=SncDCDt9mYjz5AD5tFd8sw@mail.gmail.com>
	 <2ffe8ffb9291b9390b9ed6c1e34b4fae8e8ce324.camel@j-davis.com>
	 <25787ec7-4c04-9a8a-d241-4dc9be0b1ba3@postgresql.org>
	 <605324592391599a7ecd522087edbd00720793b7.camel@j-davis.com>
	 <f353900f4c5777b02e9c7ef85326f1954ffeccea.camel@j-davis.com>
	 <52553877-7966-e25f-6c74-a974a504250e@postgresql.org>
	 <a28aba5fa6bf1abfff96e40b6d6acff8412edb15.camel@j-davis.com>
	 <74706bb8-4fb0-fbab-7e88-648ae04f2588@enterprisedb.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.44.4-0ubuntu1 
MIME-Version: 1.0
Archived-At: <https://www.postgresql.org/message-id/4e3281d1ba5950dbe3b5d3c8d511da4c5751217a.camel%40j-davis.com>
Precedence: bulk

On Mon, 2023-05-22 at 14:27 +0200, Peter Eisentraut wrote:
> The rules are for setting whatever sort order you like.=C2=A0 Maybe you
> want=20
> to sort + before - or whatever.=C2=A0 It's like, if you don't like it,
> build=20
> your own.

A build-your-own feature is fine, but it's not completely zero cost.

There some risk that rules specified for ICU version X fail to load for
ICU version Y. If that happens to your database default collation, you
are in big trouble. The risk of failing to load a language tag in a
later version, especially one returned by uloc_toLanguageTag() in
strict mode, is much lower. We can reduce the risk by allowing rules
only for CREATE COLLATION (not  CREATE DATABASE), and see what users do
with it first, and consider adding it to CREATE DATABASE later.

We can also try to explain in the docs that it's a build-it-yourself
kind of feature (use it if you see a purpose, otherwise ignore it),
though I'm not sure quite how we should word it.

And I'm skeptical that we don't have a single plausible end-to-end user
story. I just can't think of any reason someone would need something
like this, given how flexible the collation settings in the language
tags are.=C2=A0The best case I can think of is if someone is trying to make
an ICU collation that matches some non-ICU collation in another system,
which sounds hard; but perhaps it's reasonable to do in cases where it
just needs to work well-enough in some limited case.

Also, do we have an answer as to why specifying the rules as '' is not
the same as not specifying any rules[1]?

[1]
https://www.postgresql.org/message-id/36a6e89689716c2ca1fae8adc8e84601a0411=
21c.camel@j-davis.com

> The co settings are just everything else.=20
> They are not parametric, they are just some other sort order that=20
> someone spelled out explicitly.

This sounds like another case where we can't really tell the user why
they would want to use a specific "co" setting; they should only use it
if they already know they want it. Is there some way we can word that
in the documentation so that people don't misuse them?

For instance, one of them is called "emoji". I'm sure a lot of
applications use emoji (or at least might encounter them), should they
always use co-emoji, or would some people who are using emoji not want
it? Can it be combined with "ks" or other "k*" settings?

What I'm trying to avoid is users seeing something in the documentation
and using it without it really being a good fit for their problem. Then
they see something unexpected, and need to rebuild all of their indexes
or something.

> > * I don't understand what "kc" means if "ks" is not set to
> > "level1".
>=20
> There is an example here:=20
> https://peter.eisentraut.org/blog/2023/05/16/overview-of-icu-collation-se=
ttings#colcaselevel

Interesting, thank you.

Regards,
	Jeff Davis