pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pgsql: Handle the "und" locale in ICU versions 54 and older.
3+ messages / 3 participants
[nested] [flat]

* pgsql: Handle the "und" locale in ICU versions 54 and older.
@ 2023-03-23 17:09  Jeff Davis <jdavis@postgresql.org>
  0 siblings, 1 reply; 3+ messages in thread

From: Jeff Davis @ 2023-03-23 17:09 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Handle the "und" locale in ICU versions 54 and older.

The "und" locale is an alternative spelling of the root locale, but it
was not recognized until ICU 55. To maintain common behavior across
all supported ICU versions, check for "und" and replace with "root"
before opening.

Previously, the lack of support for "und" was dangerous, because
versions 54 and older fall back to the environment when a locale is
not found. If the user specified "und" for the language (which is
expected and documented), it could not only resolve to the wrong
collator, but it could unexpectedly change (which could lead to
corrupt indexes).

This effectively reverts commit d72900bded, which worked around the
problem for the built-in "unicode" collation, and is no longer
necessary.

Discussion: https://postgr.es/m/60da0cecfb512a78b8666b31631a636215d8ce73.camel@j-davis.com
Discussion: https://postgr.es/m/0c6fa66f2753217d2a40480a96bd2ccf023536a1.camel@j-davis.com
Reviewed-by: Peter Eisentraut

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/3b50275b12950280fb07193e24a4f400ed8a9fef

Modified Files
--------------
src/backend/utils/adt/pg_locale.c              | 34 ++++++++++++++++++++++++++
src/bin/initdb/initdb.c                        |  2 +-
src/test/regress/expected/collate.icu.utf8.out |  7 ++++++
src/test/regress/sql/collate.icu.utf8.sql      |  2 ++
4 files changed, 44 insertions(+), 1 deletion(-)



^ permalink  raw  reply  [nested|flat] 3+ messages in thread

* Re: pgsql: Handle the "und" locale in ICU versions 54 and older.
@ 2023-03-23 17:25  Tom Lane <tgl@sss.pgh.pa.us>
  parent: Jeff Davis <jdavis@postgresql.org>
  0 siblings, 1 reply; 3+ messages in thread

From: Tom Lane @ 2023-03-23 17:25 UTC (permalink / raw)
  To: Jeff Davis <jdavis@postgresql.org>; +Cc: pgsql-committers@lists.postgresql.org

Jeff Davis <jdavis@postgresql.org> writes:
> Handle the "und" locale in ICU versions 54 and older.
> The "und" locale is an alternative spelling of the root locale, but it
> was not recognized until ICU 55. To maintain common behavior across
> all supported ICU versions, check for "und" and replace with "root"
> before opening.

> Previously, the lack of support for "und" was dangerous, because
> versions 54 and older fall back to the environment when a locale is
> not found. If the user specified "und" for the language (which is
> expected and documented), it could not only resolve to the wrong
> collator, but it could unexpectedly change (which could lead to
> corrupt indexes).

Hmm, should we back-patch this?  Seems like existing branches would
be even more at risk than v16, because more likely to be built with
old ICU.  OTOH, we do also run the risk of breaking installations
that weren't broken before.

			regards, tom lane





^ permalink  raw  reply  [nested|flat] 3+ messages in thread

* Re: pgsql: Handle the "und" locale in ICU versions 54 and older.
@ 2023-03-24 18:10  Jeff Davis <pgsql@j-davis.com>
  parent: Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 3+ messages in thread

From: Jeff Davis @ 2023-03-24 18:10 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; Jeff Davis <jdavis@postgresql.org>; +Cc: pgsql-committers@lists.postgresql.org

On Thu, 2023-03-23 at 13:25 -0400, Tom Lane wrote:
> Hmm, should we back-patch this?  Seems like existing branches would
> be even more at risk than v16, because more likely to be built with
> old ICU.  OTOH, we do also run the risk of breaking installations
> that weren't broken before.

I wondered the same thing[1] but ultimately figured the risk outweighed
the reward. My reasoning (which I didn't post before) was:

If a user currently has a collation with locale 'und', and ICU <= 54,
they are getting their actual locale from the environment. If we
backpatch, it will silently change their locale to be the root locale,
which could be different, and break their indexes. That seems too
dangerous for a minor release.

For a major release it's more tolerable to put something like that in
the release notes.

Fortunately, I think most users now are probably using the built-in
collations, or using the empty string before an "@" to specify the root
locale, which works in all ICU versions. Users would only specify it as
"und" if they happen to know about language tags.

Regards,
	Jeff Davis

[1]
https://www.postgresql.org/message-id/9afa6dbe0d31053ad265aeba488fde784fd5b7ab.camel@j-davis.com





^ permalink  raw  reply  [nested|flat] 3+ messages in thread


end of thread, other threads:[~2023-03-24 18:10 UTC | newest]

Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-03-23 17:09 pgsql: Handle the "und" locale in ICU versions 54 and older. Jeff Davis <jdavis@postgresql.org>
2023-03-23 17:25 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-03-24 18:10   ` Jeff Davis <pgsql@j-davis.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox