From: Michael Paquier <michael@paquier.xyz>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: cees.van.zeeland@freedom.nl
Cc: pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #18362: unaccent rules and Old Greek text
Date: Mon, 26 Feb 2024 08:19:53 +0900
Message-ID: <ZdvLGeJ1BsXRkrdQ@paquier.xyz> (raw)
In-Reply-To: <CA+hUKGLpoxTRryM1MxaHMpC1c5cxoKDm0iXfogQR9KGVUG=w9w@mail.gmail.com>
References: <18362-be6d0cfe122b6354@postgresql.org>
<CA+hUKGLpoxTRryM1MxaHMpC1c5cxoKDm0iXfogQR9KGVUG=w9w@mail.gmail.com>
On Sun, Feb 25, 2024 at 04:21:36PM +1300, Thomas Munro wrote:
> On Sun, Feb 25, 2024 at 11:14 AM PG Bug reporting form
> <noreply@postgresql.org> wrote:
>> So, there are reasons to keep the current unaccent.rules as it is, but...
>> there are other reasons to add a few lines to it, f.e. after line 955 and
>> insert five greek vowels with Oxia
>> Please add:
>> ά α
>> έ ε
>> ή η
>> ί ι
>> ό ο
>> ύ υ
>> ώ ω
Correct me if I'm wrong of course, but reading a bit on the matter at
[1], letters with Tonos or Oxia are actually equivalent since 1986,
and we only include character with Tonos in our unaccent.rules.
> We don't exactly maintain this list manually, we extract it from
> Unicode source data. Can you see what needs to be adjusted in here to
> achieve that goal?
See commits like e3dd7c06e627 or 59f47fb98dab for some references.
Unfortunately, we've been using as policy to not backpatch any changes
to the in-core rules file, and you can plug in your own file. Saying
that, these additions sound like a natural addition seen from here.
> Perhaps a new range or something like that?
It seems to me that it is a bit more complicated than that, because
Unicode.data decomposes the characters with Oxia as characters with
Tonos, and then characters with Tonos are decomposed with the "base"
alphabet characters + Tonos. We do a recursive lookup at the unicode
table in get_plain_letter() and is_letter_with_marks(), so it seems to
me that we're not missing much, and I suspect that there should be no
need for a new custom range of characters..
Cees, perhaps you would like to get a shot at that?
[1]: https://en.wikipedia.org/wiki/Greek_diacritics#Unicode
--
Michael
Attachments:
[application/pgp-signature] signature.asc (832B, ../ZdvLGeJ1BsXRkrdQ@paquier.xyz/2-signature.asc)
download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: michael@paquier.xyz, thomas.munro@gmail.com, cees.van.zeeland@freedom.nl, pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #18362: unaccent rules and Old Greek text
In-Reply-To: <ZdvLGeJ1BsXRkrdQ@paquier.xyz>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox