Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1rebHD-00CGy3-0U for pgsql-bugs@arkaria.postgresql.org; Mon, 26 Feb 2024 13:44:16 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1rebH9-00Dd0A-Vu for pgsql-bugs@arkaria.postgresql.org; Mon, 26 Feb 2024 13:44:12 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1reaAr-00DKeH-7z for pgsql-bugs@lists.postgresql.org; Mon, 26 Feb 2024 12:33:37 +0000 Received: from outbound.soverin.net ([185.233.34.146]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1reaAk-001Bpc-Av for pgsql-bugs@lists.postgresql.org; Mon, 26 Feb 2024 12:33:36 +0000 Received: from smtp.freedom.nl (c04cst-smtp-frd01.int.sover.in [10.10.4.107]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by outbound.soverin.net (Postfix) with ESMTPS id 4Tk0Qb5lxWzPg; Mon, 26 Feb 2024 12:33:27 +0000 (UTC) Received: from smtp.freedom.nl (smtp.freedom.nl [10.10.4.107]) by freedom.nl (Postfix) with ESMTPSA id 4Tk0Qb1QTfz5k; Mon, 26 Feb 2024 12:33:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=freedom.nl; s=default; t=1708950807; bh=6pgvAfBasxhOaYW19wOsq5t5q3wKA1LDb9A9C2PaL58=; h=Date:Subject:To:Cc:References:From:In-Reply-To:From; b=NqoJfdZ4rk16Tpi+2/xnuFIkMms0oTCSfPTa2N3UDtsSz2lajPRYzEG84RXzGqdEI boZ5zpkp0IliiOtzV56XIT5ZDzP17WlelIq3fDJxmQKlC5uF0VbeNNtzLrJtpkzCYe 3MXPUugmlp307S4Rwk/qo+cWHYm1DuwwFRnW0asA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freedom.nl; s=default; t=1708950807; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Q8QtWoxovCNRkv5jua1kHFJEjo7OlG8NmiGiZOLHn+g=; b=OsfbPVoRoFkw7NnpYi6MUIfsIXimfxt6AiNfMtygq/8/LLU+GGJjM1Q34f4pFKyHDYjxJN L8/AuOp8SiHbs+IzL6No7j3evctn2IYbTdOmjfX7ppZlNQ1hOWbfocKxu+3AgkkeSJllAy TJY863ctEBUZhchBrSOASzjYQLIOks4= ARC-Authentication-Results: i=1; smtp.freedom.nl; auth=pass smtp.mailfrom=cees.van.zeeland@freedom.nl ARC-Seal: i=1; s=default; d=freedom.nl; t=1708950807; a=rsa-sha256; cv=none; b=XbK9lIXpka3cHHHgKC6RnbC5Dw1rJcFDT5tWPdouUXDYQtsBTwiBKmNSU4qctAr/2QQVmK zumulfRkC5iVP5q5Tlcj35Zd4AccVzOuKxDgDUKB8m5dyiRQSAHvLdr1FyIakZG50NhhZw AQWeRCT9R0QqeuL3JkjqK/smJS7EPks= Content-Type: multipart/alternative; boundary="------------9l7uPG2zmsu1dm0xYH8z7okn" Message-ID: Date: Mon, 26 Feb 2024 13:33:28 +0100 MIME-Version: 1.0 Subject: Re: BUG #18362: unaccent rules and Old Greek text To: Michael Paquier , Thomas Munro , Tom Lane Cc: pgsql-bugs@lists.postgresql.org References: <18362-be6d0cfe122b6354@postgresql.org> Content-Language: en-US X-Soverin-Authenticated: true From: Cees van Zeeland In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk This is a multi-part message in MIME format. --------------9l7uPG2zmsu1dm0xYH8z7okn Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Thomas Munro wrote: >    If I tell the script to follow such "simple" redirections, I >   get over a thousand new mappings, including those.  See attached. >   There is probably more correct terminology that I'm using here... Michael Paquier wrote: > It seems to me that it is a bit more complicated than that, because > Unicode.data decomposes the characters with Oxia as characters with > Tonos, and then characters with Tonos are decomposed with the "base" > alphabet characters + Tonos.  We do a recursive lookup at the unicode > table in get_plain_letter() and is_letter_with_marks(), so it seems to > me that we're not missing much, and I suspect that there should be no > need for a new custom range of characters.. > > Cees, perhaps you would like to get a shot at that? > > [1]: https://en.wikipedia.org/wiki/Greek_diacritics#Unicode I'm not an expert, but obviously computers make a difference between the two versions of the characters. We are talking about this series: U+1F70 - U+1F7D:    ὰ     ά     ὲ     έ     ὴ     ή     ὶ     ί ὸ     ό     ὺ     ύ     ὼ     ώ Is it possible to filter / limit in some way the redirection in the script to this range? ~ Cees --------------9l7uPG2zmsu1dm0xYH8z7okn Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit Thomas Munro <thomas.munro@gmail.com> wrote:
>    If I tell the script to follow such "simple" redirections, I
>   get over a thousand new mappings, including those.  See attached.
>   There is probably more correct terminology that I'm using here...

Michael Paquier wrote:
> It seems to me that it is a bit more complicated than that, because
> Unicode.data decomposes the characters with Oxia as characters with
> Tonos, and then characters with Tonos are decomposed with the "base"
> alphabet characters + Tonos.  We do a recursive lookup at the unicode
> table in get_plain_letter() and is_letter_with_marks(), so it seems to
> me that we're not missing much, and I suspect that there should be no
> need for a new custom range of characters..
>
> Cees, perhaps you would like to get a shot at that?
>
> [1]: https://en.wikipedia.org/wiki/Greek_diacritics#Unicode

I'm not an expert, but obviously computers make a difference between the two versions of the characters.
We are talking about this series:
U+1F70 - U+1F7D:    ὰ     ά     ὲ     έ     ὴ     ή     ὶ     ί     ὸ     ό     ὺ     ύ     ὼ     ώ        
Is it possible to filter / limit in some way the redirection in the script to this range?

~
Cees



--------------9l7uPG2zmsu1dm0xYH8z7okn--