From: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
To: pgsql-hackers <pgsql-hackers@postgresql.org>
Cc: Daniel Verite <daniel@manitou-mail.org>
Cc: Andreas Karlsson <andreas@proxel.se>
Subject: Re: Unicode normalization SQL functions
Date: Thu, 2 Apr 2020 09:45:09 +0200
Message-ID: <b93f79b0-7ea6-4807-e993-b0b3bbe8337a@2ndquadrant.com> (raw)
In-Reply-To: <7052cc8f-0164-72a8-d9a4-fd32066c938e@2ndquadrant.com>
References: <6d55838c-9353-486f-a508-77ee07d59667@manitou-mail.org>
<620b155b-a89c-d8e0-3cbb-315c3ca538f4@2ndquadrant.com>
<7052cc8f-0164-72a8-d9a4-fd32066c938e@2ndquadrant.com>
On 2020-03-26 08:25, Peter Eisentraut wrote:
> On 2020-03-24 10:20, Peter Eisentraut wrote:
>> Now I have some concerns about the size of the new table in
>> unicode_normprops_table.h, and the resulting binary size. At the very
>> least, we should probably make that #ifndef FRONTEND or something like
>> that so libpq isn't bloated by it unnecessarily. Perhaps there is a
>> better format for that table? Any ideas?
>
> I have figured this out. New patch is attached.
>
> First, I have added #ifndef FRONTEND, as mentioned above, so libpq isn't
> bloated. Second, I have changed the lookup structure to a bitfield, so
> each entry is only 32 bits instead of 64. Third, I have dropped the
> quickcheck tables for the NFD and NFKD forms. Those are by far the
> biggest tables, and you still get okay performance if you do the
> normalization check the long way, since we don't need the recomposition
> step on those cases, which is by far the slowest part. The main use
> case of all of this, I expect, is to check for NFC normalization, so
> it's okay if the other variants are not optimized to the same extent.
I have committed this version.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: peter.eisentraut@2ndquadrant.com, daniel@manitou-mail.org, andreas@proxel.se
Subject: Re: Unicode normalization SQL functions
In-Reply-To: <b93f79b0-7ea6-4807-e993-b0b3bbe8337a@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox