From: Nico Williams <nico@cryptonector.com>
To: Jeff Davis <pgsql@j-davis.com>
Cc: Alexander Borisov <lex.borisov@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Subject: Re: Improve the performance of Unicode Normalization Forms.
Date: Fri, 20 Jun 2025 13:45:22 -0500
Message-ID: <aFWsQsTmiG44+z8P@ubby> (raw)
In-Reply-To: <792ee592556431aaa2ad1570a978574f53e5be7a.camel@j-davis.com>
References: <844d3dd7-2955-4794-95d1-7f4c13cb89fc@gmail.com>
<4211ffd7fe154c4af693b98d78f4a3689ce8cc30.camel@j-davis.com>
<aFWM9coWT9JUJ8m3@ubby>
<792ee592556431aaa2ad1570a978574f53e5be7a.camel@j-davis.com>
On Fri, Jun 20, 2025 at 10:15:47AM -0700, Jeff Davis wrote:
> On Fri, 2025-06-20 at 11:31 -0500, Nico Williams wrote:
> > In the slow path you only normalize the _current character_, so you
> > only need enough buffer space for that.
>
> That's a clear win for UTF8 data. Also, if there are no changes, then
> you can just return the input buffer and not bother allocating an
> output buffer.
The latter is not relevant to string comparison or hashing, but, yeah,
if you have to produce a normalized string you can optimistically assume
it is already normalized and defer allocation until you know it isn't
normalized.
> Postgres is already form-preserving; it does not auto-normalize. (I
> have suggested that we might want to offer something like that, but
> that would be a user choice.)
Excellent, then I would advise looking into adding form-insensitive
string comparison and hashing to get f-i/f-p behavior.
> Currently, the non-deterministic collations (which offer form-
> insensitivity) are not available at the database level, so you have to
> explicitly specify the COLLATE clause on a column or query. In other
> words, Postgres is not form-insensitive by default, though there is
> work to make that possible.
TIL. Thanks.
> Databases have similar concerns as a filesystem in this respect.
I figured :)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: nico@cryptonector.com, pgsql@j-davis.com, lex.borisov@gmail.com
Subject: Re: Improve the performance of Unicode Normalization Forms.
In-Reply-To: <aFWsQsTmiG44+z8P@ubby>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox