From: Jeff Davis <pgsql@j-davis.com>
To: Jeremy Schneider <schneider@ardentperf.com>
To: pgsql-hackers@lists.postgresql.org, "Davis, Jeff" <jefdavj@amazon.com>
Subject: Re: Built-in CTYPE provider
Date: Thu, 21 Dec 2023 15:00:26 -0800
Message-ID: <d7ff0299ce370c82bef94a8b2425e7c236bc803e.camel@j-davis.com> (raw)
In-Reply-To: <8a1ae216-8150-41e2-a98d-09c57e3dc90f@ardentperf.com>
References: <ff4c2f2f9c8fc7ca27c1c24ae37ecaeaeaff6b53.camel@j-davis.com>
<8a1ae216-8150-41e2-a98d-09c57e3dc90f@ardentperf.com>
On Wed, 2023-12-20 at 15:47 -0800, Jeremy Schneider wrote:
> One other thing that comes to mind: how does the parser do case
> folding
> for relation names? Is that using OS-provided libc as of today? Or
> did
> we code it to use ICU if that's the DB default? I'm guessing libc,
> and
> global catalogs probably need to be handled in a consistent manner,
> even
> across different encodings.
The code is in downcase_identifier():
/*
* SQL99 specifies Unicode-aware case normalization, which we don't
* yet have the infrastructure for...
*/
if (ch >= 'A' && ch <= 'Z')
ch += 'a' - 'A';
else if (enc_is_single_byte && IS_HIGHBIT_SET(ch) && isupper(ch))
ch = tolower(ch);
result[i] = (char) ch;
My proposal would add the infrastructure that the comment above says is
missing.
It seems like we should be using the database collation at this point
because you don't want inconsistency between the catalogs and the
parser here. Then again, the SQL spec doesn't seem to support tailoring
of case conversions, so maybe we are avoiding it for that reason? Or
maybe we're avoiding catalog access? Or perhaps the work for ICU just
wasn't done here yet?
> (Kindof related... did you ever see the demo where I create a user
> named
> '🏃' and then I try to connect to a database with non-unicode
> encoding?
> 💥😜 ...at least it seems to be able to walk the index without
> decoding
> strings to find other users - but the way these global catalogs work
> scares me a little bit)
I didn't see that specific demo, but in general we seem to change
between pg_wchar and unicode code points too freely, so I'm not
surprised that something went wrong.
Regards,
Jeff Davis
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: pgsql@j-davis.com, schneider@ardentperf.com, jefdavj@amazon.com
Subject: Re: Built-in CTYPE provider
In-Reply-To: <d7ff0299ce370c82bef94a8b2425e7c236bc803e.camel@j-davis.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox