From: Tom Lane <tgl@sss.pgh.pa.us>
To: Michael Downey <mdowney@esri.com>
cc: "pgsql-sql@lists.postgresql.org" <pgsql-sql@lists.postgresql.org>
Subject: Re: PostgreSQL implicitly double-quoting identifier name with umlaut
In-reply-to: 
 <BY3PR05MB7985A889E17333E35AF6B110DB652@BY3PR05MB7985.namprd05.prod.outlook.com>
References: 
 <BY3PR05MB7985A889E17333E35AF6B110DB652@BY3PR05MB7985.namprd05.prod.outlook.com>
Comments: In-reply-to Michael Downey <mdowney@esri.com>
	message dated "Fri, 13 Sep 2024 23:06:11 -0000"
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-ID: <156003.1726269616.1@sss.pgh.pa.us>
Content-Transfer-Encoding: quoted-printable
Date: Fri, 13 Sep 2024 19:20:16 -0400
Message-ID: <156004.1726269616@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/156004.1726269616%40sss.pgh.pa.us>
Precedence: bulk

Michael Downey <mdowney@esri.com> writes:
> One of our internal users, using our tools, added a column called =C3=96=
rtschaft. We anticipated it would be folded to lower case.
> So we inserted our metadata for the column in our metadata with the name=
 =C3=B6rtschaft. With the system query for metadata, we
> ended up seeing query mismatches involving this column as we found the a=
ctual column name is =C3=96rtschaft
> in the database.

When working in UTF8 (or any multibyte encoding), PG's identifier
case-folding changes only ASCII letters.  I can't find anything in
our SGML docs about this, at least not where I'd expect it to be
documented.  The code is pretty clear about what it's doing though:

    /*
     * SQL99 specifies Unicode-aware case normalization, which we don't ye=
t
     * have the infrastructure for.  Instead we use tolower() to provide a
     * locale-aware translation.  However, there are some locales where th=
is
     * is not right either (eg, Turkish may do strange things with 'i' and
     * 'I').  Our current compromise is to use tolower() for characters wi=
th
     * the high bit set, as long as they aren't part of a multi-byte
     * character, and use an ASCII-only downcasing for 7-bit characters.
     */

These days the claim that no infrastructure is available is obsolete.
But I'm mighty hesitant to touch this behavior, because it'd almost
surely break peoples' apps.  We could do better on the documentation
front though.

			regards, tom lane