MIME-Version: 1.0
Date: Wed, 04 Oct 2023 14:02:50 -0400
From: Chapman Flack <chap@anastigmatix.net>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Nico Williams <nico@cryptonector.com>, Jeff Davis <pgsql@j-davis.com>,
 pgsql-hackers@postgresql.org
Subject: Re: Pre-proposal: unicode normalized text
In-Reply-To: 
 <CA+TgmoY6vNbDoBugDT=3rdJ-diyLw-kQ9sxB2JXenv48xV_tVQ@mail.gmail.com>
References: <f30b58657ceb71d5be032decf4058d454cc1df74.camel@j-davis.com>
 <ZR2fnTPDwPb0IamC@ubby21>
 <CA+TgmoY6vNbDoBugDT=3rdJ-diyLw-kQ9sxB2JXenv48xV_tVQ@mail.gmail.com>
User-Agent: Roundcube Webmail/1.6.0
Message-ID: <43864caecd65d02081d623d05ccc1683@anastigmatix.net>
Content-Type: text/plain; charset=UTF-8;
 format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: 
 <https://www.postgresql.org/message-id/43864caecd65d02081d623d05ccc1683%40anastigmatix.net>
Precedence: bulk

On 2023-10-04 13:47, Robert Haas wrote:
> On Wed, Oct 4, 2023 at 1:27 PM Nico Williams <nico@cryptonector.com> 
> wrote:
>> A UTEXT type would be helpful for specifying that the text must be
>> Unicode (in which transform?) even if the character data encoding for
>> the database is not UTF-8.
> 
> That's actually pretty thorny ... because right now client_encoding
> specifies the encoding to be used for all data sent to the client. So
> would we convert the data from UTF8 to the selected client encoding?

The SQL standard would have me able to:

CREATE TABLE foo (
   a CHARACTER VARYING CHARACTER SET UTF8,
   b CHARACTER VARYING CHARACTER SET LATIN1
)

and so on, and write character literals like

_UTF8'Hello, world!' and _LATIN1'Hello, world!'

and have those columns and data types independently contain what
they can contain, without constraints imposed by one overall
database encoding.

Obviously, we're far from being able to do that. But should it
become desirable to get closer, would it be worthwhile to also
try to follow how the standard would have it look?

Clearly, part of the job would involve making the wire protocol
able to transmit binary values and identify their encodings.

Regards,
-Chap