Date: Thu, 5 Oct 2023 14:52:37 -0500
From: Nico Williams <nico@cryptonector.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Robert Haas <robertmhaas@gmail.com>,
	Isaac Morland <isaac.morland@gmail.com>,
	Jeff Davis <pgsql@j-davis.com>,
	Chapman Flack <chap@anastigmatix.net>, pgsql-hackers@postgresql.org
Subject: Re: Pre-proposal: unicode normalized text
Message-ID: <ZR8UBTZcywbhK4JI@ubby21>
References: <f30b58657ceb71d5be032decf4058d454cc1df74.camel@j-davis.com>
 <ZR2fnTPDwPb0IamC@ubby21>
 <CA+TgmoY6vNbDoBugDT=3rdJ-diyLw-kQ9sxB2JXenv48xV_tVQ@mail.gmail.com>
 <43864caecd65d02081d623d05ccc1683@anastigmatix.net>
 <CAMsGm5eyegAkb2Eq+8d2j+BxDnSZXH0AUciwi6rA60V_ft=1dw@mail.gmail.com>
 <94a0e631b787d56fec1b17e03fe045b5e14907c1.camel@j-davis.com>
 <CAMsGm5f8k4_C6VuerSbF2gXeVwD9kMZQk43kO85Oi4oGVL7EMA@mail.gmail.com>
 <CA+TgmobAxizsgjxvZdEQxjEs6RA3qu7JLti_LdXtaXODJoWzNw@mail.gmail.com>
 <ZR8LLrk9AJVxEFbX@ubby21>
 <2816608.1696535377@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <2816608.1696535377@sss.pgh.pa.us>
Archived-At: <https://www.postgresql.org/message-id/ZR8UBTZcywbhK4JI%40ubby21>
Precedence: bulk

On Thu, Oct 05, 2023 at 03:49:37PM -0400, Tom Lane wrote:
> Nico Williams <nico@cryptonector.com> writes:
> > Text+encoding can be just like bytea with a one- or two-byte prefix
> > indicating what codeset+encoding it's in.  That'd be how to encode
> > such text values on the wire, though on disk the column's type should
> > indicate the codeset+encoding, so no need to add a prefix to the value.
> 
> The precedent of BOMs (byte order marks) suggests strongly that
> such a solution would be horrible to use.

This is just how you encode the type of the string.  You have any number
of options.  The point is that already PG can encode binary data, so if
how to encode text of disparate encodings on the wire, building on top
of the encoding of bytea is an option.