Message-ID: <ee88559cc739098a67f8d90d867f8dae9b00f82a.camel@j-davis.com>
Subject: Re: C11: should we use char32_t for unicode code points?
From: Jeff Davis <pgsql@j-davis.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Tatsuo Ishii <ishii@postgresql.org>, pgsql-hackers@postgresql.org
Date: Wed, 29 Oct 2025 13:14:21 -0700
In-Reply-To: 
 <CA+hUKGLAcZ7EdxZ-mCS1Pf0iCsCKeVYpMiYWLkH+fobWQ4ANNA@mail.gmail.com>
References: <bedcc93d06203dfd89815b10f815ca2de8626e85.camel@j-davis.com>
	 <20251024.184315.1449345234035166124.ishii@postgresql.org>
	 <435cf6bcf14bb91235da065c70d782863ca9b80e.camel@j-davis.com>
	 <CA+hUKGJ5Xh0KxLYXDZuPvw1_fHX=yuzb4xxtam1Cr6TPZZ1o+w@mail.gmail.com>
	 <efd8b17d7f97bc7653dde565e274fef32f5da65c.camel@j-davis.com>
	 <CA+hUKGLXQUYK7Cq5KbLGgTWo7pORs7yhBWO1AEnZt7xTYbLRhg@mail.gmail.com>
	 <8e5e4892c5c6b1b031b8e715dd254f21d9fb2bd9.camel@j-davis.com>
	 <CA+hUKG+hDkp1etcfy=taxJ8ybf8KapyOjqdBRPF7yaoSoSj1_w@mail.gmail.com>
	 <CA+hUKGLAcZ7EdxZ-mCS1Pf0iCsCKeVYpMiYWLkH+fobWQ4ANNA@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.52.3-0ubuntu1 
MIME-Version: 1.0
Archived-At: 
 <https://www.postgresql.org/message-id/ee88559cc739098a67f8d90d867f8dae9b00f82a.camel%40j-davis.com>
Precedence: bulk

On Thu, 2025-10-30 at 04:25 +1300, Thomas Munro wrote:
> Here are some sketch-quality patches to try out some of these ideas,
> for discussion.=C2=A0 I gave them .txt endings so as not to hijack your
> thread's CI.

I like the direction this is going. I will commit the char32_t work
anyway, so afterward feel free to hijack the thread (there's a lot of
good information here so continuing here might be more productive than
starting a new thread).

Regarding 0002, IIUC, for PG_WCHAR_UTF32, surrogates are forbidden, but
the comment about UTF-16 is a bit vague. I think we should add some
asserts to make it clear.

The basic communication mechanism between the modules is the database
encoding: it determines PgWcharEncodingScheme in both wchar.c and
pg_locale_libc.c. That seems reasonable to me, and doesn't interfere
with the other providers.

I'm still not quite sure how this fits with ICU in a single-byte
encoding, but doesn't seem worse than what we do currently.

Also, tangentially, I'm a bit anxious to do a permanent
setlocale(LC_CTYPE, "C"), and we are very close. If these two threads
are successful, I believe we can do it:

https://www.postgresql.org/message-id/90f176c5b85b9da26a3265b2630ece3552068=
566.camel%40j-davis.com

https://www.postgresql.org/message-id/d9657a6e51aa20702447bb2386b32fea62186=
70f.camel@j-davis.com

That would be a big simplification because it would isolate libc ctype
behavior to pg_locale_libc.c. That would make me feel generally more
comfortable with additional work in this area.

Regards,
	Jeff Davis