pg.ddx.io  pgsql-hackers@postgresql.org mailing list archive  

help / color / mirror / Atom feed
From: Jeff Davis <pgsql@j-davis.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Tatsuo Ishii <ishii@postgresql.org>
Cc: pgsql-hackers@postgresql.org
Subject: Re: C11: should we use char32_t for unicode code points?
Date: Wed, 29 Oct 2025 08:12:01 -0700
Message-ID: <024a6d53f246c87ee2796563f930a66fde1c0c0d.camel@j-davis.com> (raw)
In-Reply-To: <CA+hUKG+hDkp1etcfy=taxJ8ybf8KapyOjqdBRPF7yaoSoSj1_w@mail.gmail.com>
References: <bedcc93d06203dfd89815b10f815ca2de8626e85.camel@j-davis.com>
	<20251024.184315.1449345234035166124.ishii@postgresql.org>
	<435cf6bcf14bb91235da065c70d782863ca9b80e.camel@j-davis.com>
	<CA+hUKGJ5Xh0KxLYXDZuPvw1_fHX=yuzb4xxtam1Cr6TPZZ1o+w@mail.gmail.com>
	<efd8b17d7f97bc7653dde565e274fef32f5da65c.camel@j-davis.com>
	<CA+hUKGLXQUYK7Cq5KbLGgTWo7pORs7yhBWO1AEnZt7xTYbLRhg@mail.gmail.com>
	<8e5e4892c5c6b1b031b8e715dd254f21d9fb2bd9.camel@j-davis.com>
	<CA+hUKG+hDkp1etcfy=taxJ8ybf8KapyOjqdBRPF7yaoSoSj1_w@mail.gmail.com>

On Wed, 2025-10-29 at 14:00 +1300, Thomas Munro wrote:
> I wonder if the logic to select the member/semantics could be turned
> into an enum in the encoding table, to make it even clearer, and then
> that could be used as an index into a table of ctype methods obejcts
> in _libc.c.

As long as we're able to isolate that logic in the libc provider,
that's reasonable. The other providers don't need that complexity, they
just need to decode straight to UTF-32.

> You showed char16_t for Windows, but we don't ever get char16_t out
> of
> wchar.c, it's always char32_t for UTF-8 input.  It's just that
> _libc.c
> truncates to UTF-16 or short-circuits to avoid overflow on that
> platform (and in the past AIX 32-bit and maybe more), so it wouldn't
> belong in a hypothetical union or enum.

Oh, I see.

> > 
> Perhaps we could at least put the conversion in a new encoding table
> function pointer "pg_wchar_custom_to_wchar_t", so we could reserve a
> place to put that sort of optimisation in

That sounds like a good step forward. And maybe one to convert to UTF-
32 for ICU, also?

> If we do develop this idea though, one issue to contemplate is that
> EUC code points might generate more than one wchar_t, looking at
> EUC_JIS_2004[1].

Wow, that's unfortunate.


Regards,
	Jeff Davis






view thread (18+ messages)  latest in thread

Message-ID: <024a6d53f246c87ee2796563f930a66fde1c0c0d.camel@j-davis.com>
Permalink:  ../024a6d53f246c87ee2796563f930a66fde1c0c0d.camel@j-davis.com/
Also on:    postgresql.org/message-id/024a6d53f246c87ee2796563f930a66fde1c0c0d.camel@j-davis.com

 · 

reply
Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-hackers@postgresql.org
  Cc: pgsql@j-davis.com, thomas.munro@gmail.com, ishii@postgresql.org
  Subject: Re: C11: should we use char32_t for unicode code points?
  In-Reply-To: <024a6d53f246c87ee2796563f930a66fde1c0c0d.camel@j-davis.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox