Feedback-ID: i0fe9450f:Fastmail
Date: Wed, 17 Dec 2025 11:59:17 +0900
From: Michael Paquier <michael@paquier.xyz>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Robert Haas <robertmhaas@gmail.com>,
	Jeroen Vermeulen <jtvjtv@gmail.com>,
	VASUKI M <vasukianand0119@gmail.com>,
	pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #19354: JOHAB rejects valid byte sequences
Message-ID: <aUIchajpeYVTF4BT@paquier.xyz>
References: <19354-eefe6d8b3e84f9f2@postgresql.org>
 <CA+TgmoaRGSezRaA7x00X495Qho8WGTzggbDSUt-JsruXceZWug@mail.gmail.com>
 <CA+zULE4L4rA2DLAcfy=eQL7w_ZexV4P5zpQRbP=_qrhJBEOzjg@mail.gmail.com>
 <2292889.1765846569@sss.pgh.pa.us>
 <CAE2r8H5vaSyaC_t1FcpHBo-BB_=SrFj7GFnOC-SxC6WDf5c9VA@mail.gmail.com>
 <CA+zULE47EXZOp7qKYODd+mjSgDiR-WX5ZNBkwdKnj-Zc0FT58w@mail.gmail.com>
 <CA+TgmoZaoc37ohnhF5inoPxWzfoznV483xQw8Fmw+ELFScv47g@mail.gmail.com>
 <2393116.1765899706@sss.pgh.pa.us>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="G/3J6JhiVljBTI2y"
Content-Disposition: inline
In-Reply-To: <2393116.1765899706@sss.pgh.pa.us>
Archived-At: <https://www.postgresql.org/message-id/aUIchajpeYVTF4BT%40paquier.xyz>
Precedence: bulk


--G/3J6JhiVljBTI2y
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Tue, Dec 16, 2025 at 10:41:46AM -0500, Tom Lane wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
>> I'm
>> left with the conclusions that (1) nobody ever actually tried using
>> this encoding for anything real until 3 days ago and (2) we don't have
>> any testing infrastructure that verifies that the characters in the
>> mapping tables are actually accepted by pg_verifymbstr(). I wonder how
>> many other encodings we have that don't actually work?
>=20
> Indeed.  Anyone want to do some testing?

FWIW, I have been made aware a couple of weeks ago by a colleague that
SJIS and SHIFT_JIS_2004 are used by some customers, and that we are
many years behind an update of the conversion mappings in the tree
with Postgres not understanding some of the characters.  These are two
marginal in the mostly-UTF8 world we live in these days, but it's
annoying for byte sequences that should not change across the years,
just be refreshed with new data.
--
Michael

--G/3J6JhiVljBTI2y
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmlCHIUACgkQnvQgOdby
QH0hvhAAqowEaAnxc8tpL/jaaU1mnaOLW9BRRnGzvDXzzswsk0PHJoUflo6dMOaJ
ng7elIRP3LUjHTJ0I48iCNdykW27EPmdVomjMzVBoZ7QBe0q6zE8IyiTlKpLqNr7
N96LlgpncbQ/K1d2vUk0T6j9C31m7Bwi2IAtzXHwQ0NuATRaUQyT1qymGd3bpK8w
QfSkSLu7Bs6IzzDHdSYy7rUhdIM44uTksQyL0Igymoe7BrGluu9qS1V0Kx/A9vpT
YewX6GeWxQM34c44jHQo3F5Qlx2bTxrvFM6E+f7Sp13TbTISlbl/3qvk8uNiinhP
Kv0WEvVTv4vANoGzrlzCeTWq/kmc3gzKkRANUFSW/Im3YvY81DvoAj+vVZR3s2cW
K+CxYtIv4QlnLfP8jig1cQLmxcp7xTXL7AZzuNx0JYXASU3+FomEW/HVUwa93WO0
q9Z081C7vg+EZx5QxqyF0dskvH5WuEVoZ4eYEerwGiURHZ5ekxXCIQzrlpH2X6sV
ldjYmR6fEJebhhbvklQGMZq0O/FJd9xHcemFnNyMidmqLpoz8w1gQXxyLxfNoAYO
dKIi2piGkxKKKIPZj5phm4/LgPISUfNtZQtD+mVc8XIxfOl4s2ITomDV1+TKfUAR
Fjd86C2sHlMRQN354+ufBOFZFq6ArBxCnQy0pFzlyXAV14iPPcE=
=tMqf
-----END PGP SIGNATURE-----

--G/3J6JhiVljBTI2y--