Message-ID: <d2e7d63c0cc98f425652527cf260a3e6da17d2b0.camel@j-davis.com>
Subject: Re: Remaining dependency on setlocale()
From: Jeff Davis <pgsql@j-davis.com>
To: Peter Eisentraut <peter@eisentraut.org>, Thomas Munro
	 <thomas.munro@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, pgsql-hackers@postgresql.org
Date: Thu, 05 Jun 2025 22:15:34 -0700
In-Reply-To: <cd3517c7-ddb8-454e-9dd5-70e3d84ff6a2@eisentraut.org>
References: <4c5da86af36a0d5e430eee3f60ce5e06f1b5cd34.camel@j-davis.com>
	 <1841102.1722982989@sss.pgh.pa.us>
	 <CA+hUKGK7qKDHgoh1HuvMhtcpAwwVBA5BDt33vCzYYqCLfQTtzw@mail.gmail.com>
	 <CA+hUKGLLTy155HTQ6HfHu5-XPNfoFgDsRVUWevj+O6p3UmvPkg@mail.gmail.com>
	 <a70d3db4b8291b61c6937c0c003aa70e2d2c3a9a.camel@j-davis.com>
	 <CA+hUKGL82jG2PdgfQtwWG+_51TQ--6M9XNa3rtt7ub+S3Pmfsw@mail.gmail.com>
	 <b19ea499a6d78c101dc55b61bfb42069b427445c.camel@j-davis.com>
	 <fa04b56430fa4b5da7311c2faa113f406c6d192a.camel@j-davis.com>
	 <CA+hUKGKcS-Zhi8Ud+tv2TanzXBQjH+WsLyPY9DUDXob5VoYAtg@mail.gmail.com>
	 <cd3517c7-ddb8-454e-9dd5-70e3d84ff6a2@eisentraut.org>
Content-Type: multipart/mixed; boundary="=-9ReTaJJ5gP3X4yyG/q2A"
User-Agent: Evolution 3.52.3-0ubuntu1 
MIME-Version: 1.0
Archived-At: 
 <https://www.postgresql.org/message-id/d2e7d63c0cc98f425652527cf260a3e6da17d2b0.camel%40j-davis.com>
Precedence: bulk

--=-9ReTaJJ5gP3X4yyG/q2A
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, 2024-12-17 at 13:14 +0100, Peter Eisentraut wrote:
> > > > +1 to this approach. It makes things more consistent across
> > > > platforms
> > > > and avoids surprising dependencies on the global setting.
> > > >=20
> >=20
> > I think this is the best way, and I haven't seen anyone supporting
> > any
> > other idea.=C2=A0 (I'm working on those setlocale()-removal patches I
> > mentioned, more very soon...)
>=20
> I also think this is the right direction, and we'll get closer with
> the=20
> remaining patches that Thomas has lined up.
>=20
> I think at this point, we could already remove all locale settings=20
> related to LC_COLLATE.=C2=A0 Nothing uses that anymore.
>=20
> I think we will need to keep the global LC_CTYPE setting set to=20
> something useful, for example so that system error messages come out
> in=20
> the right encoding.
>=20
> But I'm concerned about the the Perl_setlocale() dance in plperl.c.=20
> Perl apparently does a setlocale(LC_ALL, "") during startup, and that
> code is a workaround to reset everything back afterwards.=C2=A0 We need t=
o
> be=20
> careful not to break that.
>=20
> (Perl has fixed that in 5.19, but the fix requires that you set
> another=20
> environment variable before launching Perl, which you can't do in a=20
> threaded system, so we'd probably need another fix eventually.=C2=A0 See=
=20
> <https://github.com/Perl/perl5/issues/8274>.)

To continue this thread, I did a symbol search in the meson build
directory like (patterns.txt attached):

  for f in `find . -name *.o`; do
    if ( nm --format=3Djust-symbols $f | \
         grep -xE -f /tmp/patterns.txt > /dev/null ); then
      echo $f; fi; done

and it output:

./contrib/fuzzystrmatch/fuzzystrmatch.so.p/dmetaphone.c.o
./contrib/fuzzystrmatch/fuzzystrmatch.so.p/fuzzystrmatch.c.o
./contrib/isn/isn.so.p/isn.c.o
./contrib/spi/refint.so.p/refint.c.o
./contrib/ltree/ltree.so.p/crc32.c.o
./src/backend/postgres_lib.a.p/commands_copyfromparse.c.o
./src/backend/postgres_lib.a.p/utils_adt_pg_locale_libc.c.o
./src/backend/postgres_lib.a.p/tsearch_wparser_def.c.o
./src/backend/postgres_lib.a.p/parser_scansup.c.o
./src/backend/postgres_lib.a.p/utils_adt_inet_net_pton.c.o
./src/backend/postgres_lib.a.p/tsearch_ts_locale.c.o
./src/bin/psql/psql.p/meson-generated_.._tab-complete.c.o
./src/interfaces/ecpg/preproc/ecpg.p/meson-generated_.._preproc.c.o
./src/interfaces/ecpg/compatlib/libecpg_compat.so.3.18.p/informix.c.o
./src/interfaces/ecpg/compatlib/libecpg_compat.a.p/informix.c.o
./src/port/libpgport_srv.a.p/pgstrcasecmp.c.o
./src/port/libpgport_shlib.a.p/pgstrcasecmp.c.o
./src/port/libpgport.a.p/pgstrcasecmp.c.o

Not a short list, but not a long list, either, so seems tractable. Note
that this misses things like isdigit() which is inlined.

A few observations while spot-checking these files:

---------------------
pgstrcasecmp.c - has code like:

       else if (IS_HIGHBIT_SET(ch) && islower(ch))
        ch =3D toupper(ch);

and comments like "Note however that the whole thing is a bit bogus for
multibyte character sets."

Most of the callers are directly comparing with ascii literals, so I'm
not sure what the point is. There are probably some more interesting
callers hidden in there.
----------------------
pg_locale_libc.c -

char2wchar and wchar2char use mbstowcs and wcstombs when the input
locale is NULL. The main culprit seems to be full text search, which
has a bunch of /* TODO */ comments. Another caller is
get_iso_localename().

There are also a couple false positives where mbstowcs_l/wcstombs_l are
emulated with uselocale() and mbstowcs/wcstombs. In that case, it's not
actually sensitive to the global setting.
-----------------------
copyfromparse.c - the input is ASCII so it can use pg_ascii_tolower()
instead of tolower()
-----------------------

Regards,
	Jeff Davis


--=-9ReTaJJ5gP3X4yyG/q2A
Content-Disposition: attachment; filename="patterns.txt"
Content-Transfer-Encoding: base64
Content-Type: text/plain; name="patterns.txt"; charset="UTF-8"

c3RyeGZybQpzdHJjb2xsCndjc2NvbGwKd2NzeGZybQppc3c/YWxwaGEKaXN3P2RpZ2l0Cmlzdz9h
bG51bQppc3c/c3BhY2UKaXN3P3VwcGVyCmlzdz9sb3dlcgp0b3c/dXBwZXIKdG93P2xvd2VyCmlz
dz9ncmFwaAppc3c/cHVuY3QKaXN3P2NudHJsCmlzdz94ZGlnaXQKaXN3P3ByaW50Cndjc3RvbWJz
Cm1ic3Rvd2NzCmQ/bj9nZXR0ZXh0CmJpbmR0ZXh0ZG9tYWluCg==


--=-9ReTaJJ5gP3X4yyG/q2A--