Feedback-ID: ie0a040ee:Fastmail
Message-ID: <dd0cdd1f-e786-426e-b336-1ffa9b2f1fc6@eisentraut.org>
Date: Wed, 17 Dec 2025 11:39:05 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: Remaining dependency on setlocale()
To: Jeff Davis <pgsql@j-davis.com>, Chao Li <li.evan.chao@gmail.com>
Cc: Thomas Munro <thomas.munro@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>,
 pgsql-hackers@postgresql.org
References: <4c5da86af36a0d5e430eee3f60ce5e06f1b5cd34.camel@j-davis.com>
 <ed394f839d813762ddcd65cfcab0e062f8b5e0b1.camel@j-davis.com>
 <CA+hUKGLYcbnHBXhHk_OFwZ03LE+D=cAirm7TVN8fMhT9rgcTbA@mail.gmail.com>
 <f040113cf384ada69558ec004a04a3ddb3e40a26.camel@j-davis.com>
 <CA+hUKGJxSDFcTZSGi++PTC=jM49q1oK=v7Lcdf96VQXXvLhoow@mail.gmail.com>
 <cdd20efc0f5418c597791a1e3a5abb68fa3ce35b.camel@j-davis.com>
 <d9657a6e51aa20702447bb2386b32fea6218670f.camel@j-davis.com>
 <b1ea6756-279a-4924-9146-e125af2057b2@eisentraut.org>
 <8186b28a1a39e61a0d833a4c25a8909ebbbabd48.camel@j-davis.com>
 <450ceb6260cad30d7afdf155d991a9caafee7c0d.camel@j-davis.com>
 <DDD50039-02B4-4033-8D22-B086BC448E1C@gmail.com>
 <78edc741a168c54d57cc898a5877b6443527a2c8.camel@j-davis.com>
 <83F6E6A9-FF29-41C9-9E15-EA5360E912D0@gmail.com>
 <5f65b85740197ba6249ea507cddf609f84a6188b.camel@j-davis.com>
 <38f3f4a6-e95e-47a7-914f-110dd6191301@eisentraut.org>
 <0e186a9a92634f0c5675a618ff5685d00cd8f836.camel@j-davis.com>
Content-Language: de-DE, en-US
From: Peter Eisentraut <peter@eisentraut.org>
In-Reply-To: <0e186a9a92634f0c5675a618ff5685d00cd8f836.camel@j-davis.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Archived-At: 
 <https://www.postgresql.org/message-id/dd0cdd1f-e786-426e-b336-1ffa9b2f1fc6%40eisentraut.org>
Precedence: bulk

On 12.12.25 21:11, Jeff Davis wrote:
>> case '\xc7':        /* C with cedilla */
>>
>> so the premise that "fuzzystrmatch is designed for ASCII" does not
>> appear to be correct.  Needs more analysis.
>>
>> (But apparently it's not multibyte aware at all, so I don't know what
>> to
>> do about that.)
> I didn't notice that, thank you. Agreed, we need a bit more discussion
> around this case as well as soundex().

Soundex is an ASCII-only algorithm, there is no expectation that the 
algorithm does anything useful with non-ASCII characters, and it doesn't 
do so now.  So I think using pg_ascii_toupper() is ok.  (Users could for 
example use unaccent to preprocess text.)

One might wonder if the presence of non-ASCII characters should be an 
error, but that doesn't have to be the subject of this thread.  I 
noticed that the Wikipedia page for Soundex even calls out PostgreSQL 
for doing things slightly different than everyone else, but I haven't 
studied the details.

For Metaphone, I found the reference implementation linked from its 
Wikipedia page, and it looks like our implementation is pretty closely 
aligned to that.  That reference implementation also contains the 
C-with-cedilla case explicitly.  The correct fix here would probably be 
to change the implementation to work on wide characters.  But I think 
for the moment you could try a shortcut like, use pg_ascii_toupper(), 
but if the encoding is LATIN1 (or LATIN9 or whichever other encodings 
also contain C-with-cedilla at that code point), then explicitly 
uppercase that one as well.  This would preserve the existing behavior.

Note that the documentation calls out: "At present, the soundex, 
metaphone, dmetaphone, and dmetaphone_alt functions do not work well 
with multibyte encodings (such as UTF-8)."