pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feed From: Peter Eisentraut <peter@eisentraut.org>
To: Jeff Davis <pgsql@j-davis.com>
To: Chao Li <li.evan.chao@gmail.com>
Cc: Thomas Munro <thomas.munro@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: pgsql-hackers@postgresql.org
Subject: Re: Remaining dependency on setlocale()
Date: Wed, 17 Dec 2025 11:39:05 +0100
Message-ID: <dd0cdd1f-e786-426e-b336-1ffa9b2f1fc6@eisentraut.org> (raw )
In-Reply-To: <0e186a9a92634f0c5675a618ff5685d00cd8f836.camel@j-davis.com >
References: <4c5da86af36a0d5e430eee3f60ce5e06f1b5cd34.camel@j-davis.com >
<ed394f839d813762ddcd65cfcab0e062f8b5e0b1.camel@j-davis.com >
<CA+hUKGLYcbnHBXhHk_OFwZ03LE+D=cAirm7TVN8fMhT9rgcTbA@mail.gmail.com >
<f040113cf384ada69558ec004a04a3ddb3e40a26.camel@j-davis.com >
<CA+hUKGJxSDFcTZSGi++PTC=jM49q1oK=v7Lcdf96VQXXvLhoow@mail.gmail.com >
<cdd20efc0f5418c597791a1e3a5abb68fa3ce35b.camel@j-davis.com >
<d9657a6e51aa20702447bb2386b32fea6218670f.camel@j-davis.com >
<b1ea6756-279a-4924-9146-e125af2057b2@eisentraut.org >
<8186b28a1a39e61a0d833a4c25a8909ebbbabd48.camel@j-davis.com >
<450ceb6260cad30d7afdf155d991a9caafee7c0d.camel@j-davis.com >
<DDD50039-02B4-4033-8D22-B086BC448E1C@gmail.com >
<78edc741a168c54d57cc898a5877b6443527a2c8.camel@j-davis.com >
<83F6E6A9-FF29-41C9-9E15-EA5360E912D0@gmail.com >
<5f65b85740197ba6249ea507cddf609f84a6188b.camel@j-davis.com >
<38f3f4a6-e95e-47a7-914f-110dd6191301@eisentraut.org >
<0e186a9a92634f0c5675a618ff5685d00cd8f836.camel@j-davis.com >
On 12.12.25 21:11, Jeff Davis wrote:
>> case '\xc7': /* C with cedilla */
>>
>> so the premise that "fuzzystrmatch is designed for ASCII" does not
>> appear to be correct. Needs more analysis.
>>
>> (But apparently it's not multibyte aware at all, so I don't know what
>> to
>> do about that.)
> I didn't notice that, thank you. Agreed, we need a bit more discussion
> around this case as well as soundex().
Soundex is an ASCII-only algorithm, there is no expectation that the
algorithm does anything useful with non-ASCII characters, and it doesn't
do so now. So I think using pg_ascii_toupper() is ok. (Users could for
example use unaccent to preprocess text.)
One might wonder if the presence of non-ASCII characters should be an
error, but that doesn't have to be the subject of this thread. I
noticed that the Wikipedia page for Soundex even calls out PostgreSQL
for doing things slightly different than everyone else, but I haven't
studied the details.
For Metaphone, I found the reference implementation linked from its
Wikipedia page, and it looks like our implementation is pretty closely
aligned to that. That reference implementation also contains the
C-with-cedilla case explicitly. The correct fix here would probably be
to change the implementation to work on wide characters. But I think
for the moment you could try a shortcut like, use pg_ascii_toupper(),
but if the encoding is LATIN1 (or LATIN9 or whichever other encodings
also contain C-with-cedilla at that code point), then explicitly
uppercase that one as well. This would preserve the existing behavior.
Note that the documentation calls out: "At present, the soundex,
metaphone, dmetaphone, and dmetaphone_alt functions do not work well
with multibyte encodings (such as UTF-8)."
view thread (96+ messages) latest in thread
Message-ID: <dd0cdd1f-e786-426e-b336-1ffa9b2f1fc6@eisentraut.org>
Permalink: ../dd0cdd1f-e786-426e-b336-1ffa9b2f1fc6@eisentraut.org/
Also on: postgresql.org/message-id/dd0cdd1f-e786-426e-b336-1ffa9b2f1fc6@eisentraut.org
copy link · copy postgr.es
reply Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: peter@eisentraut.org, pgsql@j-davis.com, li.evan.chao@gmail.com, thomas.munro@gmail.com, tgl@sss.pgh.pa.us
Subject: Re: Remaining dependency on setlocale()
In-Reply-To: <dd0cdd1f-e786-426e-b336-1ffa9b2f1fc6@eisentraut.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox