Feedback-ID: i0fe9450f:Fastmail
Date: Fri, 18 Aug 2023 15:18:58 +0900
From: Michael Paquier <michael@paquier.xyz>
To: martin@schlossarek.me, pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #18057: unaccent removes intentional spaces
Message-ID: <ZN8NUpx2f9pB+F/g@paquier.xyz>
References: <18057-62712cad01bd202c@postgresql.org>
 <ZNwRq4nV7CBXAoJN@paquier.xyz>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha512;
	protocol="application/pgp-signature"; boundary="MJNwkGkk3wx2Kn3w"
Content-Disposition: inline
In-Reply-To: <ZNwRq4nV7CBXAoJN@paquier.xyz>
Archived-At: 
 <https://www.postgresql.org/message-id/ZN8NUpx2f9pB%2BF/g%40paquier.xyz>
Precedence: bulk


--MJNwkGkk3wx2Kn3w
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

On Wed, Aug 16, 2023 at 09:00:43AM +0900, Michael Paquier wrote:
> Agreed that this looks incorrect as-is.  This goes as far as 9a206d0
> when these has been introduced, and it looks like the culprit is
> around initTrie() where the entries are loaded.  See around t_isspace,
> for example.

I was looking at the code, and my first impression was right.  All
leading and trailing whitespaces between the two characters listed in
the rule file are discarded.  The thing is that we clearly document
the parsing rules for the sake of any custom files one can feed to the
extension:
https://www.postgresql.org/docs/devel/unaccent.html

I am not sure what we can do here.  Doing nothing is certainly an
option, but I am wondering if we could put in place an extra rule
where whitespaces can be part of the translated character if it uses
double quotes, for example.  Thoughts?
--
Michael

--MJNwkGkk3wx2Kn3w
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAmTfDVIACgkQnvQgOdby
QH2++Q//fuqYViCpSwTUCnjfB9NhIm+vt8wN3J9/kL5xjcnmopRZm90pZzjA7gqB
TanSAv8hiUt5WN1sSyWsG3iB6lqVQC7gGxztB/GRauBM16DswDAMxO7hCAudc1nZ
OUoxxeWogoU1lDW6g2uwr8NEk0efLAnYDRIAkAKPVBMKk0eOU5wkEhcWwOEKNYop
gy0XjWVupeJOYIfBKkxQ8aUOPhSuLy2NEVb0mMzWlsneh7s8Pa8quKCrdvOynogz
5M4sgPWIfekNW/BkrEvlLenfYTxp0QDO1IQzRpZdrPrmfcywZ/GA1RUpHSvocGTp
gEaqe+ZueS+6J8NITIrNikeFJJZjQEAyAxgTUNb1XvU3JhELWBle4IobhQG6iEEq
7cCluHIVPEhc63cVx4ey8qd3aAWpm0NfTlVHhmz7cAEPghiO6W4Gybb5Ck9UTJBa
UeINW/2MpoIH83BZTIv/HeHlYPtGQFA/Q5OF3tMgOnfjqW5nQfgGCqXmXEDQTI92
T1RdiuOknIxYb37DMAZiLYePwY6rhc7GO/H8WumKkekVjI40jBoKwumNv2sXDBDM
n7Jg98d+0BFu6aQc+TbAQvUyLc0TnreE3lnyrWwDoLcRve1NpOBfWY0NAZKVzeYX
iiJi/6fVoftvC6ofx1oyqFYIOgK8YZVCBgbIHygOrmtlEJEQJfU=
=q5H+
-----END PGP SIGNATURE-----

--MJNwkGkk3wx2Kn3w--