Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gZ8Xo-0004hQ-WB for pgsql-bugs@arkaria.postgresql.org; Tue, 18 Dec 2018 06:07:53 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1gZ8Xm-0006ng-M1 for pgsql-bugs@arkaria.postgresql.org; Tue, 18 Dec 2018 06:07:50 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gZ8Xm-0006nV-8s for pgsql-bugs@lists.postgresql.org; Tue, 18 Dec 2018 06:07:50 +0000 Received: from out1-smtp.messagingengine.com ([66.111.4.25]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1gZ8Xe-0002QP-VR for pgsql-bugs@lists.postgresql.org; Tue, 18 Dec 2018 06:07:49 +0000 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id C2A75210F5; Tue, 18 Dec 2018 01:07:41 -0500 (EST) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Tue, 18 Dec 2018 01:07:41 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=paquier.xyz; h= date:from:to:cc:subject:message-id:references:mime-version :content-type:in-reply-to; s=fm3; bh=p+29FuCqWt8n2VlHMWn410WRi4T RXdnL3y1/T4cDh94=; b=sJuQ53Du4vI2pQBOZgCpzckYPmguJNS7O6yLC5TtRlD 9+yLD6fXQbRfhZO7B4cfkFImc3euynymiJvQO+olxo8s0rEmhWQBl2kg01K8XBP8 xCdLe2Ke3xo4FVhfyZBwMMourtNLdRjzkfoZOEwZAVtX2Nf0Gw+S+uh6isP0iXcs pPJDc1Zo3bBcK9iZoIGLmT6gq9kHB8NZsCskYpltijn+1jOYD5Vrsm57xKOgL7uo RpbjohYvRQLbQhS9zg7VaLVYsScod98ua3wsfgBwPytY1mE5tTygkn7kUTJtJvFQ ZTWK/oAgDSWFTpINukOzQmCyAN6Lc17/5PlzOqlkMTQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=p+29Fu CqWt8n2VlHMWn410WRi4TRXdnL3y1/T4cDh94=; b=oMAEF5dotZjsSIplTZQ9Px f9ihJQK4QXHEPs1oYs9s9RyfD6T+6Zl+9fWV/bUZVyWN4p5VwkUnh1+ufc1U+yfE L7miwkU7tfdcwWSdaNuNX7/Uj/hhcnsz1/PGEmg4tKKMhS0CkMHb2rbIl4qg+myh oMcVxR3qrkelkwvoDW0UJNV0xBE/DqdXuSpfo/GmZhHilfXjKF0fqnAKs8jjNI5l 6kkGeG6tf4Plz45jqY9VKlgSyfoIvHFpx0LD3OAmVNIdgS4MhqzOSmJsMIv6ifwM 9TUl3z/Y8u4yd15J5Wb8Wbt9lBGrGQfTUp/amNo4od+53y7dL7WO38o8ojhWlldA == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedtkedrudeigedgleefucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfquhhtnecuuegrihhlohhuthemucef tddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenfghrlhcuvffnffculdejtd dmnecujfgurhepfffhvffukfhfgggtuggjsehgtderredttddvnecuhfhrohhmpefoihgt hhgrvghlucfrrghquhhivghruceomhhitghhrggvlhesphgrqhhuihgvrhdrgiihiieqne cukfhppeeiuddrvdduhedrudefjedrudeivdenucfrrghrrghmpehmrghilhhfrhhomhep mhhitghhrggvlhesphgrqhhuihgvrhdrgiihiienucevlhhushhtvghrufhiiigvpedt X-ME-Proxy: Received: from paquier.xyz (c137162.net61215.cablenet.ne.jp [61.215.137.162]) by mail.messagingengine.com (Postfix) with ESMTPA id CB7E110330; Tue, 18 Dec 2018 01:07:38 -0500 (EST) Date: Tue, 18 Dec 2018 15:07:35 +0900 From: Michael Paquier To: Tom Lane Cc: Thomas Munro , hugh@whtc.ca, daniel@manitou-mail.org, PostgreSQL mailing lists Subject: Re: BUG #15548: Unaccent does not remove combining diacritical characters Message-ID: <20181218060735.GL1532@paquier.xyz> References: <29419.1544908811@sss.pgh.pa.us> <31564.1545078667@sss.pgh.pa.us> <20181218045708.GI1532@paquier.xyz> <8506.1545111362@sss.pgh.pa.us> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="B92bTrfKjyax39gr" Content-Disposition: inline In-Reply-To: <8506.1545111362@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk --B92bTrfKjyax39gr Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, Dec 18, 2018 at 12:36:02AM -0500, Tom Lane wrote: > tl;dr: I think we should convert unaccent.sql and unaccent.out > to UTF8 encoding. Then, adding more test cases for this patch > will be easy. Do you think that we could also remove the non-ASCII characters from the tests? It would be easy enough to use E'\xNN' (utf8 hex) or such in input, and show the output with bytea. That's harder to read, still we discussed about not using UTF-8 in the python script to allow folks with simple terminals to touch the code the last time this was touched (5e8d670) and the characters used could be documented as comments in the tests. -- Michael --B92bTrfKjyax39gr Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCgAdFiEEG72nH6vTowiyblFKnvQgOdbyQH0FAlwYjqcACgkQnvQgOdby QH0kvA/9Eu76bXSi4iFEGKJFpXKsJJlRFMdnjid9DFReSqGRxiCAn6ohPSmwl8bO qulmqTpw+jlNKLN4udaoguV2gqCxHnSzgFtc32RxApzbjPSqze255n6oM3mECJdR OcEhmme7GKI3PWK/wulA9pJtsJSlxZ11WhqPUnLD6KOlzxVeLhUh2zyLNSAhNw1/ SmR+1Rcfxefm+xUwTpI9QZe76DexrS1klMzLbbeGBUYKC5e/m2tA+Rgw15kyR9ab A3uX+c1CkWvHfCjX7pzb2NcghqDRfsbR+jHYWBGPLkpdepO6x5j1FaL6HPXOoamT c/rdff8r/ENjCTFt0S7FPkM2fqKRG5BSGNekmkaLLTcHDwuiCYKKUp9Nw9YLOFAm y5CSh88nTCENDdawt8TfbNl+uK2ThPQZiOH5vy7oDEdcY+Z11TJbmW9HR/ach+nG w0ayC4gdX+INvOPn1cMKkINGKea5vOhaNGCW/wfUFaio2uyQBKyq9Cp3NaAtCeW3 /HIo/U6u9GvLm1ZTzSAPm2OMi3AmcSIFywrkxnx9x6sG5kZgTRrWM4pqcj8djzQX 1Xw1aJD3RMpkK9i+z8vpc4PLmBNMXCmNOapqFKj2DrI79DmxihQCN/8ExHtpW0Hw OgKYiaXrXGkgE8VqfV9ddY6fFMLjXP6HdjQ/ifxp6vA/st0mTeU= =Hf/e -----END PGP SIGNATURE----- --B92bTrfKjyax39gr--