Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jeNHj-0002n2-BT for pgsql-bugs@arkaria.postgresql.org; Thu, 28 May 2020 18:29:43 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.92) (envelope-from ) id 1jeNHf-00025y-Ko for pgsql-bugs@arkaria.postgresql.org; Thu, 28 May 2020 18:29:39 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jeNHf-00025q-EC for pgsql-bugs@lists.postgresql.org; Thu, 28 May 2020 18:29:39 +0000 Received: from sss.pgh.pa.us ([66.207.139.130]) by makus.postgresql.org with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jeNHY-0007Eb-Uz for pgsql-bugs@lists.postgresql.org; Thu, 28 May 2020 18:29:38 +0000 Received: from pro.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.pgh.pa.us (8.15.2/8.15.2) with ESMTP id 04SITSio018153; Thu, 28 May 2020 14:29:28 -0400 From: Tom Lane To: James Lucas cc: "David G. Johnston" , PostgreSQL mailing lists , Peter Eisentraut Subject: Re: Explicit deterministic COLLATE fails with pattern matching operations on column with non-deterministic collation In-reply-to: References: <5075.1590625286@sss.pgh.pa.us> Comments: In-reply-to James Lucas message dated "Thu, 28 May 2020 13:04:06 -0500" MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <7416.1590690568.1@sss.pgh.pa.us> Date: Thu, 28 May 2020 14:29:28 -0400 Message-ID: <7417.1590690568@sss.pgh.pa.us> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk James Lucas writes: > I tried setting up a pathological test case for this, and it seems > like at least currently, even with a non-deterministic collation > statistics still count values as distinct, even if the default > collation would consider them equivalent. Not sure if that's as > intended or not? I experimented with this, and what I'm seeing is that ucol_strcollUTF8() reports that 'aaa' is different from 'aAa'. So the behavior on the Postgres side is as-expected. I suspect that the 'en-US-ks-level2' ICU locale doesn't act as you think it does. (That is, just saying that a collation is nondeterministic doesn't make it so; it only forces Postgres through slower code paths that allow for the possibility of bitwise-unequal strings being reported as equal by ICU.) Not knowing anything about ICU, I can't say more than that. [ Tested on libicu-60.3-2.el8_1 ] regards, tom lane