Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qygwc-00AIGw-Js for pgsql-hackers@arkaria.postgresql.org; Thu, 02 Nov 2023 23:17:46 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1qygwa-00G6Sm-Pq for pgsql-hackers@arkaria.postgresql.org; Thu, 02 Nov 2023 23:17:44 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qygwa-00G6Se-71 for pgsql-hackers@lists.postgresql.org; Thu, 02 Nov 2023 23:17:44 +0000 Received: from sienna.cherry.relay.mailchannels.net ([23.83.223.165]) by makus.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qygwU-003fq8-JC for pgsql-hackers@postgresql.org; Thu, 02 Nov 2023 23:17:42 +0000 X-Sender-Id: dreamhost|x-authsender|nico@cryptonector.com Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 01E8614258B; Thu, 2 Nov 2023 23:17:37 +0000 (UTC) Received: from pdx1-sub0-mail-a250.dreamhost.com (unknown [127.0.0.6]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 7AF2D14246C; Thu, 2 Nov 2023 23:17:36 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1698967056; a=rsa-sha256; cv=none; b=DjTVwpc4x550wMn8V1b93iKQ32c9MFtlPc8CN3m9yXhitbXS36vEtVVgxV8XloCnH0ptjK MBQSjJ948GgedB7S9BJIko9Dqllf4Q0pz8ETnH5hY9wA5ZhSJAzGdTrbjNqVOpDBNSb7HS Kvsq6JHHu2bypJbVEZxO28e17E0DmW+WkRHs5OIJGL+geVnr/h0QXgH6H+JnnFs1+H9hZQ PWM9FWVbhsO816eypm8+7bMG43HxB+KDZomjxnDVe+1K9iwJybzQMVX+Ph+t81hjnflqOG KrA+ZkDHs5YfHGSnzzvgU7pkHMJ2CwSCpUfdqklrLzxkz+C7KzjS+dPJm929HQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1698967056; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Z7KerFm63dCWg2rpiEbQloEmqMx1yXBwtj+QOaa4Qdc=; b=ki3tA81h2dO0mYKMHkTzAip7Eg4DAlskALCUcwa5N8pW4HpDMz1vbhV8xsmpigCYv589XF 51dMpVF4CREKCkcrhguadTxhVBZatN4wmOg51TVtPTdux6PxNZNvCHgvUP8QecnXsZn9Mu fPaiDAtxIauVwGejUOBz77P1DRtFMm+7D7MDq4YWpR6J36othlZ6mOKGqS+sTMECTw560E TDtpoMSSKdJgck1Ha71uwwg7cLe5F6NNwKbJwMfwvwT7m8D19lhIq9Zbcs5BWaJhkF3Sm1 ru8lpCHqt1/oTEptFgm2D9FchUo30DRhY0Jk+1ERdrqKuX6cfLu0a4GpGElxYA== ARC-Authentication-Results: i=1; rspamd-86646d89b6-z69w7; auth=pass smtp.auth=dreamhost smtp.mailfrom=nico@cryptonector.com X-Sender-Id: dreamhost|x-authsender|nico@cryptonector.com X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|nico@cryptonector.com X-MailChannels-Auth-Id: dreamhost X-Interest-Wiry: 43e3a77015892762_1698967056658_1568343083 X-MC-Loop-Signature: 1698967056658:4221728890 X-MC-Ingress-Time: 1698967056657 Received: from pdx1-sub0-mail-a250.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.112.232.215 (trex/6.9.2); Thu, 02 Nov 2023 23:17:36 +0000 Received: from ubby21 (075-081-095-064.res.spectrum.com [75.81.95.64]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: nico@cryptonector.com) by pdx1-sub0-mail-a250.dreamhost.com (Postfix) with ESMTPSA id 4SM0CM510vz6y; Thu, 2 Nov 2023 16:17:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cryptonector.com; s=dreamhost; t=1698967056; bh=Z7KerFm63dCWg2rpiEbQloEmqMx1yXBwtj+QOaa4Qdc=; h=Date:From:To:Cc:Subject:Content-Type; b=WEr1c16ItkDn7CL/1Yrx5+plf0kEQwD1mBLteq+trrQBgzJ5WzZMmiJ4evh0SU3Ei OKNDDeDDvoBP6Vl/hBndhUfskwImDiEQWsDplYuIwQoVDRK+7BRo2UJW+t80/gW/WC g249vhPB2Mc4M6b8FJla9LeT61sNsVmfL2PNFptuIo2SNQFV2S3xTZF1T4bwWQ04h8 QtZizP16yUfn/96th7JlVDfoKDIMF2I5gu+a9GFmqhcnJltTVjxFyhI2gliZ+tDn3z W+0tTA6t2Ikv8gV4tG6yzbkg436+frMRneglJoSWBLFjA4ecuuMxApmt54egtgsxdo yYn+StuFavbOQ== Date: Thu, 2 Nov 2023 18:17:33 -0500 From: Nico Williams To: Daniel Verite Cc: Jeff Davis , Peter Eisentraut , Robert Haas , pgsql-hackers@postgresql.org Subject: Re: Pre-proposal: unicode normalized text Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Tue, Oct 17, 2023 at 05:07:40PM +0200, Daniel Verite wrote: > > * Add a per-database option to enforce only storing assigned unicode > > code points. > > There's a problem in the fact that the set of assigned code points is > expanding with every Unicode release, which happens about every year. > > If we had this option in Postgres 11 released in 2018 it would use > Unicode 11, and in 2023 this feature would reject thousands of code > points that have been assigned since then. Yes, and that's desirable if PG were to normalize text as Jeff proposes, since then PG wouldn't know how to normalize text containing codepoints assigned after that. At that point to use those codepoints you'd have to upgrade PG -- not too unreasonable. Nico --