Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qopWO-0034ra-BS for pgsql-hackers@arkaria.postgresql.org; Fri, 06 Oct 2023 18:25:56 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1qopWL-00AC0r-Vz for pgsql-hackers@arkaria.postgresql.org; Fri, 06 Oct 2023 18:25:54 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qopWL-00AC0j-Hm for pgsql-hackers@lists.postgresql.org; Fri, 06 Oct 2023 18:25:54 +0000 Received: from insect.birch.relay.mailchannels.net ([23.83.209.93]) by makus.postgresql.org with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qopWI-000Lu4-BT for pgsql-hackers@postgresql.org; Fri, 06 Oct 2023 18:25:52 +0000 X-Sender-Id: dreamhost|x-authsender|nico@cryptonector.com Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id CDAD141149; Fri, 6 Oct 2023 18:25:47 +0000 (UTC) Received: from pdx1-sub0-mail-a234.dreamhost.com (unknown [127.0.0.6]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 5932740E7B; Fri, 6 Oct 2023 18:25:47 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1696616747; a=rsa-sha256; cv=none; b=FLUhorjP0lRmGQAXGZNjvAsDg4XCZpQSb6R2lnRByqB6gXVZI2wI3U2CfBjw0Jng6zEqVR upNqU6DmdNkwBLw8d9ZF1wFhuvwNnerjJNkcnZN/0vgDUUVi2DcEIf2QKQwCqaHcPoHMgx L4i/QdjiT/SqEtvWmXVd9ILhjdb4hco91PpnXRoFc643cBGbGSPG0C8KaSONS5lsE6IgTj 7qYntVyKp/D2uwkTDQsUPtYzVAGJUzGcg1N6S2CMJSHwiOhPufOV1UV5T52l7EOBwl5VjE kd0R8oVt9op8lJUPpTCEPsZwKMbbr8CkUvz0qhlnoRSfjF3grRARMG2kMzXYzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1696616747; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vTaflUPzhpuch1bZLwlB39Kd4Nwj3DseqRbBF0Moe2M=; b=sgmqc+z04nukVrQYi9bGIvwJyJX8v0y5juihNLhq+28qbYklKI9FC2nWP9yqD4cZk1aOZl SH/TxRvEkZxaEUjcYu7SFFHV686Uu8AqoDKKB+0zEu7VUV/Dms6DBybPggIKND5/JAgJGk /kIWYV15c+PYr6nQ3lgFtun56+ucauZcUwuqIR9tqfC9ShdTYSDS9OPq62Nzw6zmbiomSH vzz6NvyjlZ87WPNs+VoqS4dRoftsCbIDcOf/eIOACrScdRwY5ktPF5mo08M7e8cMM0yKE/ MuUXTWKwtgSq9ymoDctVQ2MQmaGPPLt1Cw1jSB8xpHB3B2h/ORaaZpWHe3eEJQ== ARC-Authentication-Results: i=1; rspamd-7d5dc8fd68-zpvhq; auth=pass smtp.auth=dreamhost smtp.mailfrom=nico@cryptonector.com X-Sender-Id: dreamhost|x-authsender|nico@cryptonector.com X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|nico@cryptonector.com X-MailChannels-Auth-Id: dreamhost X-Print-Imminent: 02e5d9334d9bef93_1696616747628_1285714473 X-MC-Loop-Signature: 1696616747628:3385606481 X-MC-Ingress-Time: 1696616747628 Received: from pdx1-sub0-mail-a234.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.101.166.218 (trex/6.9.1); Fri, 06 Oct 2023 18:25:47 +0000 Received: from ubby21 (075-081-095-064.res.spectrum.com [75.81.95.64]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: nico@cryptonector.com) by pdx1-sub0-mail-a234.dreamhost.com (Postfix) with ESMTPSA id 4S2H164NG7z38; Fri, 6 Oct 2023 11:25:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cryptonector.com; s=dreamhost; t=1696616747; bh=vTaflUPzhpuch1bZLwlB39Kd4Nwj3DseqRbBF0Moe2M=; h=Date:From:To:Cc:Subject:Content-Type:Content-Transfer-Encoding; b=ZT3DD5KjWvNzm9XLfiNtYSZqRm/F0GMwezIygaG8b46MVtCsgeZmgs8NWkyKkMzrM ofsflFqj3rlov/kN3HKr7KSylqUl5iUv8FYR9seGPegW6rcm9Z60zrePJ1gdcfj26F T/D8isunHfAueDXqtPmg80ixN+4hrLwIAanWzIiY4acFOCBTcUDcpQ8cd3nugWmIFI QRpv5yTuw4YzKd+5uomZO/RjTrUO6gZNd9ct51QHn/xPbq64l+4BIDpBGzdNhJ0q/e wH2Flv7g2rUgulB8zeu8Av1mPVxlRYAUE3tbWXxSr4Ctg2l+dQ9+0IPG/cKGrhdDT4 3Q8Ki6pyaGs6A== Date: Fri, 6 Oct 2023 13:25:44 -0500 From: Nico Williams To: Robert Haas Cc: Isaac Morland , Jeff Davis , Chapman Flack , pgsql-hackers@postgresql.org Subject: Re: Pre-proposal: unicode normalized text Message-ID: References: <43864caecd65d02081d623d05ccc1683@anastigmatix.net> <94a0e631b787d56fec1b17e03fe045b5e14907c1.camel@j-davis.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Fri, Oct 06, 2023 at 02:17:32PM -0400, Robert Haas wrote: > On Fri, Oct 6, 2023 at 1:38 PM Nico Williams wrote: > > On Fri, Oct 06, 2023 at 01:33:06PM -0400, Robert Haas wrote: > > > On Thu, Oct 5, 2023 at 3:15 PM Nico Williams wrote: > > > > Text+encoding can be just like bytea with a one- or two-byte prefix > > > > indicating what codeset+encoding it's in. That'd be how to encode > > > > such text values on the wire, though on disk the column's type should > > > > indicate the codeset+encoding, so no need to add a prefix to the value. > > > > > > Well, that would be making the encoding a per-value property, rather > > > than a per-column property like collation as I proposed. I can't see > > > > On-disk it would be just a property of the type, not part of the value. > > I mean, that's not how it works. Sure, because TEXT in PG doesn't have codeset+encoding as part of it -- it's whatever the database's encoding is. Collation can and should be a porperty of a column, since for Unicode it wouldn't be reasonable to make that part of the type. But codeset+encoding should really be a property of the type if PG were to support more than one. IMO.