Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qoBWz-000QME-EW for pgsql-hackers@arkaria.postgresql.org; Wed, 04 Oct 2023 23:43:53 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.94.2) (envelope-from ) id 1qoBWv-006dY8-DF for pgsql-hackers@arkaria.postgresql.org; Wed, 04 Oct 2023 23:43:50 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qoBWu-006dY0-LT for pgsql-hackers@lists.postgresql.org; Wed, 04 Oct 2023 23:43:49 +0000 Received: from quail.birch.relay.mailchannels.net ([23.83.209.151]) by makus.postgresql.org with esmtps (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qoBWo-0003d8-Lc for pgsql-hackers@postgresql.org; Wed, 04 Oct 2023 23:43:48 +0000 X-Sender-Id: dreamhost|x-authsender|nico@cryptonector.com Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id AF7596C1A80; Wed, 4 Oct 2023 23:43:40 +0000 (UTC) Received: from pdx1-sub0-mail-a294.dreamhost.com (unknown [127.0.0.6]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 48F016C1A6A; Wed, 4 Oct 2023 23:43:40 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1696463020; a=rsa-sha256; cv=none; b=M0qWbrHDI4Yj/pSQBw/Czzq/F1WE5jyP9eE0nThHkvUVvcn/j71xEhjx80WarqQRcsH2nE Rvfhbw6INrqKbV85oKor6hxLr/swf0xpzF4G29MWONGxMPOwOiPfOUGdVJjuOEJQG8ivfS IyUxFkIJ1GFglxIJ5FbJISnsal3C+uGXmqjFpbVTAMLtiHjHaO1gQiVxsu2IypagnOGoCG nSSBZC28iLFFvl/Y52hyh1rQVv2IdzyEyNO1V/5DqwgHWg9pbbcmTOBUDg9ij+fDbFfs6S waOHJTLQy04SJ9N5pxuBe3kwXY0fZzdkg63kqSVJ8n/i1tkZNCTPRrjrORORvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1696463020; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mz7+pHugFRoxE31JahJm1529lOqQdXe22jEVBZ4shWc=; b=A4327gxAQDxDTX2qoyDFsG2at2KBbyDmjlt5ZZs3PPv6AlEEAlhfKYa1qcAndJljB3xte4 lRzlddS96Wv5InKw/4cHD+SGveg5MoI3XWVJUzLOEwcDij0yK+HqQnmhJqbrZ/5fkkvf4i N0YlzQxBBQEfD7BouX06VfF+QNX6DuNxIWBtL5QGPE3AR1BG6Damf7DPnH6iLh+lmhC0fp 1JC6y055vqEkcXVd52EUOtfs6q1XAoMHr6XDd+GUcbaQNTbhwa2XtV8W64F/icqxm7utve vOFHLh8C8VdRjmzJK/cJhEbckagLG40dhlepPPwerNAHXDPG2H7EBvlwgXuJqA== ARC-Authentication-Results: i=1; rspamd-7d5dc8fd68-67dmm; auth=pass smtp.auth=dreamhost smtp.mailfrom=nico@cryptonector.com X-Sender-Id: dreamhost|x-authsender|nico@cryptonector.com X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|nico@cryptonector.com X-MailChannels-Auth-Id: dreamhost X-Callous-Arch: 3762be2d1dd5bd96_1696463020536_1994318131 X-MC-Loop-Signature: 1696463020536:3103331477 X-MC-Ingress-Time: 1696463020536 Received: from pdx1-sub0-mail-a294.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.106.112.200 (trex/6.9.1); Wed, 04 Oct 2023 23:43:40 +0000 Received: from ubby21 (075-081-095-064.res.spectrum.com [75.81.95.64]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: nico@cryptonector.com) by pdx1-sub0-mail-a294.dreamhost.com (Postfix) with ESMTPSA id 4S1B8q4hz6z60; Wed, 4 Oct 2023 16:43:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cryptonector.com; s=dreamhost; t=1696463020; bh=mz7+pHugFRoxE31JahJm1529lOqQdXe22jEVBZ4shWc=; h=Date:From:To:Cc:Subject:Content-Type; b=NW6Ee+t6Y+3PE3deN8M8ucEUnWVMnQHd5q2fOIG5hhjfT9GQbcU2PUp6r7wMmn8MT OGN9bYibOxiKW0mhQlLBQ7C1q1tkpUlKVqp13weW3kpfWhLNPJdI6C9KdzCW02Q3z5 JTF9FsBfJuDFwZwIi/wAUoLHfaJwWpVFH2lyhVR/5Tkl9Jo/jrPxzdJSEDv0GBLqRu xW3KhFTxoe1KsOTgp4/4dK6Raef/N3uJ4n1TwQCMwlx9xl5bcjxcOtQco/zRzUiQ41 k/eRYYcYnTT4WPgd39tQH9ve3TV51s5kEF7bXaZ7GEEiDVEWy09HwNvLPY8/4sG3ql /EKGBzLtpcE9w== Date: Wed, 4 Oct 2023 18:43:37 -0500 From: Nico Williams To: Jeff Davis Cc: Chapman Flack , Robert Haas , pgsql-hackers@postgresql.org Subject: Re: Pre-proposal: unicode normalized text Message-ID: References: <43864caecd65d02081d623d05ccc1683@anastigmatix.net> <573029acb7dd0956fb5334c3a3b71ffee16b4c0d.camel@j-davis.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <573029acb7dd0956fb5334c3a3b71ffee16b4c0d.camel@j-davis.com> List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Archived-At: Precedence: bulk On Wed, Oct 04, 2023 at 04:01:26PM -0700, Jeff Davis wrote: > On Wed, 2023-10-04 at 16:15 -0500, Nico Williams wrote: > > Better that than TEXT blobs w/ the encoding given by the `CREATE > > DATABASE` or `initdb` default! > > From an engineering perspective, yes, per-column encodings would be > more flexible. But I still don't understand who exactly would use that, > and why. Say you have a bunch of text files in different encodings for reasons (historical). And now say you want to store them in a database so you can index them and search them. Sure, you could use a filesystem, but you want an RDBMS. Well, the answer to this is "convert all those files to UTF-8". > It would take an awful lot of effort to implement and make the code > more complex, so we'd really need to see some serious demand for that. Yes, it's better to just use UTF-8. The DB could implement conversions to/from other codesets and encodings for clients that insist on it. Why would clients insist anyways? Better to do the conversions at the clients. In the middle its best to just have Unicode, and specifically UTF-8, then push all conversions to the edges of the system. Nico --