Received: from malur.postgresql.org ([217.196.149.56])
 by arkaria.postgresql.org with esmtp (Exim 4.80) (envelope-from
 <pgsql-hackers-owner+M237040=pgsql-hackers-arkari@postgresql.org>)
 id 1VggvJ-0007Uf-Ka
 for pgsql-hackers@arkaria.postgresql.org; Wed, 13 Nov 2013 20:20:25 +0000
Received: from localhost ([127.0.0.1] helo=postgresql.org)
 by malur.postgresql.org with smtp (Exim 4.80) (envelope-from
 <pgsql-hackers-owner+M237040=pgsql-hackers-arkari@postgresql.org>)
 id 1VggvJ-0003Za-11
 for pgsql-hackers@arkaria.postgresql.org; Wed, 13 Nov 2013 20:20:25 +0000
Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29])
 by malur.postgresql.org with esmtp (Exim 4.80)
 (envelope-from <kleptog@svana.org>) id 1VggvH-0003ZS-Rk
 for pgsql-hackers@postgresql.org; Wed, 13 Nov 2013 20:20:23 +0000
Received: from [2400:c400::113:212:99:188] (helo=svana.org)
 by magus.postgresql.org with esmtp (Exim 4.80)
 (envelope-from <kleptog@svana.org>)
 id 1VggvD-0002nT-W9; Wed, 13 Nov 2013 20:20:23 +0000
Received: from kleptog by svana.org with local (Exim 4.72)
 (envelope-from <kleptog@svana.org>)
 id 1Vgguk-0000Ji-SB; Thu, 14 Nov 2013 07:19:52 +1100
Date: Wed, 13 Nov 2013 21:19:50 +0100
From: Martijn van Oosterhout <kleptog@svana.org>
To: Tatsuo Ishii <ishii@postgresql.org>
Cc: tgl@sss.pgh.pa.us, maumau307@gmail.com, laurenz.albe@wien.gv.at,
 robertmhaas@gmail.com, peter_e@gmx.net, arul@fast.au.fujitsu.com,
 stark@mit.edu, Maksym.Boguk@au.fujitsu.com, hlinnakangas@vmware.com,
 pgsql-hackers@postgresql.org
Message-ID: <20131113201950.GA800@svana.org>
Reply-To: Martijn van Oosterhout <kleptog@svana.org>
References: 
 <A737B7A37273E048B164557ADEF4A58B17C57F0B@ntex2010i.host.magwien.gv.at>
 <7A2BD06B10A24EA4AA5FC5D93A70A754@maumau>
 <30151.1384101976@sss.pgh.pa.us>
 <20131112.155752.666523035722474275.t-ishii@sraoss.co.jp>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
 protocol="application/pgp-signature"; boundary="ZPt4rx8FFjLCG7dd"
Content-Disposition: inline
In-Reply-To: <20131112.155752.666523035722474275.t-ishii@sraoss.co.jp>
X-PGP-Key-ID: Length=4096; ID=0x4BE62D44
X-PGP-Key-Fingerprint: AF6F C0D6 3B8A AF7D 942B  D2BC 4BED FBE7 4BE6 2D44
User-Agent: Mutt/1.5.20 (2009-06-14)
X-SA-Exim-Connect-IP: <locally generated>
X-SA-Exim-Mail-From: kleptog@svana.org
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on calyx
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,NO_RELAYS
 autolearn=ham version=3.3.1
Subject: Re: UTF8 national character data type support WIP patch
 and list of open issues.
X-SA-Exim-Version: 4.2.1 (built Mon, 22 Mar 2010 06:26:47 +0000)
X-SA-Exim-Scanned: Yes (on svana.org)
X-Host-Lookup-Failed: Reverse DNS lookup failed for 2400:c400::113:212:99:188
 (failed)
X-Pg-Spam-Score: -1.1 (-)
List-Archive: <http://archives.postgresql.org/pgsql-hackers>
List-Help: <mailto:majordomo@postgresql.org?body=help>
List-ID: <pgsql-hackers.postgresql.org>
List-Owner: <mailto:pgsql-hackers-owner@postgresql.org>
List-Post: <mailto:pgsql-hackers@postgresql.org>
List-Subscribe: <mailto:majordomo@postgresql.org?body=sub%20pgsql-hackers>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-hackers>
X-Mailing-List: pgsql-hackers
Precedence: bulk
Sender: pgsql-hackers-owner@postgresql.org


--ZPt4rx8FFjLCG7dd
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Tue, Nov 12, 2013 at 03:57:52PM +0900, Tatsuo Ishii wrote:
> I have been thinking about this for years and I think the key idea for
> this is, implementing "universal encoding". The universal encoding
> should have following characteristics to implement N>2 encoding in a
> database.
>=20
> 1) no loss of round trip encoding conversion
>=20
> 2) no mapping table is necessary to convert from/to existing encodings
>=20
> Once we implement the universal encoding, other problem such as
> "pg_database with multiple encoding problem" can be solved easily.

Isn't this essentially what the MULE internal encoding is?

> Currently there's no such an universal encoding in the universe, I
> think the only way is, inventing it by ourselves.

This sounds like a terrible idea. In the future people are only going
to want more advanced text functions, regular expressions, indexing and
making encodings that don't exist anywhere else seems like a way to
make a lot of work for little benefit.

A better idea seems to me is to (if postgres is configured properly)
embed the non-round-trippable characters in the custom character part
of the unicode character set. In other words, adjust the mappings
tables on demand and voila.

Have a nice day,
--=20
Martijn van Oosterhout   <kleptog@svana.org>   http://svana.org/kleptog/
> He who writes carelessly confesses thereby at the very outset that he does
> not attach much importance to his own thoughts.
   -- Arthur Schopenhauer

--ZPt4rx8FFjLCG7dd
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iQIVAwUBUoPe5Uvt++dL5i1EAQgP5BAAqET6DEi7ZtKWcIJ2IpMZ/dzJh+zB1tIu
/L1DAAbLWLbZEJHPjOIoGJEfg/q+tHm3CY7SSKpvPVrobPQn4Dsn6oq0bUNt9TZA
J8zrds+FF4Zr7IjEjgYneBIzmAWsLx6WY5s7+9BQ4fuF+mDutMIEFqWGASXGs/ga
sYal6H6xAPrLPdffqF0TiRspySL71vA81mhvvdKsxk6P5f/DeCQMOxD/+GR72O8K
OX6AQ7yi+aQGa/qkshtndjboO/kV3ANcSloN+zyDMGZeD0FE1wYflipHijD3wE6Z
D5dm7N2t5lF5uWjhZqpTa9CtVwziN/I079WTLb8nsF4ya372MwsWyvywjAlFjH+a
ds771gCbOYXIxRyWrZrbjLRBB/dw0TLeEKvl1uU4PGrzY+16YUSFJc3SvqkT5hZn
qSO8SyHerOau8y/5Cls/ERx+vVXnF+bJKDVsb55avkHIS2MB97UPE/teeRTc5xtM
4KXzMfY/IUB42yPQ9BSb0Ar+bSLHoXlWrVQQGoVr5K97+7VqwkcB5lITLFLs0wew
Lku6pxUy+Vsbz7k6T4MDnwc6eH7YSokHWgyn0gW3c9M/VA4x9iv3lhbcXC3pp+K+
TKh5l8zVguhEzP73x/V+XA2yn3mZVWeMj1RPdJZdXddsdLkbTjd+vlvWBShWtAda
hrPZbjyonHU=
=C/Zv
-----END PGP SIGNATURE-----

--ZPt4rx8FFjLCG7dd--