From: Peter J. Holzer <hjp-pgsql@hjp.at>
To: pgsql-general@lists.postgresql.org
Subject: Re: Facing issue in using special characters
Date: Mon, 18 Mar 2019 22:19:23 +0100
Message-ID: <20190318211923.eaobabpawdcjm5fh@hjp.at> (raw)
In-Reply-To: <B446C5BC-7195-4BA0-80E6-A15D5CBDF365@uab.edu>
References: <VI1PR0701MB271817E2E03434C39C4EDFFAAE440@VI1PR0701MB2718.eurprd07.prod.outlook.com>
<c69d480d-89c4-f2c6-83bb-d0202785211b@gusw.net>
<B446C5BC-7195-4BA0-80E6-A15D5CBDF365@uab.edu>
On 2019-03-17 15:01:40 +0000, Warner, Gary, Jr wrote:
> Many of us have faced character encoding issues because we are not in control
> of our input sources and made the common assumption that UTF-8 covers
> everything.
UTF-8 covers "everything" in the sense that there is a round-trip from
each character in every commonly-used charset/encoding to Unicode and
back.
The actual code may of course be different. For example, the € sign is
0xA4 in iso-8859-15, but U+20AC in Unicode. So you need an
encoding/decoding step.
And "commonly-used" means just that. Unicode covers a lot of character
sets, but it can't cover every character set ever invented (I invented
my own character sets when I was sixteen. Nobody except me ever used
them and they have long succumbed to bit rot).
> In my lab, as an example, some of our social media posts have included ZawGyi
> Burmese character sets rather than Unicode Burmese. (Because Myanmar developed
> technology In a closed to the world environment, they made up their own
> non-standard character set which is very common still in Mobile phones.).
I'd be surprised if there was a character set which is "very common in
Mobile phones", even in a relatively poor country like Myanmar. Does
ZawGyi actually include characters which aren't in Unicode are are they
just encoded differently?
hp
--
_ | Peter J. Holzer | we build much bigger, better disasters now
|_|_) | | because we have much more sophisticated
| | | hjp@hjp.at | management tools.
__/ | http://www.hjp.at/ | -- Ross Anderson <https://www.edge.org/;
Attachments:
[application/pgp-signature] signature.asc (832B, ../20190318211923.eaobabpawdcjm5fh@hjp.at/2-signature.asc)
download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: hjp-pgsql@hjp.at, pgsql-general@lists.postgresql.org
Subject: Re: Facing issue in using special characters
In-Reply-To: <20190318211923.eaobabpawdcjm5fh@hjp.at>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox