From: Laurenz Albe <laurenz.albe@cybertec.at>
To: James Pang (chaolpan) <chaolpan@cisco.com>
To: pgsql-admin@lists.postgresql.org <pgsql-admin@lists.postgresql.org>
Subject: Re: database encoding and collation.
Date: Fri, 14 Jan 2022 08:17:14 +0100
Message-ID: <b12ef47c4298ca05a9e4b159ce7c71910e0206ea.camel@cybertec.at> (raw)
In-Reply-To: <PH0PR11MB5191393FFF0FDF0E510CF18CD6549@PH0PR11MB5191.namprd11.prod.outlook.com>
References: <PH0PR11MB5191DBEC457323D03B5C4592D6529@PH0PR11MB5191.namprd11.prod.outlook.com>
<68331b01ee5ed0d02158747bf0e2d2e4476cbbb8.camel@cybertec.at>
<PH0PR11MB5191393FFF0FDF0E510CF18CD6549@PH0PR11MB5191.namprd11.prod.outlook.com>
On Fri, 2022-01-14 at 02:11 +0000, James Pang (chaolpan) wrote:
> This question is about how to handle multiple languages in database , English, Japanese ,Chinese...
> The Oracle database used WE8ISO8859P1 from long time ago , when put "Chinese" letters in database,
> client application just use WE8ISO8859P1 (client encoding) to insert data into database ,
> so no conversion there. When client app query the data , it still get the data with WE8ISO8859P1 ,
> and then convert that data to UTF8 from client app side. Now, we plan migrate to Postgresql 13,
> want Postgresql database use WE8ISO8859P1 same as Oracle too, and same handling as Oracle.
> Any potential risk ?
I see the problem: the sloppy checking of Oracle allowed you to insert corrupted strings into
the Oracle database by setting client encoding equal to server encoding, and you might not
detect the problem until you try to read the data with a different client encoding.
You may be able to carry on like this in PostgreSQL, since all bytes are valid LATIN1
code points, but (like in Oracle) you will run into problems as soon as you access the
database with a different client encoding, for example with a JDBC driver.
It would be better to fix the problem. If all the data in the database are actually
encoded in a certain encoding (different from ISO-8859-1), you could simply import them
into a database with that encoding, and you would have the benefit of having correct data.
If different strings are encoded in different encodings, then you would have to repair the
data before doing that.
Yours,
Laurenz Albe
--
Cybertec | https://www.cybertec-postgresql.com
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: laurenz.albe@cybertec.at, chaolpan@cisco.com, pgsql-admin@lists.postgresql.org
Subject: Re: database encoding and collation.
In-Reply-To: <b12ef47c4298ca05a9e4b159ce7c71910e0206ea.camel@cybertec.at>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox