From: Tom Lane <tgl@sss.pgh.pa.us>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: Joel Fradkin <jfradkin@wazagua.com>
Cc: 'Salem Berhanu' <salemb4@hotmail.com>
Cc: pgsql-admin@postgresql.org
Cc: pgsql-general@postgresql.org
Subject: Re: [GENERAL] postgres & server encodings
Date: Tue, 09 Aug 2005 13:31:03 -0400
Message-ID: <17412.1123608663@sss.pgh.pa.us> (raw)
In-Reply-To: <20050809171853.GB21039@alvh.no-ip.org>
References: <16705.1123603167@sss.pgh.pa.us>
<001601c59d03$4f107af0$797ba8c0@jfradkin>
<20050809171853.GB21039@alvh.no-ip.org>
Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
> The problem only shows up when you have mixed data -- say, you have two
> applications, one website in PHP which inserts data in Latin-1, and a
> Windows app which inserts in UTF-8. In this case your data will be a
> mess to fix, and there's no way a single conversion will get it right.
> You will have to manually separate the parts that are UTF8 from the
> Latin1, and import them separately. Not a position I'd like to be in.
The only helpful tip I can think of is that you can try to import data
into a UTF8 database and see if it gets rejected as badly encoded; this
will at least give you a weak tool to separate what's what.
I'm afraid the reverse direction won't help much --- in single-byte
encodings such as Latin1 there are no encoding errors, and so you can't
do any simple filtering to check in that direction. In the end you're
going to have to eyeball a lot of data for plausibility :-(
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-admin@postgresql.org
Cc: tgl@sss.pgh.pa.us, alvherre@alvh.no-ip.org, jfradkin@wazagua.com, salemb4@hotmail.com, pgsql-general@postgresql.org
Subject: Re: [GENERAL] postgres & server encodings
In-Reply-To: <17412.1123608663@sss.pgh.pa.us>
References: <16705.1123603167@sss.pgh.pa.us> <001601c59d03$4f107af0$797ba8c0@jfradkin> <20050809171853.GB21039@alvh.no-ip.org> <17412.1123608663@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox