From: PFC <lists@boutiquenumerique.com>
To: Rajesh Mallah <mallah_rajesh@yahoo.com>
To: pgsql-sql@postgresql.org
Subject: Re: Significance of Database Encoding
Date: Sun, 15 May 2005 21:48:47 +0200
Message-ID: <op.sqt1blpqth1vuj@localhost> (raw)
In-Reply-To: <20050515183829.53021.qmail@web31011.mail.mud.yahoo.com>
References: <20050515183829.53021.qmail@web31011.mail.mud.yahoo.com>
> $ iconv -f US-ASCII -t UTF-8 < test.sql > out.sql
> iconv: illegal input sequence at position 114500
>
> Any ideas how the job can be accomplised reliably.
>
> Also my database may contain data in multiple encodings
> like WINDOWS-1251 and WINDOWS-1256 in various places
> as data has been inserted by different peoples using
> different sources and client software.
You could use a simple program like that (in Python):
output = open( "unidump", "w" )
for line in open( "your dump" ):
for encoding in "utf-8", "iso-8859-15", "whatever":
try:
output.write( unicode( line, encoding ).encode( "utf-8" ))
break
except UnicodeError:
pass
else:
print "No suitable encoding for line..."
I'd say this might work, if UTF-8 cannot absorb an apostrophe inside a
multibit character. Can it ?
Or you could do that to all your table using SELECTs but it's going to be
painful...
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-sql@postgresql.org
Cc: lists@boutiquenumerique.com, mallah_rajesh@yahoo.com
Subject: Re: Significance of Database Encoding
In-Reply-To: <op.sqt1blpqth1vuj@localhost>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox