Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com;
	b=TSFh7YFOO+N4+jF5B5f/n0FSLL2B8nw1QL/jcc2Nk15zH+XEgT4cfslSboIsKirzrDn6IJyF4izy6w3u+OwUDMlb3KHW31++hTQWS5VHNWJM+QfE+V6Tb1i73+d1Lmnn/hOi3dBvSxhoPSF+Un54STse7c5onbUsyLW9iRsyq/8=
	;
Message-ID: <20050516021650.78342.qmail@web31014.mail.mud.yahoo.com>
Date: Sun, 15 May 2005 19:16:50 -0700 (PDT)
From: Rajesh Mallah <mallah_rajesh@yahoo.com>
Subject: Re: Significance of Database Encoding
To: pgsql-sql@postgresql.org
In-Reply-To: 6667
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii


--- PFC <lists@boutiquenumerique.com> wrote:
> 
> > $ iconv -f US-ASCII -t UTF-8  < test.sql > out.sql
> > iconv: illegal input sequence at position 114500
> >
> > Any ideas how the job can be accomplised reliably.
> >
> > Also my database may contain data in multiple encodings
> > like WINDOWS-1251 and WINDOWS-1256 in various places
> > as data has been inserted by different peoples using
> > different sources and client software.
> 
> 	You could use a simple program like that (in Python):
> 
> output = open( "unidump", "w" )
> for line in open( "your dump" ):
> 	for encoding in "utf-8", "iso-8859-15", "whatever":
> 		try:
> 			output.write( unicode( line, encoding ).encode( "utf-8" ))
> 			break
> 		except UnicodeError:
> 			pass
> 	else:
> 		print "No suitable encoding for line..."


This may not work . Becuase ,conversion to utf-8 can be successfull (no runtime error)
even for an incorrect guess of the original encoding but the  result will be  an 
incorrect utf8. 

Regds
Rajesh Kumar Mallah


> 
> 	I'd say this might work, if UTF-8 cannot absorb an apostrophe inside a  
> multibit character. Can it ?
> 
> 	Or you could do that to all your table using SELECTs but it's going to be  
> painful...
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 7: don't forget to increase your free space map settings
> 


__________________________________ 
Do you Yahoo!? 
Read only the mail you want - Yahoo! Mail SpamGuard. 
http://promotions.yahoo.com/new_mail