X-Original-To: pgsql-sql-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 73322541CE for ; Sun, 15 May 2005 15:38:35 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 58684-05 for ; Sun, 15 May 2005 18:38:30 +0000 (GMT) Received: from web31011.mail.mud.yahoo.com (web31011.mail.mud.yahoo.com [68.142.201.69]) by svr1.postgresql.org (Postfix) with SMTP id 06DB3541C8 for ; Sun, 15 May 2005 15:38:28 -0300 (ADT) Received: (qmail 53023 invoked by uid 60001); 15 May 2005 18:38:29 -0000 Comment: DomainKeys? See http://antispam.yahoo.com/domainkeys DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; b=eOhCDnriZ+osM6bmw74UDK3DuLrCid6U6PqepScbmbtlb5qa0iIJh1zNZ+DYOQGgAr9J16cMdnagebcJgqWjpXXo6JqzadDwoKYyrWR2l1VPJYyfydFyaxfWDMXhpsPwvMdkzmlSR7M11j9QTpMlqEAUzFv4Vo12XoLZ6VQfkQs= ; Message-ID: <20050515183829.53021.qmail@web31011.mail.mud.yahoo.com> Received: from [210.211.162.193] by web31011.mail.mud.yahoo.com via HTTP; Sun, 15 May 2005 11:38:29 PDT Date: Sun, 15 May 2005 11:38:29 -0700 (PDT) From: Rajesh Mallah Subject: Re: Significance of Database Encoding To: PFC , pgsql-sql@postgresql.org In-Reply-To: 6667 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=4.512 tagged_above=0 required=5 tests=AWL, DNS_FROM_RFC_ABUSE, DNS_FROM_RFC_POST, FORGED_YAHOO_RCVD, RCVD_IN_NJABL_PROXY X-Spam-Level: **** X-Archive-Number: 200505/106 X-Sequence-Number: 21419 --- PFC wrote: > > > +--------------------------------------------------------------------------------------+ > > | 私はガラス > > +--------------------------------------------------------------------------------------+ > > You say it displays correctly in xterm (ie. you didn't see these in your > xterm). > There are HTML/XML unicode character entities, probably generated by your > mailer from your Unicode cut'n'paste. That is correct. Now the question is how to convert from SQL_ASCII to UNICODE. Mailing lists suggests to run recode or iconv on the dump file and restore. The problem is on running iconv with -f US-ASCII the program aborted: $ iconv -f US-ASCII -t UTF-8 < test.sql > out.sql iconv: illegal input sequence at position 114500 Any ideas how the job can be accomplised reliably. Also my database may contain data in multiple encodings like WINDOWS-1251 and WINDOWS-1256 in various places as data has been inserted by different peoples using different sources and client software. Regds Rajesh Kumar Mallah. > Using SQL ASCII to store UTF8 encoded data will work, but postgres won't > know that it's manipulating multibyte characters, so for instance the > length of a string will be its Byte length instead of correctly counting > the characters, collation rules will be funky, etc. And substring() may > well cut in the middle of an UTF8 multibyte char which will then screw > your application side processing... > Apart from that, it'll work ;) > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com