Message-ID: <3F2FE9E0.2060407@hotdogrecords.com>
Date: Tue, 05 Aug 2003 13:31:12 -0400
From: Kurt Overberg <kurt@hotdogrecords.com>
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
	rv:1.3) Gecko/20030312
MIME-Version: 1.0
To: pgsql-sql@postgresql.org, pgsql-jdbc@postgresql.org
Subject: JDBC encoding problem
References: <20030726193947.B1636@natasha.ward.six>
In-Reply-To: <20030726193947.B1636@natasha.ward.six>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit


I'm having a rather strange problem that I'm hoping someone can help me
with.  I'm using Struts 1.0/jsp on Debian linux under Tomcat 4.1.x and
the blackdown JVM  .  I'm attempting to convert my current SQL_ASCII
database to UNICODE.  I'm new to this, so am most likely making a few
mistakes.  Here's what I've done so far:

o  Converted database encoding to be UNICODE.  I'm pretty sure this part
worked okay.  (did a pg_dump, then iconv -f 8859_1 -t UTF-8, then
created new db with encoding UNICODE and reloaded- no errors upon reload)

sparky:~$ psql -l
         List of databases
    Name    |  Owner   | Encoding
-----------+----------+-----------
  unitest   | kurt     | UNICODE
  template1 | postgres | SQL_ASCII
(2 rows)


o  set client_encoding to 'UTF8';

o  In my JSP files, I set the following at the top of each:

<%@ page lanuage="java" pageEncoding="UTF-8" %>


Now, to test this, I go to a japanese page, copy some text, then paste
it into a form, that gets submitted to the server and saved into the DB.
Then I try to display what I got back from the database.  It comes out
garbled.  HOWEVER- if I leave the 'pageEncoding' out of my display .jsp
file it still comes out garbled, UNTIL I set UTF-8 manually in my
browsers Character Encoding settings (both mozilla and IE).  Then the
japanese characters render fine (just like I entered them).

Very strange.  What's confusing is that when I set the pageEncoding to
'UTF-8', the characters don't render properly, and as far as I can tell,
thats the same as manually setting the browser manually.  I must be
doing something wrong because I get the same results in IE and mozilla
(recent build).

What may be the problem- I don't do anything differently when getting
the data out of the database, just standard
resultset.getString("column");  Do I need to change that call, to handle
the potentially UTF-8 encoded strings?  I can't find anything on that at
all with google/usenet.

Any and all help, suggestions or pointers would be greatly appreciated.

Thanks!

/kurt