From software@sistemat.com.ar Sat Jul  17 19:54:22 2004
X-Original-To: pgsql-admin-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id CCB9AD1B23C;
	Sat, 17 Jul 2004 16:54:22 -0300 (ADT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 49341-06; Sat, 17 Jul 2004 19:54:21 +0000 (GMT)
Received: from mail.svhost.com.ar (mail.svhost.com.ar [200.80.239.22])
	by svr1.postgresql.org (Postfix) with ESMTP id ECA10D1B248;
	Sat, 17 Jul 2004 16:54:19 -0300 (ADT)
Received: from sistemat.com.ar (devel.sistemat.com.ar [192.168.1.40])
	by mail.svhost.com.ar (Postfix) with ESMTP
	id ABA082443B; Sat, 17 Jul 2004 16:52:38 -0300 (ART)
Message-ID: <40F98613.1060405@sistemat.com.ar>
Date: Sat, 17 Jul 2004 17:03:31 -0300
From: "Dario V. Fassi" <software@sistemat.com.ar>
Organization: SISTEMATICA ingenieria de software srl
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US;
	rv:1.4) Gecko/20030624 Netscape/7.1 (ax)
X-Accept-Language: es, es-ar, en-us, en
MIME-Version: 1.0
To: pgsql-jdbc@postgresql.org, pgsql-admin@postgresql.org
Subject: Migrate postgres databases from SQL_ASCII  to UNICODE encoding
Content-Type: multipart/alternative;
	boundary="------------030600090905070909080309"
X-Virus-Scanned: by amavisd-new at hub.org
X-Spam-Status: No, hits=1.0 tagged_above=0.0 required=5.0 tests=HTML_20_30,
	HTML_MESSAGE, HTML_TITLE_EMPTY
X-Spam-Level: *
X-Archive-Number: 200407/200
X-Sequence-Number: 14268

This is a multi-part message in MIME format.
--------------030600090905070909080309
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit


A simple question, we need to migrate many (>20) postgres databases from 
SQL_ASCII encoding to UNICODE encoding, over a 7.3.6 server.

With  Dump/Restore , we get an error (Invalid Unicode) in any field that 
has a 8 bits character coming from the SQL_ASCII , even setting the 
client_encoding to WIN, ISO-8859-1,  and others encodings.

I'm really worried about this problem since , we have many DBs to 
migrate and even with a ODBC program we can't port tables from 
Postgresql/SQL_ASCII to Postgresql/UNICODE.

Any suggestion will be *very* appreciated.

Regards,
Dario.



--------------030600090905070909080309
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1">
  <title></title>
</head>
<body text="#000000" bgcolor="#ffffff">
<br>
A simple question, we need to migrate many (&gt;20) postgres databases
from SQL_ASCII encoding to UNICODE encoding, over a 7.3.6 server.
<br>
<br>
With&nbsp; Dump/Restore , we get an error (Invalid Unicode) in
any field that has a 8 bits character coming from the SQL_ASCII , even
setting the client_encoding to WIN, ISO-8859-1,&nbsp; and others encodings.<br>
<br>
I'm really worried about this problem since , we have many DBs to
migrate and even with a ODBC program we can't port tables from
Postgresql/SQL_ASCII to Postgresql/UNICODE.<br>
<br>
Any suggestion will be *very* appreciated.<br>
<br>
Regards,<br>
Dario.<br>
<div class="moz-signature"><small><br>
</small><br>
</div>
</body>
</html>

--------------030600090905070909080309--




From tgl@sss.pgh.pa.us Sat Jul  17 21:18:17 2004
X-Original-To: pgsql-admin-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id DE2ADD1B17B;
	Sat, 17 Jul 2004 18:18:17 -0300 (ADT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 77751-05; Sat, 17 Jul 2004 21:18:17 +0000 (GMT)
Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130])
	by svr1.postgresql.org (Postfix) with ESMTP id E644CD1B1CD;
	Sat, 17 Jul 2004 18:18:14 -0300 (ADT)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
	by sss.pgh.pa.us (8.12.11/8.12.11) with ESMTP id i6HLIGSN016995;
	Sat, 17 Jul 2004 17:18:16 -0400 (EDT)
To: "Dario V. Fassi" <software@sistemat.com.ar>
Cc: pgsql-jdbc@postgresql.org, pgsql-admin@postgresql.org
Subject: Re: Migrate postgres databases from SQL_ASCII to UNICODE encoding 
In-reply-to: <40F98613.1060405@sistemat.com.ar> 
References: <40F98613.1060405@sistemat.com.ar>
Comments: In-reply-to "Dario V. Fassi" <software@sistemat.com.ar>
	message dated "Sat, 17 Jul 2004 17:03:31 -0300"
Date: Sat, 17 Jul 2004 17:18:16 -0400
Message-ID: <16994.1090099096@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
X-Virus-Scanned: by amavisd-new at hub.org
X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests=
X-Spam-Level: 
X-Archive-Number: 200407/201
X-Sequence-Number: 14269

"Dario V. Fassi" <software@sistemat.com.ar> writes:
> A simple question, we need to migrate many (>20) postgres databases from 
> SQL_ASCII encoding to UNICODE encoding, over a 7.3.6 server.

SQL_ASCII is not an encoding (it's more like the absence of knowledge
about an encoding).  What is the data actually stored as?

> With  Dump/Restore , we get an error (Invalid Unicode) in any field that 
> has a 8 bits character coming from the SQL_ASCII , even setting the 
> client_encoding to WIN, ISO-8859-1,  and others encodings.

It might work to just UPDATE pg_database to set datencoding to the
correct value reflecting what you have actually stored.  You might then
need to REINDEX any indexes on textual columns, but I don't think
anything else would go wrong.

If you have a mishmash of different encodings in a single database, then
of course there is no simple solution; you are in for some pain while
you try to fix the data.

			regards, tom lane



From software@sistemat.com.ar Sat Jul  17 23:17:09 2004
X-Original-To: pgsql-jdbc-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id BD62CD1B17B;
	Sat, 17 Jul 2004 20:17:09 -0300 (ADT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 10165-07; Sat, 17 Jul 2004 23:17:09 +0000 (GMT)
Received: from mail.svhost.com.ar (mail.svhost.com.ar [200.80.239.22])
	by svr1.postgresql.org (Postfix) with ESMTP id 48D3BD1B178;
	Sat, 17 Jul 2004 20:17:03 -0300 (ADT)
Received: from sistemat.com.ar (devel.sistemat.com.ar [192.168.1.40])
	by mail.svhost.com.ar (Postfix) with ESMTP
	id 9C5101DE13; Sat, 17 Jul 2004 20:15:24 -0300 (ART)
Message-ID: <40F9B599.6080205@sistemat.com.ar>
Date: Sat, 17 Jul 2004 20:26:17 -0300
From: "Dario V. Fassi" <software@sistemat.com.ar>
Organization: SISTEMATICA ingenieria de software srl
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US;
	rv:1.4) Gecko/20030624 Netscape/7.1 (ax)
X-Accept-Language: es, es-ar, en-us, en
MIME-Version: 1.0
To: Tom Lane <tgl@sss.pgh.pa.us>,
	Oliver Jowett <oliver@opencloud.com>, pgsql-sql@postgresql.org,
	"pgsql-jdbc@postgresql.org" <pgsql-jdbc@postgresql.org>
Subject: Re: [ADMIN] Migrate postgres databases from SQL_ASCII to UNICODE
References: <40F98613.1060405@sistemat.com.ar>
 <16994.1090099096@sss.pgh.pa.us>
In-Reply-To: <16994.1090099096@sss.pgh.pa.us>
Content-Type: multipart/alternative;
	boundary="------------020204070704000302080203"
X-Virus-Scanned: by amavisd-new at hub.org
X-Spam-Status: No, hits=0.5 tagged_above=0.0 required=5.0 tests=HTML_MESSAGE,
	HTML_TITLE_EMPTY
X-Spam-Level: 
X-Archive-Number: 200407/221
X-Sequence-Number: 10605

This is a multi-part message in MIME format.
--------------020204070704000302080203
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

Tom Lane wrote:

>"Dario V. Fassi" <software@sistemat.com.ar> writes:
>  
>
>>A simple question, we need to migrate many (>20) postgres databases from 
>>SQL_ASCII encoding to UNICODE encoding, over a 7.3.6 server.
>>    
>>
>
>SQL_ASCII is not an encoding (it's more like the absence of knowledge
>about an encoding).  What is the data actually stored as?
>  
>
>>With  Dump/Restore , we get an error (Invalid Unicode) in any field that 
>>has a 8 bits character coming from the SQL_ASCII , even setting the 
>>client_encoding to WIN, ISO-8859-1,  and others encodings.
>>    
>>
>
>It might work to just UPDATE pg_database to set datencoding to the
>correct value reflecting what you have actually stored.  You might then
>need to REINDEX any indexes on textual columns, but I don't think
>anything else would go wrong.
>
>If you have a mishmash of different encodings in a single database, then
>of course there is no simple solution; you are in for some pain while
>you try to fix the data.
>  
>

Yes you are right , the original data come from a DB2 with CodePage 
IBM-850  and was inserted  without complains in a Postgres 7.3.6 with 
SQL_ASCII.

Now we are in a Jail , because IBM-850 , isn't WIN, isn't  ISO-xx , 
isn't no one postgresql's encoding.
So when in change via pg_databases the encoding , 8 bits characters 
become garbage.
More even if we accept this garbage chars and we set encoding to e.g. 
ISO-8859-1   it's impossible go to a UNICODE because this garbage chars 
are invalid in client's encoding , so they are reject (in translation 
process as invalid unicode chars).

We are in a big problem, and the only way out I can imagine is fix the 
data by hand  :-! .

Dario,


--------------020204070704000302080203
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1">
  <title></title>
</head>
<body text="#000000" bgcolor="#ffffff">
Tom Lane wrote:<br>
<blockquote type="cite" cite="mid16994.1090099096@sss.pgh.pa.us">
  <pre wrap="">"Dario V. Fassi" <a class="moz-txt-link-rfc2396E" href="mailto:software@sistemat.com.ar">&lt;software@sistemat.com.ar&gt;</a> writes:
  </pre>
  <blockquote type="cite">
    <pre wrap="">A simple question, we need to migrate many (&gt;20) postgres databases from 
SQL_ASCII encoding to UNICODE encoding, over a 7.3.6 server.
    </pre>
  </blockquote>
  <pre wrap=""><!---->
SQL_ASCII is not an encoding (it's more like the absence of knowledge
about an encoding).  What is the data actually stored as?
  </pre>
  <blockquote type="cite">
    <pre wrap="">With  Dump/Restore , we get an error (Invalid Unicode) in any field that 
has a 8 bits character coming from the SQL_ASCII , even setting the 
client_encoding to WIN, ISO-8859-1,  and others encodings.
    </pre>
  </blockquote>
  <pre wrap=""><!---->
It might work to just UPDATE pg_database to set datencoding to the
correct value reflecting what you have actually stored.  You might then
need to REINDEX any indexes on textual columns, but I don't think
anything else would go wrong.

If you have a mishmash of different encodings in a single database, then
of course there is no simple solution; you are in for some pain while
you try to fix the data.
  </pre>
</blockquote>
<div class="moz-signature"><br>
Yes you are right , the original data come from a DB2 with CodePage
IBM-850&nbsp; and was inserted&nbsp; without complains in a Postgres 7.3.6 with
SQL_ASCII.<br>
<br>
Now we are in a Jail , because IBM-850 , isn't WIN, isn't&nbsp; ISO-xx ,
isn't no one postgresql's encoding. <br>
So when in change via pg_databases the encoding , 8 bits characters
become garbage.<br>
More even if we accept this garbage chars and we set encoding to e.g.
ISO-8859-1&nbsp;&nbsp; it's impossible go to a UNICODE because this garbage chars
are invalid in client's encoding , so they are reject (in translation
process as invalid unicode chars).<br>
<br>
We are in a big problem, and the only way out I can imagine is fix the
data by hand&nbsp; <span class="moz-smiley-s13"><span> :-! </span></span>.<br>
<br>
Dario,<br>
<br>
</div>
</body>
</html>

--------------020204070704000302080203--




From drlaz@attglobal.net Sun Jul  18 02:00:45 2004
X-Original-To: pgsql-jdbc-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id AA143D1B1B9
	for <pgsql-jdbc-postgresql.org@localhost.postgresql.org>;
	Sat, 17 Jul 2004 23:00:45 -0300 (ADT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 51934-09
	for <pgsql-jdbc-postgresql.org@localhost.postgresql.org>;
	Sun, 18 Jul 2004 02:00:46 +0000 (GMT)
Received: from smtp812.mail.sc5.yahoo.com (smtp812.mail.sc5.yahoo.com
	[66.163.170.82])
	by svr1.postgresql.org (Postfix) with SMTP id 3CEFBD1B19A
	for <pgsql-jdbc@postgresql.org>; Sat, 17 Jul 2004 23:00:39 -0300 (ADT)
Received: from unknown (HELO ANDYXP) (drlaz@sbcglobal.net@64.172.56.61 with
	login)
	by smtp812.mail.sc5.yahoo.com with SMTP; 18 Jul 2004 02:00:43 -0000
Message-ID: <075c01c46c6b$059ef320$847ba8c0@ANDYXP>
From: "Andrew Lazarus" <drlaz@attglobal.net>
To: <pgsql-jdbc@postgresql.org>
References: <40F98613.1060405@sistemat.com.ar>
 <16994.1090099096@sss.pgh.pa.us>
	<40F9B599.6080205@sistemat.com.ar>
Subject: Re: [ADMIN] Migrate postgres databases from SQL_ASCII to UNICODE
Date: Sat, 17 Jul 2004 19:00:38 -0700
MIME-Version: 1.0
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2800.1409
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409
X-Virus-Scanned: by amavisd-new at hub.org
X-Spam-Status: No, hits=0.1 tagged_above=0.0 required=5.0 tests=RCVD_IN_SORBS
X-Spam-Level: 
X-Archive-Number: 200407/225
X-Sequence-Number: 10609

If you pg_dump in ASCII format, could you figure out what has happened to
the 8-bit characters, if they are recognizable garbage (not truncated to
some genuine character) you then write a sed script. It by hand, but it
doesn't sound terrible.




From tgl@sss.pgh.pa.us Sun Jul  18 06:29:09 2004
X-Original-To: pgsql-jdbc-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id 6BA9DD1B172
	for <pgsql-jdbc-postgresql.org@localhost.postgresql.org>;
	Sun, 18 Jul 2004 03:29:09 -0300 (ADT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 26623-09
	for <pgsql-jdbc-postgresql.org@localhost.postgresql.org>;
	Sun, 18 Jul 2004 06:29:12 +0000 (GMT)
Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130])
	by svr1.postgresql.org (Postfix) with ESMTP id 7D95FD1B182
	for <pgsql-jdbc@postgresql.org>; Sun, 18 Jul 2004 03:29:05 -0300 (ADT)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
	by sss.pgh.pa.us (8.12.11/8.12.11) with ESMTP id i6I6T4OW028180;
	Sun, 18 Jul 2004 02:29:05 -0400 (EDT)
To: "Dario V. Fassi" <software@sistemat.com.ar>
Cc: Oliver Jowett <oliver@opencloud.com>, pgsql-sql@postgresql.org,
	"pgsql-jdbc@postgresql.org" <pgsql-jdbc@postgresql.org>
Subject: Re: [ADMIN] Migrate postgres databases from SQL_ASCII to UNICODE 
In-reply-to: <40F9B599.6080205@sistemat.com.ar> 
References: <40F98613.1060405@sistemat.com.ar>
 <16994.1090099096@sss.pgh.pa.us>
	<40F9B599.6080205@sistemat.com.ar>
Comments: In-reply-to "Dario V. Fassi" <software@sistemat.com.ar>
	message dated "Sat, 17 Jul 2004 20:26:17 -0300"
Date: Sun, 18 Jul 2004 02:29:04 -0400
Message-ID: <28179.1090132144@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
X-Virus-Scanned: by amavisd-new at hub.org
X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests=
X-Spam-Level: 
X-Archive-Number: 200407/227
X-Sequence-Number: 10611

"Dario V. Fassi" <software@sistemat.com.ar> writes:
>> SQL_ASCII is not an encoding (it's more like the absence of knowledge
>> about an encoding).  What is the data actually stored as?

> Yes you are right , the original data come from a DB2 with CodePage 
> IBM-850  and was inserted  without complains in a Postgres 7.3.6 with 
> SQL_ASCII.

Ugh.  You'll have to work out how to convert that codepage to one of the
encodings that PG supports.  Or else add it as a supported encoding
(I'm not sure how hard that is, but it's not out of the question).

			regards, tom lane



From peter_e@gmx.net Sun Jul  18 08:28:11 2004
X-Original-To: pgsql-sql-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id F26A2D1B178
	for <pgsql-sql-postgresql.org@localhost.postgresql.org>;
	Sun, 18 Jul 2004 05:28:11 -0300 (ADT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 56038-07
	for <pgsql-sql-postgresql.org@localhost.postgresql.org>;
	Sun, 18 Jul 2004 08:28:11 +0000 (GMT)
Received: from mail.gmx.net (mail.gmx.net [213.165.64.20])
	by svr1.postgresql.org (Postfix) with SMTP id 8AC18D1B172
	for <pgsql-sql@postgresql.org>; Sun, 18 Jul 2004 05:28:02 -0300 (ADT)
Received: (qmail 12672 invoked by uid 65534); 18 Jul 2004 08:28:07 -0000
Received: from dsl-213-023-249-064.arcor-ip.net (EHLO colt.pezone.net)
	(213.23.249.64)
	by mail.gmx.net (mp024) with SMTP; 18 Jul 2004 10:28:07 +0200
X-Authenticated: #495269
From: Peter Eisentraut <peter_e@gmx.net>
To: "Dario V. Fassi" <software@sistemat.com.ar>,
	Tom Lane <tgl@sss.pgh.pa.us>, pgsql-sql@postgresql.org
Subject: Re: [JDBC] [ADMIN] Migrate postgres databases from SQL_ASCII to
	UNICODE
Date: Sun, 18 Jul 2004 10:28:06 +0200
User-Agent: KMail/1.5.1
References: <40F98613.1060405@sistemat.com.ar>
 <16994.1090099096@sss.pgh.pa.us>
	<40F9B599.6080205@sistemat.com.ar>
In-Reply-To: <40F9B599.6080205@sistemat.com.ar>
MIME-Version: 1.0
Content-Type: text/plain;
  charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Message-Id: <200407181028.06765.peter_e@gmx.net>
X-Virus-Scanned: by amavisd-new at hub.org
X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests=
X-Spam-Level: 
X-Archive-Number: 200407/110
X-Sequence-Number: 18362

Dario V. Fassi wrote:
> Yes you are right , the original data come from a DB2 with CodePage
> IBM-850  and was inserted  without complains in a Postgres 7.3.6 with
> SQL_ASCII.

If you have a program named "recode" installed you could try using that 
to recode the dump file to a supported encoding.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/




From software@sistemat.com.ar Sun Jul  18 19:38:14 2004
X-Original-To: pgsql-jdbc-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id 434A3D1B199;
	Sun, 18 Jul 2004 16:38:14 -0300 (ADT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 46493-09; Sun, 18 Jul 2004 19:38:10 +0000 (GMT)
Received: from mail.svhost.com.ar (mail.svhost.com.ar [200.80.239.22])
	by svr1.postgresql.org (Postfix) with ESMTP id 8443AD1B170;
	Sun, 18 Jul 2004 16:38:07 -0300 (ADT)
Received: from sistemat.com.ar (devel.sistemat.com.ar [192.168.1.40])
	by mail.svhost.com.ar (Postfix) with ESMTP
	id 5AD1B20C08; Sun, 18 Jul 2004 16:36:24 -0300 (ART)
Message-ID: <40FAD3C8.3080503@sistemat.com.ar>
Date: Sun, 18 Jul 2004 16:47:20 -0300
From: "Dario V. Fassi" <software@sistemat.com.ar>
Organization: SISTEMATICA ingenieria de software srl
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US;
	rv:1.4) Gecko/20030624 Netscape/7.1 (ax)
X-Accept-Language: es, es-ar, en-us, en
MIME-Version: 1.0
To: pgsql-sql@postgresql.org
Cc: "pgsql-jdbc@postgresql.org" <pgsql-jdbc@postgresql.org>
Subject: Re: [ADMIN] Migrate postgres databases from SQL_ASCII to UNICODE
References: <40F98613.1060405@sistemat.com.ar>
 <16994.1090099096@sss.pgh.pa.us>
	<40F9B599.6080205@sistemat.com.ar> <28179.1090132144@sss.pgh.pa.us>
In-Reply-To: <28179.1090132144@sss.pgh.pa.us>
Content-Type: multipart/alternative;
	boundary="------------010008000108050309040207"
X-Virus-Scanned: by amavisd-new at hub.org
X-Spam-Status: No, hits=1.0 tagged_above=0.0 required=5.0 tests=HTML_20_30,
	HTML_MESSAGE, HTML_TITLE_EMPTY
X-Spam-Level: *
X-Archive-Number: 200407/231
X-Sequence-Number: 10615

This is a multi-part message in MIME format.
--------------010008000108050309040207
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit


Tom Lane wrote:

>"Dario V. Fassi" <software@sistemat.com.ar> writes:
>  
>
>>>SQL_ASCII is not an encoding (it's more like the absence of knowledge
>>>about an encoding).  What is the data actually stored as?
>>>      
>>>
>>Yes you are right , the original data come from a DB2 with CodePage 
>>IBM-850  and was inserted  without complains in a Postgres 7.3.6 with 
>>SQL_ASCII.
>>    
>>
>
>Ugh.  You'll have to work out how to convert that codepage to one of the
>encodings that PG supports.  Or else add it as a supported encoding
>(I'm not sure how hard that is, but it's not out of the question).
>
>			regards, tom lane
>  
>
Ok, I'm interested is do that , but someone is the development group 
would give me some assistence to get a start point to work with.
I program in Java, C, and less frequently in C++.

Dario.



--------------010008000108050309040207
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1">
  <title></title>
</head>
<body text="#000000" bgcolor="#ffffff">
<br>
Tom Lane wrote:<br>
<blockquote type="cite" cite="mid28179.1090132144@sss.pgh.pa.us">
  <pre wrap="">"Dario V. Fassi" <a class="moz-txt-link-rfc2396E"
 href="mailto:software@sistemat.com.ar">&lt;software@sistemat.com.ar&gt;</a> writes:
  </pre>
  <blockquote type="cite">
    <blockquote type="cite">
      <pre wrap="">SQL_ASCII is not an encoding (it's more like the absence of knowledge
about an encoding).  What is the data actually stored as?
      </pre>
    </blockquote>
  </blockquote>
  <blockquote type="cite">
    <pre wrap="">Yes you are right , the original data come from a DB2 with CodePage 
IBM-850  and was inserted  without complains in a Postgres 7.3.6 with 
SQL_ASCII.
    </pre>
  </blockquote>
  <pre wrap=""><!---->
Ugh.  You'll have to work out how to convert that codepage to one of the
encodings that PG supports.  Or else add it as a supported encoding
(I'm not sure how hard that is, but it's not out of the question).

			regards, tom lane
  </pre>
</blockquote>
Ok, I'm interested is do that , but someone
is the development group would give me some assistence to get a start
point to work with.<br>
I program in Java, C, and less frequently in C++.<br>
<br>
Dario.<br>
<small><br>
</small>
<div class="moz-signature"><br>
</div>
</body>
</html>

--------------010008000108050309040207--




From mstephenson@openworld.org Mon Jul  19 09:29:56 2004
X-Original-To: pgsql-jdbc-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id 662FED1B25F;
	Mon, 19 Jul 2004 06:29:56 -0300 (ADT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 48658-01; Mon, 19 Jul 2004 09:29:53 +0000 (GMT)
Received: from sadik.openworld.co.uk (sadik.openworld.co.uk [195.102.152.54])
	by svr1.postgresql.org (Postfix) with ESMTP id 7D5AED1B229;
	Mon, 19 Jul 2004 06:29:49 -0300 (ADT)
Received: from openworld.org (sokoki.openworld.co.uk [195.102.152.245])
	by sadik.openworld.co.uk (8.11.6p3/8.11.6) with ESMTP id i6J9TiE23833;
	Mon, 19 Jul 2004 09:29:45 GMT
Message-ID: <40FBA1AD.2050904@openworld.org>
Date: Mon, 19 Jul 2004 10:25:49 +0000
From: Michael Stephenson <mstephenson@openworld.org>
Organization: Open World Ltd
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040513
X-Accept-Language: en, de, fr, zh
MIME-Version: 1.0
To: "Dario V. Fassi" <software@sistemat.com.ar>
Cc: pgsql-sql@postgresql.org,
	"pgsql-jdbc@postgresql.org" <pgsql-jdbc@postgresql.org>
Subject: Re: [ADMIN] Migrate postgres databases from SQL_ASCII to UNICODE
References: <40F98613.1060405@sistemat.com.ar>
 <16994.1090099096@sss.pgh.pa.us>
	<40F9B599.6080205@sistemat.com.ar> <28179.1090132144@sss.pgh.pa.us>
	<40FAD3C8.3080503@sistemat.com.ar>
In-Reply-To: <40FAD3C8.3080503@sistemat.com.ar>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
X-Virus-Scanned: by amavisd-new at hub.org
X-Spam-Status: No, hits=0.2 tagged_above=0.0 required=5.0 tests=EXCUSE_16
X-Spam-Level: 
X-Archive-Number: 200407/233
X-Sequence-Number: 10617

What we've done in the past is just run the dump file through iconv - this has 
support for IBM-850.

http://www.gnu.org/software/libiconv/documentation/libiconv/iconv.1.html

Hope this helps,

Michael

Dario V. Fassi wrote:
> 
> Tom Lane wrote:
> 
>>"Dario V. Fassi" <software@sistemat.com.ar> writes:
>>  
>>
>>>>SQL_ASCII is not an encoding (it's more like the absence of knowledge
>>>>about an encoding).  What is the data actually stored as?
>>>>      
>>>>
>>>Yes you are right , the original data come from a DB2 with CodePage 
>>>IBM-850  and was inserted  without complains in a Postgres 7.3.6 with 
>>>SQL_ASCII.
>>>    
>>>
>>
>>Ugh.  You'll have to work out how to convert that codepage to one of the
>>encodings that PG supports.  Or else add it as a supported encoding
>>(I'm not sure how hard that is, but it's not out of the question).
>>
>>			regards, tom lane
>>  
>>
> Ok, I'm interested is do that , but someone is the development group 
> would give me some assistence to get a start point to work with.
> I program in Java, C, and less frequently in C++.
> 
> Dario.
> 
> 


-- 
Web Applications Developer
Open World Ltd, 11 Riverside Court, Riverside Road, Bath, BA2 3DZ.
Tel: +44 1225 444950  Fax: +44 1225 336738  http://www.openworld.co.uk/

CONFIDENTIALITY NOTICE
The information contained in this message is confidential, intended only for
the use of the individual or the entity named as recipient. If the reader of
this message is not that recipient, you are notified that any dissemination,
distribution or copy of this message is strictly prohibited. If you have
received this message in error, please immediately notify us by telephone on
the number above. Your co-operation is appreciated.



From xpoinsard@openpricer.com Mon Jul  19 12:46:26 2004
X-Original-To: pgsql-jdbc-postgresql.org@localhost.postgresql.org
Received: from localhost (unknown [200.46.204.144])
	by svr1.postgresql.org (Postfix) with ESMTP id 62A94D1B298
	for <pgsql-jdbc-postgresql.org@localhost.postgresql.org>;
	Mon, 19 Jul 2004 09:46:26 -0300 (ADT)
Received: from svr1.postgresql.org ([200.46.204.71])
	by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
	with ESMTP id 39600-01
	for <pgsql-jdbc-postgresql.org@localhost.postgresql.org>;
	Mon, 19 Jul 2004 12:46:23 +0000 (GMT)
Received: from main.gmane.org (main.gmane.org [80.91.224.249])
	by svr1.postgresql.org (Postfix) with ESMTP id A2A80D1B211
	for <pgsql-jdbc@postgresql.org>; Mon, 19 Jul 2004 09:46:18 -0300 (ADT)
Received: from list by main.gmane.org with local (Exim 3.35 #1 (Debian))
	id 1BmXXB-0003JZ-00
	for <pgsql-jdbc@postgresql.org>; Mon, 19 Jul 2004 14:46:21 +0200
Received: from secure.openpricer.com ([80.65.224.34])
	by main.gmane.org with esmtp (Gmexim 0.1 (Debian))
	id 1AlnuQ-0007hv-00
	for <pgsql-jdbc@postgresql.org>; Mon, 19 Jul 2004 14:46:21 +0200
Received: from xpoinsard by secure.openpricer.com with local (Gmexim 0.1
	(Debian)) id 1AlnuQ-0007hv-00
	for <pgsql-jdbc@postgresql.org>; Mon, 19 Jul 2004 14:46:21 +0200
X-Injected-Via-Gmane: http://gmane.org/
To: pgsql-jdbc@postgresql.org
From: Xavier Poinsard <xpoinsard@openpricer.com>
Subject: Re: [ADMIN] Migrate postgres databases from SQL_ASCII to UNICODE
Date: Mon, 19 Jul 2004 14:46:18 +0200
Lines: 22
Message-ID: <cdgfqq$vmn$1@sea.gmane.org>
References: <40F98613.1060405@sistemat.com.ar>
 <16994.1090099096@sss.pgh.pa.us>
	<40F9B599.6080205@sistemat.com.ar> <28179.1090132144@sss.pgh.pa.us>
	<40FAD3C8.3080503@sistemat.com.ar>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit
X-Complaints-To: usenet@sea.gmane.org
X-Gmane-NNTP-Posting-Host: secure.openpricer.com
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US;
	rv:1.7) Gecko/20040704 Debian/1.7-4
X-Accept-Language: fr-fr, en-us, en
In-Reply-To: <40FAD3C8.3080503@sistemat.com.ar>
X-Archive: encrypt
X-Virus-Scanned: by amavisd-new at hub.org
X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests=
X-Spam-Level: 
X-Archive-Number: 200407/232
X-Sequence-Number: 10616

Dario V. Fassi a écrit :
> 
> Tom Lane wrote:
> 
>>"Dario V. Fassi" <software@sistemat.com.ar> writes:
>>  
>>
>>>>SQL_ASCII is not an encoding (it's more like the absence of knowledge
>>>>about an encoding).  What is the data actually stored as?
>>>>      
>>>>
>>>Yes you are right , the original data come from a DB2 with CodePage 
>>>IBM-850  and was inserted  without complains in a Postgres 7.3.6 with 
>>>SQL_ASCII.
>>>    
>>>
>>
>>Ugh.  You'll have to work out how to convert that codepage to one of the
>>encodings that PG supports.  Or else add it as a supported encoding
>>(I'm not sure how hard that is, but it's not out of the question).
>>
did you try the recode utility ?