pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Ora2Pg and export of Multbyte UTF8 characters
3+ messages / 2 participants
[nested] [flat]

* Ora2Pg and export of Multbyte UTF8 characters
@ 2006-07-14 15:42  Chaun Keating <scubadog11@yahoo.com>
  0 siblings, 1 reply; 3+ messages in thread

From: Chaun Keating @ 2006-07-14 15:42 UTC (permalink / raw)
  To: pgsql-admin

Hello, I have gotten Ora2pg to work fairly well for
me, but I am having an issue where mutibyte characters
are being substituted with replacement characters on
export from Oracle.

I used both the output to a file and the direct import
to a PG database and see a bunch of question marks
instead of my multibyte chars.

I have pulled text directly from each database using
the exact same DBD::Oracle and DBD::Pg modules and the
Oracle set displays the multibytes appropriately, so I
believe I have elminated that module as a potential
problem.

The oracle characters from the original table look
fine when I select from the table.  An example (which
may or may not come through correctly for everyone) is
as follows:
<p>16&#12434;&#22522;&#25968;&#12392;&#12375;&#12383;&#25968;&#12290;16&#36914;&#25968;&#12391;&#12399;&#12289;0&#65374;9&#12414;&#12391;&#12398;&#26689;&#25968;&#23383;&#12434;&#36890;&#24120;&#12393;&#12362;&#12426;<p>

The postgres characters from the destination table
look like:
<p>16????????16?????0?9????????????????????????<p>

Any hints as to what I need to do to get them
corrected. When I output to a flat file it appears as
though the question marks are in the flat file so
something on export seems to be getting corrupted.

Thanks,
Chaun

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 



^ permalink  raw  reply  [nested|flat] 3+ messages in thread

* Re: Ora2Pg and export of Multbyte UTF8 characters
@ 2006-07-14 16:08  Andy Shellam <andy@andycc.net>
  parent: Chaun Keating <scubadog11@yahoo.com>
  0 siblings, 1 reply; 3+ messages in thread

From: Andy Shellam @ 2006-07-14 16:08 UTC (permalink / raw)
  To: Chaun Keating <scubadog11@yahoo.com>; +Cc: pgsql-admin

Is your database and client encoding set to UTF-8?

That's about all I can think of.



Chaun Keating wrote:
> Hello, I have gotten Ora2pg to work fairly well for
> me, but I am having an issue where mutibyte characters
> are being substituted with replacement characters on
> export from Oracle.
>
> I used both the output to a file and the direct import
> to a PG database and see a bunch of question marks
> instead of my multibyte chars.
>
> I have pulled text directly from each database using
> the exact same DBD::Oracle and DBD::Pg modules and the
> Oracle set displays the multibytes appropriately, so I
> believe I have elminated that module as a potential
> problem.
>
> The oracle characters from the original table look
> fine when I select from the table.  An example (which
> may or may not come through correctly for everyone) is
> as follows:
> <p>16&#12434;&#22522;&#25968;&#12392;&#12375;&#12383;&#25968;&#12290;16&#36914;&#25968;&#12391;&#12399;&#12289;0&#65374;9&#12414;&#12391;&#12398;&#26689;&#25968;&#23383;&#12434;&#36890;&#24120;&#12393;&#12362;&#12426;<p>
>
> The postgres characters from the destination table
> look like:
> <p>16????????16?????0?9????????????????????????<p>
>
> Any hints as to what I need to do to get them
> corrected. When I output to a flat file it appears as
> though the question marks are in the flat file so
> something on export seems to be getting corrupted.
>
> Thanks,
> Chaun
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around 
> http://mail.yahoo.com 
>
> ---------------------------(end of broadcast)---------------------------
> TIP 6: explain analyze is your friend
>
> !DSPAM:14,44b7bf7434536342414476!
>
>
>   




^ permalink  raw  reply  [nested|flat] 3+ messages in thread

* Re: Ora2Pg and export of Multbyte UTF8 characters
@ 2006-07-14 18:29  Chaun Keating <scubadog11@yahoo.com>
  parent: Andy Shellam <andy@andycc.net>
  0 siblings, 0 replies; 3+ messages in thread

From: Chaun Keating @ 2006-07-14 18:29 UTC (permalink / raw)
  To: pgsql-admin

Ah, I've actually run into this problem before.  Seems
I'd forgotten to explicitly set the NLS_LANG parameter
in the Ora2Pg script.  Once I set it to
AMERICAN_AMERICA.UTF8 it seems to work just dandy.

--- Andy Shellam <andy@andycc.net> wrote:

> Is your database and client encoding set to UTF-8?
> 
> That's about all I can think of.
> 
> 
> 
> Chaun Keating wrote:
> > Hello, I have gotten Ora2pg to work fairly well
> for
> > me, but I am having an issue where mutibyte
> characters
> > are being substituted with replacement characters
> on
> > export from Oracle.
> >
> > I used both the output to a file and the direct
> import
> > to a PG database and see a bunch of question marks
> > instead of my multibyte chars.
> >
> > I have pulled text directly from each database
> using
> > the exact same DBD::Oracle and DBD::Pg modules and
> the
> > Oracle set displays the multibytes appropriately,
> so I
> > believe I have elminated that module as a
> potential
> > problem.
> >
> > The oracle characters from the original table look
> > fine when I select from the table.  An example
> (which
> > may or may not come through correctly for
> everyone) is
> > as follows:
> >
>
<p>16を基数とした数。16進数では、0~9までの桁数字を通常どおり<p>
> >
> > The postgres characters from the destination table
> > look like:
> > <p>16????????16?????0?9????????????????????????<p>
> >
> > Any hints as to what I need to do to get them
> > corrected. When I output to a flat file it appears
> as
> > though the question marks are in the flat file so
> > something on export seems to be getting corrupted.
> >
> > Thanks,
> > Chaun
> >
> > __________________________________________________
> > Do You Yahoo!?
> > Tired of spam?  Yahoo! Mail has the best spam
> protection around 
> > http://mail.yahoo.com 
> >
> > ---------------------------(end of
> broadcast)---------------------------
> > TIP 6: explain analyze is your friend
> >
> > !DSPAM:14,44b7bf7434536342414476!
> >
> >
> >   
> 
> 
> ---------------------------(end of
> broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will
> ignore your desire to
>        choose an index scan if your joining column's
> datatypes do not
>        match
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 



^ permalink  raw  reply  [nested|flat] 3+ messages in thread


end of thread, other threads:[~2006-07-14 18:29 UTC | newest]

Thread overview: 3+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2006-07-14 15:42 Ora2Pg and export of Multbyte UTF8 characters Chaun Keating <scubadog11@yahoo.com>
2006-07-14 16:08 ` Andy Shellam <andy@andycc.net>
2006-07-14 18:29   ` Chaun Keating <scubadog11@yahoo.com>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox