pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
Character encoding conversion
4+ messages / 4 participants
[nested] [flat]

* Character encoding conversion
@ 2008-02-09 14:28  Mike Blackwell <maiku41@sbcglobal.net>
  0 siblings, 3 replies; 4+ messages in thread

From: Mike Blackwell @ 2008-02-09 14:28 UTC (permalink / raw)
  To: pgsql-admin

I have a database which was originally created with LATIN1 encoding. 
I'd like to move it to UTF8.  The data will load ok (COPY)  but I am
getting 'invalid byte sequence for encoding..." messages when accessing
the data.

Is there a way to automatically convert the offending characters, or to
easily locate them in a pg_dump file so they can be converted by hand?

Mike





^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: Character encoding conversion
@ 2008-02-09 14:39  Shoaib Mir <shoaibmir@gmail.com>
  parent: Mike Blackwell <maiku41@sbcglobal.net>
  2 siblings, 0 replies; 4+ messages in thread

From: Shoaib Mir @ 2008-02-09 14:39 UTC (permalink / raw)
  To: Mike Blackwell <maiku41@sbcglobal.net>; +Cc: pgsql-admin

On Feb 9, 2008 7:28 PM, Mike Blackwell <maiku41@sbcglobal.net> wrote:

> I have a database which was originally created with LATIN1 encoding.
> I'd like to move it to UTF8.  The data will load ok (COPY)  but I am
> getting 'invalid byte sequence for encoding..." messages when accessing
> the data.
>
> Is there a way to automatically convert the offending characters, or to
> easily locate them in a pg_dump file so they can be converted by hand?
>
>

Try using 'iconv' for your dump file and convert it all to UTF8 first before
restoring to a UTF8 database.

-- 
Shoaib Mir
Fujitsu Australia Software Technology
shoaibm[@]fast.fujitsu.com.au

^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: Character encoding conversion
@ 2008-02-09 14:46  Michael Fuhr <mike@fuhr.org>
  parent: Mike Blackwell <maiku41@sbcglobal.net>
  2 siblings, 0 replies; 4+ messages in thread

From: Michael Fuhr @ 2008-02-09 14:46 UTC (permalink / raw)
  To: Mike Blackwell <maiku41@sbcglobal.net>; +Cc: pgsql-admin

On Sat, Feb 09, 2008 at 08:28:58AM -0600, Mike Blackwell wrote:
> I have a database which was originally created with LATIN1 encoding. 
> I'd like to move it to UTF8.  The data will load ok (COPY)  but I am
> getting 'invalid byte sequence for encoding..." messages when accessing
> the data.

What's the complete error message?  What's the output of the following
query in a session that gets the error?

select name, setting from pg_settings where name ~ 'encoding';

> Is there a way to automatically convert the offending characters, or to
> easily locate them in a pg_dump file so they can be converted by hand?

You said the data loaded without error, right?  Let's see if we can
figure out why queries are failing before considering what corrective
action to take.

-- 
Michael Fuhr



^ permalink  raw  reply  [nested|flat] 4+ messages in thread

* Re: Character encoding conversion
@ 2008-02-09 18:14  Tom Lane <tgl@sss.pgh.pa.us>
  parent: Mike Blackwell <maiku41@sbcglobal.net>
  2 siblings, 0 replies; 4+ messages in thread

From: Tom Lane @ 2008-02-09 18:14 UTC (permalink / raw)
  To: Mike Blackwell <maiku41@sbcglobal.net>; +Cc: pgsql-admin

Mike Blackwell <maiku41@sbcglobal.net> writes:
> I have a database which was originally created with LATIN1 encoding. 
> I'd like to move it to UTF8.  The data will load ok (COPY)  but I am
> getting 'invalid byte sequence for encoding..." messages when accessing
> the data.

How exactly did you move the data?  And what PG version are we talking
about?

> Is there a way to automatically convert the offending characters, or to
> easily locate them in a pg_dump file so they can be converted by hand?

If you've got client_encoding and server_encoding set up correctly,
it should happen automatically during the COPY.  I think you messed
that up somehow, but it's not clear just how.

			regards, tom lane



^ permalink  raw  reply  [nested|flat] 4+ messages in thread


end of thread, other threads:[~2008-02-09 18:14 UTC | newest]

Thread overview: 4+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2008-02-09 14:28 Character encoding conversion Mike Blackwell <maiku41@sbcglobal.net>
2008-02-09 14:39 ` Shoaib Mir <shoaibmir@gmail.com>
2008-02-09 14:46 ` Michael Fuhr <mike@fuhr.org>
2008-02-09 18:14 ` Tom Lane <tgl@sss.pgh.pa.us>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox