pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Bruce Momjian <pgman@candle.pha.pa.us>
To: Thomas Mueller <news-exp-dec05@tmueller.com>
Cc: pgsql-admin@postgresql.org
Subject: Re: 8.1: import of 8.0 dump fails with UTF-8 error
Date: Sun, 11 Dec 2005 21:48:23 -0500 (EST)
Message-ID: <200512120248.jBC2mNB14188@candle.pha.pa.us> (raw)
In-Reply-To: <dnhkhn$a2e$1@sea.gmane.org>

Thomas Mueller wrote:
> Hello,
> 
> I did a pg_dumpall of all my Pg 8.0.3 databases, removed 8.0, installed
> 8.1 and tried to import the dump. One table in one database failed with:
> 
> ERROR:  invalid UTF-8 byte sequence detected near byte 0x83
> CONTEXT:  COPY pwd_name, line 22428, column name: "t.tonnement"
> 
> So I exported that database with 8.0 as Inserts to a text file and tried
> to fix it using iconv, but that fails as well:
> 
> # iconv -f UTF-8 -t UTF-8 dump.sql > dump-fixed.sql
> iconv: illegal input sequence at position 2588882
> 
> How can I fix the sql script to import it?
> I have Debian Linux 3.1.

We have updated the 8.1.0 release notes to mention a fix:

       Some users are having problems loading UTF-8 data into 8.1.X.
       This is because previous versions allowed invalid UTF-8 byte
       sequences to be entered into the database, and this release
       properly accepts only valid UTF-8 sequences. One way to correct a
       dumpfile is to run the command <command>iconv -c -f UTF-8 -t
       UTF-8 -o cleanfile.sql dumpfile.sql</>. The <literal>-c</> option
       removes invalid character sequences. A diff of the two files will
       show the sequences that are invalid. <command>iconv</> reads the
       entire input file into memory so it might be necessary to use
       <application>split</> to break up the dump into multiple smaller
       files for processing.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073



view thread (2+ messages)

Message-ID: <200512120248.jBC2mNB14188@candle.pha.pa.us>
Permalink:  ../200512120248.jBC2mNB14188@candle.pha.pa.us/
Also on:    postgresql.org/message-id/200512120248.jBC2mNB14188@candle.pha.pa.us

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-admin@postgresql.org
  Cc: pgman@candle.pha.pa.us, news-exp-dec05@tmueller.com
  Subject: Re: 8.1: import of 8.0 dump fails with UTF-8 error
  In-Reply-To: <200512120248.jBC2mNB14188@candle.pha.pa.us>
  References: <dnhkhn$a2e$1@sea.gmane.org> <200512120248.jBC2mNB14188@candle.pha.pa.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox