X-Original-To: pgsql-admin-postgresql.org@postgresql.org Received: from localhost (mx1.hub.org [200.46.208.251]) by postgresql.org (Postfix) with ESMTP id 6D2E79FA682 for ; Tue, 25 Jul 2006 16:59:09 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.208.251]) (amavisd-new, port 10024) with ESMTP id 88639-03 for ; Tue, 25 Jul 2006 16:59:04 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey- Received: from mail19.bluewin.ch (mail19.bluewin.ch [195.186.18.65]) by postgresql.org (Postfix) with ESMTP id C86EA9FB1B7 for ; Tue, 25 Jul 2006 16:59:04 -0300 (ADT) Received: from [192.168.1.74] (83.78.158.96) by mail19.bluewin.ch (Bluewin 7.3.110.2) id 449253F8008E3748 for pgsql-admin@postgresql.org; Tue, 25 Jul 2006 19:59:03 +0000 From: Ivo Rossacher To: pgsql-admin@postgresql.org Subject: Re: invalid UTF-8 byte sequences and iconv Date: Tue, 25 Jul 2006 21:58:47 +0200 User-Agent: KMail/1.8.2 References: <44C66B33.3070504@wulfsberg.com> In-Reply-To: <44C66B33.3070504@wulfsberg.com> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200607252158.47484.rossacher@bluewin.ch> X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=0 tagged_above=0 required=5 tests= X-Spam-Level: X-Archive-Number: 200607/213 X-Sequence-Number: 22459 In earlier version of postgres the database did allow to store invalid byte sequences. The newer versions do check correctly for the byte sequences and do not allow invalid sequences. So if your dump is really in UTF8 already you will have to search for the invalid sequences in the dump and replace them with the correct one. (if you have a lot of them and a big dump recode might by of help for you). If the dump is not UTF8 you have to pass the correct encoding to iconv in the procedure you described. Best regards Ivo Am Dienstag, 25. Juli 2006 21.04 schrieb Karen Springer: > Hi, > > We have set up a new server and are needing to move our database from > 7.3 to 8.1.4. On restore I'm getting the 'invalid UTF-8 byte sequence' > error message. If I use the command iconv -c -f UTF-8 -t UTF-8 -o > cleanfile.sql dumpfile.sql, then the characters are deleted and the > restore goes smoothly. The problem is that we want those characters. > They are for example the degree symbol and the micro symbol. Is there > anyway to bring these characters over? Thanks in advance. > > Karen > > ---------------------------(end of broadcast)--------------------------- > TIP 5: don't forget to increase your free space map settings