X-Original-To: pgsql-admin-postgresql.org@postgresql.org Received: from localhost (wm.hub.org [200.46.204.128]) by postgresql.org (Postfix) with ESMTP id EBF249FB2D8 for ; Tue, 8 Aug 2006 23:54:25 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.128]) (amavisd-new, port 10024) with ESMTP id 20216-02 for ; Wed, 9 Aug 2006 02:54:25 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey- Received: from noel.decibel.org (noel.decibel.org [67.100.216.10]) by postgresql.org (Postfix) with ESMTP id 158E89FB219 for ; Tue, 8 Aug 2006 23:54:24 -0300 (ADT) Received: from [192.168.42.113] (natpool.bovine.net [67.100.216.14]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (No client certificate requested) by noel.decibel.org (Postfix) with ESMTP id 9ADDA5643F; Tue, 8 Aug 2006 21:54:23 -0500 (CDT) In-Reply-To: References: Mime-Version: 1.0 (Apple Message framework v752.2) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <773783AD-1DEC-411E-9BF1-16293664A090@pervasive.com> Cc: "pgsql admin" Content-Transfer-Encoding: 7bit From: Jim Nasby Subject: Re: Strange UTF8 issue with pg_dump/pg_restore Date: Tue, 8 Aug 2006 11:48:37 -0500 To: Benjamin Krajmalnik X-Mailer: Apple Mail (2.752.2) X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200608/101 X-Sequence-Number: 22644 On Aug 7, 2006, at 9:08 PM, Benjamin Krajmalnik wrote: > I have a database whose records are inserted via ODBC. > I turned on logging, and the client is explicitly setting the > client encoding to UTF. > > Rows are inserted into the tables. When I do a pg_dump, I do not > get any errors. When I try tor restore, I am getting an error > regarding an invalid UTF-8 character sequence. > Of course, this renders compressed backups useless, since I have no > way of manually fixing the file. The process of fixing the file > is, of course, painful. > > Is this a known issue? I am running PG 8.1.4. The problem > happened on PG Windows. > Tomorrow I will try to see if the problem exists in the FreeBSD > compile. If I get it to reproduce, I will be able to send some > data dumps, but will need to know who to send it to. Are the dumps from an older version? There's a known issue that older versions weren't doing proper checking, in which case you can have invalid UTF8 characters. The fix is to run the dump through iconv. -- Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com Pervasive Software http://pervasive.com work: 512-231-6117 vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461