Received: from localhost (maia-1.hub.org [200.46.204.191]) by postgresql.org (Postfix) with ESMTP id 0525B9FB36B for ; Sat, 7 Apr 2007 18:09:35 -0300 (ADT) Received: from postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.191]) (amavisd-maia, port 10024) with ESMTP id 16808-06 for ; Sat, 7 Apr 2007 18:09:31 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey-1.7.4 Received: from crestone.coronasolutions.com (crestone.coronasolutions.com [66.45.104.24]) by postgresql.org (Postfix) with ESMTP id B8AA09FB1E3 for ; Sat, 7 Apr 2007 18:09:31 -0300 (ADT) Received: from localhost (localhost.localdomain [127.0.0.1]) by crestone.coronasolutions.com (Postfix) with ESMTP id 940996445DF; Sat, 7 Apr 2007 15:09:29 -0600 (MDT) Received: from crestone.coronasolutions.com ([127.0.0.1]) by localhost (crestone.coronasolutions.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 21168-03; Sat, 7 Apr 2007 15:09:25 -0600 (MDT) Received: from mac.local (c-67-166-27-159.hsd1.co.comcast.net [67.166.27.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by crestone.coronasolutions.com (Postfix) with ESMTP id 9D6CB6445DB for ; Sat, 7 Apr 2007 15:09:25 -0600 (MDT) Message-ID: <46180885.4050504@drivefaster.net> Date: Sat, 07 Apr 2007 15:09:25 -0600 From: Dan Harris User-Agent: Thunderbird 2.0b2 (Macintosh/20070116) MIME-Version: 1.0 To: pgsql-admin@postgresql.org Subject: Can't restore a pg_dump due to encoding errors Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new-20030616-p10 (Debian) at coronasolutions.com X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200704/67 X-Sequence-Number: 25131 Hello all: I'm in a bit of a bind here. Today is my maintenance window for upgrading from 8.0.3 to 8.2.3. I did a pg_dumpall overnight ( resulting in a 72GB file ) and then a pg_dump on one of my small databases this morning to test pg_restore. When trying to reload, I'm getting the following error: psql:sac.bak:2793776: ERROR: invalid byte sequence for encoding "UTF8": 0x92 A google search came up with this link: http://www.hostingforum.ca/280472-invalid-byte-sequence-encoding-utf8.html Which sounds very simliar to what's going on here. Again the dump was done on 8.0.3, which sounds like it was before the anti-sql injection patch that would refuse to import "invalid" characters. The database ( for now ) *should* only be storing English/Latin characters. Although, the old server had LOCALE='C' previously ( this was unnecessary so I'm trying to drop that today ). I don't know if this would matter, but that's the only thing that was different from a bog standard Postgres setup besides the obvious tuning changes. I have never changed any of the encoding settings on client or server from the defaults. So far, I tried editing the dump file and changing the client_encoding to 'UTF8' at the top, hoping this might make a difference but did not. The advice in that thread seems to be "fix the bad data". But I am not seeing how I can tell exactly what or where this bad data is. Luckily, this small dump file is manageable by opening in vim and pointing to the line 2793776 the error references, however the line referenced is just the "\." at the end of a huge COPY section. I don't know which line inside that secion is the problem and there's enough of them that it'd be pretty impractical for me to visually scan the file looking for something ( I'm not even quite sure what to look for at this point ). And, if that is the only solution and I run into a problem restoring the 72GB dump file, I'm going to be in some serious trouble, especially if I am not alerted to the problem until *after* it's finished looking at all the COPY statements! What say the sages? Thanks -Dan