From: Jim Nasby <jnasby@pervasive.com>
To: Benjamin Krajmalnik <kraj@illumen.com>
Cc: Pgsql-Admin \(E-mail\) <pgsql-admin@postgresql.org>
Subject: Re: Strange UTF8 issue with pg_dump/pg_restore
Date: Wed, 9 Aug 2006 13:20:06 -0500
Message-ID: <4D27CB1096EF1C408F4BFAB0046EC7B6099FF5@ausmailid.aus.pervasive.com> (raw)
Please include the list in your replies so other people can help.
-----Original Message-----
From: Benjamin Krajmalnik [mailto:kraj@illumen.com]
Sent: Tuesday, August 08, 2006 10:31 PM
To: Jim Nasby
Subject: RE: [ADMIN] Strange UTF8 issue with pg_dump/pg_restore
I had corrupt rows inserted on 8.1.4/Windows. I had truncated the table, and it startd collecting data again.
I have since moved to 8.1.4/FreeBSD. I have been running the data for 2 days now, so the odds of having a corrupt record is pretty good if the problem exists. I will wait a few more days and will pg_dump the table which had the problems and attempt to restor it on a dev server to see if I see a problem.
Is iconv a utility that comes woth PG? How can I run the data through it (I am doing pg_dump in custom format).
No, it's a unix utility. You'd have to dump in plain text output, and then run that through iconv. There's info about this in the 8.1 release notes.
_____
From: Jim Nasby [mailto:jnasby@pervasive.com]
Sent: Tue 8/8/2006 10:48 AM
To: Benjamin Krajmalnik
Cc: pgsql admin
Subject: Re: [ADMIN] Strange UTF8 issue with pg_dump/pg_restore
On Aug 7, 2006, at 9:08 PM, Benjamin Krajmalnik wrote:
> I have a database whose records are inserted via ODBC.
> I turned on logging, and the client is explicitly setting the
> client encoding to UTF.
>
> Rows are inserted into the tables. When I do a pg_dump, I do not
> get any errors. When I try tor restore, I am getting an error
> regarding an invalid UTF-8 character sequence.
> Of course, this renders compressed backups useless, since I have no
> way of manually fixing the file. The process of fixing the file
> is, of course, painful.
>
> Is this a known issue? I am running PG 8.1.4. The problem
> happened on PG Windows.
> Tomorrow I will try to see if the problem exists in the FreeBSD
> compile. If I get it to reproduce, I will be able to send some
> data dumps, but will need to know who to send it to.
Are the dumps from an older version? There's a known issue that older
versions weren't doing proper checking, in which case you can have
invalid UTF8 characters. The fix is to run the dump through iconv.
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com <http://pervasive.com/; work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-admin@postgresql.org
Cc: jnasby@pervasive.com, kraj@illumen.com
Subject: Re: Strange UTF8 issue with pg_dump/pg_restore
In-Reply-To: <4D27CB1096EF1C408F4BFAB0046EC7B6099FF5@ausmailid.aus.pervasive.com>
References: <4D27CB1096EF1C408F4BFAB0046EC7B6099FF5@ausmailid.aus.pervasive.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox