DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=message-id:date:from:to:subject:in-reply-to:mime-version
	:content-type:content-transfer-encoding:content-disposition
	:references;
	b=jpe8EH48JNrk1wM7lPNY9WsZkeBtNgB55kimR/5ScJicemvLwSizQ/T9QyrYspW/0O
	XlBlnNov02adS6SrTlYKW2OfEmXGIK4GazbpyzKrhdDjHIQXiYlWWQWE7l/Pu6dj3oPy
	xdYIvDNDHbYvCnL/RTTw1AYO8REMTezSsDCV4=
Message-ID: <57a815bf0807110137j198cb9e9gcb7d07405c42f42e@mail.gmail.com>
Date: Fri, 11 Jul 2008 18:37:39 +1000
From: "Mikel Lindsaar" <raasdnil@gmail.com>
To: pgsql-admin@postgresql.org
Subject: Re: Importing data - possible UTF8 import bug?
In-Reply-To: <57a815bf0807101902o193644ear50e211e67663d874@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <57a815bf0807101902o193644ear50e211e67663d874@mail.gmail.com>

OK, I'm mailing the list the results of my problem so future people can find it.

The error was

ERROR: invalid byte sequence for encoding "UTF8": 0xa2

with many different types of 0x... lines.

The problem was indeed a bug, but one that sat between the keyboard
and screen (that is, me), not with the COPY command.  I didn't read
the COPY docs well enough, in there it clearly states that a backslash
followed by digits will be interpreted as a character with that
numeric code (in the table).

As the data I was importing contained addresses, it had a unit number
and street number, like this; 2\554, so this was being interpreted as
the number 2 followed by a character represented by \554 which was an
invalid sequence and so rightly so, Copy failed and complained about
an invalid char sequence.

Going through the data set and replacing the backslashes with forward
slashes (which works in my case) or if you need to be non destructive,
replcaing the single backslash with a double backslash, handles the
problem.

Sorry all for the noise.

Mikel


-- 
http://lindsaar.net/
Rails, RSpec and Life blog....