Received: from maia.hub.org (maia-3.hub.org [200.46.204.243]) by mail.postgresql.org (Postfix) with ESMTP id 670701337B61 for ; Sun, 13 Feb 2011 08:03:40 -0400 (AST) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.243]) (amavisd-maia, port 10024) with ESMTP id 51192-08 for ; Sun, 13 Feb 2011 12:03:31 +0000 (UTC) X-Greylist: delayed 00:54:41.414052 by SQLgrey-1.7.6 Received: from panda.hostingbay.net (panda.hostingbay.net [74.86.150.75]) by mail.postgresql.org (Postfix) with ESMTP id 755D91337B49 for ; Sun, 13 Feb 2011 08:03:31 -0400 (AST) Received: from cpe-121-223-230-214.static.vic.bigpond.net.au ([121.223.230.214] helo=[10.0.0.26]) by panda.hostingbay.net with esmtpa (Exim 4.69) (envelope-from ) id 1PoZot-0006ik-6X; Sun, 13 Feb 2011 22:08:48 +1100 Subject: Re: UTF characters compromising data import. Mime-Version: 1.0 (Apple Message framework v1082) Content-Type: text/plain; charset=utf-8 From: Gavin Beau Baumanis In-Reply-To: Date: Sun, 13 Feb 2011 22:08:41 +1100 Cc: pgsql-sql@postgresql.org Content-Transfer-Encoding: quoted-printable Message-Id: <54974F5C-F995-4980-96CE-87956639C55C@thespidernet.com> References: To: Jasen Betts X-Mailer: Apple Mail (2.1082) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - panda.hostingbay.net X-AntiAbuse: Original Domain - postgresql.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - thespidernet.com X-Source: X-Source-Args: X-Source-Dir: X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-1.9 tagged_above=-5 required=5 tests=BAYES_00=-1.9 X-Spam-Level: X-Archive-Number: 201102/61 X-Sequence-Number: 35137 Hi and thanks for the replies, I have had some luck. I did find the encoding used originally to create the text files I am = trying to import. I have managed to use the client_encoding environmental variable and = then successfully did manage to import the data. Gavin. On 12/02/2011, at 8:15 PM, Jasen Betts wrote: > On 2011-02-08, Gavin Beau Baumanis wrote: >=20 >> I understand the error message - but what I don't know is what I >> need to set the encoding to - in order to import / use the data. =20= >=20 > if you run it through =20 >=20 > iconv --from-code=3DASCII -to-code=3DUTF8 -c >=20 > it'll strip out all the non-ascii symbols, without knowing the > encoding it's impossible to assign any useful meaning to them. > This step may render your data useless, it would be much better to > find out what the encoding should be. >=20 > perhaps you can figure it out by observation? >=20 > --=20 > =E2=9A=82=E2=9A=83 100% natural >=20 > --=20 > Sent via pgsql-sql mailing list (pgsql-sql@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-sql