Received: from maia.hub.org (maia-2.hub.org [200.46.204.251]) by mail.postgresql.org (Postfix) with ESMTP id 11DD8133740B for ; Sat, 12 Feb 2011 05:15:39 -0400 (AST) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.251]) (amavisd-maia, port 10024) with ESMTP id 48284-05 for ; Sat, 12 Feb 2011 09:15:31 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from smtpcorp.com (smtpcorp.com [174.143.60.32]) by mail.postgresql.org (Postfix) with ESMTP id EEA77133616B for ; Sat, 12 Feb 2011 05:15:30 -0400 (AST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=smtpcorp.com; s=main; h=Content-Transfer-Encoding:Content-Type:Mime-Version:References:Message-ID:Date:Subject:From:To; bh=hIT2GBZLMNN8Cm8DdOK9irvge8o99gBnIIdBs/64gdU=; b=GrhkqWhW/qn/2ZQTPrMgwiTbcfCsscFA7sio1NGd3V6tyPXEiEFPUP4U5bnevBoB8RI9aXtscQcXh3ZUHt9E/oCtttuzUyrWC98AdcHpOO+j86bxBNDaVrQB0jHyLpqW; Received: from ip-118-90-143-71.xdsl.xnet.co.nz ([118.90.143.71] helo=gonzo) by smtpcorp.com with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.70) id 1PoBZj-0004Nt-1f; mid 144 for pgsql-sql@postgresql.org; Sat, 12 Feb 2011 09:15:31 +0000 Received: from news by gonzo with local (Exim 4.72) (envelope-from ) id 1PoBZQ-0007nd-5E for pgsql-sql@postgresql.org; Sat, 12 Feb 2011 22:15:12 +1300 To: pgsql-sql@postgresql.org Path: not-for-mail From: Jasen Betts Newsgroups: gmane.comp.db.postgresql.sql Subject: Re: UTF characters compromising data import. Date: 12 Feb 2011 09:15:11 GMT Organization: Dis (not Dat) Organisation Lines: 18 Message-ID: References: NNTP-Posting-Host: localhost Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Trace: reversiblemaps.ath.cx 1297502111 29983 127.0.0.1 (12 Feb 2011 09:15:11 GMT) X-Face: ?)Aw4rXwN5u0~$nqKj`xPz>xHCwgi^q+^?Ri*+R(&uv2=E1Q0Zk(>h!~o2ID@6{uf8s; a +M[5[U[QT7xFN%^gR"=tuJw%TXXR'Fp~W; (T"1(739R%m0Yyyv*gkGoPA.$b,D.w:z+<'"=-lVT?6 {T?=R^:W5g|E2#EhjKCa+nt":4b}dU7GYB*HBxn&Td$@f%.kl^:7X8rQWd[NTc"P"u6nkisze/Q; 8 "9Z{peQF,w)7UjV$c|RO/mQW/NMgWfr5*$-Z%u46"/00mx-,\R'fLPe.)^ User-Agent: slrn/pre1.0.0-18 (Linux) X-Smtpcorp-Track: 22855370.1.18809960 X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-1.9 tagged_above=-5 required=5 tests=BAYES_00=-1.9 X-Spam-Level: X-Archive-Number: 201102/60 X-Sequence-Number: 35136 On 2011-02-08, Gavin Beau Baumanis wrote: > I understand the error message - but what I don't know is what I > need to set the encoding to - in order to import / use the data. if you run it through iconv --from-code=ASCII -to-code=UTF8 -c it'll strip out all the non-ascii symbols, without knowing the encoding it's impossible to assign any useful meaning to them. This step may render your data useless, it would be much better to find out what the encoding should be. perhaps you can figure it out by observation? -- ⚂⚃ 100% natural