Received: from maia.hub.org (maia-5.hub.org [200.46.204.29]) by mail.postgresql.org (Postfix) with ESMTP id 1712F1337B80 for ; Sun, 27 Feb 2011 23:14:25 -0400 (AST) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.29]) (amavisd-maia, port 10024) with ESMTP id 85032-06 for ; Mon, 28 Feb 2011 03:14:19 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from www.postnewspapers.com.au (www.postnewspapers.com.au [173.230.139.50]) by mail.postgresql.org (Postfix) with ESMTP id 8D8641337B9E for ; Sun, 27 Feb 2011 23:14:17 -0400 (AST) Received: from localhost (localhost [127.0.0.1]) by www.postnewspapers.com.au (Postfix) with ESMTP id 8D999AD1; Sun, 27 Feb 2011 22:14:17 -0500 (EST) X-Virus-Scanned: amavisd-new at postnewspapers.com.au.com Received: from www.postnewspapers.com.au ([127.0.0.1]) by localhost (www.postnewspapers.com.au [127.0.0.1]) (amavisd-new, port 10024) with LMTP id vZaqiq-narf3; Sun, 27 Feb 2011 22:14:16 -0500 (EST) Received: from mail.postnewspapers.com.au (ppp171-128.static.internode.on.net [150.101.171.128]) by www.postnewspapers.com.au (Postfix) with ESMTP id A0F9019; Sun, 27 Feb 2011 22:14:15 -0500 (EST) Received: from [10.0.0.53] (wallace.localnet [10.0.0.53]) by mail.postnewspapers.com.au (Postfix) with ESMTP id 516E6AF0E4; Mon, 28 Feb 2011 11:14:14 +0800 (WST) Message-ID: <4D6B1305.5010400@postnewspapers.com.au> Date: Mon, 28 Feb 2011 11:14:13 +0800 From: Craig Ringer User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.13) Gecko/20101208 Thunderbird/3.1.7 MIME-Version: 1.0 To: AI Rumman CC: pgsql-general General Subject: Re: ERROR: invalid byte sequence for encoding "UTF8": 0xc35c References: In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-1.91 tagged_above=-5 required=5 tests=BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01 X-Spam-Level: X-Archive-Number: 201102/930 X-Sequence-Number: 173196 On 27/02/11 20:47, AI Rumman wrote: > I am getting error in Postgresql 9.0.1. > > update import_details_test > set data_row = '["4","1 Monor JoÃ\u083ão S. AntÃ\u0083ão ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Because your email client may have transformed the text encoding, I can't make any certain conclusions about what you're actually sending to the database, but it's highly likely that you're sending latin-1 encoded text to the database while your client_encoding is set to 'utf8'. The marked text is most likely the problem... but I think there's more wrong with it than just being latin-1 encoded. That kind of mangling often comes about when utf-8 text has been incorrectly interpreted as latin-1 and modified, or when something has incorrectly tried to do utf8<->latin-1 conversions more than once. You really need to figure out what encoding your input is in, convert it to a known encoding like utf-8 *once*, and keep it that way. If you're using Python, which I suspect you might be, the "".decode() function is useful. For example, I can convert a latin-1 encoded byte string to a python Unicode string with: "somelatin1string".decode("latin-1") Sometimes you can get away with just "SET client_encoding=latin-1" but in this case your string data looks like it's been mangled by more than just a single encoding mis-interpretation, so you'll probably just silently insert corrupt data by doing that. Don't. Fix your code so it knows what the text encoding of the input is. If you are, in fact, using Python, it's a really good idea to always "".decode() all your inputs so your internal processing is done in Unicode (UTF-16, in fact). Similarly, Qt programmers should convert everything to unicode QString as soon as possible and use that for all internal manipulation. It'll save a lot of pain.