From superman0920@gmail.com Thu Mar 29 03:00:52 2012 Received: from makus.postgresql.org (makus.postgresql.org [98.129.198.125]) by mail.postgresql.org (Postfix) with ESMTP id 5C851FD81BD for ; Thu, 29 Mar 2012 00:00:52 -0300 (ADT) Received: from mail-yx0-f174.google.com ([209.85.213.174]) by makus.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1SD5bX-0002Dj-5y for pgsql-admin@postgresql.org; Thu, 29 Mar 2012 03:00:51 +0000 Received: by yenl12 with SMTP id l12so1211049yen.19 for ; Wed, 28 Mar 2012 20:00:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:reply-to:subject:x-priority:x-guid:x-has-attach :x-mailer:mime-version:message-id:content-type; bh=GYFShFihvaUfzN9Ibt6Q3wOXT7cniXUXZxkj2nLMPZI=; b=E44VjTVly1IM3zEKP769/wi0WLABnR3jM6Vc+0SYWTYFPkrXLdPKVRYMp7a7UVABcS ogu6t8T+uwdBygXmuq967xwBBHJK9w5Yl5QbkhGWjYSXSg8vWtC9ePnYY5qZ+6nWUz4b 5mEd1pFEpH76eXaFlGeNl0UbsYitJ0ta+FVaW0KXZr22fHOgcfy+cPYhfE6/E5TPTo12 m9TCyPbeoemCmpsid2NQEoUdKjt3hCCr3HHmj7Bd9+RpmxeFPIgAQg64UdlSetFybyUO 5rHaWk3pwydYHADGIn4YYsfIGk+4DzmUroNg5pSmQeQWhloF21CiYEIEOSbTguaEyRvI xaeQ== Received: by 10.68.229.169 with SMTP id sr9mr1708930pbc.164.1332990038103; Wed, 28 Mar 2012 20:00:38 -0700 (PDT) Received: from imsuperman-PC ([218.61.139.119]) by mx.google.com with ESMTPS id x1sm3968263pbp.50.2012.03.28.20.00.35 (version=SSLv3 cipher=OTHER); Wed, 28 Mar 2012 20:00:37 -0700 (PDT) Date: Thu, 29 Mar 2012 11:00:36 +0800 From: superman0920 To: pgsql-admin Reply-To: superman0920 Subject: about encoding X-Priority: 3 X-GUID: 5E7C6932-0BA8-4D9A-89D7-A53EECD29699 X-Has-Attach: no X-Mailer: Foxmail 7.0.1.88[cn] Mime-Version: 1.0 Message-ID: <201203291100302033066@gmail.com> Content-Type: multipart/alternative; boundary="----=_001_NextPart333461281737_=----" X-Pg-Spam-Score: -2.3 (--) X-Archive-Number: 201203/145 X-Sequence-Number: 39333 This is a multi-part message in MIME format. ------=_001_NextPart333461281737_=---- Content-Type: text/plain; charset="gb2312" Content-Transfer-Encoding: base64 aGVsbG8NCmkgd2FudCB0byBpbnNlcnQgYSByZXBvcnQgdG8gcG9zdGdyZXNxbKOsdGhlIHJlcG9y dCBjb250YWluIHNvbWV0aGluZyBDaGluZXNlIGNoYXJhY3RlcnMgYW5kIHRoZSBwb3N0Z3Jlc3Fs IGlzIHV0Zi04Lg0KDQp0aGUgcmVzcG9uc2UgZnJvbSBkYiBpcyB0aGlzOiANCkVSUk9SOiAgaW52 YWxpZCBieXRlIHNlcXVlbmNlIGZvciBlbmNvZGluZyAiVVRGOCI6IDB4YjEgDQoNCmhvdyBjYW4g aSBmaXggaXQgPyANCg0KdGh4DQoNCg0KDQoNCnN1cGVybWFuMDkyMA== ------=_001_NextPart333461281737_=---- Content-Type: text/html; charset="gb2312" Content-Transfer-Encoding: quoted-printable

hello

i want to insert a report to postgresql=A3=ACthe report contain somet= hing=20 Chinese characters and the postgresql is utf-8.

the response from db is this:

ERROR: invalid byte sequence= for encoding "UTF8": 0xb1=20

how can i fix it ?

thx

superman0920

------=_001_NextPart333461281737_=------ From superman0920@gmail.com Thu Mar 29 03:22:55 2012 Received: from makus.postgresql.org (makus.postgresql.org [98.129.198.125]) by mail.postgresql.org (Postfix) with ESMTP id 2839B1D983CB for ; Thu, 29 Mar 2012 00:22:55 -0300 (ADT) Received: from mail-pb0-f46.google.com ([209.85.160.46]) by makus.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1SD5wp-0002Wf-V6 for pgsql-admin@postgresql.org; Thu, 29 Mar 2012 03:22:54 +0000 Received: by pbcun15 with SMTP id un15so2734897pbc.19 for ; Wed, 28 Mar 2012 20:22:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:reply-to:subject:x-priority:x-guid:x-has-attach :x-mailer:mime-version:message-id:content-type; bh=CkPdH9H9jTRSq3NHNvgCY6v1G4jT0Nh7T1q2zkFZ2lY=; b=XVa5ffz4btc0wuL4iRdaS8bWJ3s/lpJQqxBs8qgN8CQ06xx52d+pEXvev8/QhaWl9s DJGA+sJZTrXiaOZckT7KHId89++NDql0Tgnwds5P6Q9UXNHFLbwLvZQXYTgMldngmb6V 4RrVnYNn8JVdE8q3flwRpG73WRu9KBTEDDVy7RS9EjMIrLBQ8uYCE3ZjbP/56hp1npCC E9KA96TmUbXCw/iyWooTATeQqCB5A1PUB3ejATIMpvMkD0HrfhqtRkoygSEdc1IRwfHU 2qXHNLhal6yXbAdr/blfYXEVzldKn9ER49NWyz6TA8rBIpjetCZQPC0wuh+SixjV/3h+ +33Q== Received: by 10.68.131.1 with SMTP id oi1mr1853271pbb.102.1332991359143; Wed, 28 Mar 2012 20:22:39 -0700 (PDT) Received: from imsuperman-PC ([218.61.139.119]) by mx.google.com with ESMTPS id f2sm4019461pbr.16.2012.03.28.20.22.35 (version=SSLv3 cipher=OTHER); Wed, 28 Mar 2012 20:22:38 -0700 (PDT) Date: Thu, 29 Mar 2012 11:22:36 +0800 From: superman0920 To: pgsql-admin Reply-To: superman0920 Subject: about encoding X-Priority: 3 X-GUID: C8BBC1C6-48CC-42AA-AE2F-A950AEB26122 X-Has-Attach: no X-Mailer: Foxmail 7.0.1.88[cn] Mime-Version: 1.0 Message-ID: <201203291122317598487@gmail.com> Content-Type: multipart/alternative; boundary="----=_001_NextPart626812110821_=----" X-Pg-Spam-Score: -2.3 (--) X-Archive-Number: 201203/129 X-Sequence-Number: 39317 This is a multi-part message in MIME format. ------=_001_NextPart626812110821_=---- Content-Type: text/plain; charset="gb2312" Content-Transfer-Encoding: base64 aGVsbG8NCmkgd2FudCB0byBpbnNlcnQgYSByZXBvcnQgdG8gcG9zdGdyZXNxbKOsdGhlIHJlcG9y dCBjb250YWluIHNvbWV0aGluZyBDaGluZXNlIGNoYXJhY3RlcnMgYW5kIHRoZSBwb3N0Z3Jlc3Fs IGlzIHV0Zi04Lg0KDQp0aGUgcmVzcG9uc2UgZnJvbSBkYiBpcyB0aGlzOiANCkVSUk9SOiAgaW52 YWxpZCBieXRlIHNlcXVlbmNlIGZvciBlbmNvZGluZyAiVVRGOCI6IDB4YjEgDQoNCmhvdyBjYW4g aSBmaXggaXQgPyANCg0KdGh4DQoNCg0KDQoNCnN1cGVybWFuMDkyMA== ------=_001_NextPart626812110821_=---- Content-Type: text/html; charset="gb2312" Content-Transfer-Encoding: quoted-printable

hello

i want to insert a report to postgresql=A3=ACthe report contain somet= hing=20 Chinese characters and the postgresql is utf-8.

the response from db is this:

ERROR: invalid byte sequence= for encoding "UTF8": 0xb1=20

how can i fix it ?

thx

superman0920

------=_001_NextPart626812110821_=------ From Kevin.Grittner@wicourts.gov Thu Mar 29 14:37:38 2012 Received: from magus.postgresql.org (magus.postgresql.org [87.238.57.229]) by mail.postgresql.org (Postfix) with ESMTP id A3529E27B4A for ; Thu, 29 Mar 2012 11:37:38 -0300 (ADT) Received: from gwmta.wicourts.gov ([165.219.244.99] helo=gw.wicourts.gov) by magus.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1SDGTk-0000DX-TO for pgsql-admin@postgresql.org; Thu, 29 Mar 2012 14:37:37 +0000 Received: from Courts-MTA by gw.wicourts.gov with Novell_GroupWise; Thu, 29 Mar 2012 09:37:19 -0500 Message-Id: <4F742D48020000250004692A@gw.wicourts.gov> X-Mailer: Novell GroupWise Internet Agent 8.0.2 Date: Thu, 29 Mar 2012 09:37:12 -0500 From: "Kevin Grittner" To: "superman0920" , "pgsql-admin" Subject: Re: about encoding References: <201203291122317598487@gmail.com> In-Reply-To: <201203291122317598487@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Pg-Spam-Score: -4.2 (----) X-Archive-Number: 201203/134 X-Sequence-Number: 39322 superman0920 wrote: =20 > i want to insert a report to postgresql=EF=BC=8Cthe report contain > something Chinese characters and the postgresql is utf-8. >=20 > the response from db is this:=20 > ERROR: invalid byte sequence for encoding =22UTF8=22: 0xb1=20 =20 That=27s not a valid byte sequence for a character under the UTF8 character encoding scheme. You said that PostgreSQL is using utf-8, but what encoding is used for the report? Is your client connecting using the character encoding scheme used by the report? =20 -Kevin From scrawford@pinpointresearch.com Mon Apr 2 15:55:38 2012 Received: from makus.postgresql.org (makus.postgresql.org [98.129.198.125]) by mail.postgresql.org (Postfix) with ESMTP id B7AB81782AF6 for ; Mon, 2 Apr 2012 12:55:38 -0300 (ADT) Received: from cerberus.pinpointresearch.com ([66.7.238.130] helo=polaris.pinpointresearch.com) by makus.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1SEjbV-0002g5-CL for pgsql-admin@postgresql.org; Mon, 02 Apr 2012 15:55:38 +0000 Received: from betelgeuse.pinpointresearch.com (betelgeuse.pinpointresearch.com [192.168.1.179]) by polaris.pinpointresearch.com (Postfix) with ESMTP id BD4DAE00EC83; Mon, 2 Apr 2012 08:55:23 -0700 (PDT) Message-ID: <4F79CBEC.7080805@pinpointresearch.com> Date: Mon, 02 Apr 2012 08:55:24 -0700 From: Steve Crawford User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.27) Gecko/20120216 Lightning/1.0b2 Thunderbird/3.1.19 MIME-Version: 1.0 To: superman0920 CC: pgsql-admin Subject: Re: about encoding References: <201203291100302033066@gmail.com> In-Reply-To: <201203291100302033066@gmail.com> Content-Type: multipart/alternative; boundary="------------030006060201060101020103" X-Pg-Spam-Score: -1.9 (-) X-Archive-Number: 201204/2 X-Sequence-Number: 39338 This is a multi-part message in MIME format. --------------030006060201060101020103 Content-Type: text/plain; charset=GB2312 Content-Transfer-Encoding: quoted-printable On 03/28/2012 08:00 PM, superman0920 wrote: > hello > i want to insert a report to postgresql=A3=ACthe report contain somethi= ng > Chinese characters and the postgresql is utf-8. > the response from db is this: > ERROR: invalid byte sequence for encoding "UTF8": 0xb1 > how can i fix it ? > Remove the bad character?? You need to provide a bit more information, but the likely cause is that the data you are inserting is not UTF8 encoded. Check the system that is generating the data you wish to insert and make sure it is set to write its output as UTF8. If you cannot set it to write UTF8, then you need to determine what character-set it is using and tell PostgreSQL to use that. See "SET client_encoding..." Note that UTF8 is the character set that PostgreSQL is using internally but you can read and write other sets through the client_encoding setting. Also be aware that UTF8 is just one of many ways of encoding the various Unicode code-points. People sometimes conversationally interchange Unicode and UTF8 but they are not the same. Make sure that the data you are receiving is specifically UTF8 and not just some unknown Unicode encoding. If you are still stuck after checking/fixing the above you could actually have bad data in your input or, possibly, an incorrect byte-order issue though that is unlikely in most situations. Cheers, Steve --------------030006060201060101020103 Content-Type: text/html; charset=GB2312 Content-Transfer-Encoding: quoted-printable On 03/28/2012 08:00 PM, superman0920 wrote:

hello

i want to insert a report to postgresql=A3=ACthe report contai= n something Chinese characters and the postgresql is utf-8.

the response from db is this:

ERROR: invalid&nbs= p;byte sequence for encoding "UTF8": 0xb1

how can i fix it ?

<snark>Remove the bad character??</snark>

You need to provide a bit more information, but the likely cause is that the data you are inserting is not UTF8 encoded. Check the system that is generating the data you wish to insert and make sure it is set to write its output as UTF8.

If you cannot set it to write UTF8, then you need to determine what character-set it is using and tell PostgreSQL to use that. See "SET client_encoding..." Note that UTF8 is the character set that PostgreSQL is using internally but you can read and write other sets through the client_encoding setting.

Also be aware that UTF8 is just one of many ways of encoding the various Unicode code-points. People sometimes conversationally interchange Unicode and UTF8 but they are not the same. Make sure that the data you are receiving is specifically UTF8 and not just some unknown Unicode encoding.

If you are still stuck after checking/fixing the above you could actually have bad data in your input or, possibly, an incorrect byte-order issue though that is unlikely in most situations.

Cheers,
Steve

--------------030006060201060101020103--