Received: from maia.hub.org (unknown [200.46.204.183]) by mail.postgresql.org (Postfix) with ESMTP id 3FFFE633197 for ; Mon, 22 Mar 2010 05:48:51 -0300 (ADT) Received: from mail.postgresql.org ([200.46.204.86]) by maia.hub.org (mx1.hub.org [200.46.204.183]) (amavisd-maia, port 10024) with ESMTP id 28540-06 for ; Mon, 22 Mar 2010 08:48:40 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from hermes.vectorsf.com (firewall.vectorsf.com [62.82.80.62]) by mail.postgresql.org (Postfix) with ESMTP id 66F34632DF6 for ; Mon, 22 Mar 2010 05:48:40 -0300 (ADT) Received: from localhost (localhost.localdomain [127.0.0.1]) by hermes.vectorsf.com (Postfix) with ESMTP id 6970111C0047 for ; Mon, 22 Mar 2010 09:48:36 +0100 (CET) X-Virus-Scanned: amavisd-new at hermes.vectorsf.com Received: from hermes.vectorsf.com ([127.0.0.1]) by localhost (hermes.vectorsf.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ji3iZimecYRS for ; Mon, 22 Mar 2010 09:48:31 +0100 (CET) Received: from hermes.vectorsf.com (hermes.vectorsf.com [10.70.4.30]) by hermes.vectorsf.com (Postfix) with ESMTP id 3523E11C0012 for ; Mon, 22 Mar 2010 09:48:31 +0100 (CET) From: =?iso-8859-1?Q?I=F1igo_Martinez_Lasala?= To: Subject: Bad encoded chars in being inserted into database Date: Mon, 22 Mar 2010 09:48:31 +0100 (CET) Message-ID: <050f01cac99c$7dafe100$790fa300$@com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_NextPart_000_0510_01CAC9A4.DF744900" X-Mailer: Microsoft Office Outlook 12.0 X-Mailer: Zimbra 5.0.20_GA_3127.RHEL4_64 (ZimbraConnectorForOutlook/5.0.3191.20) Content-Language: es Thread-Index: AcrJnH1ewzMtxpQ8RNe8stLAPzTRcg== X-Originating-IP: [10.70.7.99] X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=-0.184 tagged_above=-10 required=5 tests=BAYES_40=-0.185, HTML_MESSAGE=0.001 X-Spam-Level: X-Archive-Number: 201003/147 X-Sequence-Number: 33946 This is a multi-part message in MIME format. ------=_NextPart_000_0510_01CAC9A4.DF744900 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 8bit Hi everybody. I have a doubt about how postgres deal with bad encoded characters into database. We have several gforge application. They are using postgres as database. If we export a database and import again, we have to deal with several bad encoded chars. These bad chars always come from copy & paste emails from Lotus Notes mail client. OK, I understand the Notes client people is using is an ancient application and does not deal very well with some Unicode chars… What I cannot understand is why postgres accept these bad enconded characters into database, exports them without problema but does not allow them when importing again. This has been happening since postgers 7.3. However, until 7.4.XX (y don’t remember what minor version) you could import database without ERRORs. However, since 7.4.XX it’s impossible and it’s imperative to clean bad characters (using iconv, for example) prior importing tables. I agree with this postgres policy, but what I don’t is that you can INSERT them via application. That is, no bad characters should be inserted into database. The check should be made for both import and insert procedures so no bad chars would appear into database. Any suggestion / appreciation about this? We are using php4/5-pgsql module from several distros (CentOS 4/5, debian 4/5 and Ubuntu 8.04LTS) so I discard a pgsql problem and anyway database should deal with this… ------=_NextPart_000_0510_01CAC9A4.DF744900 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable

Hi everybody.

 

I have a doubt about how postgres deal with bad = encoded characters into database.

We have several gforge application. They are using = postgres as database.

If we export a database and import again, we have = to deal with several bad encoded chars. These bad chars always come from copy = & paste emails from Lotus Notes mail client. OK, I understand the Notes = client people is using is an ancient application and does not deal very well = with some Unicode chars…

What I cannot understand is why postgres accept = these bad enconded characters into database, exports them without problema but does not = allow them when importing again.

This has been happening since postgers 7.3. = However, until 7.4.XX (y don’t remember what minor version) you could import = database without ERRORs. However, since 7.4.XX it’s impossible and = it’s imperative to clean bad characters (using iconv, for example) prior = importing tables.

I agree with this postgres policy, but what I = don’t is that you can INSERT them via application. That is, no bad characters = should be inserted into database. The check should be made for both import and insert = procedures so no bad chars would appear into database.

 

Any suggestion / appreciation about = this?

 

We are using php4/5-pgsql module from several = distros (CentOS 4/5, debian 4/5 and Ubuntu 8.04LTS) so I discard a pgsql problem and = anyway database should deal with this…

 

------=_NextPart_000_0510_01CAC9A4.DF744900--