DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type:content-transfer-encoding;
	b=NSF6xUAOfGN+N8pU1u0BY8S4CbAswm1Kaun9nWzLJizHRlqkS2vV+bd6JrLlfIUwWk
	/Ng+5LiRRXbDOXQTENVxsgV/GQuCQBunrBuOFORIYsAqChtDjnax/zUle/MPCu2nBRei
	Wa80H4s+pE4hNT0HJK3cuKOkFckhqdgRoxits=
MIME-Version: 1.0
In-Reply-To: <050f01cac99c$7dafe100$790fa300$@com>
References: <AcrJnH1ewzMtxpQ8RNe8stLAPzTRcg==>
	<050f01cac99c$7dafe100$790fa300$@com>
Date: Mon, 22 Mar 2010 16:05:44 -0600
Message-ID: <dcc563d11003221505w7a12d704j3d0ce0b50a5f83c0@mail.gmail.com>
Subject: Re: Bad encoded chars in being inserted into database
From: Scott Marlowe <scott.marlowe@gmail.com>
To: =?ISO-8859-1?Q?I=F1igo_Martinez_Lasala?= <imartinez@vectorsf.com>
Cc: pgsql-admin@postgresql.org
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

On Mon, Mar 22, 2010 at 2:48 AM, I=F1igo Martinez Lasala
<imartinez@vectorsf.com> wrote:
> Hi everybody.
>
>
>
> I have a doubt about how postgres deal with bad encoded characters into
> database.
>
> We have several gforge application. They are using postgres as database.
>
> If we export a database and import again, we have to deal with several ba=
d
> encoded chars. These bad chars always come from copy & paste emails from
> Lotus Notes mail client. OK, I understand the Notes client people is usin=
g
> is an ancient application and does not deal very well with some Unicode
> chars=85
>
> What I cannot understand is why postgres accept these bad enconded
> characters into database, exports them without problema but does not allo=
w
> them when importing again.
>
> This has been happening since postgers 7.3. However, until 7.4.XX (y don=
=92t
> remember what minor version) you could import database without ERRORs.
> However, since 7.4.XX it=92s impossible and it=92s imperative to clean ba=
d
> characters (using iconv, for example) prior importing tables.

This is because postgresql's support for UTF-8 encoding (and all
encoding really) has gotten tighter over time, so that the filter to
catch improperly encoded UTF has gotten better with each major
release.