From: Rural Hunter <ruralhunter@gmail.com>
To: pgsql-admin@postgresql.org <pgsql-admin@postgresql.org>
Subject: Re: invalid byte sequence for encoding "UTF8": 0xf481 - how could this happen?
Date: Mon, 16 Apr 2012 16:52:02 +0800
Message-ID: <4F8BDDB2.9020801@gmail.com> (raw)
In-Reply-To: <D960CB61B694CF459DCFB4B0128514C207BB6CE6@exadv11.host.magwien.gv.at>
References: <4F88DB40.9070609@gmail.com>
<D960CB61B694CF459DCFB4B0128514C207BB6CE6@exadv11.host.magwien.gv.at>
=>SELECT convert_to(content, 'UTF8') FROM tmp_article;
This works. My pg is at latest 9.1.3 on ubuntu 10.04 server. We have
millions of data in the db but this is the only one we met the problem.
The bad data is inserted in recent days and we upgraded to 9.1.3 right
after it was released.
δΊ 2012/4/16 16:31, Albe Laurenz ει:
> Rural Hunter wrote:
>> My db is in utf-8, I have a row in my table say tmp_article and I
> wanted
>> to generate ts_vector from the article content:
>> select to_tsvector(content) from tmp_article;
>> But I got this error:
>> ERROR: invalid byte sequence for encoding "UTF8": 0xf481
>>
>> I am wondering how this could happen. I think if there was invalid
> UTF8
>> bytes in the content, it shouldn't have been able to inserted into the
>> tmp_article table as I sometimes see similar errors when inserting
>> records to tmp_article. Am I right?
> You are right in theory. A lot depends on your PostgreSQL version,
> because
> the efforts to prevent invalid strings from entering the database have
> led to changes over the versions. Older versions are more permissive.
>
> To test the theory that the contents of the table are bad, you can
> test if the same happens if you
>
> SELECT convert_to(content, 'UTF8') FROM tmp_article;
>
> Yours,
> Laurenz Albe
>
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: ruralhunter@gmail.com
Subject: Re: invalid byte sequence for encoding "UTF8": 0xf481 - how could this happen?
In-Reply-To: <4F8BDDB2.9020801@gmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox