Message-ID: <489303E6.40508@hagander.net>
Date: Fri, 01 Aug 2008 14:39:02 +0200
From: Magnus Hagander <magnus@hagander.net>
User-Agent: Thunderbird 2.0.0.16 (X11/20080724)
MIME-Version: 1.0
To: "Joshua D. Drake" <jd@commandprompt.com>
CC: Tom Lane <tgl@sss.pgh.pa.us>, Bruce Momjian <bruce@momjian.us>,
	PostgreSQL www <pgsql-www@postgresql.org>
Subject: Re: Email search failure
References: <200807312019.m6VKJvR02505@momjian.us>
	<23897.1217539560@sss.pgh.pa.us>
	<48927FEF.8070200@commandprompt.com>
In-Reply-To: <48927FEF.8070200@commandprompt.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

Joshua D. Drake wrote:
> Tom Lane wrote:
>> Bruce Momjian <bruce@momjian.us> writes:
>>> Why is the email below now appearing in a search?
>>
>> Probably because nothing has gotten indexed for a month or more.
>> Whoever is supposed to maintain the archive indexer has been
>> on vacation since it broke ...
> 
> That would be Magnus and you are correct. He just got back. The problem
> (last I checked) is an issue with Russian emails.

Looking at it now. That clearly wasn't the only problem, because there
was a "sleep 1800" process that had been running since July 3. Logfiles
weren't touched etc. Just restarting it fixed that part, which clearly
somebody else could've done as well ;)

I found the bug with the Russian emails, btw. It seems mhonarc encoded
the invalid UTF8 sequences inside valid HTML escape entities And the
code applied the "fix broken UTF8" logic *before* it unescaped the HTML
entities. Now it does it both before and after..

Oh, and this should never have affected messages on -hackers for
example, because it was always processed before ru-general. It would hit
the PUG lists, -www, -patches and a few others.

//Magnus