Message-ID: <44F44EF5.3090102@commandprompt.com>
Date: Tue, 29 Aug 2006 07:28:05 -0700
From: "Joshua D. Drake" <jd@commandprompt.com>
Organization: Command Prompt, Inc.
User-Agent: Thunderbird 1.5.0.5 (X11/20060728)
MIME-Version: 1.0
To: Dave Page <dpage@vale-housing.co.uk>
CC: PostgreSQL WWW <pgsql-www@postgresql.org>
Subject: Re: A counter productive conversation about search.
References: 
 <E7F85A1B5FF8D44C8A1AF6885BC9A0E40154C88D@ratbert.vale-housing.co.uk>
In-Reply-To: 
 <E7F85A1B5FF8D44C8A1AF6885BC9A0E40154C88D@ratbert.vale-housing.co.uk>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit


>> Other options include lucene, and rolling our own.
> 
> Is Lucene capable of handling the size of our index? This has always

I am going to say, "yes" without any actual knowledge because of Lucene 
but that is because I am putting more trust in the fact that it is an 
Apache project then anything. I will check.

> been the problem we've had with other projects like MnogoSearch. They
> work well until you load them up with the archives after which they
> simply can't cope without ridiculous amounts of hardware.
> 
>> Rolling our own really wouldn't be that hard "if" we can create a 
>> reasonably smart web page grabber. We have all the tools 
>> (tsearch2 and 
>> pg_pgtrm) to easily do the searches.
>>
>> So is anyone up for helping develop a page grabber?
> 
> We have one - it builds the static version of the main site by spidering
> it hourly.

Should we look at that then?

> 
> Regards, Dave.
> 


-- 

    === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 || 24x7/Emergency: +1.800.492.2240
    Providing the most comprehensive  PostgreSQL solutions since 1997
              http://www.commandprompt.com/