Received: from localhost (unknown [200.46.204.183]) by developer.postgresql.org (Postfix) with ESMTP id 740A72E004D for ; Wed, 18 Jun 2008 09:49:57 -0300 (ADT) Received: from developer.postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.183]) (amavisd-maia, port 10024) with ESMTP id 23861-04 for ; Wed, 18 Jun 2008 09:49:45 -0300 (ADT) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from fk-out-0910.google.com (fk-out-0910.google.com [209.85.128.190]) by developer.postgresql.org (Postfix) with ESMTP id 95D7B2E0037 for ; Wed, 18 Jun 2008 09:49:50 -0300 (ADT) Received: by fk-out-0910.google.com with SMTP id e30so362905fke.3 for ; Wed, 18 Jun 2008 05:49:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to :subject:mime-version:content-type; bh=LlJNnrBYIfm458dwqeFdvnzXRJrdnSp3arKYDgx4C5s=; b=euDvfNsni8WBWrryq7MtssIML5VydjxEXLtpbF1StwHRHlanvDkrtKtRZJJl8j22LW FwwDQc5V49/UGaaeHUcLY1h4XwvRhB5N8WyxmcAfuGgQ5UwK8FHnWm4bJd3vp7StjotA rPM5jrQVsEQWXkkEpXOP/e4Hn8VxxDXmkqEis= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type; b=LGs8KeDHviQWrlgRmwNeadwyiYG7y2b2/dlbg/nbhBoQrYNYj+f83nKUPZK7fXXuWS X70Wx1f6fDW+Yy4TczSPXtqoSFX3LU9DdtQScwrsXdbp8Mmrssw8nncp67oyACyz7ZGU orM1XbcFk9CQV37647QKLDQXsIp9MctchwkyE= Received: by 10.82.156.10 with SMTP id d10mr33191bue.72.1213793388817; Wed, 18 Jun 2008 05:49:48 -0700 (PDT) Received: by 10.82.162.16 with HTTP; Wed, 18 Jun 2008 05:49:48 -0700 (PDT) Message-ID: <269b27950806180549k323833c7n38a0d9f434542bf2@mail.gmail.com> Date: Wed, 18 Jun 2008 14:49:48 +0200 From: Sabbiolina To: pgsql-admin@postgresql.org Subject: full-text search question MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_505_19418740.1213793388850" X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=0.001 tagged_above=0 required=5 tests=HTML_MESSAGE=0.001 X-Spam-Level: X-Archive-Number: 200806/137 X-Sequence-Number: 29475 ------=_Part_505_19418740.1213793388850 Content-Type: text/plain; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Hello, I've seen that the default parser for the full-text search can identify e-mail addresses, hosts, URLs=85 but I have a serious problem with it: Suppose I index the following sentence "the search engine I use the most is www.google.com" And I search "google" no result is found. Instead if I search "www.google.com" the record is found correctly. I guess the reason is because the parser treats www.google.com as a single token (of type 'host') but as everyone can easily see the result of this is a major problem. In fact the word "google" actually is in the above sentence, and the end-user of the database obviously asks me "why does your FTS not find that record when I can clearly see that my search term is there?" Reading the docs I've seen that the parser can produce multiple tokens for the same word (for example the word "make-up" produces 4 tokens: make-up, make, -, up)=85 why not doing the same with URLs and e-mails? Why www.google.com is only treated as a unique word? Why not producing multiple tokens like www.google.com, www, ., google, ., com? (obviously www and . ca= n be nulled or stopworded). Does anybody know of a better parser for Postgres? Or at least a trick to make its FTS find the record above by searching only a part of the URL? ------=_Part_505_19418740.1213793388850 Content-Type: text/html; charset=WINDOWS-1252 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline

Hello,

 

I've seen that the default pars= er for the full-text search can identify e-mail addresses, hosts, URLs=85 but I have a serious problem with it:

 

Suppose I index the following s= entence "the search engine I use the most is www.goog= le.com"

 

And I search "google" no result is found.

Instead if I search "www.google.com" the record is found correctly.=

 

I guess the reason is because t= he parser treats www.google.com as a single to= ken (of type 'host') but as everyone can easily see the result of this is a major problem. In fact the word "google" actually is in the above sentence, and the end-user of the database obviously asks me "why does your FTS not find that record when I can clearly see that my search term is there?"

 

Reading the docs I've seen that= the parser can produce multiple tokens for the same word (for example the word "make-u= p" produces 4 tokens: make-up, make, -, up)=85 why not doing the same with URLs and e-mails? Why www.google.com= is only treated as a unique word? Why not producing multiple tokens like www.google.com, www, ., google, ., com? (obviously www and . can be nulled or stopworded).

 

Does anybody know of a better parser for Postgres? Or at least a trick to make its FTS find the record above by sear= ching only a part of the URL? ------=_Part_505_19418740.1213793388850--