MIME-Version: 1.0
References: 
 <CALDQ5NxzgeXHRCD4dS_6qz+nn01ivi3i1ZEtD2DmC779i0=iSQ@mail.gmail.com>
 <ab4184b7ab84623be10c4676e090cc27ae78b355.camel@cybertec.at>
 <Ye79wNIXsyhwwwce@momjian.us> <2150096.1643057249@sss.pgh.pa.us>
 <CAF4Au4yttKJ1KAP-cO+HMLQ2_66vmx0dLTBUbE4W8Aa64foafg@mail.gmail.com>
 <22d5245c9c5a9aa05a0510bdd52458812140a870.camel@cybertec.at>
 <2257661.1643127753@sss.pgh.pa.us>
 <79b3eb6e-152e-3c56-7b71-51d091c0f6d9@postgrespro.ru>
 <2274255.1643133268@sss.pgh.pa.us>
 <CALDQ5NzFfKCDvmbr6otF+ePH=oijN3xBeqjMen4boitUppTMBA@mail.gmail.com>
 <a73f39bc-94f9-e8c6-9428-9ce94b33a4a7@postgrespro.ru>
In-Reply-To: <a73f39bc-94f9-e8c6-9428-9ce94b33a4a7@postgrespro.ru>
From: James Addison <jay@jp-hosting.net>
Date: Wed, 26 Jan 2022 08:28:43 +0000
Message-ID: 
 <CALDQ5NwjHE6jjmxVPSq00FbTiVVKcb9+fX7nMnrRXtHNZGt+2g@mail.gmail.com>
Subject: Re: Mailing list search engine: surprising missing results?
To: Ivan Panchenko <i.panchenko@postgrespro.ru>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, pgsql-www@lists.postgresql.org
Content-Type: text/plain; charset="UTF-8"
Archived-At: 
 <https://www.postgresql.org/message-id/CALDQ5NwjHE6jjmxVPSq00FbTiVVKcb9%2BfX7nMnrRXtHNZGt%2B2g%40mail.gmail.com>
Precedence: bulk

On Tue, 25 Jan 2022 at 21:23, Ivan Panchenko <i.panchenko@postgrespro.ru> wrote:
>
> On 25.01.2022 23:48, James Addison wrote:
> > I'm uncertain why parsing hyphenated query text produces compound tokens?
>
> Because in some cases user wants to search the full hyphenated words,
> not parts of them.

That makes sense, although to refer back to a previous suggestion of
yours, we could allow matching on the full hyphenated words by
emitting an 'OR' condition from the parsed query, instead of 'AND'
(perhaps using an argument?).

In other words:

# expected query to achieve a match (from your previous post in this thread)
'boyers-moore' | ('boyers' & 'moore')

# actual query that does not result in a match today (plainto_tsquery
for 'boyer-moore')
'boyer-moore' & 'boyer' & 'moore'

> >> It seems to me that in both cases we'd be better off generating
> >> "'boyers' <-> 'moore'", without the compound token at all.
> >> Maybe there's a case for the weaker 'boyers' & 'moore' translation,
> >> but I think if people wanted that they'd just enter separate words.
>
> Matching the compond token might be significant for ranking. (?)

Yes that does seem likely.  The knowledge that there is an exact-match
token in the results could be important for various use cases
(including relevance scoring).

> Probably, there is no universal *to_tsquery function and no universal
> parser to fit all users.

That seems possible too, yep.