From: Tom Lane <tgl@sss.pgh.pa.us>
To: Laurenz Albe <laurenz.albe@cybertec.at>
cc: Oleg Bartunov <obartunov@postgrespro.ru>,
 Bruce Momjian <bruce@momjian.us>,
        James Addison <jay@jp-hosting.net>,
        PostgreSQL WWW <pgsql-www@postgresql.org>
Subject: Re: Mailing list search engine: surprising missing results?
In-reply-to: <22d5245c9c5a9aa05a0510bdd52458812140a870.camel@cybertec.at>
References: 
 <CALDQ5NxzgeXHRCD4dS_6qz+nn01ivi3i1ZEtD2DmC779i0=iSQ@mail.gmail.com>
 <ab4184b7ab84623be10c4676e090cc27ae78b355.camel@cybertec.at>
 <Ye79wNIXsyhwwwce@momjian.us> <2150096.1643057249@sss.pgh.pa.us>
 <CAF4Au4yttKJ1KAP-cO+HMLQ2_66vmx0dLTBUbE4W8Aa64foafg@mail.gmail.com>
 <22d5245c9c5a9aa05a0510bdd52458812140a870.camel@cybertec.at>
Comments: In-reply-to Laurenz Albe <laurenz.albe@cybertec.at>
	message dated "Tue, 25 Jan 2022 13:43:48 +0100"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <2257660.1643127753.1@sss.pgh.pa.us>
Date: Tue, 25 Jan 2022 11:22:33 -0500
Message-ID: <2257661.1643127753@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/2257661.1643127753%40sss.pgh.pa.us>
Precedence: bulk

Laurenz Albe <laurenz.albe@cybertec.at> writes:
> On Tue, 2022-01-25 at 14:04 +0300, Oleg Bartunov wrote:
>> On Mon, Jan 24, 2022 at 11:47 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> Bruce Momjian <bruce@momjian.us> writes:
>>>> On Mon, Jan 24, 2022 at 08:27:41AM +0100, Laurenz Albe wrote:
>>>>> The reason is that the 'moore' in 'boyer-moore' is stemmed, since it
>>>>> is at the end of the word, while the 'moore' in 'Boyer-Moore-Horspool'
>>>>> isn't:

> Not quite.  The problem is question is the "'boyer-moore':1".
> If that were "'boyer-moor':1" instead, the problem would disappear.

Actually, when I try this here, it seems like the stemming *is*
consistent:

regression=# SELECT to_tsvector('english', 'Boyer-Moore-Horspool');
                       to_tsvector                        
----------------------------------------------------------
 'boyer':2 'boyer-moore-horspool':1 'horspool':4 'moor':3
(1 row)

regression=# SELECT to_tsvector('english', 'Boyer-Moore');
            to_tsvector            
-----------------------------------
 'boyer':2 'boyer-moor':1 'moor':3
(1 row)

If you try variants of that where the first or third term is stemmable,
say

regression=# SELECT to_tsvector('english', 'Boyers-Moore-Horspool');
                        to_tsvector                        
-----------------------------------------------------------
 'boyer':2 'boyers-moore-horspool':1 'horspool':4 'moor':3
(1 row)

it sure appears that each component word is stemmed independently
already.  So I think the original explanation here is wrong and
we need to probe more closely.

			regards, tom lane