From: Tom Lane <tgl@sss.pgh.pa.us>
To: Pavel Borisov <pashkin.elfe@gmail.com>
cc: mnelson@binarykeep.com,
        PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>
Subject: Re: Text search prefix matching and stop words
In-reply-to: 
 <CALT9ZEG-i0prBw5N7pMAPqL_Kj=g_xK-oKjumE6-q0TVvOfB4A@mail.gmail.com>
References: <039b712e-e02f-4470-b6b4-7f810cc1bcb5@www.fastmail.com>
 <CALT9ZEG-i0prBw5N7pMAPqL_Kj=g_xK-oKjumE6-q0TVvOfB4A@mail.gmail.com>
Comments: In-reply-to Pavel Borisov <pashkin.elfe@gmail.com>
	message dated "Sat, 09 Oct 2021 00:30:41 +0400"
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-ID: <8754.1633727187.1@sss.pgh.pa.us>
Date: Fri, 08 Oct 2021 17:06:27 -0400
Message-ID: <8755.1633727187@sss.pgh.pa.us>
Archived-At: 
 <https://www.postgresql.org/message-id/8755.1633727187%40sss.pgh.pa.us>
Precedence: bulk

Pavel Borisov <pashkin.elfe@gmail.com> writes:
>> Prefix matching should not omit stop words, as matching lexemes may
>> legitimately begin with stop words.

> I am not sure that it is a bug. I think this is a way how to_tsquery
> conversion work: stopwords first then template processing.

I concur with the OP that this is a bug, or at least that it'd be nice
if it worked better.  But I'm not sure we can make it better.  The basic
design of our text search stuff combined the functions of normalization
and stop-word-suppression into a single dictionary stack, so that it's
impossible to ask for just one of those to happen.  But if we skip
applying the dictionaries at all for a prefix item, then word
normalization doesn't happen, which would create a different set of
unexpected-failure-to-match conditions.  (So your proposed workaround
of casting directly to tsquery just moves the problem somewhere else.)

I think we could only fix this with a dictionary API change that
allows telling the dictionaries not to suppress stopwords.  Not
sure how practical that is.  If we'd had the prefix-match feature
from the beginning, maybe it'd have occurred to us that we needed
that API option ... but we didn't.

			regards, tom lane