From: Aleksandr Parfenov <a.parfenov@postgrespro.ru>
To: Dmitry Ivanov <d.ivanov@postgrespro.ru>
Cc: Aleksander Alekseev <a.alekseev@postgrespro.ru>
Cc: Thomas Munro <thomas.munro@enterprisedb.com>
Cc: David Steele <david@pgmasters.net>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Subject: Re: new function for tsquery creartion
Date: Tue, 3 Apr 2018 17:13:20 +0300
Message-ID: <20180403171320.400cd24a@asp437-24-g082ur> (raw)
In-Reply-To: <94e41ae5e36b640d3172729c4c9b5b4c@postgrespro.ru>
References: <fe931111ff7e9ad79196486ada79e268@postgrespro.ru>
<20180302030605.5ur6ktkb2vyb46pb@alap3.anarazel.de>
<152025433949.1548.16466871234054596674.pgcf@coridan.postgresql.org>
<34280367-0dab-7f73-8d7c-d21c7276814b@pgmasters.net>
<8df9de83119eb295f3424c573d64ff84@postgrespro.ru>
<20180326092405.71820cf2@asp437-24-g082ur>
<a1686708fc7d43d9a2de19c1697ac57e@postgrespro.ru>
<CAEepm=3dQmh7NCC_ap-L4yMiVRcZuPXtJUv5_mOnYdoqsnUKvg@mail.gmail.com>
<6754a224654a4f0339195b83b133d551@postgrespro.ru>
<9153fb21831d0ba7c832960f37df2688@postgrespro.ru>
<20180328092115.GA32581@e733.localdomain>
<8829c490d0e17b6033b3fd622dfeed1e@postgrespro.ru>
<cbd6ea6e79b8247765a6c493943a5799@postgrespro.ru>
<b8355fbb948314e0dd4dd2594debc1bf@postgrespro.ru>
<1f33416d7173df5504e84e63680bd27e@postgrespro.ru>
<94e41ae5e36b640d3172729c4c9b5b4c@postgrespro.ru>
On Tue, 03 Apr 2018 14:28:37 +0300
Dmitry Ivanov <d.ivanov@postgrespro.ru> wrote:
> I'm sorry, I totally forgot to fix a few more things, the patch is
> attached below.
The patch looks good to me except two things.
I'm not sure about the different result for these queries:
SELECT websearch_to_tsquery('simple', 'cat or ');
websearch_to_tsquery
----------------------
'cat'
(1 row)
SELECT websearch_to_tsquery('simple', 'cat or');
websearch_to_tsquery
----------------------
'cat' & 'or'
(1 row)
But I don't have strong opinion about these queries, since input in
both of them looks broken in terms of operator usage.
I found an odd behavior of the query creation function in case:
SELECT websearch_to_tsquery('english', '"pg_class pg"');
websearch_to_tsquery
-----------------------------
( 'pg' & 'class' ) <-> 'pg'
(1 row)
This query means that lexemes 'pg' and 'class' should be at the same
distance from the last 'pg'. In other words, they should have the same
position. But default parser will interpret pg_class as two separate
words during text parsing/vector creation.
The bug wasn't introduced in the patch and can be found in current
master. During the discussion of the patch with Dmitry, he noticed that
to_tsquery() function shares same odd behavior:
select to_tsquery('english', ' pg_class <-> pg');
to_tsquery
-----------------------------
( 'pg' & 'class' ) <-> 'pg'
(1 row)
This oddity caused by they implementation of makepol. In makepol, each
token (parsed by query parser) is sent to FTS parser and in case of
further separation of the token, it uses operator selected in functions
to_tsquery and friends. So it doesn't change over the runtime.
I see two different ways to solve it:
1) Use the same operator inside the parenthesizes. This will mean to
parse it as few parts of one word.
2) Remove parenthesizes. This will mean to parse it as few separate
words.
I prefer the second way since in some languages words can be separated
by some special symbol or not separated by any symbols at all and
should be extracted by special FTS parser. It also allows us to parse
such words as one by using the special parser (as it done for hyphenated
word).
But in the example with websearch_to_tsquery, I think it should use
the same operator for quoted part of the query. For example, we can
update the operator in makepol before sending it to pushval
(pushval_morph) to do so.
It looks like there should be two separated patches, one for
websearch_to_tsquery and another one for fixing odd behavior of the
query construction. But since the first one may depend on the
bugfix, to solve case with quotes, I will mark it as Waiting on
Author.
--
Aleksandr Parfenov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: a.parfenov@postgrespro.ru, d.ivanov@postgrespro.ru, a.alekseev@postgrespro.ru, thomas.munro@enterprisedb.com, david@pgmasters.net, pgsql-hackers@lists.postgresql.org
Subject: Re: new function for tsquery creartion
In-Reply-To: <20180403171320.400cd24a@asp437-24-g082ur>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox