Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1j4jvt-0000wS-RO for pgsql-bugs@arkaria.postgresql.org; Thu, 20 Feb 2020 11:23:54 +0000 Received: from localhost ([127.0.0.1] helo=malur.postgresql.org) by malur.postgresql.org with esmtp (Exim 4.89) (envelope-from ) id 1j4jvs-0006Gu-5i for pgsql-bugs@arkaria.postgresql.org; Thu, 20 Feb 2020 11:23:52 +0000 Received: from makus.postgresql.org ([2001:4800:3e1:1::229]) by malur.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.89) (envelope-from ) id 1j4h5a-0001wr-Vo for pgsql-bugs@lists.postgresql.org; Thu, 20 Feb 2020 08:21:43 +0000 Received: from forward106p.mail.yandex.net ([77.88.28.109]) by makus.postgresql.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1j4h5W-0003w4-OW for pgsql-bugs@lists.postgresql.org; Thu, 20 Feb 2020 08:21:41 +0000 Received: from forward103o.mail.yandex.net (forward103o.mail.yandex.net [37.140.190.177]) by forward106p.mail.yandex.net (Yandex) with ESMTP id 58BDD1C83BE6; Thu, 20 Feb 2020 11:21:31 +0300 (MSK) Received: from mxback7j.mail.yandex.net (mxback7j.mail.yandex.net [IPv6:2a02:6b8:0:1619::110]) by forward103o.mail.yandex.net (Yandex) with ESMTP id BBE105F81292; Thu, 20 Feb 2020 11:21:30 +0300 (MSK) Received: from sas1-5ebd8269dbc4.qloud-c.yandex.net (sas1-5ebd8269dbc4.qloud-c.yandex.net [2a02:6b8:c14:3611:0:640:5ebd:8269]) by mxback7j.mail.yandex.net (mxback/Yandex) with ESMTP id r7nAFC2MRg-LUq09slD; Thu, 20 Feb 2020 11:21:30 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1582186890; bh=bvIxvV8Qxmx5w/T7JEQzIyWgEWWA+Z4M94DiEYRlEx4=; h=In-Reply-To:Subject:To:From:References:Date:Message-ID; b=bsW5majkqG8FRkYO1i3jbTWhHVK1Z5B160k3RQiquwYQcmYEt8lqTjMenRx1+9J5N MI+CSncH86ODHOFKashv8mQG1w4lnRhVnI6UFREhBPltWOJ8o1i1mdgoXX8SUWvjVL ycSfx51CnmQIggC+uDAaGyguGniUfPQBBSdCFMkg= Authentication-Results: mxback7j.mail.yandex.net; dkim=pass header.i=@yandex.ru Received: by sas1-5ebd8269dbc4.qloud-c.yandex.net (smtp/Yandex) with ESMTPSA id USI9tm4WqN-LTBetHXk; Thu, 20 Feb 2020 11:21:29 +0300 (using TLSv1.1 with cipher ECDHE-RSA-AES128-SHA (128/128 bits)) (Client certificate not present) Date: Thu, 20 Feb 2020 11:21:24 +0300 From: egocenter Message-ID: <9210322148.20200220112124@yandex.ru> To: Artur Zakirov , pgsql-bugs@lists.postgresql.org Subject: Re: Full text search bug ('russian' regconfig) In-Reply-To: <0f991eaf-2394-41a2-9d8e-c36aef35fbb1@gmail.com> References: <1453547361.20200219113508@yandex.ru> <0f991eaf-2394-41a2-9d8e-c36aef35fbb1@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: quoted-printable List-Id: List-Help: List-Subscribe: List-Post: List-Owner: List-Archive: Precedence: bulk =0D=0AHello, Artur! Thanks for the answer, ok, it's strange that only 1 word is affected that way (as if two lexemes e= xist for 1 word)... *I use double to_tsvector to eliminate words duplicates. in the example below ts_title =3D 'histori':2 'watcom':1,3 and it gives 2 entries in '=E3=EE=F0=EE=E4 - watcom' via ts_rank_cd I need to count UNIQUE words entries but it seems to be no luck with std fu= nctionality (I see 2 ways: custom ts_rank function OR to_tsvector / edit tsvector and l= eave only first position for 'watcom': ts_title =3D 'histori':2 'watcom':1). If you have any idea on that situation, I would highly appreciate it! Thank= s in advance) --------- SELECT round((ts_rank_cd(ts_title, web_query_or)/0.1)::NUMERIC, 0) AS titl= e_entries_count, -- 2, but should be 1=20 * FROM (SELECT to_tsvector('russian', 'watcom history | watcom') AS ts_title, websearch_to_tsquery('russian', REPLACE('=E3=EE=F0=EE=E4 - watcom', '-= ' , '')) AS web_query_and, -- =F2=E8=F0=E5 =E7=E0=EC=E5=ED=E5=ED=EE =E4=EB= =FF =EE=F2=EC=E5=ED=FB =E5=E3=EE =EA=EE=ED=E2=E5=F0=F2=E0=F6=E8=E8 =E2 =EE= =F2=F0=E8=F6=E0=ED=E8=E5 ! REPLACE(websearch_to_tsquery(:reg_config, REPLACE('=E3=EE=F0=EE=E4 - w= atcom', '- ' , ''))::TEXT, '&', '|')::tsquery AS web_query_or=20 ) AS main; --=20 > Hello > On 2/19/2020 5:35 PM, egocenter wrote: >> Text search doesn't work correct with the EQUAL string in text and query= (russian dictionary config), >> as you can see in example ts_vector receives different from ts_query lex= emes for identical text: >>=20 >> tsv =3D '=E4=E0=ED':1 '=EC=E0=E3=E0=E7=E8=ED':2 '=ED=F3=E6=ED':3 '=EF=EE= =F1=E5=F9=E0':4 '=F2=EE=F7=ED':5 >> tsq =3D '=ED=F3=E6=ED' & '=F2=EE=F7=ED' & '=E4=E0=ED' & '=EF=EE=F1=E5=F9= =E0=E5=EC' & '=EC=E0=E3=E0=E7=E8=ED' > It is because you call to_tsvector() two times. 'russian' is a Snowball > dictionary and it uses stemming algorithms to cut words ending. Your=20 > query works if to_tsvector() isn't called twice on the same text: > =3D# SELECT > web_query_and @@ ts_title, > web_query_and @@ '=E7=E0=F7=E5=EC =ED=F3=E6=ED=FB =F2=EE=F7=ED=FB=E5 = =E4=E0=ED=ED=FB=E5 =EE =EF=EE=F1=E5=F9=E0=E5=EC=EE=F1=F2=E8 =EC=E0=E3=E0=E7= =E8=ED=EE=E2', > * > FROM > (SELECT > to_tsvector('russian', '=E7=E0=F7=E5=EC =ED=F3=E6=ED=FB =F2=EE=F7=ED= =FB=E5 =E4=E0=ED=ED=FB=E5 =EE =EF=EE=F1=E5=F9=E0=E5=EC=EE=F1=F2=E8=20 > =EC=E0=E3=E0=E7=E8=ED=EE=E2') AS ts_title, > websearch_to_tsquery('russian', '=E7=E0=F7=E5=EC =ED=F3=E6=ED=FB =F2= =EE=F7=ED=FB=E5 =E4=E0=ED=ED=FB=E5 =EE=20 > =EF=EE=F1=E5=F9=E0=E5=EC=EE=F1=F2=E8 =EC=E0=E3=E0=E7=E8=ED=EE=E2?') AS we= b_query_and > ) AS main; > It gives 'true' for the first column.