From: Euler Taveira <euler@eulerto.com>
To: bosamia.karan@gmail.com
To: pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #18580: The pg_similarity appears to be wrong
Date: Thu, 15 Aug 2024 12:13:06 -0300
Message-ID: <f90f08b5-ba6d-4b39-9653-ede0f70a1be9@app.fastmail.com> (raw)
In-Reply-To: <18580-11fe381a9adde140@postgresql.org>
References: <18580-11fe381a9adde140@postgresql.org>
On Mon, Aug 12, 2024, at 6:58 AM, PG Bug reporting form wrote:
> SELECT *
> FROM (
> SELECT
> *,
> similarity(provision_clean_description, 'Policies The General Partner
> shall promptly notify the Investor of any proposed changes in the Funds
> leverage policies including adjustments to leverage ratios') AS sim_tim
> FROM provision_database
> ) pd
> WHERE sim_tim <= 1 and sim_tim > 0.7 and firm_id=18;
>
> This both sentences giving similarity score as 1 despite the fact that the
> sentence 1. has Policies as the starting word(do not include the starting
> hyphen in the sentences):
> - Policies The General Partner shall promptly notify the Investor of any
> proposed changes in the Funds leverage policies including adjustments to
> leverage ratios
> - The General Partner shall promptly notify the Investor of any proposed
> changes in the Funds leverage policies including adjustments to leverage
> ratios
>
This is not a bug.
That's how trigram works. The documentation [1] explains that the words don't
need to be in the same order because it counts the number of common trigrams.
Trigrams are extracted ignoring non-alphanumeric characters. Trigrams are
case-insensitive. You can check the trigrams extracted using the show_trgm()
function.
--
-- return the non-common trigrams
--
WITH a AS (
SELECT x FROM unnest(show_trgm('Policies The General Partner shall promptly
notify the Investor of any proposed changes in the Funds leverage policies
including adjustments to leverage ratios')) x),
b AS (
SELECT x FROM unnest(show_trgm('The General Partner shall promptly notify the
Investor of any proposed changes in the Funds leverage policies including
adjustments to leverage ratios')) x)
SELECT * FROM a FULL JOIN b ON (a.x = b.x) WHERE a.x IS NULL OR b.x IS NULL;
[1] https://www.postgresql.org/docs/current/pgtrgm.html#PGTRGM-CONCEPTS
--
Euler Taveira
EDB https://www.enterprisedb.com/
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-bugs@postgresql.org
Cc: euler@eulerto.com, bosamia.karan@gmail.com, pgsql-bugs@lists.postgresql.org
Subject: Re: BUG #18580: The pg_similarity appears to be wrong
In-Reply-To: <f90f08b5-ba6d-4b39-9653-ede0f70a1be9@app.fastmail.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox