pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: pgsql-committers@lists.postgresql.org
Subject: pgsql: Fix ts_headline() to handle ORs and phrase queries more honestly
Date: Thu, 19 Jan 2023 21:21:52 +0000
Message-ID: <E1pIcM4-004VcH-1M@gemulon.postgresql.org> (raw)

Fix ts_headline() to handle ORs and phrase queries more honestly.

This patch largely reverts what I did in commits c9b0c678d and
78e73e875.  The maximum cover length limit that I added in 78e73e875
(to band-aid over c9b0c678d's performance issues) creates too many
user-visible behavior discrepancies, as complained of for example in
bug #17691.  The real problem with hlCover() is not what I thought
at the time, but more that it seems to have been designed with only
AND tsquery semantics in mind.  It doesn't work quite right for OR,
and even less so for NOT or phrase queries.  However, we can improve
that situation by building a variant of TS_execute() that returns a
list of match locations.  We already get an ExecPhraseData struct
representing match locations for the primitive case of a simple match,
as well as one for a phrase match; we just need to add some logic to
combine these for AND and OR operators.  The result is a list of
ExecPhraseDatas, which hlCover can regard as having simple AND
semantics, so that its old algorithm works correctly.

There's still a lot not to like about ts_headline's behavior, but
I think the remaining issues have to do with the heuristics used
in mark_hl_words and mark_hl_fragments (which, likewise, were not
revisited when phrase search was added).  Improving those is a task
for another day.

Patch by me; thanks to Alvaro Herrera for review.

Discussion: https://postgr.es/m/840.1669405935@sss.pgh.pa.us

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/5a617d75d3b31414f378dd764a11db1a08fa79bb

Modified Files
--------------
src/backend/tsearch/wparser_def.c     | 260 ++++++++++++++++++++++------------
src/backend/utils/adt/tsvector_op.c   | 181 ++++++++++++++++++++++-
src/include/tsearch/ts_utils.h        |   3 +
src/test/regress/expected/tsearch.out | 145 +++++++++++++++++--
src/test/regress/sql/tsearch.sql      |  77 ++++++++++
5 files changed, 560 insertions(+), 106 deletions(-)



Message-ID: <E1pIcM4-004VcH-1M@gemulon.postgresql.org>
Permalink:  ../E1pIcM4-004VcH-1M@gemulon.postgresql.org/
Also on:    postgresql.org/message-id/E1pIcM4-004VcH-1M@gemulon.postgresql.org

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-committers@postgresql.org
  Cc: tgl@sss.pgh.pa.us
  Subject: Re: pgsql: Fix ts_headline() to handle ORs and phrase queries more honestly
  In-Reply-To: <E1pIcM4-004VcH-1M@gemulon.postgresql.org>
  References: <E1pIcM4-004VcH-1M@gemulon.postgresql.org>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox