pg.ddx.io  pgsql-committers@postgresql.org mailing list archive  
help / color / mirror / Atom feed
pgsql: Fix ts_headline() edge cases for empty query and empty search te
9+ messages / 2 participants
[nested] [flat]

Commits from this thread:
  2023-04-06 5499706bdf d6ac2348b8 1bee3ca238 16188d88ec 027d29edf4 83965ef0bd  Stabilize just-added regression test cases.
* pgsql: Fix ts_headline() edge cases for empty query and empty search te
@ 2023-04-06 19:52 Tom Lane <tgl@sss.pgh.pa.us>
  2023-04-06 21:38 ` Re: pgsql: Fix ts_headline() edge cases for empty query and empty search te Andres Freund <andres@anarazel.de>
  0 siblings, 1 reply; 9+ messages in thread

From: Tom Lane @ 2023-04-06 19:52 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix ts_headline() edge cases for empty query and empty search text.

tsquery's GETQUERY() macro is only safe to apply to a tsquery
that is known non-empty; otherwise it gives a pointer to garbage.
Before commit 5a617d75d, ts_headline() avoided this pitfall, but
only in a very indirect, nonobvious way.  (hlCover could not reach
its TS_execute call, because if the query contains no lexemes
then hlFirstIndex would surely return -1.)  After that commit,
it fell into the trap, resulting in weird errors such as
"unrecognized operator" and/or valgrind complaints.  In HEAD,
fix this by not calling TS_execute_locations() at all for an
empty query.  In the back branches, add a defensive check to
hlCover() --- that's not fixing any live bug, but I judge the
code a bit too fragile as-is.

Also, both mark_hl_fragments() and mark_hl_words() were careless
about the possibility of empty search text: in the cases where
no match has been found, they'd end up telling mark_fragment() to
mark from word indexes 0 to 0 inclusive, even when there is no
word 0.  This is harmless since we over-allocated the prs->words
array, but it does annoy valgrind.  Fix so that the end index is -1
and thus mark_fragment() will do nothing in such cases.

Bottom line is that this fixes a live bug in HEAD, but in the
back branches it's only getting rid of a valgrind nitpick.
Back-patch anyway.

Per report from Alexander Lakhin.

Discussion: https://postgr.es/m/c27f642d-020b-01ff-ae61-086af287c4fd@gmail.com

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/029dea882a7aa34f46732473eed7c917505e6481

Modified Files
--------------
src/backend/tsearch/wparser_def.c     | 21 ++++++++++++++-------
src/test/regress/expected/tsearch.out | 21 +++++++++++++++++++++
src/test/regress/sql/tsearch.sql      |  6 ++++++
3 files changed, 41 insertions(+), 7 deletions(-)



^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: pgsql: Fix ts_headline() edge cases for empty query and empty search te
  2023-04-06 19:52 pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
@ 2023-04-06 21:38 ` Andres Freund <andres@anarazel.de>
  2023-04-06 21:47   ` Re: pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 1 reply; 9+ messages in thread

From: Andres Freund @ 2023-04-06 21:38 UTC (permalink / raw)
  To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-committers@lists.postgresql.org

Hi,

On 2023-04-06 19:52:55 +0000, Tom Lane wrote:
> Fix ts_headline() edge cases for empty query and empty search text.

Unfortunately there appears to be some portability issue with this. Causes a
test output difference on macos. Noticed it when testing something I was about
to push, and I was very confused for a second how my change could have a
portability aspect :)

https://cirrus-ci.com/build/6629557554380800

https://api.cirrus-ci.com/v1/artifact/task/4764833480966144/testrun/build/testrun/regress/regress/re...

inline patch 1: /Users/admin/pgsql/build/testrun/regress/regress/results/tsearch.out (+3/-0 lines) download inline-1.patchdiff -U3 /Users/admin/pgsql/src/test/regress/expected/tsearch.out /Users/admin/pgsql/build/testrun/regress/regress/results/tsearch.out --- /Users/admin/pgsql/src/test/regress/expected/tsearch.out 2023-04-06 20:03:11 +++ /Users/admin/pgsql/build/testrun/regress/regress/results/tsearch.out 2023-04-06 20:05:56 @@ -2133,6 +2133,9 @@ NOTICE: text-search query doesn't contain lexemes: "" LINE 2: '', ''::tsquery); ^ +NOTICE: text-search query doesn't contain lexemes: "" +LINE 2: '', ''::tsquery); + ^ ts_headline -------------
Sure looks like somehow there's duplicate NOTICEs? Greetings, Andres Freund ^ permalink raw reply [nested|flat] 9+ messages in thread

* Re: pgsql: Fix ts_headline() edge cases for empty query and empty search te
  2023-04-06 19:52 pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
  2023-04-06 21:38 ` Re: pgsql: Fix ts_headline() edge cases for empty query and empty search te Andres Freund <andres@anarazel.de>
@ 2023-04-06 21:47   ` Tom Lane <tgl@sss.pgh.pa.us>
  2023-04-06 22:04     ` Re: pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 1 reply; 9+ messages in thread

From: Tom Lane @ 2023-04-06 21:47 UTC (permalink / raw)
  To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-committers@lists.postgresql.org

Andres Freund <andres@anarazel.de> writes:
> On 2023-04-06 19:52:55 +0000, Tom Lane wrote:
>> Fix ts_headline() edge cases for empty query and empty search text.

> Unfortunately there appears to be some portability issue with this. Causes a
> test output difference on macos.

Huh ... looking.

			regards, tom lane





^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* Re: pgsql: Fix ts_headline() edge cases for empty query and empty search te
  2023-04-06 19:52 pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
  2023-04-06 21:38 ` Re: pgsql: Fix ts_headline() edge cases for empty query and empty search te Andres Freund <andres@anarazel.de>
  2023-04-06 21:47   ` Re: pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
@ 2023-04-06 22:04     ` Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 9+ messages in thread

From: Tom Lane @ 2023-04-06 22:04 UTC (permalink / raw)
  To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-committers@lists.postgresql.org

I wrote:
> Andres Freund <andres@anarazel.de> writes:
>> On 2023-04-06 19:52:55 +0000, Tom Lane wrote:
>>> Fix ts_headline() edge cases for empty query and empty search text.

>> Unfortunately there appears to be some portability issue with this. Causes a
>> test output difference on macos.

> Huh ... looking.

It's not about macOS, it's about that build script using
-DRANDOMIZE_ALLOCATED_MEMORY, which causes coerce_type() to
call tsqueryin() twice.  So you get the bleat twice.  Ugh.

I can work around it in these test cases by using to_tsquery()
instead of just "''::tsquery", but maybe we should rethink
something.

			regards, tom lane





^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* pgsql: Fix ts_headline() edge cases for empty query and empty search te
@ 2023-04-06 19:52 Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 9+ messages in thread

From: Tom Lane @ 2023-04-06 19:52 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix ts_headline() edge cases for empty query and empty search text.

tsquery's GETQUERY() macro is only safe to apply to a tsquery
that is known non-empty; otherwise it gives a pointer to garbage.
Before commit 5a617d75d, ts_headline() avoided this pitfall, but
only in a very indirect, nonobvious way.  (hlCover could not reach
its TS_execute call, because if the query contains no lexemes
then hlFirstIndex would surely return -1.)  After that commit,
it fell into the trap, resulting in weird errors such as
"unrecognized operator" and/or valgrind complaints.  In HEAD,
fix this by not calling TS_execute_locations() at all for an
empty query.  In the back branches, add a defensive check to
hlCover() --- that's not fixing any live bug, but I judge the
code a bit too fragile as-is.

Also, both mark_hl_fragments() and mark_hl_words() were careless
about the possibility of empty search text: in the cases where
no match has been found, they'd end up telling mark_fragment() to
mark from word indexes 0 to 0 inclusive, even when there is no
word 0.  This is harmless since we over-allocated the prs->words
array, but it does annoy valgrind.  Fix so that the end index is -1
and thus mark_fragment() will do nothing in such cases.

Bottom line is that this fixes a live bug in HEAD, but in the
back branches it's only getting rid of a valgrind nitpick.
Back-patch anyway.

Per report from Alexander Lakhin.

Discussion: https://postgr.es/m/c27f642d-020b-01ff-ae61-086af287c4fd@gmail.com

Branch
------
REL_15_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/f976a77787ebfd0595a3aa19ac6bb587cc6391e2

Modified Files
--------------
src/backend/tsearch/wparser_def.c     |  8 ++++++--
src/test/regress/expected/tsearch.out | 21 +++++++++++++++++++++
src/test/regress/sql/tsearch.sql      |  6 ++++++
3 files changed, 33 insertions(+), 2 deletions(-)



^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* pgsql: Fix ts_headline() edge cases for empty query and empty search te
@ 2023-04-06 19:52 Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 9+ messages in thread

From: Tom Lane @ 2023-04-06 19:52 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix ts_headline() edge cases for empty query and empty search text.

tsquery's GETQUERY() macro is only safe to apply to a tsquery
that is known non-empty; otherwise it gives a pointer to garbage.
Before commit 5a617d75d, ts_headline() avoided this pitfall, but
only in a very indirect, nonobvious way.  (hlCover could not reach
its TS_execute call, because if the query contains no lexemes
then hlFirstIndex would surely return -1.)  After that commit,
it fell into the trap, resulting in weird errors such as
"unrecognized operator" and/or valgrind complaints.  In HEAD,
fix this by not calling TS_execute_locations() at all for an
empty query.  In the back branches, add a defensive check to
hlCover() --- that's not fixing any live bug, but I judge the
code a bit too fragile as-is.

Also, both mark_hl_fragments() and mark_hl_words() were careless
about the possibility of empty search text: in the cases where
no match has been found, they'd end up telling mark_fragment() to
mark from word indexes 0 to 0 inclusive, even when there is no
word 0.  This is harmless since we over-allocated the prs->words
array, but it does annoy valgrind.  Fix so that the end index is -1
and thus mark_fragment() will do nothing in such cases.

Bottom line is that this fixes a live bug in HEAD, but in the
back branches it's only getting rid of a valgrind nitpick.
Back-patch anyway.

Per report from Alexander Lakhin.

Discussion: https://postgr.es/m/c27f642d-020b-01ff-ae61-086af287c4fd@gmail.com

Branch
------
REL_14_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/34ad3aedb0ce4f36fd55aa916200a68a185bf1ed

Modified Files
--------------
src/backend/tsearch/wparser_def.c     |  8 ++++++--
src/test/regress/expected/tsearch.out | 21 +++++++++++++++++++++
src/test/regress/sql/tsearch.sql      |  6 ++++++
3 files changed, 33 insertions(+), 2 deletions(-)



^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* pgsql: Fix ts_headline() edge cases for empty query and empty search te
@ 2023-04-06 19:52 Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 9+ messages in thread

From: Tom Lane @ 2023-04-06 19:52 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix ts_headline() edge cases for empty query and empty search text.

tsquery's GETQUERY() macro is only safe to apply to a tsquery
that is known non-empty; otherwise it gives a pointer to garbage.
Before commit 5a617d75d, ts_headline() avoided this pitfall, but
only in a very indirect, nonobvious way.  (hlCover could not reach
its TS_execute call, because if the query contains no lexemes
then hlFirstIndex would surely return -1.)  After that commit,
it fell into the trap, resulting in weird errors such as
"unrecognized operator" and/or valgrind complaints.  In HEAD,
fix this by not calling TS_execute_locations() at all for an
empty query.  In the back branches, add a defensive check to
hlCover() --- that's not fixing any live bug, but I judge the
code a bit too fragile as-is.

Also, both mark_hl_fragments() and mark_hl_words() were careless
about the possibility of empty search text: in the cases where
no match has been found, they'd end up telling mark_fragment() to
mark from word indexes 0 to 0 inclusive, even when there is no
word 0.  This is harmless since we over-allocated the prs->words
array, but it does annoy valgrind.  Fix so that the end index is -1
and thus mark_fragment() will do nothing in such cases.

Bottom line is that this fixes a live bug in HEAD, but in the
back branches it's only getting rid of a valgrind nitpick.
Back-patch anyway.

Per report from Alexander Lakhin.

Discussion: https://postgr.es/m/c27f642d-020b-01ff-ae61-086af287c4fd@gmail.com

Branch
------
REL_13_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/b7d6c51d4693765354a1f60e6eb45010bddf023d

Modified Files
--------------
src/backend/tsearch/wparser_def.c     |  8 ++++++--
src/test/regress/expected/tsearch.out | 21 +++++++++++++++++++++
src/test/regress/sql/tsearch.sql      |  6 ++++++
3 files changed, 33 insertions(+), 2 deletions(-)



^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* pgsql: Fix ts_headline() edge cases for empty query and empty search te
@ 2023-04-06 19:52 Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 9+ messages in thread

From: Tom Lane @ 2023-04-06 19:52 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix ts_headline() edge cases for empty query and empty search text.

tsquery's GETQUERY() macro is only safe to apply to a tsquery
that is known non-empty; otherwise it gives a pointer to garbage.
Before commit 5a617d75d, ts_headline() avoided this pitfall, but
only in a very indirect, nonobvious way.  (hlCover could not reach
its TS_execute call, because if the query contains no lexemes
then hlFirstIndex would surely return -1.)  After that commit,
it fell into the trap, resulting in weird errors such as
"unrecognized operator" and/or valgrind complaints.  In HEAD,
fix this by not calling TS_execute_locations() at all for an
empty query.  In the back branches, add a defensive check to
hlCover() --- that's not fixing any live bug, but I judge the
code a bit too fragile as-is.

Also, both mark_hl_fragments() and mark_hl_words() were careless
about the possibility of empty search text: in the cases where
no match has been found, they'd end up telling mark_fragment() to
mark from word indexes 0 to 0 inclusive, even when there is no
word 0.  This is harmless since we over-allocated the prs->words
array, but it does annoy valgrind.  Fix so that the end index is -1
and thus mark_fragment() will do nothing in such cases.

Bottom line is that this fixes a live bug in HEAD, but in the
back branches it's only getting rid of a valgrind nitpick.
Back-patch anyway.

Per report from Alexander Lakhin.

Discussion: https://postgr.es/m/c27f642d-020b-01ff-ae61-086af287c4fd@gmail.com

Branch
------
REL_12_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/a1fb4bd8562e0a7f162ff8a08c3c2ecdeb792f80

Modified Files
--------------
src/backend/tsearch/wparser_def.c     |  8 ++++++--
src/test/regress/expected/tsearch.out | 21 +++++++++++++++++++++
src/test/regress/sql/tsearch.sql      |  6 ++++++
3 files changed, 33 insertions(+), 2 deletions(-)



^ permalink  raw  reply  [nested|flat] 9+ messages in thread

* pgsql: Fix ts_headline() edge cases for empty query and empty search te
@ 2023-04-06 19:52 Tom Lane <tgl@sss.pgh.pa.us>
  0 siblings, 0 replies; 9+ messages in thread

From: Tom Lane @ 2023-04-06 19:52 UTC (permalink / raw)
  To: pgsql-committers@lists.postgresql.org

Fix ts_headline() edge cases for empty query and empty search text.

tsquery's GETQUERY() macro is only safe to apply to a tsquery
that is known non-empty; otherwise it gives a pointer to garbage.
Before commit 5a617d75d, ts_headline() avoided this pitfall, but
only in a very indirect, nonobvious way.  (hlCover could not reach
its TS_execute call, because if the query contains no lexemes
then hlFirstIndex would surely return -1.)  After that commit,
it fell into the trap, resulting in weird errors such as
"unrecognized operator" and/or valgrind complaints.  In HEAD,
fix this by not calling TS_execute_locations() at all for an
empty query.  In the back branches, add a defensive check to
hlCover() --- that's not fixing any live bug, but I judge the
code a bit too fragile as-is.

Also, both mark_hl_fragments() and mark_hl_words() were careless
about the possibility of empty search text: in the cases where
no match has been found, they'd end up telling mark_fragment() to
mark from word indexes 0 to 0 inclusive, even when there is no
word 0.  This is harmless since we over-allocated the prs->words
array, but it does annoy valgrind.  Fix so that the end index is -1
and thus mark_fragment() will do nothing in such cases.

Bottom line is that this fixes a live bug in HEAD, but in the
back branches it's only getting rid of a valgrind nitpick.
Back-patch anyway.

Per report from Alexander Lakhin.

Discussion: https://postgr.es/m/c27f642d-020b-01ff-ae61-086af287c4fd@gmail.com

Branch
------
REL_11_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/bc428b12ac3ac7f4444b5d6f792cdbc3ee3c9388

Modified Files
--------------
src/backend/tsearch/wparser_def.c     |  8 ++++++--
src/test/regress/expected/tsearch.out | 21 +++++++++++++++++++++
src/test/regress/sql/tsearch.sql      |  6 ++++++
3 files changed, 33 insertions(+), 2 deletions(-)



^ permalink  raw  reply  [nested|flat] 9+ messages in thread


end of thread, other threads:[~2023-04-06 22:04 UTC | newest]

Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-04-06 19:52 pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
2023-04-06 21:38 ` Andres Freund <andres@anarazel.de>
2023-04-06 21:47   ` Tom Lane <tgl@sss.pgh.pa.us>
2023-04-06 22:04     ` Tom Lane <tgl@sss.pgh.pa.us>
2023-04-06 19:52 pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
2023-04-06 19:52 pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
2023-04-06 19:52 pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
2023-04-06 19:52 pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
2023-04-06 19:52 pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox