pg.ddx.io pgsql-committers@postgresql.org mailing list archive
help / color / mirror / Atom feedpgsql: Fix ts_headline() edge cases for empty query and empty search te
9+ messages / 2 participants
[nested] [flat]
Commits from this thread:
2023-04-06 5499706bdf d6ac2348b8 1bee3ca238 16188d88ec 027d29edf4 83965ef0bd Stabilize just-added regression test cases.
* pgsql: Fix ts_headline() edge cases for empty query and empty search te
@ 2023-04-06 19:52 Tom Lane <tgl@sss.pgh.pa.us>
2023-04-06 21:38 ` Re: pgsql: Fix ts_headline() edge cases for empty query and empty search te Andres Freund <andres@anarazel.de>
0 siblings, 1 reply; 9+ messages in thread
From: Tom Lane @ 2023-04-06 19:52 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Fix ts_headline() edge cases for empty query and empty search text.
tsquery's GETQUERY() macro is only safe to apply to a tsquery
that is known non-empty; otherwise it gives a pointer to garbage.
Before commit 5a617d75d, ts_headline() avoided this pitfall, but
only in a very indirect, nonobvious way. (hlCover could not reach
its TS_execute call, because if the query contains no lexemes
then hlFirstIndex would surely return -1.) After that commit,
it fell into the trap, resulting in weird errors such as
"unrecognized operator" and/or valgrind complaints. In HEAD,
fix this by not calling TS_execute_locations() at all for an
empty query. In the back branches, add a defensive check to
hlCover() --- that's not fixing any live bug, but I judge the
code a bit too fragile as-is.
Also, both mark_hl_fragments() and mark_hl_words() were careless
about the possibility of empty search text: in the cases where
no match has been found, they'd end up telling mark_fragment() to
mark from word indexes 0 to 0 inclusive, even when there is no
word 0. This is harmless since we over-allocated the prs->words
array, but it does annoy valgrind. Fix so that the end index is -1
and thus mark_fragment() will do nothing in such cases.
Bottom line is that this fixes a live bug in HEAD, but in the
back branches it's only getting rid of a valgrind nitpick.
Back-patch anyway.
Per report from Alexander Lakhin.
Discussion: https://postgr.es/m/c27f642d-020b-01ff-ae61-086af287c4fd@gmail.com
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/029dea882a7aa34f46732473eed7c917505e6481
Modified Files
--------------
src/backend/tsearch/wparser_def.c | 21 ++++++++++++++-------
src/test/regress/expected/tsearch.out | 21 +++++++++++++++++++++
src/test/regress/sql/tsearch.sql | 6 ++++++
3 files changed, 41 insertions(+), 7 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Fix ts_headline() edge cases for empty query and empty search te
2023-04-06 19:52 pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
@ 2023-04-06 21:38 ` Andres Freund <andres@anarazel.de>
2023-04-06 21:47 ` Re: pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 9+ messages in thread
From: Andres Freund @ 2023-04-06 21:38 UTC (permalink / raw)
To: Tom Lane <tgl@sss.pgh.pa.us>; +Cc: pgsql-committers@lists.postgresql.org
Hi,
On 2023-04-06 19:52:55 +0000, Tom Lane wrote:
> Fix ts_headline() edge cases for empty query and empty search text.
Unfortunately there appears to be some portability issue with this. Causes a
test output difference on macos. Noticed it when testing something I was about
to push, and I was very confused for a second how my change could have a
portability aspect :)
https://cirrus-ci.com/build/6629557554380800
https://api.cirrus-ci.com/v1/artifact/task/4764833480966144/testrun/build/testrun/regress/regress/re...
inline patch 1: /Users/admin/pgsql/build/testrun/regress/regress/results/tsearch.out (+3/-0 lines) download inline-1.patch
diff -U3 /Users/admin/pgsql/src/test/regress/expected/tsearch.out /Users/admin/pgsql/build/testrun/regress/regress/results/tsearch.out
--- /Users/admin/pgsql/src/test/regress/expected/tsearch.out 2023-04-06 20:03:11
+++ /Users/admin/pgsql/build/testrun/regress/regress/results/tsearch.out 2023-04-06 20:05:56
@@ -2133,6 +2133,9 @@
NOTICE: text-search query doesn't contain lexemes: ""
LINE 2: '', ''::tsquery);
^
+NOTICE: text-search query doesn't contain lexemes: ""
+LINE 2: '', ''::tsquery);
+ ^
ts_headline
-------------
Sure looks like somehow there's duplicate NOTICEs?
Greetings,
Andres Freund
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Fix ts_headline() edge cases for empty query and empty search te
2023-04-06 19:52 pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
2023-04-06 21:38 ` Re: pgsql: Fix ts_headline() edge cases for empty query and empty search te Andres Freund <andres@anarazel.de>
@ 2023-04-06 21:47 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-04-06 22:04 ` Re: pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 9+ messages in thread
From: Tom Lane @ 2023-04-06 21:47 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-committers@lists.postgresql.org
Andres Freund <andres@anarazel.de> writes:
> On 2023-04-06 19:52:55 +0000, Tom Lane wrote:
>> Fix ts_headline() edge cases for empty query and empty search text.
> Unfortunately there appears to be some portability issue with this. Causes a
> test output difference on macos.
Huh ... looking.
regards, tom lane
^ permalink raw reply [nested|flat] 9+ messages in thread
* Re: pgsql: Fix ts_headline() edge cases for empty query and empty search te
2023-04-06 19:52 pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
2023-04-06 21:38 ` Re: pgsql: Fix ts_headline() edge cases for empty query and empty search te Andres Freund <andres@anarazel.de>
2023-04-06 21:47 ` Re: pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
@ 2023-04-06 22:04 ` Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 9+ messages in thread
From: Tom Lane @ 2023-04-06 22:04 UTC (permalink / raw)
To: Andres Freund <andres@anarazel.de>; +Cc: pgsql-committers@lists.postgresql.org
I wrote:
> Andres Freund <andres@anarazel.de> writes:
>> On 2023-04-06 19:52:55 +0000, Tom Lane wrote:
>>> Fix ts_headline() edge cases for empty query and empty search text.
>> Unfortunately there appears to be some portability issue with this. Causes a
>> test output difference on macos.
> Huh ... looking.
It's not about macOS, it's about that build script using
-DRANDOMIZE_ALLOCATED_MEMORY, which causes coerce_type() to
call tsqueryin() twice. So you get the bleat twice. Ugh.
I can work around it in these test cases by using to_tsquery()
instead of just "''::tsquery", but maybe we should rethink
something.
regards, tom lane
^ permalink raw reply [nested|flat] 9+ messages in thread
* pgsql: Fix ts_headline() edge cases for empty query and empty search te
@ 2023-04-06 19:52 Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 9+ messages in thread
From: Tom Lane @ 2023-04-06 19:52 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Fix ts_headline() edge cases for empty query and empty search text.
tsquery's GETQUERY() macro is only safe to apply to a tsquery
that is known non-empty; otherwise it gives a pointer to garbage.
Before commit 5a617d75d, ts_headline() avoided this pitfall, but
only in a very indirect, nonobvious way. (hlCover could not reach
its TS_execute call, because if the query contains no lexemes
then hlFirstIndex would surely return -1.) After that commit,
it fell into the trap, resulting in weird errors such as
"unrecognized operator" and/or valgrind complaints. In HEAD,
fix this by not calling TS_execute_locations() at all for an
empty query. In the back branches, add a defensive check to
hlCover() --- that's not fixing any live bug, but I judge the
code a bit too fragile as-is.
Also, both mark_hl_fragments() and mark_hl_words() were careless
about the possibility of empty search text: in the cases where
no match has been found, they'd end up telling mark_fragment() to
mark from word indexes 0 to 0 inclusive, even when there is no
word 0. This is harmless since we over-allocated the prs->words
array, but it does annoy valgrind. Fix so that the end index is -1
and thus mark_fragment() will do nothing in such cases.
Bottom line is that this fixes a live bug in HEAD, but in the
back branches it's only getting rid of a valgrind nitpick.
Back-patch anyway.
Per report from Alexander Lakhin.
Discussion: https://postgr.es/m/c27f642d-020b-01ff-ae61-086af287c4fd@gmail.com
Branch
------
REL_15_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/f976a77787ebfd0595a3aa19ac6bb587cc6391e2
Modified Files
--------------
src/backend/tsearch/wparser_def.c | 8 ++++++--
src/test/regress/expected/tsearch.out | 21 +++++++++++++++++++++
src/test/regress/sql/tsearch.sql | 6 ++++++
3 files changed, 33 insertions(+), 2 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
* pgsql: Fix ts_headline() edge cases for empty query and empty search te
@ 2023-04-06 19:52 Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 9+ messages in thread
From: Tom Lane @ 2023-04-06 19:52 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Fix ts_headline() edge cases for empty query and empty search text.
tsquery's GETQUERY() macro is only safe to apply to a tsquery
that is known non-empty; otherwise it gives a pointer to garbage.
Before commit 5a617d75d, ts_headline() avoided this pitfall, but
only in a very indirect, nonobvious way. (hlCover could not reach
its TS_execute call, because if the query contains no lexemes
then hlFirstIndex would surely return -1.) After that commit,
it fell into the trap, resulting in weird errors such as
"unrecognized operator" and/or valgrind complaints. In HEAD,
fix this by not calling TS_execute_locations() at all for an
empty query. In the back branches, add a defensive check to
hlCover() --- that's not fixing any live bug, but I judge the
code a bit too fragile as-is.
Also, both mark_hl_fragments() and mark_hl_words() were careless
about the possibility of empty search text: in the cases where
no match has been found, they'd end up telling mark_fragment() to
mark from word indexes 0 to 0 inclusive, even when there is no
word 0. This is harmless since we over-allocated the prs->words
array, but it does annoy valgrind. Fix so that the end index is -1
and thus mark_fragment() will do nothing in such cases.
Bottom line is that this fixes a live bug in HEAD, but in the
back branches it's only getting rid of a valgrind nitpick.
Back-patch anyway.
Per report from Alexander Lakhin.
Discussion: https://postgr.es/m/c27f642d-020b-01ff-ae61-086af287c4fd@gmail.com
Branch
------
REL_14_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/34ad3aedb0ce4f36fd55aa916200a68a185bf1ed
Modified Files
--------------
src/backend/tsearch/wparser_def.c | 8 ++++++--
src/test/regress/expected/tsearch.out | 21 +++++++++++++++++++++
src/test/regress/sql/tsearch.sql | 6 ++++++
3 files changed, 33 insertions(+), 2 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
* pgsql: Fix ts_headline() edge cases for empty query and empty search te
@ 2023-04-06 19:52 Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 9+ messages in thread
From: Tom Lane @ 2023-04-06 19:52 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Fix ts_headline() edge cases for empty query and empty search text.
tsquery's GETQUERY() macro is only safe to apply to a tsquery
that is known non-empty; otherwise it gives a pointer to garbage.
Before commit 5a617d75d, ts_headline() avoided this pitfall, but
only in a very indirect, nonobvious way. (hlCover could not reach
its TS_execute call, because if the query contains no lexemes
then hlFirstIndex would surely return -1.) After that commit,
it fell into the trap, resulting in weird errors such as
"unrecognized operator" and/or valgrind complaints. In HEAD,
fix this by not calling TS_execute_locations() at all for an
empty query. In the back branches, add a defensive check to
hlCover() --- that's not fixing any live bug, but I judge the
code a bit too fragile as-is.
Also, both mark_hl_fragments() and mark_hl_words() were careless
about the possibility of empty search text: in the cases where
no match has been found, they'd end up telling mark_fragment() to
mark from word indexes 0 to 0 inclusive, even when there is no
word 0. This is harmless since we over-allocated the prs->words
array, but it does annoy valgrind. Fix so that the end index is -1
and thus mark_fragment() will do nothing in such cases.
Bottom line is that this fixes a live bug in HEAD, but in the
back branches it's only getting rid of a valgrind nitpick.
Back-patch anyway.
Per report from Alexander Lakhin.
Discussion: https://postgr.es/m/c27f642d-020b-01ff-ae61-086af287c4fd@gmail.com
Branch
------
REL_13_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/b7d6c51d4693765354a1f60e6eb45010bddf023d
Modified Files
--------------
src/backend/tsearch/wparser_def.c | 8 ++++++--
src/test/regress/expected/tsearch.out | 21 +++++++++++++++++++++
src/test/regress/sql/tsearch.sql | 6 ++++++
3 files changed, 33 insertions(+), 2 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
* pgsql: Fix ts_headline() edge cases for empty query and empty search te
@ 2023-04-06 19:52 Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 9+ messages in thread
From: Tom Lane @ 2023-04-06 19:52 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Fix ts_headline() edge cases for empty query and empty search text.
tsquery's GETQUERY() macro is only safe to apply to a tsquery
that is known non-empty; otherwise it gives a pointer to garbage.
Before commit 5a617d75d, ts_headline() avoided this pitfall, but
only in a very indirect, nonobvious way. (hlCover could not reach
its TS_execute call, because if the query contains no lexemes
then hlFirstIndex would surely return -1.) After that commit,
it fell into the trap, resulting in weird errors such as
"unrecognized operator" and/or valgrind complaints. In HEAD,
fix this by not calling TS_execute_locations() at all for an
empty query. In the back branches, add a defensive check to
hlCover() --- that's not fixing any live bug, but I judge the
code a bit too fragile as-is.
Also, both mark_hl_fragments() and mark_hl_words() were careless
about the possibility of empty search text: in the cases where
no match has been found, they'd end up telling mark_fragment() to
mark from word indexes 0 to 0 inclusive, even when there is no
word 0. This is harmless since we over-allocated the prs->words
array, but it does annoy valgrind. Fix so that the end index is -1
and thus mark_fragment() will do nothing in such cases.
Bottom line is that this fixes a live bug in HEAD, but in the
back branches it's only getting rid of a valgrind nitpick.
Back-patch anyway.
Per report from Alexander Lakhin.
Discussion: https://postgr.es/m/c27f642d-020b-01ff-ae61-086af287c4fd@gmail.com
Branch
------
REL_12_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/a1fb4bd8562e0a7f162ff8a08c3c2ecdeb792f80
Modified Files
--------------
src/backend/tsearch/wparser_def.c | 8 ++++++--
src/test/regress/expected/tsearch.out | 21 +++++++++++++++++++++
src/test/regress/sql/tsearch.sql | 6 ++++++
3 files changed, 33 insertions(+), 2 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
* pgsql: Fix ts_headline() edge cases for empty query and empty search te
@ 2023-04-06 19:52 Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 9+ messages in thread
From: Tom Lane @ 2023-04-06 19:52 UTC (permalink / raw)
To: pgsql-committers@lists.postgresql.org
Fix ts_headline() edge cases for empty query and empty search text.
tsquery's GETQUERY() macro is only safe to apply to a tsquery
that is known non-empty; otherwise it gives a pointer to garbage.
Before commit 5a617d75d, ts_headline() avoided this pitfall, but
only in a very indirect, nonobvious way. (hlCover could not reach
its TS_execute call, because if the query contains no lexemes
then hlFirstIndex would surely return -1.) After that commit,
it fell into the trap, resulting in weird errors such as
"unrecognized operator" and/or valgrind complaints. In HEAD,
fix this by not calling TS_execute_locations() at all for an
empty query. In the back branches, add a defensive check to
hlCover() --- that's not fixing any live bug, but I judge the
code a bit too fragile as-is.
Also, both mark_hl_fragments() and mark_hl_words() were careless
about the possibility of empty search text: in the cases where
no match has been found, they'd end up telling mark_fragment() to
mark from word indexes 0 to 0 inclusive, even when there is no
word 0. This is harmless since we over-allocated the prs->words
array, but it does annoy valgrind. Fix so that the end index is -1
and thus mark_fragment() will do nothing in such cases.
Bottom line is that this fixes a live bug in HEAD, but in the
back branches it's only getting rid of a valgrind nitpick.
Back-patch anyway.
Per report from Alexander Lakhin.
Discussion: https://postgr.es/m/c27f642d-020b-01ff-ae61-086af287c4fd@gmail.com
Branch
------
REL_11_STABLE
Details
-------
https://git.postgresql.org/pg/commitdiff/bc428b12ac3ac7f4444b5d6f792cdbc3ee3c9388
Modified Files
--------------
src/backend/tsearch/wparser_def.c | 8 ++++++--
src/test/regress/expected/tsearch.out | 21 +++++++++++++++++++++
src/test/regress/sql/tsearch.sql | 6 ++++++
3 files changed, 33 insertions(+), 2 deletions(-)
^ permalink raw reply [nested|flat] 9+ messages in thread
end of thread, other threads:[~2023-04-06 22:04 UTC | newest]
Thread overview: 9+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-04-06 19:52 pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
2023-04-06 21:38 ` Andres Freund <andres@anarazel.de>
2023-04-06 21:47 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-04-06 22:04 ` Tom Lane <tgl@sss.pgh.pa.us>
2023-04-06 19:52 pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
2023-04-06 19:52 pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
2023-04-06 19:52 pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
2023-04-06 19:52 pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
2023-04-06 19:52 pgsql: Fix ts_headline() edge cases for empty query and empty search te Tom Lane <tgl@sss.pgh.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox