pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedSlow fulltext query plan
5+ messages / 3 participants
[nested] [flat]
* Slow fulltext query plan
@ 2012-04-12 22:09 Benoit Delbosc <bdelbosc@nuxeo.com>
2012-04-12 22:25 ` Re: Slow fulltext query plan Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 5+ messages in thread
From: Benoit Delbosc @ 2012-04-12 22:09 UTC (permalink / raw)
To: pgsql-performance
Hi,
I would like to understand why the following query execution don't use
any fulltext indexes
and takes more than 300s (using lot of temporary files):
EXPLAIN ANALYZE SELECT hierarchy.id
FROM hierarchy
JOIN fulltext ON fulltext.id = hierarchy.id,
TO_TSQUERY('whatever') query1,
TO_TSQUERY('whatever') query2
WHERE (query1 @@ nx_to_tsvector(fulltext.fulltext)) OR (query2 @@
nx_to_tsvector(fulltext.fulltext_title));
The query plan is here:
http://explain.depesz.com/s/YgP
While if I replace the query2 by query1 in the second clause:
EXPLAIN ANALYZE SELECT hierarchy.id
FROM hierarchy
JOIN fulltext ON fulltext.id = hierarchy.id,
TO_TSQUERY('whatever') query1,
TO_TSQUERY('whatever') query2
WHERE (query1 @@ nx_to_tsvector(fulltext.fulltext)) OR (query1 @@
nx_to_tsvector(fulltext.fulltext_title));
It is 5 order of magniude faster (15ms) using the gin indexes:
http://explain.depesz.com/s/RLa
The nx_to_tsvector is an immutable function with the following code:
SELECT TO_TSVECTOR('english', SUBSTR($1, 1, 250000))
Here is the list of indexes:
hierarchy: "hierarchy_pk" PRIMARY KEY, btree (id)
fulltext: "fulltext_fulltext_idx" gin
(nx_to_tsvector(fulltext::character varying))
fulltext: "fulltext_fulltext_title_idx" gin
(nx_to_tsvector(fulltext_title::character varying))
fulltext and fulltext_title are text type.
And some PostgreSQL configuration:
PostgreSQL 9.1.2 on x86_64-unknown-linux-gnu
shared_buffers: 4GB
effective_cache_size: 10GB
work_mem: 20MB
Thanks for your work and enlightenment
ben
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Slow fulltext query plan
2012-04-12 22:09 Slow fulltext query plan Benoit Delbosc <bdelbosc@nuxeo.com>
@ 2012-04-12 22:25 ` Tom Lane <tgl@sss.pgh.pa.us>
2012-04-12 22:56 ` Re: Slow fulltext query plan Benoit Delbosc <bdelbosc@nuxeo.com>
0 siblings, 1 reply; 5+ messages in thread
From: Tom Lane @ 2012-04-12 22:25 UTC (permalink / raw)
To: Benoit Delbosc <bdelbosc@nuxeo.com>; +Cc: pgsql-performance
Benoit Delbosc <bdelbosc@nuxeo.com> writes:
> EXPLAIN ANALYZE SELECT hierarchy.id
> FROM hierarchy
> JOIN fulltext ON fulltext.id = hierarchy.id,
> TO_TSQUERY('whatever') query1,
> TO_TSQUERY('whatever') query2
> WHERE (query1 @@ nx_to_tsvector(fulltext.fulltext)) OR (query2 @@
> nx_to_tsvector(fulltext.fulltext_title));
Is there a reason why you're writing the query in such a
non-straightforward way, rather than just
EXPLAIN ANALYZE SELECT hierarchy.id
FROM hierarchy
JOIN fulltext ON fulltext.id = hierarchy.id
WHERE (TO_TSQUERY('whatever') @@ nx_to_tsvector(fulltext.fulltext))
OR (TO_TSQUERY('whatever') @@ nx_to_tsvector(fulltext.fulltext_title));
?
regards, tom lane
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Slow fulltext query plan
2012-04-12 22:09 Slow fulltext query plan Benoit Delbosc <bdelbosc@nuxeo.com>
2012-04-12 22:25 ` Re: Slow fulltext query plan Tom Lane <tgl@sss.pgh.pa.us>
@ 2012-04-12 22:56 ` Benoit Delbosc <bdelbosc@nuxeo.com>
2012-04-14 01:16 ` Re: Slow fulltext query plan Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 1 reply; 5+ messages in thread
From: Benoit Delbosc @ 2012-04-12 22:56 UTC (permalink / raw)
To: pgsql-performance
On 13/04/2012 00:25, Tom Lane wrote:
> Benoit Delbosc<bdelbosc@nuxeo.com> writes:
>> EXPLAIN ANALYZE SELECT hierarchy.id
>> FROM hierarchy
>> JOIN fulltext ON fulltext.id = hierarchy.id,
>> TO_TSQUERY('whatever') query1,
>> TO_TSQUERY('whatever') query2
>> WHERE (query1 @@ nx_to_tsvector(fulltext.fulltext)) OR (query2 @@
>> nx_to_tsvector(fulltext.fulltext_title));
> Is there a reason why you're writing the query in such a
> non-straightforward way, rather than just
>
> EXPLAIN ANALYZE SELECT hierarchy.id
> FROM hierarchy
> JOIN fulltext ON fulltext.id = hierarchy.id
> WHERE (TO_TSQUERY('whatever') @@ nx_to_tsvector(fulltext.fulltext))
> OR (TO_TSQUERY('whatever') @@ nx_to_tsvector(fulltext.fulltext_title));
>
> ?
>
This query is written by a framework, also I thought that is a common
pattern that can be found in the documentation:
http://www.postgresql.org/docs/9.1/interactive/textsearch-controls.html
if you think this a wrong way to do it then I will try to fix the framework.
btw your version takes 15ms :)
Thanks
ben
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Slow fulltext query plan
2012-04-12 22:09 Slow fulltext query plan Benoit Delbosc <bdelbosc@nuxeo.com>
2012-04-12 22:25 ` Re: Slow fulltext query plan Tom Lane <tgl@sss.pgh.pa.us>
2012-04-12 22:56 ` Re: Slow fulltext query plan Benoit Delbosc <bdelbosc@nuxeo.com>
@ 2012-04-14 01:16 ` Tom Lane <tgl@sss.pgh.pa.us>
0 siblings, 0 replies; 5+ messages in thread
From: Tom Lane @ 2012-04-14 01:16 UTC (permalink / raw)
To: Benoit Delbosc <bdelbosc@nuxeo.com>; +Cc: pgsql-performance
Benoit Delbosc <bdelbosc@nuxeo.com> writes:
> On 13/04/2012 00:25, Tom Lane wrote:
>> Is there a reason why you're writing the query in such a
>> non-straightforward way, rather than just
>>
>> EXPLAIN ANALYZE SELECT hierarchy.id
>> FROM hierarchy
>> JOIN fulltext ON fulltext.id = hierarchy.id
>> WHERE (TO_TSQUERY('whatever') @@ nx_to_tsvector(fulltext.fulltext))
>> OR (TO_TSQUERY('whatever') @@ nx_to_tsvector(fulltext.fulltext_title));
> This query is written by a framework, also I thought that is a common
> pattern that can be found in the documentation:
> http://www.postgresql.org/docs/9.1/interactive/textsearch-controls.html
Well, "common pattern" would be stretching it. Anyway I've concluded
that this is in fact a planner bug. There will be a fix in 9.2, but I'm
not going to take the risk of back-patching it, so you might want to
think about changing that framework.
regards, tom lane
^ permalink raw reply [nested|flat] 5+ messages in thread
* Re: Slow fulltext query plan
@ 2012-04-16 13:11 Florent Guillaume <fg@nuxeo.com>
0 siblings, 0 replies; 5+ messages in thread
From: Florent Guillaume @ 2012-04-16 13:11 UTC (permalink / raw)
To: pgsql-performance
Tom Lane wrote:
> Benoit Delbosc<bdelbosc@nuxeo.com> writes:
>> On 13/04/2012 00:25, Tom Lane wrote:
>>> Is there a reason why you're writing the query in such a
>>> non-straightforward way, rather than just
>>>
>>> EXPLAIN ANALYZE SELECT hierarchy.id
>>> FROM hierarchy
>>> JOIN fulltext ON fulltext.id = hierarchy.id
>>> WHERE (TO_TSQUERY('whatever') @@ nx_to_tsvector(fulltext.fulltext))
>>> OR (TO_TSQUERY('whatever') @@ nx_to_tsvector(fulltext.fulltext_title));
>>
>> This query is written by a framework, also I thought that is a common
>> pattern that can be found in the documentation:
>> http://www.postgresql.org/docs/9.1/interactive/textsearch-controls.html
>
> Well, "common pattern" would be stretching it. Anyway I've concluded
> that this is in fact a planner bug. There will be a fix in 9.2, but I'm
> not going to take the risk of back-patching it, so you might want to
> think about changing that framework.
FYI the reason why we have queries that look like what Benoit
describes is that we often use the query alias twice, once for
TO_TSVECTOR and once for TS_RANK_CD, for instance:
SELECT hierarchy.id, TS_RANK_CD(fulltext, query1, 32) as nxscore
FROM hierarchy
JOIN fulltext ON fulltext.id = hierarchy.id,
TO_TSQUERY('whatever') query1,
TO_TSQUERY('whatever') query2
WHERE (query1 @@ nx_to_tsvector(fulltext.fulltext)) OR (query2 @@
nx_to_tsvector(fulltext.fulltext_title))
ORDER BY nxscore DESC;
(as is also described in the doc mentioned btw).
Florent
--
Florent Guillaume, Director of R&D, Nuxeo
Open Source, Java EE based, Enterprise Content Management (ECM)
http://www.nuxeo.com http://www.nuxeo.org +33 1 40 33 79 87
^ permalink raw reply [nested|flat] 5+ messages in thread
end of thread, other threads:[~2012-04-16 13:11 UTC | newest]
Thread overview: 5+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2012-04-12 22:09 Slow fulltext query plan Benoit Delbosc <bdelbosc@nuxeo.com>
2012-04-12 22:25 ` Tom Lane <tgl@sss.pgh.pa.us>
2012-04-12 22:56 ` Benoit Delbosc <bdelbosc@nuxeo.com>
2012-04-14 01:16 ` Tom Lane <tgl@sss.pgh.pa.us>
2012-04-16 13:11 Re: Slow fulltext query plan Florent Guillaume <fg@nuxeo.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox