From: Oleg Bartunov <oleg@sai.msu.su>
To: Rick Jansen <rick@rockingstone.nl>
Cc: Mike Rylander <mrylander@gmail.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Tsearch2 performance on big database
Date: Thu, 24 Mar 2005 13:51:42 +0300 (MSK)
Message-ID: <Pine.GSO.4.62.0503241343580.5508@ra.sai.msu.su> (raw)
In-Reply-To: <42429940.8040607@rockingstone.nl>
References: <42400F57.3040604@rockingstone.nl>
<Pine.GSO.4.62.0503221533400.5508@ra.sai.msu.su>
<b918cf3d050322044847d3f9e1@mail.gmail.com>
<42401CBF.50403@rockingstone.nl>
<b918cf3d0503220730665fe92c@mail.gmail.com>
<Pine.GSO.4.62.0503221840480.5508@ra.sai.msu.su>
<42404263.40604@rockingstone.nl>
<Pine.GSO.4.62.0503221907320.5508@ra.sai.msu.su>
<42412E4B.4010407@rockingstone.nl>
<Pine.GSO.4.62.0503231236390.5508@ra.sai.msu.su>
<42429940.8040607@rockingstone.nl>
On Thu, 24 Mar 2005, Rick Jansen wrote:
> Oleg Bartunov wrote:
>> from my notes
>> http://www.sai.msu.su/~megera/oddmuse/index.cgi/Tsearch_V2_Notes
>>
>> It's usefull to see words statistics, for example, to check how good
>> your dictionaries work or how did you configure pg_ts_cfgmap. Also, you
>> may notice probable stop words relevant for your collection. Tsearch
>> provides stat() function:
>>
>> .......................
>>
>> Don't hesitate to read it and if you find some bugs or know better wording
>> I'd be glad to improve my notes.
>>
>
> Thanks, but that stat() query takes way too long.. I let it run for like
> 4 hours and still nothing. The database I am testing tsearch2 on is also
> the production database (mysql) server so I have to be careful not to
> use too many resources :o
stat() is indeed a bigdog, it was designed for developers needs,
so we recommend to save results in table.
>
> Anyway, here's my pg_ts_cfgmap now (well the relevant bits):
>
> default_english | lhword | {en_ispell,en_stem}
> default_english | lpart_hword | {en_ispell,en_stem}
> default_english | lword | {en_ispell,en_stem}
>
> Is it normal that queries for single words (or perhaps they are words
> that are common) take a really long time? Like this:
>
'hispanic' isn't common, I see you get only 674 rows and
'buckingham & palace' returns 185 rows. Did you run 'vacuum analyze' ?
I see a big discrepancy between estimated rows (8041) and actual rows.
> ilab=# explain analyze select count(*) from books where description_fti @@
> to_tsquery('default', 'hispanic');
> QUERY PLAN
> ------------------------------------------------------------------------------------------------------------------------------------------
> Aggregate (cost=20369.81..20369.81 rows=1 width=0) (actual
> time=261512.031..261512.031 rows=1 loops=1)
> -> Index Scan using idxfti_idx on books (cost=0.00..20349.70 rows=8041
> width=0) (actual time=45777.760..261509.288 rows=674 loops=1)
> Index Cond: (description_fti @@ '\'hispan\''::tsquery)
> Total runtime: 261518.529 ms
> (4 rows)
>
> ilab=# explain analyze select titel from books where description_fti @@
> to_tsquery('default', 'buckingham & palace');
> QUERY PLAN
> ------------------------------------------------------------------------------------------------------------------------------------
> Index Scan using idxfti_idx on books (cost=0.00..20349.70 rows=8041
> width=57) (actual time=18992.045..48863.385 rows=185 loops=1)
> Index Cond: (description_fti @@ '\'buckingham\' & \'palac\''::tsquery)
> Total runtime: 48863.874 ms
> (3 rows)
>
>
> I dont know what happened, these queries were a lot faster 2 days ago..what
> the feck is going on?!
>
> Rick
>
>
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: oleg@sai.msu.su, rick@rockingstone.nl, mrylander@gmail.com
Subject: Re: Tsearch2 performance on big database
In-Reply-To: <Pine.GSO.4.62.0503241343580.5508@ra.sai.msu.su>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox