X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id A9F8353674 for ; Thu, 24 Mar 2005 10:41:12 +0000 (GMT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 63200-05 for ; Thu, 24 Mar 2005 10:41:09 +0000 (GMT) Received: from mx1.rockingstone.nl (mx1.rockingstone.nl [82.94.241.100]) by svr1.postgresql.org (Postfix) with ESMTP id 82D3653669 for ; Thu, 24 Mar 2005 10:41:06 +0000 (GMT) Received: from localhost (web1 [127.0.0.1]) by mx1.rockingstone.nl (Postfix) with ESMTP id 06A5CFB0FB; Thu, 24 Mar 2005 11:47:11 +0100 (CET) Received: from mx1.rockingstone.nl ([127.0.0.1]) by localhost (web1 [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 26892-17; Thu, 24 Mar 2005 11:47:09 +0100 (CET) Received: from [10.10.20.219] (217-19-26-53.dsl.cambrium.nl [217.19.26.53]) by mx1.rockingstone.nl (Postfix) with ESMTP id 8CC45FB077; Thu, 24 Mar 2005 11:47:08 +0100 (CET) Message-ID: <42429940.8040607@rockingstone.nl> Date: Thu, 24 Mar 2005 11:41:04 +0100 From: Rick Jansen User-Agent: Mozilla Thunderbird 1.0 (X11/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Oleg Bartunov Cc: Mike Rylander , pgsql-performance@postgresql.org Subject: Re: Tsearch2 performance on big database References: <42400F57.3040604@rockingstone.nl> <42401CBF.50403@rockingstone.nl> <42404263.40604@rockingstone.nl> <42412E4B.4010407@rockingstone.nl> In-Reply-To: X-Enigmail-Version: 0.90.0.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at rockingstone.nl X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0 tagged_above=0 required=5 tests= X-Spam-Level: X-Archive-Number: 200503/501 X-Sequence-Number: 11206 Oleg Bartunov wrote: > from my notes > http://www.sai.msu.su/~megera/oddmuse/index.cgi/Tsearch_V2_Notes > > It's usefull to see words statistics, for example, to check how good > your dictionaries work or how did you configure pg_ts_cfgmap. Also, you > may notice probable stop words relevant for your collection. Tsearch > provides stat() function: > > ....................... > > Don't hesitate to read it and if you find some bugs or know better wording > I'd be glad to improve my notes. > Thanks, but that stat() query takes way too long.. I let it run for like 4 hours and still nothing. The database I am testing tsearch2 on is also the production database (mysql) server so I have to be careful not to use too many resources :o Anyway, here's my pg_ts_cfgmap now (well the relevant bits): default_english | lhword | {en_ispell,en_stem} default_english | lpart_hword | {en_ispell,en_stem} default_english | lword | {en_ispell,en_stem} Is it normal that queries for single words (or perhaps they are words that are common) take a really long time? Like this: ilab=# explain analyze select count(*) from books where description_fti @@ to_tsquery('default', 'hispanic'); QUERY PLAN ------------------------------------------------------------------------------------------------------------------------------------------ Aggregate (cost=20369.81..20369.81 rows=1 width=0) (actual time=261512.031..261512.031 rows=1 loops=1) -> Index Scan using idxfti_idx on books (cost=0.00..20349.70 rows=8041 width=0) (actual time=45777.760..261509.288 rows=674 loops=1) Index Cond: (description_fti @@ '\'hispan\''::tsquery) Total runtime: 261518.529 ms (4 rows) ilab=# explain analyze select titel from books where description_fti @@ to_tsquery('default', 'buckingham & palace'); QUERY PLAN ------------------------------------------------------------------------------------------------------------------------------------ Index Scan using idxfti_idx on books (cost=0.00..20349.70 rows=8041 width=57) (actual time=18992.045..48863.385 rows=185 loops=1) Index Cond: (description_fti @@ '\'buckingham\' & \'palac\''::tsquery) Total runtime: 48863.874 ms (3 rows) I dont know what happened, these queries were a lot faster 2 days ago..what the feck is going on?! Rick -- Systems Administrator for Rockingstone IT http://www.rockingstone.com http://www.megabooksearch.com - Search many book listing sites at once