Message-ID: <4354913B.2020001@modgraph-usa.com>
Date: Mon, 17 Oct 2005 23:07:55 -0700
From: "Craig A. James" <cjames@modgraph-usa.com>
User-Agent: Mozilla Thunderbird 1.0.6 (X11/20050716)
MIME-Version: 1.0
To: Oleg Bartunov <oleg@sai.msu.su>
Cc: pgsql-performance@postgresql.org
Subject: Re: tsearch2/GIST performance factors?
References: <4351166C.9010102@modgraph-usa.com>
	<Pine.GSO.4.63.0510172127510.10366@ra.sai.msu.su>
In-Reply-To: <Pine.GSO.4.63.0510172127510.10366@ra.sai.msu.su>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Oleg wrote:
> Did you consider *decreasing* SIGLENINT ? Size of index will diminish
> and performance could be increased. I use in current project SIGLENINT=15

The default value for SIGLENINT actually didn't work at all.  It was only by increasing it that I got any performance at all.  An examination of the GIST indexes showed that most of the first level and many of the second level bitmaps were saturated.

> tsearch2's index is a lossy index, read 
> http://www.sai.msu.su/~megera/oddmuse/index.cgi/Tsearch_V2_internals
> so search results  should be rechecked !

Yes, thanks.  We do indeed recheck the actual results.  The tests I'm running are just on the raw index performance - how long does it take to "select ... where dockeys @@ to_tsquery(...)".

> We have our TODO http://www.sai.msu.su/~megera/oddmuse/index.cgi/todo
> and hope to find sponsorhips for fts project for 8.2 release.
> Unfortunately, I didn't find spare time to package tsearchd for you,
> it should certainly help you.

At this point we may not have time to try tsearchd, and unfortunately we're not in a position to sponsor anything yet.

My original question is still bothering me.  Is it normal for a keyword that occurs in more than about 2% of the documents to cause such inconsistent performance?  Is there any single thing I might look at that would help improve performance (like, do I need more memory?  More shared memory?  Different config parameters?)

Thanks,
Craig