To: "Kevin Grittner" <Kevin.Grittner@wicourts.gov>
cc: jesper@krogh.cc, pgsql-performance@postgresql.org, oleg@sai.msu.su,
	teodor@sigaev.ru
Subject: Re: Queryplan within FTS/GIN index -search. 
In-reply-to: <4AEFF056020000250002C195@gw.wicourts.gov> 
References: 
 <4AEF0369020000250002C183@gw.wicourts.gov><4AEF0369020000250002C183@gw.wicourts.gov>
	<11587.1257207482@sss.pgh.pa.us>
	<4AEFF056020000250002C195@gw.wicourts.gov>
Comments: In-reply-to "Kevin Grittner" <Kevin.Grittner@wicourts.gov>
	message dated "Tue, 03 Nov 2009 08:56:53 -0600"
Date: Tue, 03 Nov 2009 10:35:44 -0500
Message-ID: <4730.1257262544@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

"Kevin Grittner" <Kevin.Grittner@wicourts.gov> writes:
> Perhaps I'm missing something.  My point was that there are words
> which are too common to be useful for index searches, yet uncommon
> enough to usefully limit the results.  These words could typically
> benefit from tsearch2 style parsing and dictionaries; so declaring
> them as stop words would be bad from a functional perspective, yet
> searching an index for them would be bad from a performance
> perspective.

Right, but the original complaint in this thread was that a GIN index is
slow about searching for very common terms.  The answer to that clearly
is to not index common terms, rather than worry about making the case
a bit faster.

It may well be that Jesper's identified a place where the GIN code could
be improved --- it seems like having the top-level search logic be more
aware of the AND/OR structure of queries would be useful.  But the
particular example shown here doesn't make a very good case for that,
because it's hard to tell how much of a penalty would be taken in more
realistic examples.

			regards, tom lane