To: "Steinar H. Gunderson" <sgunderson@bigfoot.com>
cc: pgsql-performance@postgresql.org, Teodor Sigaev <teodor@sigaev.ru>,
	Oleg Bartunov <oleg@sai.msu.su>
Subject: Re: [GENERAL] Creation of tsearch2 index is very slow 
In-reply-to: <20060121000519.GA29755@uio.no> 
References: <26153.1137784469@sss.pgh.pa.us>
 <20060120201036.GE31908@svana.org>
	<26583.1137788505@sss.pgh.pa.us> <20060120205132.GF31908@svana.org>
	<26958.1137791955@sss.pgh.pa.us> <20060120213754.GG31908@svana.org>
	<7.0.1.0.2.20060120174141.039b03e8@earthlink.net>
	<6111.1137797436@sss.pgh.pa.us> <20060120225751.GA27230@uio.no>
	<6532.1137801157@sss.pgh.pa.us> <20060121000519.GA29755@uio.no>
Comments: In-reply-to "Steinar H. Gunderson" <sgunderson@bigfoot.com>
	message dated "Sat, 21 Jan 2006 01:05:19 +0100"
Date: Fri, 20 Jan 2006 19:23:10 -0500
Message-ID: <6743.1137802990@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

"Steinar H. Gunderson" <sgunderson@bigfoot.com> writes:
> On Fri, Jan 20, 2006 at 06:52:37PM -0500, Tom Lane wrote:
>> It's also worth considering that the entire approach is a heuristic,
>> really --- getting the furthest-apart pair of seeds doesn't guarantee
>> an optimal split as far as I can see.  Maybe there's some totally
>> different way to do it.

> For those of us who don't know what tsearch2/gist is trying to accomplish
> here, could you provide some pointers? :-)

Well, we're trying to split an index page that's gotten full into two
index pages, preferably with approximately equal numbers of items in
each new page (this isn't a hard requirement though).  I think the true
figure of merit for a split is how often will subsequent searches have
to descend into *both* of the resulting pages as opposed to just one
--- the less often that is true, the better.  I'm not very clear on
what tsearch2 is doing with these bitmaps, but it looks like an upper
page's downlink has the union (bitwise OR) of the one-bits in the values
on the lower page, and you have to visit the lower page if this union
has a nonempty intersection with the set you are looking for.  If that's
correct, what you really want is to divide the values so that the unions
of the two sets have minimal overlap ... which seems to me to have
little to do with what the code does at present.

Teodor, Oleg, can you clarify what's needed here?

			regards, tom lane