From: Martijn van Oosterhout <kleptog@svana.org>
To: Oleg Bartunov <oleg@sai.msu.su>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Ron <rjpeace@earthlink.net>
Cc: pgsql-performance@postgresql.org
Subject: Re: [GENERAL] Creation of tsearch2 index is very slow
Date: Sat, 21 Jan 2006 16:04:24 +0100
Message-ID: <20060121150424.GB9955@svana.org> (raw)
In-Reply-To: <Pine.GSO.4.63.0601211619541.14417@ra.sai.msu.su>
References: <26153.1137784469@sss.pgh.pa.us>
<20060120201036.GE31908@svana.org>
<26583.1137788505@sss.pgh.pa.us>
<20060120205132.GF31908@svana.org>
<26958.1137791955@sss.pgh.pa.us>
<20060120213754.GG31908@svana.org>
<7.0.1.0.2.20060120174141.039b03e8@earthlink.net>
<6111.1137797436@sss.pgh.pa.us>
<20060121130830.GA9955@svana.org>
<Pine.GSO.4.63.0601211619541.14417@ra.sai.msu.su>
On Sat, Jan 21, 2006 at 04:29:13PM +0300, Oleg Bartunov wrote:
> Martijn, you're right! We want not only to split page to very
> different parts, but not to increase the number of sets bits in
> resulted signatures, which are union (OR'ed) of all signatures
> in part. We need not only fast index creation (thanks, Tom !),
> but a better index. Some information is available here
> http://www.sai.msu.su/~megera/oddmuse/index.cgi/Tsearch_V2_internals
> There are should be more detailed document, but I don't remember where:)
I see how it works, what I don't quite get is whether the "inverted
index" you refer to is what we're working with here, or just what's in
tsearchd?
> >That's harder though (this algorithm does approximate it sort of)
> >and I havn't come up with an algorithm yet
>
> Don't ask how hard we thought :)
Well, looking at how other people are struggling with it, it's
definitly a Hard Problem. One thing though, I don't think the picksplit
algorithm as is really requires you to strictly have the longest
distance, just something reasonably long. So I think the alternate
algorithm I posted should produce equivalent results. No idea how to
test it though...
Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.
Attachments:
signature.asc (application/pgp-signature, 188B) download
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: kleptog@svana.org, oleg@sai.msu.su, tgl@sss.pgh.pa.us, rjpeace@earthlink.net
Subject: Re: [GENERAL] Creation of tsearch2 index is very slow
In-Reply-To: <20060121150424.GB9955@svana.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox