From: Oleg Bartunov <oleg@sai.msu.su>
To: Kris Kiger <kris@musicrebellion.com>
Cc: pgsql-admin@postgresql.org
Subject: Re: tsearch2 poor performance
Date: Wed, 29 Sep 2004 19:08:03 +0400 (MSD)
Message-ID: <Pine.GSO.4.58.0409291905020.14980@ra.sai.msu.su> (raw)
In-Reply-To: <415AC7A2.9020906@musicrebellion.com>
References: <71E37EF6B7DCC1499CEA0316A2568328DC9DF0@loki.wc.globexplorer.net>
<4158409E.8060109@musicrebellion.com>
<Pine.GSO.4.58.0409272040090.14980@ra.sai.msu.su>
<41584E49.5040301@musicrebellion.com>
<Pine.GSO.4.58.0409272243140.14980@ra.sai.msu.su>
<41586264.5020302@musicrebellion.com>
<Pine.GSO.4.58.0409291600370.14980@ra.sai.msu.su>
<415AC7A2.9020906@musicrebellion.com>
Kris,
is't possible to share your "paragraph generator" ? It'd be better than
downloading the whole dump :)
On Wed, 29 Sep 2004, Kris Kiger wrote:
> Oleg, the data I have right now was generated using a random paragraph
> generator. The words are real words, but there are only 508 distinct
> keywords in the 3,000,000 records that tsearch2 will pick up, using
> default settings. I was using this data set for the purpose of testing
> tsearch2's capabilities, so it's not real world data. If you still want
> it, let me know where to send it and I will send you a dump of the DB.
>
> Kris
>
>
>
> Oleg Bartunov wrote:
>
> >Kris,
> >
> >we're working on prototype of tsearchd - full text search daemon, which
> >maintain static inverted index outside of postgresql using the same
> >parser, dictionary tsearch2 does. This approach could scale up
> >fts capability preserving access to metadata, so yo may have
> >"archive" part of your collection (tsearchd) and "online", which could be
> >searchable with tsearch2.
> >
> >Here is what we have right now:
> >
> >pages ( tid integer, fts_index tsvector)
> >
> >1. Create index
> >select count(tdindex(tid,fts_index)) from pages;
> >2. Flush index
> >select tdflush();
> >3. Search
> >select pages.tid, rank(fts_index,to_tsquery('supernovae & magellan')) as rank
> >from pages, tdsearch(to_tsquery('supernovae & magellan')) as idx where
> >tid=idx order by rank desc;
> >
> >If it's possible, you could share your data, so we could test our
> >prototype on real data.
> >
> >
> > Oleg
> >
> >
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 7: don't forget to increase your free space map settings
>
Regards,
Oleg
_____________________________________________________________
Oleg Bartunov, sci.researcher, hostmaster of AstroNet,
Sternberg Astronomical Institute, Moscow University (Russia)
Internet: oleg@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(095)939-16-83, +007(095)939-23-83
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-admin@postgresql.org
Cc: oleg@sai.msu.su, kris@musicrebellion.com
Subject: Re: tsearch2 poor performance
In-Reply-To: <Pine.GSO.4.58.0409291905020.14980@ra.sai.msu.su>
References: <71E37EF6B7DCC1499CEA0316A2568328DC9DF0@loki.wc.globexplorer.net> <4158409E.8060109@musicrebellion.com> <Pine.GSO.4.58.0409272040090.14980@ra.sai.msu.su> <41584E49.5040301@musicrebellion.com> <Pine.GSO.4.58.0409272243140.14980@ra.sai.msu.su> <41586264.5020302@musicrebellion.com> <Pine.GSO.4.58.0409291600370.14980@ra.sai.msu.su> <415AC7A2.9020906@musicrebellion.com> <Pine.GSO.4.58.0409291905020.14980@ra.sai.msu.su>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox