From: Kris Kiger <kris@musicrebellion.com>
To: pgsql-admin@postgresql.org
Subject: Re: tsearch2 poor performance
Date: Wed, 29 Sep 2004 09:33:06 -0500
Message-ID: <415AC7A2.9020906@musicrebellion.com> (raw)
References: <71E37EF6B7DCC1499CEA0316A2568328DC9DF0@loki.wc.globexplorer.net>
<4158409E.8060109@musicrebellion.com>
<Pine.GSO.4.58.0409272040090.14980@ra.sai.msu.su>
<41584E49.5040301@musicrebellion.com>
<Pine.GSO.4.58.0409272243140.14980@ra.sai.msu.su>
<41586264.5020302@musicrebellion.com>
<Pine.GSO.4.58.0409291600370.14980@ra.sai.msu.su>
Oleg, the data I have right now was generated using a random paragraph
generator. The words are real words, but there are only 508 distinct
keywords in the 3,000,000 records that tsearch2 will pick up, using
default settings. I was using this data set for the purpose of testing
tsearch2's capabilities, so it's not real world data. If you still want
it, let me know where to send it and I will send you a dump of the DB.
Kris
Oleg Bartunov wrote:
>Kris,
>
>we're working on prototype of tsearchd - full text search daemon, which
>maintain static inverted index outside of postgresql using the same
>parser, dictionary tsearch2 does. This approach could scale up
>fts capability preserving access to metadata, so yo may have
>"archive" part of your collection (tsearchd) and "online", which could be
>searchable with tsearch2.
>
>Here is what we have right now:
>
>pages ( tid integer, fts_index tsvector)
>
>1. Create index
>select count(tdindex(tid,fts_index)) from pages;
>2. Flush index
>select tdflush();
>3. Search
>select pages.tid, rank(fts_index,to_tsquery('supernovae & magellan')) as rank
>from pages, tdsearch(to_tsquery('supernovae & magellan')) as idx where
>tid=idx order by rank desc;
>
>If it's possible, you could share your data, so we could test our
>prototype on real data.
>
>
> Oleg
>
>
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients from your mail client; the link fills in
the headers below: reply via email
Some clients, webmail especially, drop In-Reply-To and References from
a link, and the reply then starts a new thread: use the mbox below.
To: pgsql-admin@postgresql.org
Cc: kris@musicrebellion.com
Subject: Re: tsearch2 poor performance
In-Reply-To: <415AC7A2.9020906@musicrebellion.com>
References: <71E37EF6B7DCC1499CEA0316A2568328DC9DF0@loki.wc.globexplorer.net> <4158409E.8060109@musicrebellion.com> <Pine.GSO.4.58.0409272040090.14980@ra.sai.msu.su> <41584E49.5040301@musicrebellion.com> <Pine.GSO.4.58.0409272243140.14980@ra.sai.msu.su> <41586264.5020302@musicrebellion.com> <Pine.GSO.4.58.0409291600370.14980@ra.sai.msu.su> <415AC7A2.9020906@musicrebellion.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox