pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: psql-mail@freeuk.com
To: pgsql-performance@postgresql.org
Subject: Large Text Search Help
Date: Wed, 08 Oct 2003 16:48:17 +0100
Message-ID: <E1A7GXx-0007J5-PM@buckaroo.freeuk.net> (raw)

Hi,
I am trying to design a large text search database.

It will have upwards of 6 million documents, along with meta data on 
each.

I am currently looking at tsearch2 to provide fast text searching and 
also playing around with different hardware configurations.

1. With tsearch2 I get very good query times up until I insert more 
records. For example with 100,000 records tsearch2 returns in around 6 
seconds, with 200,000 records tsearch2 returns in just under a minute. 
Is this due to the indices fitting entirely in memory with 100,000 
records?

2. As well as whole word matching i also need to be able to do 
substring matching. Is the FTI module the way to approach this?

3. I have just begun to look into distibuted queries. Is there an 
existing solution for distibuting a postgresql database amongst 
multiple servers, so each has the same schema but only a subset of the 
total data?

Any other helpful comments or sugestions on how to improve query times 
using different hardware or software techniques would be appreciated.

Thanks,

Mat



view thread (2+ messages)  latest in thread

Message-ID: <E1A7GXx-0007J5-PM@buckaroo.freeuk.net>
Permalink:  ../E1A7GXx-0007J5-PM@buckaroo.freeuk.net/
Also on:    postgresql.org/message-id/E1A7GXx-0007J5-PM@buckaroo.freeuk.net

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: psql-mail@freeuk.com
  Subject: Re: Large Text Search Help
  In-Reply-To: <E1A7GXx-0007J5-PM@buckaroo.freeuk.net>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox