pg.ddx.io  pgsql-sql@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Clayton Cottingham <drfrog@smartt.com>
Cc: pgsql-sql@postgreSQL.org
Subject: Re: [SQL] quest:optimization with indexes
Date: Thu, 12 Aug 1999 10:34:21 -0400
Message-ID: <18216.934468461@sss.pgh.pa.us> (raw)
In-Reply-To: <37B20977.B560E075@smartt.com>

Clayton Cottingham <drfrog@smartt.com> writes:
> what about helper software like glimpse, as per my other thread?
> i was told to use glimpse, because full text index {fti} isnt robust
> enough to handle large db's

AFAIK there isn't any real good way to use glimpse in combination with
Postgres.  Glimpse is a file-granularity indexer --- that is, if you are
searching for a word, say "foobar", then glimpse's index tells it which
files within a large directory tree contain that word, and then it goes
and runs "grep foobar" on each such file to find the actual occurrences
of that word.  Glimpse works great for large, slowly changing
collections of files (I say "slowly changing" because updating the index
is fairly expensive).  For instance, I use it all the time to search the
Postgres source files.  But it'd be quite useless for a Postgres
database where all the data is in one table (= one file).  Also, there's
no natural equivalent of file granularity in a database; if you did
adapt glimpse to know about the inner structure of a Postgres database,
you'd end up indexing at the record level, which'd be a mighty
inefficient way to use glimpse.

If you do have a moderately large collection of slowly-changing
documents, I'd suggest storing them as a plain tree of files, and using
glimpse and Postgres separately as index-maintaining tools.  (That is,
your database would just contain references to the files, plus whatever
information you wanted to manipulate with SQL operations.)

			regards, tom lane



view thread (7+ messages)

Message-ID: <18216.934468461@sss.pgh.pa.us>
Permalink:  ../18216.934468461@sss.pgh.pa.us/
Also on:    postgresql.org/message-id/18216.934468461@sss.pgh.pa.us

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients from your mail client; the link fills in
  the headers below: reply via email
  Some clients, webmail especially, drop In-Reply-To and References from
  a link, and the reply then starts a new thread: use the mbox below.

  To: pgsql-sql@postgresql.org
  Cc: tgl@sss.pgh.pa.us, drfrog@smartt.com
  Subject: Re: [SQL] quest:optimization with indexes
  In-Reply-To: <18216.934468461@sss.pgh.pa.us>
  References: <37B20977.B560E075@smartt.com> <18216.934468461@sss.pgh.pa.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox