Received: from sss.sss.pgh.pa.us (sss.pgh.pa.us [209.114.166.2]) by hub.org (8.9.3/8.9.3) with ESMTP id KAA18358 for ; Thu, 12 Aug 1999 10:36:00 -0400 (EDT) (envelope-from tgl@sss.pgh.pa.us) Received: from sss.sss.pgh.pa.us (localhost [127.0.0.1]) by sss.sss.pgh.pa.us (8.9.1/8.9.1) with ESMTP id KAA18218; Thu, 12 Aug 1999 10:34:21 -0400 (EDT) To: Clayton Cottingham cc: pgsql-sql@postgreSQL.org Subject: Re: [SQL] quest:optimization with indexes In-reply-to: Your message of Wed, 11 Aug 1999 23:38:31 +0000 <37B20977.B560E075@smartt.com> Date: Thu, 12 Aug 1999 10:34:21 -0400 Message-ID: <18216.934468461@sss.pgh.pa.us> From: Tom Lane Clayton Cottingham writes: > what about helper software like glimpse, as per my other thread? > i was told to use glimpse, because full text index {fti} isnt robust > enough to handle large db's AFAIK there isn't any real good way to use glimpse in combination with Postgres. Glimpse is a file-granularity indexer --- that is, if you are searching for a word, say "foobar", then glimpse's index tells it which files within a large directory tree contain that word, and then it goes and runs "grep foobar" on each such file to find the actual occurrences of that word. Glimpse works great for large, slowly changing collections of files (I say "slowly changing" because updating the index is fairly expensive). For instance, I use it all the time to search the Postgres source files. But it'd be quite useless for a Postgres database where all the data is in one table (= one file). Also, there's no natural equivalent of file granularity in a database; if you did adapt glimpse to know about the inner structure of a Postgres database, you'd end up indexing at the record level, which'd be a mighty inefficient way to use glimpse. If you do have a moderately large collection of slowly-changing documents, I'd suggest storing them as a plain tree of files, and using glimpse and Postgres separately as index-maintaining tools. (That is, your database would just contain references to the files, plus whatever information you wanted to manipulate with SQL operations.) regards, tom lane