X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 1100ED1B1BD for ; Wed, 2 Jun 2004 22:33:30 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 30602-01 for ; Thu, 3 Jun 2004 01:33:32 +0000 (GMT) Received: from houston.familyhealth.com.au (fhnet.arach.net.au [203.22.197.21]) by svr1.postgresql.org (Postfix) with ESMTP id 003E6D1B1AE for ; Wed, 2 Jun 2004 22:33:27 -0300 (ADT) Received: from [192.168.0.40] (work-40.internal [192.168.0.40]) by houston.familyhealth.com.au (8.12.9p1/8.12.9) with ESMTP id i531XNWL094135; Thu, 3 Jun 2004 09:33:24 +0800 (WST) (envelope-from chriskl@familyhealth.com.au) Message-ID: <40BE8216.5000802@familyhealth.com.au> Date: Thu, 03 Jun 2004 09:42:46 +0800 From: Christopher Kings-Lynne User-Agent: Mozilla Thunderbird 0.6 (Windows/20040502) X-Accept-Language: en-us, en MIME-Version: 1.0 To: "W.H. van Atteveldt" Cc: pgsql-performance@postgresql.org Subject: Re: Postgres query optimization with varchar fields References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at hub.org X-Archive-Number: 200406/23 X-Sequence-Number: 7081 > I am investigating whether it is useful to directly query a database > containing a rather large text corpus (order of magnitude 100k - 1m > newspaper articles, so around 100 million words), or whether I should > use third party text indexing services. I want to know things such as: > how often is a certain word (or pattern) mentioned in an article and how > often it is mentioned with the condition that another word is nearby > (same article or n words distant). You really want to use the contrib/tsearch2 module that comes already with PostgreSQL. cd contrib/tsearch2 gmake install psql < tsearch2.sql more README.tsearch2 Chris