From: Jaume Sabater <jsabater@linuxsilo.net>
To: pgsql-admin@postgresql.org
Subject: Using text search for locations, or a computed btree index
Date: Thu, 4 Dec 2008 17:21:34 +0000 (GMT)
Message-ID: <16902748.7851228411294539.JavaMail.root@zimbra.linuxsilo.net> (raw)
In-Reply-To: <31612419.7831228410455018.JavaMail.root@zimbra.linuxsilo.net>
Hello everyone!
The guys creating the web site at the company I work for want an incremental search for locations. So, I need to find all locations that start with Uni, then Unit, then Unite, and so on. Problem is that the tables, although partitioned, have got several milion of tuples. Here is what I have in mind:
1. There are no stop words ("Palma of Majorca" is "Palma of Majorca", not "Palma Majorca", although this is not the best example).
2. There are no synonyms that have to be matched to other words.
3. No phrases have to be mapped to single words.
4. No variations of words need to be mapped to a canonical form (although this may be false when we add alternate names of locations)
5. I need to take out all accented characters and change all words to its lowercased form for faster searching and easier locating.
So, I've been reading all the text search documentation available online and I think that I should be:
1. Creating an extra column with the "normalized" version of the location name.
2. Creating a GIN index on that extra column.
3. Creating a synonyms (variations) dictionary with the alternate names (touristic names that have no political equivalent but people know and use to search)
What I don't know how to do is to just tell tsvector (and tsquery when searching) to do what I need and not what I don't need. I can't find a way in the online documentation to tell the engine which pieces I want from the whole puzzle.
Alternatively, provided we don't use alternate names, I could use a computed btree index that does those needed operations: lowercase, substitute accented characters by their equivalents with a regular expression, and store that into an extra colum, perhaps?
Any hints would be welcome. Thanks in advance.
--
Jaume Sabater
http://linuxsilo.net/
"Ubi sapientas ibi libertas"
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-admin@postgresql.org
Cc: jsabater@linuxsilo.net
Subject: Re: Using text search for locations, or a computed btree index
In-Reply-To: <16902748.7851228411294539.JavaMail.root@zimbra.linuxsilo.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox