DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type:content-transfer-encoding;
	b=xlVt6IIH18K5tYAoEFJvW1wbo4aoITG8VPb3TZxUnpinprA5oId03/9yksQM4sif6q
	tEI/6wLyid3uMuAv+QStw10q8e/vmIfoHKwat+b2kw+oh7+4+rfpzlhCp8S4QReejrF3
	tyi33SYOjY54dOTPdgGmwDdu8yjkqHbLAP5oo=
MIME-Version: 1.0
In-Reply-To: <20110525221342.14e34046@dick.coachhouse>
References: <20110522210526.0f8b66fa@dick.coachhouse>
	<20110522230041.28d1bc35@dick.coachhouse>
	<BANLkTikGzsrvXprLXeN3TxZwA=M8fzkn=g@mail.gmail.com>
	<4DD9BB87.7070000@postnewspapers.com.au>
	<ircnj2$cpt$1@news.albasani.net>
	<20110523163341.56edb1aa@dick.coachhouse>
	<irgo1v$dbv$1@news.albasani.net> <4DDD1F7C.5040008@gmail.com>
	<20110525221342.14e34046@dick.coachhouse>
Date: Thu, 26 May 2011 10:15:50 +1200
Message-ID: <BANLkTimBKGkhdzYCZqBR=pxa9bNhUBXWLg@mail.gmail.com>
Subject: Re: extracting location info from string
From: Andrej <andrej.groups@gmail.com>
To: Tarlika Elisabeth Schmitz <postgresql3@numerixtechnology.de>
Cc: pgsql-sql@postgresql.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On 26 May 2011 09:13, Tarlika Elisabeth Schmitz
<postgresql3@numerixtechnology.de> wrote:
> On Wed, 25 May 2011 09:25:48 -0600
> Rob Sargent <robjsargent@gmail.com> wrote:
>
>>
>>
>>On 05/24/2011 10:57 AM, Lew wrote:
>>> Tarlika Elisabeth Schmitz wrote:
>>>
>>>> CREATE TABLE person
>>>> (
>>>> id integer NOT NULL,
>>>> "name" character varying(256) NOT NULL,
>>>> "location" character varying(256),
>>>> CONSTRAINT person_pkey PRIMARY KEY (id)
>>>> );
>>>>
>>>> this was just a TEMPORARY table I created for quick analysis
>>>> of my CSV data (now renamed to temp_person).
>
> CREATE TABLE country
> (
> =A0id character varying(3) NOT NULL, -- alpha-3 code
> =A0"name" character varying(50) NOT NULL,
> =A0CONSTRAINT country_pkey PRIMARY KEY (id)
> );
>
>
>>To minimize the ultimately quite necessary human adjudication, one
>>might make good use of what is often termed "crowd sourcing": =A0Keep
>>all the distinct "hand entered" values and a map to the final human
>>assessment.
>
> I was wondering how to do just that. I don't think it would be a good
> idea to hard code this into the clean-up script. Take, for instance,
> variations of COUNTRY.NAME spelling. Where would I store these?
>
> I could do with a concept for this problem, which applies to a lot of
> string-type info.

I'd start w/ downloading a list as mentioned here:
http://answers.google.com/answers/threadview?id=3D596822

And run it through a wee perl script using
http://search.cpan.org/~maurice/Text-DoubleMetaphone-0.07/DoubleMetaphone.p=
m
to make phonetic matches ...

Then I'd run your own data through DoubleMetaphone, and clean up
matches if not too many false positives show up.


Cheers,
Andrej