Received: from localhost (unknown [200.46.204.184])
	by developer.postgresql.org (Postfix) with ESMTP id 373D92E0045
	for <pgsql-sql-postgresql.org@developer.postgresql.org>;
	Wed, 16 Apr 2008 00:23:42 -0300 (ADT)
Received: from developer.postgresql.org ([200.46.204.71])
	by localhost (mx1.hub.org [200.46.204.184]) (amavisd-maia, port 10024)
	with ESMTP id 38004-07
	for <pgsql-sql-postgresql.org@developer.postgresql.org>;
	Wed, 16 Apr 2008 00:23:36 -0300 (ADT)
X-Greylist: from auto-whitelisted by SQLgrey-1.7.5
Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130])
	by developer.postgresql.org (Postfix) with ESMTP id 1D7B82E0035
	for <pgsql-sql@postgresql.org>; Wed, 16 Apr 2008 00:23:34 -0300 (ADT)
Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1])
	by sss.pgh.pa.us (8.14.2/8.14.2) with ESMTP id m3G3NW1j021482;
	Tue, 15 Apr 2008 23:23:32 -0400 (EDT)
To: Andreas <maps.on@gmx.net>
cc: pgsql-sql@postgresql.org
Subject: Re: How to find double entries 
In-reply-to: <48056F5F.30507@gmx.net> 
References: <48056F5F.30507@gmx.net>
Comments: In-reply-to Andreas <maps.on@gmx.net>
	message dated "Wed, 16 Apr 2008 05:15:43 +0200"
Date: Tue, 15 Apr 2008 23:23:32 -0400
Message-ID: <21481.1208316212@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
X-Virus-Scanned: Maia Mailguard 1.0.1
X-Archive-Number: 200804/87
X-Sequence-Number: 30626

Andreas <maps.on@gmx.net> writes:
> I'd like to identify and then merge records of e.g.   'google', 'gogle', 
> 'guugle' 

> Then I want to match abbrevations like  'A-Company Ltd.', 'a company 
> ltd.', 'A-Company Limited'

> Is there a way to do this?
> It would be OK just to list candidats up to be manually checked afterwards.

There are some functions in contrib/fuzzystrmatch that seem like they'd
help you find candidate duplicates.  contrib/pg_trgm and text search
might also offer promising tools.

What's really a duplicate sounds like a judgment call here, so you
probably shouldn't even think of automating it completely.

			regards, tom lane