Received: from localhost (unknown [200.46.204.184])
	by developer.postgresql.org (Postfix) with ESMTP id A5B892E0038
	for <pgsql-sql-postgresql.org@developer.postgresql.org>;
	Wed, 16 Apr 2008 11:50:14 -0300 (ADT)
Received: from developer.postgresql.org ([200.46.204.71])
	by localhost (mx1.hub.org [200.46.204.184]) (amavisd-maia, port 10024)
	with ESMTP id 54818-08
	for <pgsql-sql-postgresql.org@developer.postgresql.org>;
	Wed, 16 Apr 2008 11:49:55 -0300 (ADT)
X-Greylist: from auto-whitelisted by SQLgrey-1.7.5
Received: from www.postnewspapers.com.au (www.postnewspapers.com.au
	[202.61.230.242])
	by developer.postgresql.org (Postfix) with ESMTP id 8E6322E0030
	for <pgsql-sql@postgresql.org>; Wed, 16 Apr 2008 11:50:04 -0300 (ADT)
Received: from mail.postnewspapers.com.au
	(202-89-185-120.static.dsl.amnet.net.au [202.89.185.120])
	by www.postnewspapers.com.au (Postfix) with ESMTP id 1BBD5D55AB;
	Wed, 16 Apr 2008 22:49:59 +0800 (WST)
Received: from [10.0.0.53] (wallace.localnet [10.0.0.53])
	by mail.postnewspapers.com.au (Postfix) with ESMTP id 3BDFC1E0434;
	Wed, 16 Apr 2008 22:50:02 +0800 (WST)
Message-ID: <48062003.3050409@postnewspapers.com.au>
Date: Wed, 16 Apr 2008 23:49:23 +0800
From: Craig Ringer <craig@postnewspapers.com.au>
User-Agent: Icedove 1.5.0.12 (X11/20070607)
MIME-Version: 1.0
To: Vivek Khera <vivek@khera.org>
Cc: pgsql-sql@postgresql.org
Subject: Re: How to find double entries
References: <48056F5F.30507@gmx.net> <21481.1208316212@sss.pgh.pa.us>
	<82308532-B9D5-481C-9256-75BDCB69E74B@khera.org>
In-Reply-To: <82308532-B9D5-481C-9256-75BDCB69E74B@khera.org>
X-Enigmail-Version: 0.94.2.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Virus-Scanned: Maia Mailguard 1.0.1
X-Archive-Number: 200804/97
X-Sequence-Number: 30636

Vivek Khera wrote:
> 
> On Apr 15, 2008, at 11:23 PM, Tom Lane wrote:
>> What's really a duplicate sounds like a judgment call here, so you
>> probably shouldn't even think of automating it completely.
> 
> I did a consulting gig about 10 years ago for a company that made
> software to normalize street addresses and names.  Literally dozens of
> people worked there, and that was their primary software product.  It is
> definitely not a trivial task, as the rules can be extremely complex.