Received: from localhost (unknown [200.46.204.184]) by developer.postgresql.org (Postfix) with ESMTP id 0485A2E0062 for ; Wed, 16 Apr 2008 11:27:58 -0300 (ADT) Received: from developer.postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.184]) (amavisd-maia, port 10024) with ESMTP id 50635-04 for ; Wed, 16 Apr 2008 11:27:42 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey-1.7.5 Received: from yertle.kcilink.com (thingy.kcilink.com [74.92.149.59]) by developer.postgresql.org (Postfix) with ESMTP id 44A7E2E0054 for ; Wed, 16 Apr 2008 11:27:51 -0300 (ADT) Received: from host-121.int.kcilink.com (host-121.int.kcilink.com [192.168.7.121]) by yertle.kcilink.com (Postfix) with ESMTP id F11358A03A for ; Wed, 16 Apr 2008 10:27:49 -0400 (EDT) Message-Id: <82308532-B9D5-481C-9256-75BDCB69E74B@khera.org> From: Vivek Khera To: pgsql-sql@postgresql.org In-Reply-To: <21481.1208316212@sss.pgh.pa.us> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v919.2) Subject: Re: How to find double entries Date: Wed, 16 Apr 2008 10:27:49 -0400 References: <48056F5F.30507@gmx.net> <21481.1208316212@sss.pgh.pa.us> X-Mailer: Apple Mail (2.919.2) X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200804/96 X-Sequence-Number: 30635 On Apr 15, 2008, at 11:23 PM, Tom Lane wrote: > What's really a duplicate sounds like a judgment call here, so you > probably shouldn't even think of automating it completely. I did a consulting gig about 10 years ago for a company that made software to normalize street addresses and names. Literally dozens of people worked there, and that was their primary software product. It is definitely not a trivial task, as the rules can be extremely complex.