Received: from localhost (unknown [200.46.204.184]) by developer.postgresql.org (Postfix) with ESMTP id 3F7812E004A for ; Wed, 16 Apr 2008 00:23:42 -0300 (ADT) Received: from developer.postgresql.org ([200.46.204.71]) by localhost (mx1.hub.org [200.46.204.184]) (amavisd-maia, port 10024) with ESMTP id 34696-10 for ; Wed, 16 Apr 2008 00:23:36 -0300 (ADT) X-Greylist: from auto-whitelisted by SQLgrey-1.7.5 Received: from www.postnewspapers.com.au (www.postnewspapers.com.au [202.61.230.242]) by developer.postgresql.org (Postfix) with ESMTP id 24BE22E0037 for ; Wed, 16 Apr 2008 00:23:35 -0300 (ADT) Received: from mail.postnewspapers.com.au (202-89-185-120.static.dsl.amnet.net.au [202.89.185.120]) by www.postnewspapers.com.au (Postfix) with ESMTP id D8FE29E9; Wed, 16 Apr 2008 11:23:31 +0800 (WST) Received: from [10.0.0.53] (wallace.localnet [10.0.0.53]) by mail.postnewspapers.com.au (Postfix) with ESMTP id 0FD701E0435; Wed, 16 Apr 2008 11:23:35 +0800 (WST) Message-ID: <48057F21.30607@postnewspapers.com.au> Date: Wed, 16 Apr 2008 12:22:57 +0800 From: Craig Ringer User-Agent: Icedove 1.5.0.12 (X11/20070607) MIME-Version: 1.0 To: Andreas Cc: pgsql-sql@postgresql.org Subject: Re: How to find double entries References: <48056F5F.30507@gmx.net> In-Reply-To: <48056F5F.30507@gmx.net> X-Enigmail-Version: 0.94.2.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Virus-Scanned: Maia Mailguard 1.0.1 X-Archive-Number: 200804/88 X-Sequence-Number: 30627 Andreas wrote: > Hi, > > how can I find double entries in varchar columns where the content is > not 100% identical because of a spelling error or the person considered > it "looked nicer" that way? When doing some near-duplicate elimination as part of converting a legacy data set to PostgreSQL I found the `fuzzystrmatch' contrib module immensely helpful. http://www.postgresql.org/docs/current/static/fuzzystrmatch.html -- Craig Ringer