Received: from localhost (maia-2.hub.org [200.46.204.187])
	by postgresql.org (Postfix) with ESMTP id 569849FB674
	for <pgsql-performance-postgresql.org@postgresql.org>;
	Fri, 25 May 2007 13:56:34 -0300 (ADT)
Received: from postgresql.org ([200.46.204.71])
	by localhost (mx1.hub.org [200.46.204.187]) (amavisd-maia, port 10024)
	with ESMTP id 80804-04 for
	<pgsql-performance-postgresql.org@postgresql.org>;
	Fri, 25 May 2007 13:56:29 -0300 (ADT)
X-Greylist: from auto-whitelisted by SQLgrey-1.7.5
Received: from mail.envysolutions.com (unknown [206.248.142.186])
	by postgresql.org (Postfix) with ESMTP id 6B7E19FB2EA
	for <pgsql-performance@postgresql.org>;
	Fri, 25 May 2007 13:56:30 -0300 (ADT)
Received: from localhost (localhost.localdomain [127.0.0.1])
	by mail.envysolutions.com (Postfix) with ESMTP id EC2A254055;
	Fri, 25 May 2007 12:56:33 -0400 (EDT)
X-Virus-Scanned: amavisd-new at mail.envysolutions.com
Received: from mail.envysolutions.com ([127.0.0.1])
	by localhost (mark.mielke.cc [127.0.0.1]) (amavisd-new, port 10024)
	with ESMTP id RPScAUxchthV; Fri, 25 May 2007 12:56:33 -0400 (EDT)
Received: by mail.envysolutions.com (Postfix, from userid 500)
	id C477C54058; Fri, 25 May 2007 12:56:33 -0400 (EDT)
Date: Fri, 25 May 2007 12:56:33 -0400
From: mark@mark.mielke.cc
To: Richard Huxton <dev@archonet.com>
Cc: Mark Lewis <mark.lewis@mir3.com>,
	James Mansion <james@mansionfamily.plus.com>,
	Magnus Hagander <magnus@hagander.net>,
	Alexander Staubo <alex@purefiction.net>, Andy <frum@ar-sd.net>,
	pgsql-performance@postgresql.org
Subject: Re: LIKE search and performance
Message-ID: <20070525165633.GA13579@mark.mielke.cc>
References: <009501c79d4c$445428f0$0b00a8c0@mpsro.dom>
	<88daf38c0705230905u1a981102k691b47bd36011a80@mail.gmail.com>
	<4655DE75.4080506@mansionfamily.plus.com>
	<4655E64B.5050503@hagander.net>
	<4655FB8A.1040707@mansionfamily.plus.com>
	<1180040560.31471.262.camel@archimedes>
	<20070524215450.GA3858@mark.mielke.cc>
	<46569AA5.10205@archonet.com>
	<20070525141630.GA12162@mark.mielke.cc>
	<4657023A.20209@archonet.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <4657023A.20209@archonet.com>
User-Agent: Mutt/1.4.2.2i
X-Virus-Scanned: Maia Mailguard 1.0.1
X-Spam-Status: No, hits=0.766 tagged_above=0 required=5 tests=AWL=-0.145,
	BAYES_40=-0.185, FORGED_RCVD_HELO=0.135, NO_REAL_NAME=0.961
X-Spam-Level: 
X-Archive-Number: 200705/522
X-Sequence-Number: 24912

On Fri, May 25, 2007 at 04:35:22PM +0100, Richard Huxton wrote:
> >I notice you did not provide a real life example as requested. :-)
> OK - any application that allows user-built queries: <choose column: 
> foo> <choose filter: contains> <choose target: "bar">
> Want another? Any application that has a "search by name" box - users 
> can (and do) put one letter in and hit enter.
> Unfortunately you don't always have control over the selectivity of 
> queries issued.

The database has 10 million records. The user enters "bar" and it
translates to "%bar%". You are suggesting that we expect bar to match
1 million+ records? :-)

I hope not. I would define this as bad process. I would also use "LIMIT"
to something like "100".

> >This seems like an ivory tower restriction. Not allowing best performance
> >in a common situation vs not allowing worst performance in a not-so-common
> >situation.
> What best performance plan are you thinking of? I'm assuming we're 
> talking about trailing-wildcard matches here, rather than "contains" 
> style matches.

"Trailing-wildcard" already uses B-Tree index, does it not?

I am speaking of contains, as contains is the one that was said to
require a seqscan. I am questioning why it requires a seqscan. The
claim was made that with MVCC, the index is insufficient to check
for visibility and that the table would need to be accessed anyways,
therefore a seqscan is required. I question whether a like '%bar%'
should be considered a high selectivity query in the general case.
I question whether a worst case should be assumed.

Perhaps I question too much? :-)

Cheers,
mark

-- 
mark@mielke.cc / markm@ncf.ca / markm@nortel.com     __________________________
.  .  _  ._  . .   .__    .  . ._. .__ .   . . .__  | Neighbourhood Coder
|\/| |_| |_| |/    |_     |\/|  |  |_  |   |/  |_   | 
|  | | | | \ | \   |__ .  |  | .|. |__ |__ | \ |__  | Ottawa, Ontario, Canada

  One ring to rule them all, one ring to find them, one ring to bring them all
                       and in the darkness bind them...

                           http://mark.mielke.cc/