Message-ID: <4657273A.3080005@archonet.com>
Date: Fri, 25 May 2007 19:13:14 +0100
From: Richard Huxton <dev@archonet.com>
User-Agent: Thunderbird 2.0.0.0 (X11/20070326)
MIME-Version: 1.0
To: Gregory Stark <stark@enterprisedb.com>
Cc: mark@mark.mielke.cc, Mark Lewis <mark.lewis@mir3.com>,
	James Mansion <james@mansionfamily.plus.com>,
	Magnus Hagander <magnus@hagander.net>,
	Alexander Staubo <alex@purefiction.net>, Andy <frum@ar-sd.net>,
	pgsql-performance@postgresql.org
Subject: Re: LIKE search and performance
References: <009501c79d4c$445428f0$0b00a8c0@mpsro.dom>
	<88daf38c0705230905u1a981102k691b47bd36011a80@mail.gmail.com>
	<4655DE75.4080506@mansionfamily.plus.com>	<4655E64B.5050503@hagander.net>
	<4655FB8A.1040707@mansionfamily.plus.com>
	<1180040560.31471.262.camel@archimedes>
	<20070524215450.GA3858@mark.mielke.cc>
	<46569AA5.10205@archonet.com>	<20070525141630.GA12162@mark.mielke.cc>
	<4657023A.20209@archonet.com>	<20070525165633.GA13579@mark.mielke.cc>
	<46571CEC.7070202@archonet.com>
	<87y7jckcp6.fsf@oxford.xeocode.com>
In-Reply-To: <87y7jckcp6.fsf@oxford.xeocode.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Gregory Stark wrote:
> "Richard Huxton" <dev@archonet.com> writes:
> 
>> Now you and I can look at a substring and probably make a good guess how common
>> it is (assuming we know the targets are British surnames or Japanese towns). PG
>> needs one number - or rather, it picks one number for each length of
>> search-string (afaik).
> 
> I don't think that's true. Postgres calculates the lower and upper bound
> implied by the search pattern and then uses the histogram to estimate how
> selective that range is. It's sometimes surprisingly good but obviously it's
> not perfect.

Sorry - I'm obviously picking my words badly today.

I meant for the "contains" substring match. It gives different (goes 
away and checks...yes) predictions based on string length. So it guesses 
that LIKE '%aaa%' will match more than LIKE '%aaaa%'. Of course, if we 
were matching surnames you and I could say that this is very unlikely, 
but without some big statistics table I guess there's not much more PG 
can do.

For a trailing wildcard LIKE 'aaa%' it can and does as you say convert 
this into something along the lines of (>= 'aaa' AND < 'aab'). Although 
IIRC that depends if your locale allows such (not sure, I don't really 
use non-C/non-English locales enough).

-- 
   Richard Huxton
   Archonet Ltd