To: "Joshua Shanks" <jjshanks@gmail.com>
Cc: "Tom Lane" <tgl@sss.pgh.pa.us>,
	"Craig Ringer" <craig@postnewspapers.com.au>,
	<pgsql-performance@postgresql.org>
Subject: Re: query planner not using the correct index
In-Reply-To: <84f0acdb0808071501n15b0182ga7eca7b1be37d705@mail.gmail.com>
	(Joshua Shanks's message of "Thu\, 7 Aug 2008 17\:01\:53 -0500")
User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.1 (gnu/linux)
References: <84f0acdb0808061435q5ebb607cjc311f1eb4da841a@mail.gmail.com>
	<489A623A.4090400@postnewspapers.com.au>
	<84f0acdb0808062014p29cf7482ied332225a0a92c63@mail.gmail.com>
	<489A6AF5.50908@postnewspapers.com.au>
	<84f0acdb0808070811l43a3da9eq240d272f8ddd105c@mail.gmail.com>
	<27696.1218143560@sss.pgh.pa.us>
	<84f0acdb0808071501n15b0182ga7eca7b1be37d705@mail.gmail.com>
From: Gregory Stark <stark@enterprisedb.com>
Organization: EnterpriseDB
Date: Thu, 07 Aug 2008 23:38:07 +0100
Message-ID: <874p5wsb74.fsf@oxford.xeocode.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

"Joshua Shanks" <jjshanks@gmail.com> writes:

> Those 3 values in reality and in the stats account for 98% of the
> rows. actual distinct values are around 350

Measuring n_distinct from a sample is inherently difficult and unreliable.
When 98% of your table falls into those categories it's leaving very few
chances for the sample to find many other distinct values. 

I haven't seen the whole thread, if you haven't tried already you could try
raising the statistics target for these columns -- that's usually necessary
anyways when you have a very skewed distribution like this.

> It seems like the planner would want to get the result set from
> bars.bars_id condition and if it is big using the index on the join to
> avoid the separate sorting, but if it is small (0-5 rows which is our
> normal case) use the primary key index to join and then just quickly
> sort. Is there any reason the planner doesn't do this?

Yeah, Heikki's suggested having a kind of "branch" plan node that knows how
where the break-point is between two plans and can call the appropriate one.
We don't have anything like that yet.

-- 
  Gregory Stark
  EnterpriseDB          http://www.enterprisedb.com
  Ask me about EnterpriseDB's On-Demand Production Tuning