To: "Guillaume Smet" <guillaume.smet@gmail.com>
cc: "Kevin McArthur" <Kevin@stormtide.ca>, pgsql-performance@postgresql.org
Subject: Re: Bad Planner Statistics for Uneven distribution. 
In-reply-to: <1d4e0c10607211500t62be09f2v8284ceb5176b01bc@mail.gmail.com> 
References: <003b01c6acf1$a66d8910$0701a8c0@kdesktop>
	<8073.1153517393@sss.pgh.pa.us>
	<1d4e0c10607211500t62be09f2v8284ceb5176b01bc@mail.gmail.com>
Comments: In-reply-to "Guillaume Smet" <guillaume.smet@gmail.com>
	message dated "Sat, 22 Jul 2006 00:00:01 +0200"
Date: Sat, 22 Jul 2006 13:03:58 -0400
Message-ID: <27058.1153587838@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

"Guillaume Smet" <guillaume.smet@gmail.com> writes:
> Isn't there any way to make PostgreSQL have a better estimation here:
> ->  Index Scan using models_brands_brand on models_brands
> (cost=0.00..216410.97 rows=92372 width=0) (actual time=0.008..0.008
> rows=0 loops=303)
>            Index Cond: (brand = $0)

Note that the above plan extract is pretty misleading, because it
doesn't account for the implicit "LIMIT 1" of an EXISTS() clause.
What the planner is *actually* imputing to this plan is 216410.97/92372
cost units, or about 2.34.  However that applies to the seqscan variant
as well.

I think the real issue with Kevin's example is that when doing an
EXISTS() on a brand_id that doesn't actually exist in the table, the
seqscan plan has worst-case behavior (ie, scan the whole table) while
the indexscan plan still manages to be cheap.  Because his brands table
has so many brand_ids that aren't in the table, that case dominates the
results.  Not sure how we could factor that risk into the cost
estimates.  The EXISTS code could probably special-case it reasonably
well for the simplest seqscan and indexscan subplans, but I don't see
what to do with more general subqueries (like joins).

			regards, tom lane