Message-ID: <4942BCD2.9070306@cheapcomplexdevices.com>
Date: Fri, 12 Dec 2008 11:34:42 -0800
From: Ron Mayer <rm_pg@cheapcomplexdevices.com>
User-Agent: Thunderbird 2.0.0.17 (X11/20080925)
MIME-Version: 1.0
To: Gregory Stark <stark@enterprisedb.com>
CC: Simon Riggs <simon@2ndQuadrant.com>, Robert Haas <robertmhaas@gmail.com>,
	Tom Lane <tgl@sss.pgh.pa.us>,
	Kevin Grittner <Kevin.Grittner@wicourts.gov>,
	"jd@commandprompt.com" <jd@commandprompt.com>,
	Josh Berkus <josh@agliodbs.com>,
	Greg Smith <gsmith@gregsmith.com>, pgsql-hackers@postgresql.org
Subject: Re: benchmarking the query planner
References: <603c8f070812061019r5aca3aajeaa5493b11c5feff@mail.gmail.com>
	<603c8f070812092025k6325e0fcud001b205a507a634@mail.gmail.com>
	<4313.1228931932@sss.pgh.pa.us>
	<603c8f070812101828p1bd7df49yb8c916adedb49fe1@mail.gmail.com>
	<9565.1229018968@sss.pgh.pa.us>	<1229033073.13078.179.camel@hp_dx2400_1>
	<87ej0etkfh.fsf@oxford.xeocode.com>	<1229039044.13078.193.camel@hp_dx2400_1>
	<4136ffa0812111823u645b6ec9wdca60b3da4b00499@mail.gmail.com>
	<1229072675.13078.196.camel@hp_dx2400_1>
	<603c8f070812120344m67ef2c1fs41806cfb4ff9e396@mail.gmail.com>
	<1229093975.13078.275.camel@ebony.2ndQuadrant>
	<87tz99s7bu.fsf@oxford.xeocode.com>
	<1229100057.8673.29.camel@ebony.2ndQuadrant>
	<87hc59s4qx.fsf@oxford.xeocode.com>
In-Reply-To: <87hc59s4qx.fsf@oxford.xeocode.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Gregory Stark wrote:
> Simon Riggs <simon@2ndQuadrant.com> writes:
>> The amount of I/O could stay the same, just sample all rows on block. [....]
> 
> It will also introduce strange biases. For instance in a clustered table it'll
> think there are a lot more duplicates than there really are because it'll see
> lots of similar values.

But for ndistinct - it seems it could only help things.  If the
ndistinct guesser just picks
   max(the-current-one-row-per-block-guess,
       a-guess-based-on-all-the-rows-on-the-blocks)
it seems we'd be no worse off for clustered tables; and much
better off for randomly organized tables.


In some ways I fear *not* sampling all rows on the block also
introduces strange biases by largely overlooking the fact that
the table's clustered.

In my tables clustered on zip-code we don't notice info like
"state='AZ' is present in well under 1% of blocks in the table",
while if we did scan all rows on the blocks it might guess this.
But I guess a histogram of blocks would be additional stat rather
than an improved one.