From: Andrew Dunstan <andrew@dunslane.net>
To: Mischa Sandberg <mischa.sandberg@telus.net>
Cc: josh@agliodbs.com, pgsql-perform <pgsql-performance@postgresql.org>
Cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Bad n_distinct estimation; hacks suggested?
Date: Wed, 27 Apr 2005 09:43:31 -0400
Message-ID: <426F9703.4010108@dunslane.net> (raw)
In-Reply-To: <1114580284.426f253cc0087@webmail.telus.net>
References: <200504191209.05181.josh@agliodbs.com>
<19276.1114442580@sss.pgh.pa.us>
<1114454941.21529.245.camel@localhost.localdomain>
<200504251213.18565.josh@agliodbs.com>
<426D565E.8040400@dunslane.net>
<1114580284.426f253cc0087@webmail.telus.net>
Mischa Sandberg wrote:
>
>Perhaps I can save you some time (yes, I have a degree in Math). If I
>understand correctly, you're trying extrapolate from the correlation
>between a tiny sample and a larger sample. Introducing the tiny sample
>into any decision can only produce a less accurate result than just
>taking the larger sample on its own; GIGO. Whether they are consistent
>with one another has no relationship to whether the larger sample
>correlates with the whole population. You can think of the tiny sample
>like "anecdotal" evidence for wonderdrugs.
>
>
>
Ok, good point.
I'm with Tom though in being very wary of solutions that require even
one-off whole table scans. Maybe we need an additional per-table
statistics setting which could specify the sample size, either as an
absolute number or as a percentage of the table. It certainly seems that
where D/N ~ 0.3, the estimates on very large tables at least are way way
out.
Or maybe we need to support more than one estimation method.
Or both ;-)
cheers
andrew
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: andrew@dunslane.net, mischa.sandberg@telus.net, pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Bad n_distinct estimation; hacks suggested?
In-Reply-To: <426F9703.4010108@dunslane.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox