pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: josh@agliodbs.com
Cc: Greg Stark <gsstark@mit.edu>
Cc: Marko Ristola <marko.ristola@kolumbus.fi>
Cc: pgsql-perform <pgsql-performance@postgresql.org>
Cc: pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] Bad n_distinct estimation; hacks suggested?
Date: Sun, 24 Apr 2005 00:48:59 -0400
Message-ID: <25382.1114318139@sss.pgh.pa.us> (raw)
In-Reply-To: <200504231639.11897.josh@agliodbs.com>
References: <200504191209.05181.josh@agliodbs.com>
	<200504221336.08325.josh@agliodbs.com>
	<8764ydapky.fsf@stark.xeocode.com>
	<200504231639.11897.josh@agliodbs.com>

Josh Berkus <josh@agliodbs.com> writes:
> Overall, our formula is inherently conservative of n_distinct.   That is, I 
> believe that it is actually computing the *smallest* number of distinct 
> values which would reasonably produce the given sample, rather than the 
> *median* one.  This is contrary to the notes in analyze.c, which seem to 
> think that we're *overestimating* n_distinct.  

Well, the notes are there because the early tests I ran on that formula
did show it overestimating n_distinct more often than not.  Greg is
correct that this is inherently a hard problem :-(

I have nothing against adopting a different formula, if you can find
something with a comparable amount of math behind it ... but I fear
it'd only shift the failure cases around.

			regards, tom lane



view thread (53+ messages)  latest in thread

Message-ID: <25382.1114318139@sss.pgh.pa.us>
Permalink:  ../25382.1114318139@sss.pgh.pa.us/
Also on:    postgresql.org/message-id/25382.1114318139@sss.pgh.pa.us

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: tgl@sss.pgh.pa.us, josh@agliodbs.com, gsstark@mit.edu, marko.ristola@kolumbus.fi, pgsql-hackers@postgresql.org
  Subject: Re: [HACKERS] Bad n_distinct estimation; hacks suggested?
  In-Reply-To: <25382.1114318139@sss.pgh.pa.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox