X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id EDFC553406; Tue, 3 May 2005 18:33:14 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 03749-05; Tue, 3 May 2005 21:33:11 +0000 (GMT) Received: from priv-edtnes56.telusplanet.net (outbound01.telus.net [199.185.220.220]) by svr1.postgresql.org (Postfix) with ESMTP id 45B1E5322A; Tue, 3 May 2005 18:33:09 -0300 (ADT) Received: from localhost ([199.185.220.240]) by priv-edtnes56.telusplanet.net (InterMail vM.6.01.04.04 201-2131-118-104-20050224) with ESMTP id <20050503213311.KSVN13846.priv-edtnes56.telusplanet.net@localhost>; Tue, 3 May 2005 15:33:11 -0600 Received: from 209.17.183.249 ( [209.17.183.249]) as user a3a18850@204.209.205.53 by webmail.telus.net with HTTP; Tue, 3 May 2005 14:33:10 -0700 Message-ID: <1115155990.4277ee16aba34@webmail.telus.net> Date: Tue, 3 May 2005 14:33:10 -0700 From: Mischa Sandberg To: Markus Schaber Cc: pgsql-perform , pgsql-hackers@postgresql.org Subject: Re: [PERFORM] Bad n_distinct estimation; hacks suggested? References: <200504191209.05181.josh@agliodbs.com> <1114580284.426f253cc0087@webmail.telus.net> <426F9703.4010108@dunslane.net> <200504270825.17550.josh@agliodbs.com> <4277774F.7040205@logix-tt.com> In-Reply-To: <4277774F.7040205@logix-tt.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit User-Agent: Internet Messaging Program (IMP) 3.1-cvs X-Originating-IP: 209.17.183.249 X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=1.071 tagged_above=0 required=5 tests=AWL, DNS_FROM_RFC_POST, FORGED_RCVD_HELO X-Spam-Level: * X-Archive-Number: 200505/254 X-Sequence-Number: 67130 Quoting Markus Schaber : > Hi, Josh, > > Josh Berkus wrote: > > > Yes, actually. We need 3 different estimation methods: > > 1 for tables where we can sample a large % of pages (say, >= 0.1) > > 1 for tables where we sample a small % of pages but are "easily > estimated" > > 1 for tables which are not easily estimated by we can't afford to > sample a > > large % of pages. > > > > If we're doing sampling-based estimation, I really don't want > people to lose > > sight of the fact that page-based random sampling is much less > expensive than > > row-based random sampling. We should really be focusing on > methods which > > are page-based. Okay, although given the track record of page-based sampling for n-distinct, it's a bit like looking for your keys under the streetlight, rather than in the alley where you dropped them :-) How about applying the distinct-sampling filter on a small extra data stream to the stats collector? -- Engineers think equations approximate reality. Physicists think reality approximates the equations. Mathematicians never make the connection.