X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 4383A545D0 for ; Tue, 26 Apr 2005 21:19:27 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 61279-03 for ; Wed, 27 Apr 2005 00:19:23 +0000 (GMT) Received: from tht.net (vista.tht.net [216.126.88.2]) by svr1.postgresql.org (Postfix) with ESMTP id A560C54589 for ; Tue, 26 Apr 2005 21:19:21 -0300 (ADT) Received: from 216-154-15-113.dsl.look.ca (216-154-15-113.dsl.look.ca [216.154.15.113]) by tht.net (Postfix) with ESMTP id 1D0BE76A29; Tue, 26 Apr 2005 20:19:30 -0400 (EDT) Subject: Re: [PERFORM] Bad n_distinct estimation; hacks suggested? From: Rod Taylor To: Greg Stark Cc: pgsql-hackers@postgresql.org, Gurmeet Manku In-Reply-To: <87oec15d9m.fsf@stark.xeocode.com> References: <87u0lt5efo.fsf@stark.xeocode.com> <1114557011.77587.107.camel@home> <87oec15d9m.fsf@stark.xeocode.com> Content-Type: text/plain Date: Tue, 26 Apr 2005 20:16:32 -0400 Message-Id: <1114560992.77587.112.camel@home> Mime-Version: 1.0 X-Mailer: Evolution 2.2.2 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0 tagged_above=0 required=5 tests= X-Spam-Level: X-Archive-Number: 200504/873 X-Sequence-Number: 66742 On Tue, 2005-04-26 at 19:28 -0400, Greg Stark wrote: > Rod Taylor writes: > > > On Tue, 2005-04-26 at 19:03 -0400, Greg Stark wrote: > > > This one looks *really* good. > > > > > > http://www.aladdin.cs.cmu.edu/papers/pdfs/y2001/dist_sampl.pdf > > > > > > It does require a single full table scan > > > > Ack.. Not by default please. > > > > I have a few large append-only tables (vacuum isn't necessary) which do > > need stats rebuilt periodically. > > The algorithm can also naturally be implemented incrementally. Which would be > nice for your append-only tables. But that's not Postgres's current philosophy > with statistics. Perhaps some trigger function that you could install yourself > to update statistics for a newly inserted record would be useful. If when we have partitions, that'll be good enough. If partitions aren't available this would be quite painful to anyone with large tables -- much as the days of old used to be painful for ANALYZE. --