Received: from localhost (unknown [200.46.204.183]) by mail.postgresql.org (Postfix) with ESMTP id 8773764FCBD for ; Fri, 12 Dec 2008 12:59:18 -0400 (AST) Received: from mail.postgresql.org ([200.46.204.86]) by localhost (mx1.hub.org [200.46.204.183]) (amavisd-maia, port 10024) with ESMTP id 08600-02 for ; Fri, 12 Dec 2008 12:59:15 -0400 (AST) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from outmail137085.authsmtp.co.uk (outmail137085.authsmtp.co.uk [62.13.137.85]) by mail.postgresql.org (Postfix) with ESMTP id 08C2864FCAB for ; Fri, 12 Dec 2008 12:59:14 -0400 (AST) Received: from mail-c188.authsmtp.com (mail-c188.authsmtp.com [62.13.128.25]) by punt4.authsmtp.com (8.14.2/8.14.2/Kp) with ESMTP id mBCGwYrv070419; Fri, 12 Dec 2008 16:58:34 GMT Received: from [192.168.0.3] (85-211-226-82.dyn.gotadsl.co.uk [85.211.226.82]) (authenticated bits=0) by mail.authsmtp.com (8.14.2/8.14.2/Kp) with ESMTP id mBCGwU4h054598; Fri, 12 Dec 2008 16:58:30 GMT Subject: Re: benchmarking the query planner From: Simon Riggs To: Tom Lane Cc: Gregory Stark , Robert Haas , Kevin Grittner , "jd@commandprompt.com" , Josh Berkus , Greg Smith , pgsql-hackers@postgresql.org In-Reply-To: <6558.1229098576@sss.pgh.pa.us> References: <603c8f070812061019r5aca3aajeaa5493b11c5feff@mail.gmail.com> <603c8f070812092025k6325e0fcud001b205a507a634@mail.gmail.com> <4313.1228931932@sss.pgh.pa.us> <603c8f070812101828p1bd7df49yb8c916adedb49fe1@mail.gmail.com> <9565.1229018968@sss.pgh.pa.us> <1229033073.13078.179.camel@hp_dx2400_1> <87ej0etkfh.fsf@oxford.xeocode.com> <1229039044.13078.193.camel@hp_dx2400_1> <4136ffa0812111823u645b6ec9wdca60b3da4b00499@mail.gmail.com> <1229072675.13078.196.camel@hp_dx2400_1> <603c8f070812120344m67ef2c1fs41806cfb4ff9e396@mail.gmail.com> <1229093975.13078.275.camel@ebony.2ndQuadrant> <87tz99s7bu.fsf@oxford.xeocode.com> <6558.1229098576@sss.pgh.pa.us> Content-Type: text/plain Date: Fri, 12 Dec 2008 16:58:14 +0000 Message-Id: <1229101094.8673.44.camel@ebony.2ndQuadrant> Mime-Version: 1.0 X-Mailer: Evolution 2.12.0 Content-Transfer-Encoding: 7bit X-Server-Quench: 1adc376c-c86e-11dd-aecc-001871e930f4 X-AuthRoute: OCdxZQATClZOTQEd DAteCiNZVAwpPBRK HVkIKg5MJUcNSQVJ NksafhtFaQdbaVxS HGQLW1xEUVt7XGt/ aAsfZQBDYEtPQQdo VlZBQE1QEQdtHhxP W2xtJV4Xc11GfXp1 bE4sXHBeXwp7JBIu QUlSRnBVbWE1dWEe BBRFJlYHIx5Kfh5N bVEqXXANN2FJBC9q VzwTFhsSEA9kHWxf TwcGLVcbW0UKGDF0 WxAPVSwyEFcUDz03 JhogJ0R0 X-Authentic-SMTP: 61633235383639.cat.dmpriest.net.uk:1378/Kp X-Report-SPAM: If SPAM / abuse - report it at: http://www.authsmtp.com/abuse X-Virus-Status: No virus detected - but ensure you scan with your own anti-virus system! X-Virus-Scanned: Maia Mailguard 1.0.1 X-Spam-Status: No, hits=0 tagged_above=0 required=5 tests=none X-Spam-Level: X-Archive-Number: 200812/956 X-Sequence-Number: 129585 On Fri, 2008-12-12 at 11:16 -0500, Tom Lane wrote: > Perhaps a better plan is to try to de-emphasize use of ndistinct, > though I concede I have no idea how to do that. We don't actually care about the accuracy of the ndistinct much, just the accuracy of our answer to the question "given work_mem = X, is it better to use a hash plan". So we just need to scan the table until we can answer that question accurately enough. i.e. a variable sized sample. Perhaps we could store a probability distribution for various values of work_mem, rather than a single ndistinct value. Anyway, definitely handwaving now to stimulate ideas. -- Simon Riggs www.2ndQuadrant.com PostgreSQL Training, Services and Support