pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Richard Huxton <dev@archonet.com>
To: Greg Stark <gsstark@mit.edu>
Cc: Josh Berkus <josh@agliodbs.com>
Cc: Tambet Matiisen <t.matiisen@aprote.ee>
Cc: pgsql-performance@postgresql.org
Subject: Re: What about utility to calculate planner cost constants?
Date: Tue, 22 Mar 2005 17:53:05 +0000
Message-ID: <42405B81.20107@archonet.com> (raw)
In-Reply-To: <87y8cfbqlf.fsf@stark.xeocode.com>
References: <A66A11DBF5525341AEF6B8DE39CDE77008805C@black.aprote.com>
	<200503210951.06044.josh@agliodbs.com>
	<87acowcmut.fsf@stark.xeocode.com>
	<424007E8.1020700@archonet.com>
	<87y8cfbqlf.fsf@stark.xeocode.com>

Greg Stark wrote:
> Richard Huxton <dev@archonet.com> writes:
> 
>>You'd only need to log them if they diverged from expected anyway. That should
>>result in fairly low activity pretty quickly (or we're wasting our time).
>>Should they go to the stats collector rather than logs?
> 
> I think you need to log them all. Otherwise when you go to analyze the numbers
> and come up with ideal values you're going to be basing your optimization on a
> skewed subset.

I can see your thinking, I must admit I was thinking of a more iterative 
process: estimate deltas, change config, check, repeat. I'm not 
convinced there are "ideal" values with a changing workload - for 
example, random_page_cost will presumably vary depending on how much 
contention there is for random seeks. Certainly, effective_cache size 
could vary.

> I don't know whether the stats collector or the logs is better suited to this.
> 
>>>(Also, currently explain analyze has overhead that makes this impractical.
>>>Ideally it could subtract out its overhead so the solutions would be accurate
>>>enough to be useful)
>>
>>Don't we only need the top-level figures though? There's no need to record
>>timings for each stage, just work completed.
> 
> I guess you only need top level values. But you also might want some flag if
> the row counts for any node were far off. In that case perhaps you would want
> to discard the data point.

I think you'd need to adjust work-estimates by actual-rows / estimated-rows.

I _was_ trying to think of a clever way of using row mis-estimates to 
correct statistics automatically. This was triggered by the discussion a 
few weeks ago about hints to the planner and the recent talk about plan 
cacheing. Some sort of feedback loop so the planner would know its 
estimates were off should be a big win from an ease-of-use point of 
view. Didn't look easy to do though :-(

--
   Richard Huxton
   Archonet Ltd



view thread (21+ messages)  latest in thread

Message-ID: <42405B81.20107@archonet.com>
Permalink:  ../42405B81.20107@archonet.com/
Also on:    postgresql.org/message-id/42405B81.20107@archonet.com

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: dev@archonet.com, gsstark@mit.edu, josh@agliodbs.com, t.matiisen@aprote.ee
  Subject: Re: What about utility to calculate planner cost constants?
  In-Reply-To: <42405B81.20107@archonet.com>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox