pg.ddx.io  pgsql-performance@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Greg Stark <gsstark@mit.edu>
Cc: Richard Huxton <dev@archonet.com>
Cc: David Brown <time@bigpond.net.au>
Cc: pgsql-performance@postgresql.org
Subject: Re: Seqscan rather than Index
Date: Fri, 17 Dec 2004 12:44:41 -0500
Message-ID: <21110.1103305481@sss.pgh.pa.us> (raw)
In-Reply-To: <87acsdkjte.fsf@stark.xeocode.com>
References: <20041217011853.9E8FF3CADB0@svr1.postgresql.org>
	<41C2D936.3060901@archonet.com>
	<87acsdkjte.fsf@stark.xeocode.com>

Greg Stark <gsstark@mit.edu> writes:
> Postgres is also more pessimistic about the efficiency of index scans. It's
> willing to use a sequential scan down to well below 5% selectivity when other
> databases use the more traditional rule of thumb of 10%.

However, other databases are probably basing their analysis on a
different execution model.  Since we have to visit both heap and index
in all cases, we do indeed have a larger penalty for index use.

I've looked pretty closely at the cost model for index access, believe me.
It's not pessimistic; if anything it is undercharging for index access.
(For one thing it treats the index's internal fetches as sequential
access, when in reality they are probably random.)

I think the one effect that's not being modeled is amortization of index
fetches across successive queries.  The cost model is pretty much based
on the assumption that each query starts from ground zero, whereas in
reality a heavily used index will certainly have all its upper levels in
RAM, and if it's not too large the leaf pages might all be cached too.
I wouldn't want to switch the planner over to making that assumption
exclusively, but we could talk about having a cost parameter that dials
the assumption up or down.

Awhile back I tried rewriting btcostestimate to charge zero for
accessing the metapage and the upper index levels, but charge
random_page_cost for fetching leaf pages.  For small indexes this came
out with worse (larger) numbers than we have now, which is not the
direction we want to go in :-(.  So I think that we have to somehow
honestly model caching of index pages across queries.

Of course, to be completely fair such a modification should account for
caching of heap pages as well, so it would also bring down the estimates
for seqscans.  But I'd be willing to accept a model that considers only
caching of index pages as a zero-order approximation.

			regards, tom lane



view thread (16+ messages)  latest in thread

Message-ID: <21110.1103305481@sss.pgh.pa.us>
Permalink:  ../21110.1103305481@sss.pgh.pa.us/
Also on:    postgresql.org/message-id/21110.1103305481@sss.pgh.pa.us

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-performance@postgresql.org
  Cc: tgl@sss.pgh.pa.us, gsstark@mit.edu, dev@archonet.com, time@bigpond.net.au
  Subject: Re: Seqscan rather than Index
  In-Reply-To: <21110.1103305481@sss.pgh.pa.us>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox