X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 44FC73CCF06 for ; Fri, 17 Dec 2004 17:45:14 +0000 (GMT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 83121-09 for ; Fri, 17 Dec 2004 17:45:06 +0000 (GMT) Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by svr1.postgresql.org (Postfix) with ESMTP id 74F1B3CCF1E for ; Fri, 17 Dec 2004 17:45:05 +0000 (GMT) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.13.1/8.13.1) with ESMTP id iBHHifxt021111; Fri, 17 Dec 2004 12:44:41 -0500 (EST) To: Greg Stark Cc: Richard Huxton , David Brown , pgsql-performance@postgresql.org Subject: Re: Seqscan rather than Index In-reply-to: <87acsdkjte.fsf@stark.xeocode.com> References: <20041217011853.9E8FF3CADB0@svr1.postgresql.org> <41C2D936.3060901@archonet.com> <87acsdkjte.fsf@stark.xeocode.com> Comments: In-reply-to Greg Stark message dated "17 Dec 2004 10:47:57 -0500" Date: Fri, 17 Dec 2004 12:44:41 -0500 Message-ID: <21110.1103305481@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.005 tagged_above=0 required=5 tests=AWL X-Spam-Level: X-Archive-Number: 200412/214 X-Sequence-Number: 9589 Greg Stark writes: > Postgres is also more pessimistic about the efficiency of index scans. It's > willing to use a sequential scan down to well below 5% selectivity when other > databases use the more traditional rule of thumb of 10%. However, other databases are probably basing their analysis on a different execution model. Since we have to visit both heap and index in all cases, we do indeed have a larger penalty for index use. I've looked pretty closely at the cost model for index access, believe me. It's not pessimistic; if anything it is undercharging for index access. (For one thing it treats the index's internal fetches as sequential access, when in reality they are probably random.) I think the one effect that's not being modeled is amortization of index fetches across successive queries. The cost model is pretty much based on the assumption that each query starts from ground zero, whereas in reality a heavily used index will certainly have all its upper levels in RAM, and if it's not too large the leaf pages might all be cached too. I wouldn't want to switch the planner over to making that assumption exclusively, but we could talk about having a cost parameter that dials the assumption up or down. Awhile back I tried rewriting btcostestimate to charge zero for accessing the metapage and the upper index levels, but charge random_page_cost for fetching leaf pages. For small indexes this came out with worse (larger) numbers than we have now, which is not the direction we want to go in :-(. So I think that we have to somehow honestly model caching of index pages across queries. Of course, to be completely fair such a modification should account for caching of heap pages as well, so it would also bring down the estimates for seqscans. But I'd be willing to accept a model that considers only caching of index pages as a zero-order approximation. regards, tom lane