X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id DD1AB8BA083 for ; Thu, 17 Feb 2005 03:14:10 +0000 (GMT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 98092-06 for ; Thu, 17 Feb 2005 03:14:03 +0000 (GMT) Received: from floppy.pyrenet.fr (news.pyrenet.fr [194.250.190.2]) by svr1.postgresql.org (Postfix) with ESMTP id 0A7A08BA0BE for ; Thu, 17 Feb 2005 03:14:06 +0000 (GMT) Received: by floppy.pyrenet.fr (Postfix, from userid 106) id DC4F731E18; Thu, 17 Feb 2005 04:14:06 +0100 (MET) From: Ron Mayer X-Newsgroups: pgsql.performance Subject: Re: seq scan cache vs. index cache smackdown Date: Wed, 16 Feb 2005 19:22:19 -0800 Organization: Hub.Org Networking Services Lines: 39 Message-ID: <42140DEB.2030207@cheapcomplexdevices.com> References: <9326.1108452962@sss.pgh.pa.us> <87vf8u2qgw.fsf@stark.xeocode.com> <200502150922.59176.josh@agliodbs.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@news.hub.org To: Josh Berkus User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206) X-Accept-Language: en-us, en In-Reply-To: <200502150922.59176.josh@agliodbs.com> To: pgsql-performance@postgresql.org X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0 tagged_above=0 required=5 tests= X-Spam-Level: X-Archive-Number: 200502/249 X-Sequence-Number: 10578 Josh Berkus wrote: > > Now you can see why other DBMSs don't use the OS disk cache. ... > ...as long as we use the OS disk cache, we can't > eliminate checkpoint spikes, at least on Linux. Wouldn't the VM settings like the ones under /proc/sys/vm and/or the commit=XXX mount option if using ext3 be a good place to control this? It seems if you wanted, by setting /proc/sys/vm/dirty_background_ratio and /proc/sys/vm/dirty_expire_centisecs very low you'd be constantly flushing dirty pages. Has anyone experimented with these kinds of values: /proc/sys/vm/dirty_ratio /* the generator of dirty data writes back at this ratio */ /proc/sys/vm/dirty_background_ratio /* start background writeback */ /proc/sys/vm/dirty_writeback_centisecs /* the interval between [some style of] writebacks */ /proc/sys/vm/dirty_expire_centisecs /* the number of centiseconds that data is allowed to remain dirty I tried these to workaround the opposite kind of problem.... on a laptop running linux under vmware I wanted to avoid having it do writes quickly to make each individual transaction go faster; at the expense of a big spike in IO that the sales guy would trigger explicitly before talking a while. Setting each of those very high and using a commit=600 mount option made the whole demo run with very little IO except for the explicit sync; but I never took the time to understand which setting mattered to me or why. >>It seems inevitable that Postgres will eventually eliminate that redundant >>layer of buffering. Since mmap is not workable, that means using O_DIRECT >>to read table and index data.