X-Original-To: pgsql-performance-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id B841432B43C for ; Sat, 9 Oct 2004 04:20:03 +0100 (BST) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 45254-05 for ; Sat, 9 Oct 2004 03:19:58 +0000 (GMT) Received: from news.hub.org (news.hub.org [200.46.204.72]) by svr1.postgresql.org (Postfix) with ESMTP id 1634632B429 for ; Sat, 9 Oct 2004 04:19:57 +0100 (BST) Received: from news.hub.org (news.hub.org [200.46.204.72]) by news.hub.org (8.12.9/8.12.9) with ESMTP id i993Ju2B046079 for ; Sat, 9 Oct 2004 03:19:56 GMT (envelope-from news@news.hub.org) Received: (from news@localhost) by news.hub.org (8.12.9/8.12.9/Submit) id i992nwbS038767 for pgsql-performance@postgresql.org; Sat, 9 Oct 2004 02:49:58 GMT From: Christopher Browne X-Newsgroups: comp.databases.postgresql.performance Subject: Re: First set of OSDL Shared Mem scalability results, some wierdness ... Date: Fri, 08 Oct 2004 22:10:19 -0400 Organization: cbbrowne Computing Inc Lines: 48 Message-ID: References: <200410081443.16109.josh@agliodbs.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Complaints-To: usenet@news.hub.org X-message-flag: Outlook is rather hackable, isn't it? X-Home-Page: http://www.cbbrowne.com/info/ X-Affero: http://svcs.affero.net/rm.php?r=cbbrowne User-Agent: Gnus/5.1006 (Gnus v5.10.6) XEmacs/21.4 (Security Through Obscurity, linux) Cancel-Lock: sha1:MSmAo22qNssFcOru6grzK7aLro0= To: pgsql-performance@postgresql.org X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200410/118 X-Sequence-Number: 8594 josh@agliodbs.com (Josh Berkus) wrote: > I've been trying to peg the "sweet spot" for shared memory using > OSDL's equipment. With Jan's new ARC patch, I was expecting that > the desired amount of shared_buffers to be greatly increased. This > has not turned out to be the case. That doesn't surprise me. My primary expectation would be that ARC would be able to make small buffers much more effective alongside vacuums and seq scans than they used to be. That does not establish anything about the value of increasing the size buffer caches... > This result is so surprising that I want people to take a look at it > and tell me if there's something wrong with the tests or some > bottlenecking factor that I've not seen. I'm aware of two conspicuous scenarios where ARC would be expected to _substantially_ improve performance: 1. When it allows a VACUUM not to throw useful data out of the shared cache in that VACUUM now only 'chews' on one page of the cache; 2. When it allows a Seq Scan to not push useful data out of the shared cache, for much the same reason. I don't imagine either scenario are prominent in the OSDL tests. Increasing the number of cache buffers _is_ likely to lead to some slowdowns: - Data that passes through the cache also passes through kernel cache, so it's recorded twice, and read twice... - The more cache pages there are, the more work is needed for PostgreSQL to manage them. That will notably happen anywhere that there is a need to scan the cache. - If there are any inefficiencies in how the OS kernel manages shared memory, as their size scales, well, that will obviously cause a slowdown. -- If this was helpful, rate me http://www.ntlug.org/~cbbrowne/internet.html "One World. One Web. One Program." -- MICROS~1 hype "Ein Volk, ein Reich, ein Fuehrer" -- Nazi hype (One people, one country, one leader)