X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 3639F3A4143 for ; Mon, 25 Oct 2004 18:14:45 +0100 (BST) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 72181-08 for ; Mon, 25 Oct 2004 17:14:42 +0000 (GMT) Received: from stark.xeocode.com (gsstark.mtl.istop.com [66.11.160.162]) by svr1.postgresql.org (Postfix) with ESMTP id 7FCE73A3AEC for ; Mon, 25 Oct 2004 18:14:42 +0100 (BST) Received: from localhost ([127.0.0.1] helo=stark.xeocode.com) by stark.xeocode.com with smtp (Exim 3.36 #1 (Debian)) id 1CM8QZ-0001sm-00; Mon, 25 Oct 2004 13:14:39 -0400 To: pgsql-hackers@postgresql.org Subject: Re: [PATCHES] ARC Memory Usage analysis References: <1098471059.20926.49.camel@localhost.localdomain> <41796115.2020501@Yahoo.com> <20041022200955.GC4382@it.is.rice.edu> <417D1D01.3090401@Yahoo.com> <20651.1098720192@sss.pgh.pa.us> In-Reply-To: <20651.1098720192@sss.pgh.pa.us> From: Greg Stark Organization: The Emacs Conspiracy; member since 1992 Date: 25 Oct 2004 13:14:39 -0400 Message-ID: <87mzyavhxc.fsf@stark.xeocode.com> Lines: 25 User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.3 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200410/829 X-Sequence-Number: 60332 Tom Lane writes: > However, I'm still really nervous about the idea of using > effective_cache_size to control the ARC algorithm. That number is > usually entirely bogus. It wouldn't be too hard to have a port-specific function that tries to guess the total amount of memory. That isn't always right but it's at least a better ballpark default than a fixed arbitrary value. However I wonder about another approach entirely. If postgres timed how long reads took it shouldn't find it very hard to distinguish between a cached buffer being copied and an actual i/o operation. It should be able to track the percentage of time that buffers requested are in the kernel's cache and use that directly instead of the estimated cache size. Adding two gettimeofdays to every read call would be annoyingly expensive. But a port-specific function to check the cpu instruction counter could be useful. It doesn't have to be an accurate measurement of time (such as on some multi-processor machines) as long as it's possible to distinguish when a slow disk operation has occurred from when no disk operation has occurred. -- greg