X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 2390B3A3C35 for ; Mon, 25 Oct 2004 22:53:43 +0100 (BST) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 95841-08 for ; Mon, 25 Oct 2004 21:53:32 +0000 (GMT) Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by svr1.postgresql.org (Postfix) with ESMTP id 2DFF13A422C for ; Mon, 25 Oct 2004 22:53:32 +0100 (BST) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.13.1/8.13.1) with ESMTP id i9PLrQWC000791; Mon, 25 Oct 2004 17:53:26 -0400 (EDT) To: Greg Stark Cc: pgsql-hackers@postgresql.org Subject: Re: [PATCHES] ARC Memory Usage analysis In-reply-to: <87hdoiv62b.fsf@stark.xeocode.com> References: <1098471059.20926.49.camel@localhost.localdomain> <41796115.2020501@Yahoo.com> <20041022200955.GC4382@it.is.rice.edu> <417D1D01.3090401@Yahoo.com> <20651.1098720192@sss.pgh.pa.us> <87mzyavhxc.fsf@stark.xeocode.com> <87hdoiv62b.fsf@stark.xeocode.com> Comments: In-reply-to Greg Stark message dated "25 Oct 2004 17:30:52 -0400" Date: Mon, 25 Oct 2004 17:53:25 -0400 Message-ID: <790.1098741205@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.0 tagged_above=0.0 required=5.0 tests= X-Spam-Level: X-Archive-Number: 200410/853 X-Sequence-Number: 60356 Greg Stark writes: > So I would suggest using something like 100us as the threshold for > determining whether a buffer fetch came from cache. I see no reason to hardwire such a number. On any hardware, the distribution is going to be double-humped, and it will be pretty easy to determine a cutoff after minimal accumulation of data. The real question is whether we can afford a pair of gettimeofday() calls per read(). This isn't a big issue if the read actually results in I/O, but if it doesn't, the percentage overhead could be significant. If we assume that the effective_cache_size value isn't changing very fast, maybe it would be good enough to instrument only every N'th read (I'm imagining N on the order of 100) for this purpose. Or maybe we need only instrument reads that are of blocks that are close to where the ARC algorithm thinks the cache edge is. One small problem is that the time measurement gives you only a lower bound on the time the read() actually took. In a heavily loaded system you might not get the CPU back for long enough to fool you about whether the block came from cache or not. Another issue is what we do with the effective_cache_size value once we have a number we trust. We can't readily change the size of the ARC lists on the fly. regards, tom lane