User-Agent: Microsoft-Entourage/11.2.5.060620
Date: Thu, 21 Sep 2006 20:46:41 -0700
Subject: Re: Large tables (was: RAID 0 not as fast as
From: "Luke Lonergan" <llonergan@greenplum.com>
To: mark@mark.mielke.cc,
	"Guy Thornley" <guy@esphion.com>
cc: "Markus Schaber" <schabi@logix-tt.com>,
	pgsql-performance@postgresql.org
Message-ID: <C138AAB1.2524%llonergan@greenplum.com>
Thread-Topic: [PERFORM] Large tables (was: RAID 0 not as fast as
Thread-Index: Acbd+bbL9XY0c0nsEduGXQAWy4o9DA==
In-Reply-To: <20060922034037.GB2820@mark.mielke.cc>
MIME-Version: 1.0
Content-Type: text/plain;
 charset=us-ascii
Content-Transfer-Encoding: 7bit

Mark,

On 9/21/06 8:40 PM, "mark@mark.mielke.cc" <mark@mark.mielke.cc> wrote:

> I'd advise against using this call unless it can be shown that the page
> will not be used in the future, or at least, that the page is less useful
> than all other pages currently in memory. This is what the call really means.
> It means, "There is no value to keeping this page in memory".

Yes, it's a bit subtle.

I think the topic is similar to "cache bypass", used in cache capable vector
processors (Cray, Convex, Multiflow, etc) in the 90's.  When you are
scanning through something larger than the cache, it should be marked
"non-cacheable" and bypass caching altogether.  This avoids a copy, and
keeps the cache available for things that can benefit from it.

WRT the PG buffer cache, the rule would have to be: "if the heap scan is
going to be larger than "effective_cache_size", then issue the
posix_fadvise(BLOCK_NOT_NEEDED) call".  It doesn't sound very efficient to
do this in block/extent increments though, and it would possibly mess with
subsets of the block space that would be re-used for other queries.

- Luke