Date: Fri, 21 Apr 2006 15:15:11 +0800
From: Wu Fengguang <wfg@mail.ustc.edu.cn>
To: "Jim C. Nasby" <jnasby@pervasive.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Introducing a new linux readahead framework
Message-ID: <20060421071511.GA4885@mail.ustc.edu.cn>
Mail-Followup-To: Wu Fengguang <wfg@mail.ustc.edu.cn>,
	"Jim C. Nasby" <jnasby@pervasive.com>,
	pgsql-performance@postgresql.org
References: <20060421013826.GA5217@mail.ustc.edu.cn>
	<20060421043147.GV49405@pervasive.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20060421043147.GV49405@pervasive.com>
User-Agent: Mutt/1.5.11+cvs20060126

On Thu, Apr 20, 2006 at 11:31:47PM -0500, Jim C. Nasby wrote:
> > In adaptive readahead, the context based method may be of particular
> > interest to postgresql users. It works by peeking into the file cache
> > and check if there are any history pages present or accessed. In this
> > way it can detect almost all forms of sequential / semi-sequential read
> > patterns, e.g.
> > 	- parallel / interleaved sequential scans on one file
> > 	- sequential reads across file open/close
> > 	- mixed sequential / random accesses
> > 	- sparse / skimming sequential read
> > 
> > It also have methods to detect some less common cases:
> > 	- reading backward
> > 	- seeking all over reading N pages
> 
> Are there any ways to inform the kernel that you either are or aren't
> doing a sequential read? It seems that in some cases it would be better

This call will disable readahead totally for fd:
        posix_fadvise(fd, any, any, POSIX_FADV_RANDOM);

This one will reenable it:
        posix_fadvise(fd, any, any, POSIX_FADV_NORMAL);

This one will enable readahead _and_ set max readahead window to
2*max_readahead_kb:
        posix_fadvise(fd, any, any, POSIX_FADV_SEQUENTIAL);

> to bypass a bunch of tricky logic trying to determine that it's doing a
> sequential read. A sequential scan in PostgreSQL would be such a case.

You do not need to worry about the detecting `overhead' on sequential
scans :) The adaptive readahead framework has a fast code path(the
stateful method) to handle normal sequential reads, the detection of
which is really trivial.

> The opposite example would be an index scan of a highly uncorrelated
> index, which would produce mostly random reads from the table. In that
> case, reading ahead probably makes very little sense, though your logic
> might have a better idea of the access pattern than PostgreSQL does.

As for the index scans, the failsafe code path(i.e. the context based
one) will normally be used, and it does have a little overhead in
looking up the page cache(about 0.4% more CPU time). However, the
penalty of random disk access is so large that if ever it helps
reducing a small fraction of disk accesses, you wins.

Thanks,
Wu