User-Agent: Microsoft-Entourage/11.2.1.051004
Date: Wed, 01 Feb 2006 09:42:12 -0800
Subject: Re: Huge Data sets, simple queries
From: "Luke Lonergan" <llonergan@greenplum.com>
To: "PFC" <lists@peufeu.com>,
	"Jeffrey W. Baker" <jwbaker@acm.org>
cc: pgsql-performance@postgresql.org
Message-ID: <C00632F4.1B71F%llonergan@greenplum.com>
Thread-Topic: [PERFORM] Huge Data sets, simple queries
Thread-Index: AcYnDgi8W6A/b6ncQWWiAoo3F5S6bAASMw0g
In-Reply-To: <op.s4ad01lhcigqcu@apollo13>
MIME-Version: 1.0
Content-Type: text/plain;
 charset=us-ascii
Content-Transfer-Encoding: 7bit

PFC,


On 2/1/06 1:01 AM, "PFC" <lists@peufeu.com> wrote:

>         3- Read one block of A, then one block of B, then one block of A, etc.
> Essentially this is the same as the threaded case, except there's only one
> thread.
>                 53 seconds total (with heavy seeking noise from the hdd).
> 
>         I was half expecting 3 to take the same as 2. It simulates, for
> instance, 
> scanning a table and its index, or scanning 2 sort bins. Well, maybe one
> day...

This is actually interesting overall - I think what this might be showing is
that the Linux SW RAID1 is alternating I/Os to the mirror disks from
different processes (LWP or HWP both maybe?), but not within one process.

>         It would be nice if the Kernel had an API for applications to tell it
> "I'm gonna need these blocks in the next seconds, can you read them in the
> order you like (fastest), from whatever disk you like, and cache them for
> me please; so that I can read them in the order I like, but very fast ?"

More control is always good IMO, but for now there's I/O reordering in the
SCSI layer and readahead tuning.  There is POSIX fadvise() also to tell the
underlying I/O layer what the access pattern looks like.

- Luke