DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=dk20050327;
 d=earthlink.net;
	b=ZD7w/7wn+TfSbCif48o8IpHsDIgOXWhq16BU2u3it+z2c7GldnbRH5v/YeXx/4hU;
	h=Received:Message-Id:X-Mailer:Date:To:From:Subject:Cc:In-Reply-To:References:Mime-Version:Content-Type:X-ELNK-Trace:X-Originating-IP;
Message-Id: <7.0.1.0.2.20060920153217.03d6be30@earthlink.net>
Date: Wed, 20 Sep 2006 15:35:11 -0400
To: Markus Schaber <schabi@logix-tt.com>
From: Ron <rjpeace@earthlink.net>
Subject: Re: Large tables (was: RAID 0 not as fast as
Cc: pgsql-performance@postgresql.org
In-Reply-To: <4511821E.2030902@logix-tt.com>
References: <C136C28E.4E9%llonergan@greenplum.com>
	<4511821E.2030902@logix-tt.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed

IMHO, AIO is the architecturally cleaner and more elegant solution.

We in fact have a project on the boards to do this but funding (as 
yet) has not been found.

My $.02,
Ron


At 02:02 PM 9/20/2006, Markus Schaber wrote:
>Hi, Luke,
>
>Luke Lonergan wrote:
>
> >> Do you think that adding some posix_fadvise() calls to the backend to
> >> pre-fetch some blocks into the OS cache asynchroneously could improve
> >> that situation?
> >
> > Nope - this requires true multi-threading of the I/O, there need to be
> > multiple seek operations running simultaneously.  The current executor
> > blocks on each page request, waiting for the I/O to happen before 
> requesting
> > the next page.  The OS can't predict what random page is to be requested
> > next.
>
>I thought that posix_fadvise() with POSIX_FADV_WILLNEED was exactly
>meant for this purpose?
>
>My idea was that the executor could posix_fadvise() the blocks it will
>need in the near future, and later, when it actually issues the blocking
>read, the block is there already. This could even give speedups in the
>single-spindle case, as the I/O scheduler could already fetch the next
>blocks while the executor processes the current one.
>
>But there must be some details in the executor that prevent this.
>
> > We can implement multiple scanners (already present in MPP), or we could
> > implement AIO and fire off a number of simultaneous I/O requests for
> > fulfillment.
>
>AIO is much more intrusive to implement, so I'd preferrably look
>whether posix_fadvise() could improve the situation.
>
>Thanks,
>Markus
>
>---------------------------(end of broadcast)---------------------------
>TIP 3: Have you checked our extensive FAQ?
>
>                http://www.postgresql.org/docs/faq