To: "Simon Riggs" <simon@2ndquadrant.com>
Cc: "Pg Hackers" <pgsql-hackers@postgresql.org>
Subject: Re: Commitfest patches
In-Reply-To: <1206697482.4285.1518.camel@ebony.site> (Simon Riggs's message of
	"Fri\, 28 Mar 2008 09\:44\:42 +0000")
User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.1 (gnu/linux)
References: <87wsnnz046.fsf@oxford.xeocode.com>
	<1206697482.4285.1518.camel@ebony.site>
From: Gregory Stark <stark@enterprisedb.com>
Organization: EnterpriseDB
Date: Fri, 28 Mar 2008 11:26:54 +0000
Message-ID: <87od8zytq9.fsf@oxford.xeocode.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

"Simon Riggs" <simon@2ndquadrant.com> writes:

> On Fri, 2008-03-28 at 09:08 +0000, Gregory Stark wrote:
>
>> A more invasive form of this patch would be to assign and pin a buffer when
>> the preread is done. That would men subsequently we would have a pinned buffer
>> ready to go and not need to go back to the buffer manager a second time. We
>> would instead just "complete" the i/o by issuing a normal read call.
>
> So if posix_fadvise did nothing or there was a longer than optimal
> delay, this would be a net loss.
>
> You'd need reasonable evidence that the posix_fadvise facility was a win
> on all platforms and recent release levels before we could agree with
> that.

I posted a test program and asked for others to test it on various platforms
and various sized raid arrays. matthew@flymine.org was the only person who
bothered to run it and he only found a 16x speedup on his hardware.

 http://archives.postgresql.org/pgsql-performance/2008-01/msg00285.php

(unfortunately our mail archives seem to have failed to archive his message,
this is my followup to it)

> I think we need a more thorough examination of this area before we
> commit anything. Maybe you've done this, but I haven't seen the
> analysis. Can you say more, please? Or at least say what you don't know,
> so other experts listening can fill in the blanks.

For heaven's sake. I've been posting about this for months. Search the
archives for "RAID arrays and performance", "There's random access and then
there's random access", and "Bitmap index scan preread using posix_fadvise".

I described which interfaces worked on Linux and Solaris based on empirical
tests. I posted source code for synthetic benchmarks so we could test it on a
wide range of hardware. I posted graphs based on empirical results. I posted
mathematical formulas analysing just how much preread would be expected to
exercise a raid array fully. I'm not sure what else I can do to effect a more
thorough examination.

-- 
  Gregory Stark
  EnterpriseDB          http://www.enterprisedb.com
  Ask me about EnterpriseDB's PostGIS support!