DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma;
	h=message-id:date:from:to:subject:cc:in-reply-to:mime-version
	:content-type:content-transfer-encoding:content-disposition
	:references;
	b=OHoxuOHUYBUQfUIrpAS74FYudX3WZFwpAaRDrv2ylEqYd4aMhJKzYgkiFrrPfv/dQl
	QAL/uHyunsKrIEJY6zUy8/isj0l9r6+NGTs2VEVSkaMAlDUAiTY5rzcJUki4R3GmNe+y
	rNJnUMOkCTTNLqASZ2TzogRnZgYv/g24UihRg=
Message-ID: <603c8f070901022013u242179fbq4ba97e2f5aea3715@mail.gmail.com>
Date: Fri, 2 Jan 2009 23:13:10 -0500
From: "Robert Haas" <robertmhaas@gmail.com>
To: "Greg Stark" <stark@enterprisedb.com>
Subject: Re: posix_fadvise v22
Cc: "Greg Smith" <gsmith@gregsmith.com>, "Tom Lane" <tgl@sss.pgh.pa.us>,
	"ITAGAKI Takahiro" <itagaki.takahiro@oss.ntt.co.jp>,
	Postgres <pgsql-hackers@postgresql.org>
In-Reply-To: <4136ffa0901021902m3d9565e8v6d62a75c4a81511a@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <FDDBA24E-FF4D-4654-BA75-692B3BA71B97@enterprisedb.com>
	<87k5a6tmy7.fsf@oxford.xeocode.com>
	<603c8f070901011230t1afa6644racb715b975078054@mail.gmail.com>
	<10658.1230843315@sss.pgh.pa.us>
	<603c8f070901011843p121e40afu35c4a7bdc51c051c@mail.gmail.com>
	<Pine.GSO.4.64.0901020335150.16680@westnet.com>
	<603c8f070901021436v71f10188g1d747b493c89f2f5@mail.gmail.com>
	<4136ffa0901021502w78f0611dx63929ac0fbbdc2f4@mail.gmail.com>
	<603c8f070901021742s61dcbaf2r51deff1219196944@mail.gmail.com>
	<4136ffa0901021902m3d9565e8v6d62a75c4a81511a@mail.gmail.com>

> Any chance you could put back the code in explain.c which showed
> whether posix_fadvise is actually getting used? Another thing I did
> when testing was attaching with strace to see if posix_fadvise (the
> syscall on linux was actually fadvise64 iirc) is actually getting
> called.

I tried changing this:
        returnCode = posix_fadvise(VfdCache[file].fd, offset, amount,
POSIX_FADV_WILLNEED);
to this:
        returnCode = 0;

When I did that, it when back from 50 s to 33 s, which I think means
that posix_fadvise is getting called and that that is what is making
it slower.

> And is this on a system with multiple spindles? How many?

Latitude D830 laptop.  Single disk.  Fedora 9.  kernel-2.6.27.9-73.fc9.x86_64.

> And how much of the data is in shared buffers or in filesystem cache?
> Is this consistent for repeated queries? Is it only when you're
> repeating a query for dates that you've already selected?

I stopped the cluster, dropped the page cache, and restarted the
cluster just before testing.  Repeated tests are fast due to caching
effects.  shared_buffers is 240MB.  System has 2GB RAM, steady state
is about 1GB of page cache.

> And how fast is it with effective_io_concurrency set to 1,2,3,5,6,7,8,...?

I do not currently have this information.  :-)

I will try to run some more tests over the weekend, but I'm too tired
now and am starting to make mistakes.

> Do you see the same effect if you use a self-contained test case
> instead of the TPC-H data so I can try it?

Not sure exactly what you have in mind here.  If you send a script or
something to reproduce I will try it.

...Robert