From: Simon Riggs <simon@2ndquadrant.com>
To: Gavin Sherry <swm@linuxworld.com.au>
Cc: David Boreham <david_list@boreham.org>
Cc: pgsql-hackers@postgresql.org
Subject: Re: ice-broker scan thread
Date: Tue, 29 Nov 2005 22:19:27 +0000
Message-ID: <1133302767.2906.462.camel@localhost.localdomain> (raw)
In-Reply-To: <Pine.LNX.4.58.0511300826120.23317@linuxworld.com.au>
References: <Pine.LNX.4.58.0511282217470.13586@josh.db>
<Pine.LNX.4.58.0511291429330.18112@linuxworld.com.au>
<20051129110214.GA31333@svana.org>
<438C76DA.4060801@boreham.org>
<20051129164651.GF31333@svana.org>
<438C8FD9.3090302@boreham.org>
<Pine.LNX.4.58.0511300826120.23317@linuxworld.com.au>
On Wed, 2005-11-30 at 08:30 +1100, Gavin Sherry wrote:
> On Tue, 29 Nov 2005, David Boreham wrote:
>
> >
> > >By default when you use aio you get the version in libc (-lrt IIRC)
> > >which has the issue I mentioned, probably because it's probably
> > >optimised for the lots-of-network-connections type program where
> > >multiple outstanding requests on a single fd are not meaningful. You
> > >can however link in some other library which gives you kernel support.
> > >However, I don't have a new enough kernel to have the kernel support so
> > >I havn't tested that.
> > >
> > >
> > Actually, after reading up on the current state of things, I'm not sure you
> > can even get POSIX aio on top of kernel aio in Linux. There are also a
> > few limitations in the 2.6 aio implementation that might prove troublesome:
> > for example it only works with O_DIRECT.
> >
> > libaio gives userland access to the kernel aio api (which is different
> > from POSIX aio).
>
> Yes. The O_DIRECT issue is my biggest concern about Linux at the moment.
> That being said, the plan is to only pre-fetch the next N blocks, where N
> < 32, and to read them into the local buffer cache. In a situation where
> space in the cache low (and prefetched pages might be pushed out before we
> even get to read them), we need to provide such information to the
> readahead mechanism so that it can reduce the number of blocks which it
> prefetches.
My understanding was that Linux at least has a reasonable readahead
mechanism that works on the scale you suggest.
I think its fair to assume that anybody that wants this can afford
sufficient memory to make it worthwhile. Multiple processes per scan
implies (low numbers of users or I/O overkill).
Best Regards, Simon Riggs
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: simon@2ndquadrant.com, swm@linuxworld.com.au, david_list@boreham.org
Subject: Re: ice-broker scan thread
In-Reply-To: <1133302767.2906.462.camel@localhost.localdomain>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox