From: Qingqing Zhou <zhouqq@cs.toronto.edu>
To: pgsql-hackers@postgresql.org
Subject: Re: ice-broker scan thread
Date: Tue, 29 Nov 2005 00:55:36 -0500
Message-ID: <dmgqf2$2e82$1@news.hub.org> (raw)
References: <Pine.LNX.4.58.0511282217470.13586@josh.db>
<Pine.LNX.4.58.0511291429330.18112@linuxworld.com.au>
<Pine.LNX.4.58.0511282350370.13833@josh.db>
<438BE53B.1090106@boreham.org>
"David Boreham" <david_list@boreham.org> wrote
>>
> I don't think your NT overlapped I/O code is quite right. At least
> I think it will issue reads at a high rate without waiting for any of them
> to complete. Beyond some point that has to give the kernel gut-rot.
>
[also with reply to Gavin] look up dictionary for "gut-rot", got it ... Uh,
this behavior is intended - I try to push enough requests shortly to kernel
so that it understands that I am doing sequential scan, so it would pull the
data from disk to file system cache more efficiently. Some file systems may
have "free-behind" mechanism, but our main thread (who really process the
query) should be fast enough before the data vanished.
>
> You could re-write your program to have a single thread but use aio.
> In that case it should show the same read ahead benefit that you see
> with the thread.
>
I guess this is also Gavin's point - I understand that will be two different
methodologies to handle "read-ahead". If no other thread/process involved,
then the main thread will be responsible to grab a free buffer page from
bufferpool and ask the kernel to put the data there by sync IO (current
PostgreSQL does) or async IOs. And that's what I want to avoid. I'd like to
use a dedicated thread/process to "break the ice" only, i.e., pull data from
disk to file system cache, so that the main thread will only issue *logical*
read.
Regards,
Qingqing
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-hackers@postgresql.org
Cc: zhouqq@cs.toronto.edu
Subject: Re: ice-broker scan thread
In-Reply-To: <dmgqf2$2e82$1@news.hub.org>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox