pg.ddx.io pgsql-hackers@postgresql.org mailing list archive
help / color / mirror / Atom feedSafe/Fast I/O ...
20+ messages / 10 participants
[nested] [flat]
* Safe/Fast I/O ...
@ 1998-04-12 03:32 The Hermit Hacker <scrappy@hub.org>
0 siblings, 1 reply; 20+ messages in thread
From: The Hermit Hacker @ 1998-04-12 03:32 UTC (permalink / raw)
To: pgsql-hackers
Has anyone looked into this? I'm just getting ready to download it and
play with it, see what's involved in using it. From what I can see, its
essentially an optimized stdio library...
URL is at: http://www.research.att.com/sw/tools/sfio
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] Safe/Fast I/O ...
@ 1998-04-12 05:41 Matthew N. Dodd <winter@jurai.net>
parent: The Hermit Hacker <scrappy@hub.org>
0 siblings, 1 reply; 20+ messages in thread
From: Matthew N. Dodd @ 1998-04-12 05:41 UTC (permalink / raw)
To: The Hermit Hacker <scrappy@hub.org>; +Cc: pgsql-hackers
On Sun, 12 Apr 1998, The Hermit Hacker wrote:
> Has anyone looked into this? I'm just getting ready to download it and
> play with it, see what's involved in using it. From what I can see, its
> essentially an optimized stdio library...
>
> URL is at: http://www.research.att.com/sw/tools/sfio
Using mmap and/or AIO would be better...
FreeBSD and Solaris support AIO I believe. Given past trends Linux will
as well.
/*
Matthew N. Dodd | A memory retaining a love you had for life
winter@jurai.net | As cruel as it seems nothing ever seems to
http://www.jurai.net/~winter | go right - FLA M 3.1:53
*/
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] Safe/Fast I/O ...
@ 1998-04-12 05:57 The Hermit Hacker <scrappy@hub.org>
parent: Matthew N. Dodd <winter@jurai.net>
0 siblings, 1 reply; 20+ messages in thread
From: The Hermit Hacker @ 1998-04-12 05:57 UTC (permalink / raw)
To: Matthew N. Dodd <winter@jurai.net>; +Cc: pgsql-hackers
On Sun, 12 Apr 1998, Matthew N. Dodd wrote:
> On Sun, 12 Apr 1998, The Hermit Hacker wrote:
> > Has anyone looked into this? I'm just getting ready to download it and
> > play with it, see what's involved in using it. From what I can see, its
> > essentially an optimized stdio library...
> >
> > URL is at: http://www.research.att.com/sw/tools/sfio
>
> Using mmap and/or AIO would be better...
>
> FreeBSD and Solaris support AIO I believe. Given past trends Linux will
> as well.
I hate to have to ask, but how is MMAP or AIO better then sfio? I
haven't had enough time to research any of this, and am just starting to
look at it...
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] Safe/Fast I/O ...
@ 1998-04-12 06:08 Matthew N. Dodd <winter@jurai.net>
parent: The Hermit Hacker <scrappy@hub.org>
0 siblings, 2 replies; 20+ messages in thread
From: Matthew N. Dodd @ 1998-04-12 06:08 UTC (permalink / raw)
To: The Hermit Hacker <scrappy@hub.org>; +Cc: pgsql-hackers
On Sun, 12 Apr 1998, The Hermit Hacker wrote:
> I hate to have to ask, but how is MMAP or AIO better then sfio? I
> haven't had enough time to research any of this, and am just starting to
> look at it...
If its simple to compile and works as a drop in replacement AND is faster,
I see no reason why PostgreSQL shouldn't try to link with it.
Keep in mind though that in order to use MMAP or AIO you'd be
restructuring the code to be more efficient rather than doing more of the
same old thing but optimized.
Only testing will prove me right or wrong though. :)
/*
Matthew N. Dodd | A memory retaining a love you had for life
winter@jurai.net | As cruel as it seems nothing ever seems to
http://www.jurai.net/~winter | go right - FLA M 3.1:53
*/
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] Safe/Fast I/O ...
@ 1998-04-12 06:37 The Hermit Hacker <scrappy@hub.org>
parent: Matthew N. Dodd <winter@jurai.net>
1 sibling, 2 replies; 20+ messages in thread
From: The Hermit Hacker @ 1998-04-12 06:37 UTC (permalink / raw)
To: Matthew N. Dodd <winter@jurai.net>; +Cc: pgsql-hackers
On Sun, 12 Apr 1998, Matthew N. Dodd wrote:
> On Sun, 12 Apr 1998, The Hermit Hacker wrote:
> > I hate to have to ask, but how is MMAP or AIO better then sfio? I
> > haven't had enough time to research any of this, and am just starting to
> > look at it...
>
> If its simple to compile and works as a drop in replacement AND is faster,
> I see no reason why PostgreSQL shouldn't try to link with it.
That didn't really answer the question :(
> Keep in mind though that in order to use MMAP or AIO you'd be
> restructuring the code to be more efficient rather than doing more of the
> same old thing but optimized.
I don't know anything about AIO, so if you can give me a pointer
to where I can read up on it, please do...
...but, with MMAP, unless I'm mistaken, you'd essentially be
reading the file(s) into memory and then manipulating the file(s) there.
Which means one helluva large amount of RAM being required...no?
Using stdio vs sfio, to read a 1.2million line file, the time to
complete goes from 7sec to 5sec ... that makes for a substantial savings
in time, if its applicable.
the problem, as I see it right now, is the docs for it suck ...
so, right now, I'm fumbling through figuring it all out :)
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] Safe/Fast I/O ...
@ 1998-04-12 07:47 David Gould <dg@illustra.com>
parent: The Hermit Hacker <scrappy@hub.org>
1 sibling, 0 replies; 20+ messages in thread
From: dg@illustra.com @ 1998-04-12 07:47 UTC (permalink / raw)
To: scrappy@hub.org; +Cc: winter@jurai.net; pgsql-hackers
Marc G. Fournier wrote:
> On Sun, 12 Apr 1998, Matthew N. Dodd wrote:
> > On Sun, 12 Apr 1998, The Hermit Hacker wrote:
> > > I hate to have to ask, but how is MMAP or AIO better then sfio? I
> > > haven't had enough time to research any of this, and am just starting to
> > > look at it...
> >
> > If its simple to compile and works as a drop in replacement AND is faster,
> > I see no reason why PostgreSQL shouldn't try to link with it.
>
> That didn't really answer the question :(
>
> > Keep in mind though that in order to use MMAP or AIO you'd be
> > restructuring the code to be more efficient rather than doing more of the
> > same old thing but optimized.
>
> I don't know anything about AIO, so if you can give me a pointer
> to where I can read up on it, please do...
>
> ...but, with MMAP, unless I'm mistaken, you'd essentially be
> reading the file(s) into memory and then manipulating the file(s) there.
> Which means one helluva large amount of RAM being required...no?
>
> Using stdio vs sfio, to read a 1.2million line file, the time to
> complete goes from 7sec to 5sec ... that makes for a substantial savings
> in time, if its applicable.
>
> the problem, as I see it right now, is the docs for it suck ...
> so, right now, I'm fumbling through figuring it all out :)
One of the options when building perl5 is to use sfio instead of stdio. I
haven't tried it, but they seem to think it works.
That said, The only place I see this helping pgsql is in copyin and copyout
as these use the stdio: fread(), fwrite(), etc interfaces.
Everywhere else we use the system call IO interfaces: read(), write(),
recv(), send(), select() etc, and do our own buffering.
My prediction is that sfio vs stdio will have undetectable performance
impact on sql performance and only very minor impact on copyin, copyout (as
most of the overhead is in pgsql, not libc).
As far as IO, the problem we have is fsync(). To get rid of it means doing
a real writeahead log system and (maybe) aio to the log. As soon as we get
real logging then we don't need to force datapages out so we can get rid
of all the fsync and (given how slow we are otherwise) completely eliminate
IO as a bottleneck.
Pgsql was built for comfort, not for speed. Fine tuning and code
tweeking and microoptimization is fine as far as it goes. But there is
probably a maximum 2x speed up to be had that way. Total.
We need a 10x speedup to play with serious databases. This will take real
architectural changes.
If you are interested in what is necessary, I highly recommend the book
"Transaction Processing" by Jim Gray (and someone whose name escapes me
just now). It is a great big thing and will take a while to get through, but
is is decently written and very well worth the time. It pretty much gives
away the whole candy store as far as building high performance, reliable,
and scalable database and TP systems. I wish it had been available 10
years ago when I got into the DB game.
-dg
David Gould dg@illustra.com 510.628.3783 or 510.305.9468
Informix Software (No, really) 300 Lakeside Drive Oakland, CA 94612
- Linux. Not because it is free. Because it is better.
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] Safe/Fast I/O ...
@ 1998-04-12 13:28 Bruce Momjian <maillist@candle.pha.pa.us>
parent: Matthew N. Dodd <winter@jurai.net>
1 sibling, 3 replies; 20+ messages in thread
From: Bruce Momjian @ 1998-04-12 13:28 UTC (permalink / raw)
To: winter@jurai.net; +Cc: scrappy@hub.org; pgsql-hackers
>
> On Sun, 12 Apr 1998, The Hermit Hacker wrote:
> > I hate to have to ask, but how is MMAP or AIO better then sfio? I
> > haven't had enough time to research any of this, and am just starting to
> > look at it...
>
> If its simple to compile and works as a drop in replacement AND is faster,
> I see no reason why PostgreSQL shouldn't try to link with it.
>
> Keep in mind though that in order to use MMAP or AIO you'd be
> restructuring the code to be more efficient rather than doing more of the
> same old thing but optimized.
>
> Only testing will prove me right or wrong though. :)
As David Gould mentioned, we need to do pre-fetching of data pages
somehow.
When doing a sequential scan on a table, the OS is doing a one-page
prefetch, which is probably enough. The problem is index scans of the
table. Those are not sequential in the main heap table (unless it is
clustered on the index), so a prefetch would help here a lot.
That is where we need async i/o. I am looking in BSDI, and I don't see
any way to do async i/o. The only way I can think of doing it is via
threads.
--
Bruce Momjian | 830 Blythe Avenue
maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
+ If your life is a hard drive, | (610) 353-9879(w)
+ Christ can be your backup. | (610) 853-3000(h)
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] Safe/Fast I/O ...
@ 1998-04-12 14:06 Bruce Momjian <maillist@candle.pha.pa.us>
parent: Bruce Momjian <maillist@candle.pha.pa.us>
2 siblings, 2 replies; 20+ messages in thread
From: Bruce Momjian @ 1998-04-12 14:06 UTC (permalink / raw)
To: maillist@candle.pha.pa.us; +Cc: winter@jurai.net; scrappy@hub.org; pgsql-hackers
> As David Gould mentioned, we need to do pre-fetching of data pages
> somehow.
>
> When doing a sequential scan on a table, the OS is doing a one-page
> prefetch, which is probably enough. The problem is index scans of the
> table. Those are not sequential in the main heap table (unless it is
> clustered on the index), so a prefetch would help here a lot.
>
> That is where we need async i/o. I am looking in BSDI, and I don't see
> any way to do async i/o. The only way I can think of doing it is via
> threads.
I found it. It is an fcntl option. From man fcntl:
O_ASYNC Enable the SIGIO signal to be sent to the process group when
I/O is possible, e.g., upon availability of data to be read.
Who else supports this?
--
Bruce Momjian | 830 Blythe Avenue
maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
+ If your life is a hard drive, | (610) 353-9879(w)
+ Christ can be your backup. | (610) 853-3000(h)
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] Safe/Fast I/O ...
@ 1998-04-12 14:22 Bruce Momjian <maillist@candle.pha.pa.us>
parent: Bruce Momjian <maillist@candle.pha.pa.us>
2 siblings, 1 reply; 20+ messages in thread
From: Bruce Momjian @ 1998-04-12 14:22 UTC (permalink / raw)
To: maillist@candle.pha.pa.us; +Cc: winter@jurai.net; scrappy@hub.org; pgsql-hackers
> When doing a sequential scan on a table, the OS is doing a one-page
> prefetch, which is probably enough. The problem is index scans of the
> table. Those are not sequential in the main heap table (unless it is
> clustered on the index), so a prefetch would help here a lot.
>
> That is where we need async i/o. I am looking in BSDI, and I don't see
> any way to do async i/o. The only way I can think of doing it is via
> threads.
O_ASYNC Enable the SIGIO signal to be sent to the process group when
I/O is possible, e.g., upon availability of data to be read.
Now I am questioning this. I am not sure this acually for file i/o, or
only tty i/o.
--
Bruce Momjian | 830 Blythe Avenue
maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
+ If your life is a hard drive, | (610) 353-9879(w)
+ Christ can be your backup. | (610) 853-3000(h)
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] Safe/Fast I/O ...
@ 1998-04-12 17:24 The Hermit Hacker <scrappy@hub.org>
parent: Bruce Momjian <maillist@candle.pha.pa.us>
1 sibling, 0 replies; 20+ messages in thread
From: The Hermit Hacker @ 1998-04-12 17:24 UTC (permalink / raw)
To: Bruce Momjian <maillist@candle.pha.pa.us>; +Cc: winter@jurai.net; pgsql-hackers
On Sun, 12 Apr 1998, Bruce Momjian wrote:
> > As David Gould mentioned, we need to do pre-fetching of data pages
> > somehow.
> >
> > When doing a sequential scan on a table, the OS is doing a one-page
> > prefetch, which is probably enough. The problem is index scans of the
> > table. Those are not sequential in the main heap table (unless it is
> > clustered on the index), so a prefetch would help here a lot.
> >
> > That is where we need async i/o. I am looking in BSDI, and I don't see
> > any way to do async i/o. The only way I can think of doing it is via
> > threads.
>
> I found it. It is an fcntl option. From man fcntl:
>
> O_ASYNC Enable the SIGIO signal to be sent to the process group when
> I/O is possible, e.g., upon availability of data to be read.
>
> Who else supports this?
FreeBSD...
Marc G. Fournier
Systems Administrator @ hub.org
primary: scrappy@hub.org secondary: scrappy@{freebsd|postgresql}.org
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] Safe/Fast I/O ...
@ 1998-04-12 17:37 Jordan Henderson <jordanh@ccia.com>
parent: Bruce Momjian <maillist@candle.pha.pa.us>
0 siblings, 1 reply; 20+ messages in thread
From: Jordan Henderson @ 1998-04-12 17:37 UTC (permalink / raw)
To: maillist@candle.pha.pa.us; +Cc: pgsql-hackers
>> When doing a sequential scan on a table, the OS is doing a one-page
>> prefetch, which is probably enough. The problem is index scans of the
>> table. Those are not sequential in the main heap table (unless it is
>> clustered on the index), so a prefetch would help here a lot.
>>
>> That is where we need async i/o. I am looking in BSDI, and I don't see
>> any way to do async i/o. The only way I can think of doing it is via
>> threads.
>
>
> O_ASYNC Enable the SIGIO signal to be sent to the process group when
> I/O is possible, e.g., upon availability of data to be read.
>
>Now I am questioning this. I am not sure this acually for file i/o, or
>only tty i/o.
>
async file calls:
aio_cancel
aio_error
aio_read
aio_return -- gets status of pending io call
aio_suspend
aio_write
And yes the Gray book is great!
Jordan Henderson
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] Safe/Fast I/O ...
@ 1998-04-12 17:38 Matthew N. Dodd <winter@jurai.net>
parent: Bruce Momjian <maillist@candle.pha.pa.us>
1 sibling, 0 replies; 20+ messages in thread
From: Matthew N. Dodd @ 1998-04-12 17:38 UTC (permalink / raw)
To: Bruce Momjian <maillist@candle.pha.pa.us>; +Cc: scrappy@hub.org; pgsql-hackers
On Sun, 12 Apr 1998, Bruce Momjian wrote:
> I found it. It is an fcntl option. From man fcntl:
>
> O_ASYNC Enable the SIGIO signal to be sent to the process group when
> I/O is possible, e.g., upon availability of data to be read.
>
> Who else supports this?
FreeBSD, and NetBSD appearto.
Linux and Solaris appear not to.
I was really speaking of the POSIX 1003.1B AIO/LIO calls when I originally
brought this up. (aio_read/aio_write)
/*
Matthew N. Dodd | A memory retaining a love you had for life
winter@jurai.net | As cruel as it seems nothing ever seems to
http://www.jurai.net/~winter | go right - FLA M 3.1:53
*/
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] Safe/Fast I/O ...
@ 1998-04-12 17:42 Bruce Momjian <maillist@candle.pha.pa.us>
parent: Jordan Henderson <jordanh@ccia.com>
0 siblings, 1 reply; 20+ messages in thread
From: Bruce Momjian @ 1998-04-12 17:42 UTC (permalink / raw)
To: jordanh@ccia.com; +Cc: pgsql-hackers
>
> async file calls:
> aio_cancel
> aio_error
> aio_read
> aio_return -- gets status of pending io call
> aio_suspend
> aio_write
Can you elaborate on this? Does it cause a read() to return right away,
and signal when data is ready?
>
> And yes the Gray book is great!
>
> Jordan Henderson
>
>
--
Bruce Momjian | 830 Blythe Avenue
maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
+ If your life is a hard drive, | (610) 353-9879(w)
+ Christ can be your backup. | (610) 853-3000(h)
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] Safe/Fast I/O ...
@ 1998-04-12 19:25 David Gould <dg@illustra.com>
parent: Bruce Momjian <maillist@candle.pha.pa.us>
0 siblings, 0 replies; 20+ messages in thread
From: dg@illustra.com @ 1998-04-12 19:25 UTC (permalink / raw)
To: maillist@candle.pha.pa.us; +Cc: jordanh@ccia.com; pgsql-hackers
> > async file calls:
> > aio_cancel
> > aio_error
> > aio_read
> > aio_return -- gets status of pending io call
> > aio_suspend
> > aio_write
>
> Can you elaborate on this? Does it cause a read() to return right away,
> and signal when data is ready?
These are posix calls. Many systems support them and they are fairly easy
to emulate (with threads or io processes) on systems that don't. If we
are going to do Async IO, I suggest that we code to the posix interface and
build emulators for the systems that don't have the posix calls.
I think there is an implementation of this for Linux, but it is a separate
package, not part of the base system as far as I know. Of course with Linux
anything you know it didn't do two weeks ago, it will do next week...
Here is the Solaris man page for aio_read() and aio_write:
-dg
-----------------------------------------------------------------------------
SunOS 5.5.1 Last change: 19 Aug 1993 1
aio_read(3R) Realtime Library aio_read(3R)
NAME
aio_read, aio_write - asynchronous read and write operations
SYNOPSIS
cc [ flag ... ] file ... -lposix4 [ library ... ]
#include <aio.h>
int aio_read(struct aiocb *aiocbp);
int aio_write(struct aiocb *aiocbp);
struct aiocb {
int aio_fildes; /* file descriptor */
volatile void *aio_buf; /* buffer location */
size_t aio_nbytes; /* length of transfer
*/
off_t aio_offset; /* file offset */
int aio_reqprio; /* request priority
offset */
struct sigevent aio_sigevent; /* signal number and
offset */
int aio_lio_opcode; /* listio operation */
};
struct sigevent {
int sigev_notify; /* notification mode */
int sigev_signo; /* signal number */
union sigval sigev_value; /* signal value */
};
union sigval {
int sival_int; /* integer value */
void *sival_ptr; /* pointer value */
};
MT-LEVEL
MT-Safe
DESCRIPTION
aio_read() queues an asynchronous read request, and returns
control immediately. Rather than blocking until completion,
the read operation continues concurrently with other
activity of the process.
Upon enqueuing the request, the calling process reads
aiocbp->nbytes from the file referred to by aiocbp->fildes
into the buffer pointed to by aiocbp->aio_buf.
aiocbp->offset marks the absolute position from the begin-
ning of the file (in bytes) at which the read begins.
aio_write() queues an asynchronous write request, and
returns control immediately. Rather than blocking until
completion, the write operation continues concurrently with
other activity of the process.
Upon enqueuing the request, the calling process writes
aiocbp->nbytes from the buffer pointed to by aiocbp-
>aio_buf into the file referred to by aiocbp->fildes. If
O_APPEND is set for aiocbp->fildes, aio_write() operations
append to the file in the same order as the calls were made.
If O_APPEND is not set for the file descriptor, then the
write operation will occur at the absolute position from the
beginning of the file plus aiocbp->offset (in bytes).
These asynchronous operations are submitted at a priority
equal to the calling process' scheduling priority minus
aiocbp->aio_reqprio.
aiocb->aio_sigevent defines both the signal to be generated
and how the calling process will be notified upon I/O com-
pletion. If aio_sigevent.sigev_notify is SIGEV_NONE, then
no signal will be posted upon I/O completion, but the error
status and the return status for the operation will be set
appropriately. If aio_sigevent.sigev_notify is
SIGEV_SIGNAL, then the signal specified in
aio_sigevent.sigev_signo will be sent to the process. If
the SA_SIGINFO flag is set for that signal number, then the
signal will be queued to the process and the value specified
in aio_sigevent.sigev_value will be the si_value component
of the generated signal (see siginfo(5)).
RETURN VALUES
If the I/O operation is successfully queued, aio_read() and
aio_write() return 0, otherwise, they return -1, and set
errno to indicate the error condition. aiocbp may be used
as an argument to aio_error(3R) and aio_return(3R) in order
to determine the error status and the return status of the
asynchronous operation while it is proceeding.
ERRORS
EAGAIN The requested asynchronous I/O operation was
not queued due to system resource limita-
tions.
ENOSYS aio_read() or aio_write() is not supported by
this implementation.
EBADF If the calling function is aio_read(), and
aiocbp->fildes is not a valid file descriptor
open for reading. If the calling function is
aio_write(), and aiocbp->fildes is not a
valid file descriptor open for writing.
EINVAL The file offset value implied by aiocbp->aio_offset
would be invalid,
aiocbp->aio_reqprio is not a valid value,
or aiocbp->aio_nbytes is an invalid value.
ECANCELED The requested I/O was canceled before the I/O
completed due to an explicit aio_cancel(3R)
request.
EINVAL The file offset value implied by aiocbp-
>aio_offset would be invalid.
SEE ALSO
close(2), exec(2), exit(2), fork(2), lseek(2), read(2),
write(2), aio_cancel(3R), aio_return(3R), lio_listio(3R),
siginfo(5)
NOTES
For portability, the application should set aiocb- >aio_reqprio
to 0.
Applications compiled under Solaris 2.3 and 2.4 and using
POSIX aio must be recompiled to work correctly when Solaris
supports the Asynchronous Input and Output option.
BUGS
In Solaris 2.5, these functions always return - 1 and set
errno to ENOSYS, because this release does not support the
Asynchronous Input and Output option. It is our intention
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] Safe/Fast I/O ...
@ 1998-04-13 00:50 ocie@paracel.com
parent: Bruce Momjian <maillist@candle.pha.pa.us>
2 siblings, 0 replies; 20+ messages in thread
From: ocie@paracel.com @ 1998-04-13 00:50 UTC (permalink / raw)
To: maillist@candle.pha.pa.us; +Cc: winter@jurai.net; scrappy@hub.org; pgsql-hackers
Bruce Momjian wrote:
>
> >
> > On Sun, 12 Apr 1998, The Hermit Hacker wrote:
> > > I hate to have to ask, but how is MMAP or AIO better then sfio? I
> > > haven't had enough time to research any of this, and am just starting to
> > > look at it...
> >
> > If its simple to compile and works as a drop in replacement AND is faster,
> > I see no reason why PostgreSQL shouldn't try to link with it.
> >
> > Keep in mind though that in order to use MMAP or AIO you'd be
> > restructuring the code to be more efficient rather than doing more of the
> > same old thing but optimized.
> >
> > Only testing will prove me right or wrong though. :)
>
> As David Gould mentioned, we need to do pre-fetching of data pages
> somehow.
>
> When doing a sequential scan on a table, the OS is doing a one-page
> prefetch, which is probably enough. The problem is index scans of the
> table. Those are not sequential in the main heap table (unless it is
> clustered on the index), so a prefetch would help here a lot.
>
> That is where we need async i/o. I am looking in BSDI, and I don't see
> any way to do async i/o. The only way I can think of doing it is via
> threads.
I have heard the glibc version 2.0 will support the Posix AIO spec.
Solaris currently has AN implementation of AIO, but it is not the
POSIX one. This prefetch could be done in another process or thread,
rather than tying the code to a given AIO implementation.
Ocie
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] Safe/Fast I/O ...
@ 1998-04-13 01:23 Michal Mosiewicz <mimo@interdata.com.pl>
parent: The Hermit Hacker <scrappy@hub.org>
1 sibling, 0 replies; 20+ messages in thread
From: Michal Mosiewicz @ 1998-04-13 01:23 UTC (permalink / raw)
To: The Hermit Hacker <scrappy@hub.org>; +Cc: pgsql-hackers
The Hermit Hacker wrote:
> ...but, with MMAP, unless I'm mistaken, you'd essentially be
> reading the file(s) into memory and then manipulating the file(s) there.
> Which means one helluva large amount of RAM being required...no?
Not exactly. Memory mapping is used only to map file into some memory
addresses but not put into memory. Disk sectors are copied into memory
on demand. If some mmaped page is accessed - it is copied from disk into
memory.
The main reason of using memory mapping is that you don't have to create
unnecessary buffers. Normally, for every operation you have to create
some in-memory buffer, copy the data there, do some operations, put the
data back into file. In case of memory mapping you may avoid of creating
of unnecessary buffers, and moreover you may call your system functions
less frequently. There are also additional savings. (Less memory
copying, reusing memory if several processes map the same file)
I don't think there exist more efficient solutions.
Mike
--
WWW: http://www.lodz.pdi.net/~mimo tel: Int. Acc. Code + 48 42 148340
add: Michal Mosiewicz * Bugaj 66 m.54 * 95-200 Pabianice * POLAND
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] Safe/Fast I/O ...
@ 1998-04-13 16:26 Garrett Wollman <wollman@khavrinen.lcs.mit.edu>
0 siblings, 0 replies; 20+ messages in thread
From: Garrett Wollman @ 1998-04-13 16:26 UTC (permalink / raw)
To: pgsql-hackers
[Please forgive me for the way this post is put together; I'm not
actually on your mailing-list, but was just perusing the archives.]
Michal Mosiewicz <mimo@interdata.com.pl> writes:
> The main reason of using memory mapping is that you don't have to create
> unnecessary buffers. Normally, for every operation you have to create
> some in-memory buffer, copy the data there, do some operations, put the
> data back into file. In case of memory mapping you may avoid of creating
> of unnecessary buffers, and moreover you may call your system functions
> less frequently. There are also additional savings. (Less memory
> copying, reusing memory if several processes map the same file)
Additionally, if your operating system is at all reasonable, using
memory mapping allows you to take advantage of all the work that has
gone into tuning your VM system. If you map a large file, and then
access in some way that shows reasonable locality, the VM system will
probably be able to do a better job of page replacement on a
system-wide basis than you could do with a cache built into your
application. (A good system will also provide other benefits, such as
pre-faulting and extended read ahead.)
Of course, it does have one significant drawback: memory-mapped regions
do not automatically extend when their underlying files do. So, for
interacting with a structure that shows effectively linear access and
growth, asynchronous I/O is more likely to be a benefit, since AIO can
extend a file asynchronously, whereas other mechanisms will block
while the file is being extended. (Depending on the system, this may
not be true for multi-threaded programs.)
-GAWollman
--
Garrett A. Wollman | O Siem / We are all family / O Siem / We're all the same
wollman@lcs.mit.edu | O Siem / The fires of freedom
Opinions not those of| Dance in the burning flame
MIT, LCS, CRS, or NSA| - Susan Aglukark and Chad Irschick
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] Safe/Fast I/O ...
@ 1998-04-14 22:25 Paul M. Aoki <aoki@CS.Berkeley.EDU>
0 siblings, 0 replies; 20+ messages in thread
From: aoki@CS.Berkeley.EDU @ 1998-04-14 22:25 UTC (permalink / raw)
To: pgsql-hackers@hub.org; +Cc: wollman@khavrinen.lcs.mit.edu
> Date: Mon, 13 Apr 1998 12:26:59 -0400 (EDT)
> From: Garrett Wollman <wollman@khavrinen.lcs.mit.edu>
> Subject: Re: [HACKERS] Safe/Fast I/O ...
>
> [Please forgive me for the way this post is put together; I'm not
> actually on your mailing-list, but was just perusing the archives.]
> if your operating system is at all reasonable, using
> memory mapping allows you to take advantage of all the work that has
> gone into tuning your VM system. If you map a large file, and then
> access in some way that shows reasonable locality, the VM system will
> probably be able to do a better job of page replacement on a
> system-wide basis than you could do with a cache built into your
> application.
not necessarily. in this case, the application (the database) has
several very different page access patterns, some of which (e.g.,
non-boustrophedonic nested-loops join, index leaf accesses) *do not*
exhibit reasonable locality and therefore benefit from the ability to
turn on hate-hints or MRU paging on a selective basis. database query
processing is one of the classic examples why "one size does not fit
all" when it comes to page replacement -- no amount of "tuning" of an
LRU/clock algorithm will help if the access pattern is wrong.
stonebraker's 20-year-old CACM flame on operating system services for
databases has motivated a lot of work, e.g., microkernel external
pagers and the more recent work at stanford and princeton on
application-specific paging, but many older VM systems still don't
have a working madvise().. meaning that a *portable* database still
has to implement its own buffer cache if it wants to exploit its
application-specific paging behavior.
--
Paul M. Aoki | University of California at Berkeley
aoki@CS.Berkeley.EDU | Dept. of EECS, Computer Science Division #1776
| Berkeley, CA 94720-1776
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] Safe/Fast I/O ...
@ 1998-04-22 11:21 Andrew Martin <martin@biochemistry.ucl.ac.uk>
0 siblings, 1 reply; 20+ messages in thread
From: Andrew Martin @ 1998-04-22 11:21 UTC (permalink / raw)
To: Bruce Momjian <maillist@candle.pha.pa.us>; +Cc: pgsql-hackers
> > As David Gould mentioned, we need to do pre-fetching of data pages
> > somehow.
> >
> > When doing a sequential scan on a table, the OS is doing a one-page
> > prefetch, which is probably enough. The problem is index scans of the
> > table. Those are not sequential in the main heap table (unless it is
> > clustered on the index), so a prefetch would help here a lot.
> >
> > That is where we need async i/o. I am looking in BSDI, and I don't see
> > any way to do async i/o. The only way I can think of doing it is via
> > threads.
>
> I found it. It is an fcntl option. From man fcntl:
>
> O_ASYNC Enable the SIGIO signal to be sent to the process group when
> I/O is possible, e.g., upon availability of data to be read.
>
> Who else supports this?
>
under Irix:
man fcntl:
F_SETFL Set file status flags to the third argument, arg, taken as an
object of type int. Only the following flags can be set [see
fcntl(5)]: FAPPEND, FSYNC, FNDELAY, FNONBLK, FDIRECT, and
FASYNC. Since arg is used as a bit vector to set the flags,
values for all the flags must be specified in arg. (Typically,
arg may be constructed by obtaining existing values by F_GETFL
and then changing the particular flags.) FAPPEND is equivalent
to O_APPEND; FSYNC is equivalent to O_SYNC; FNDELAY is
equivalent to O_NDELAY; FNONBLK is equivalent to O_NONBLOCK;
and FDIRECT is equivalent to O_DIRECT. FASYNC is equivalent to
calling ioctl with the FIOASYNC command (except that with ioctl
all flags need not be specified). This enables the SIGIO
facilities and is currently supported only on sockets.
....but then I can find no details of FIOASYNC on the ioctl page or pages
referenced therein.
Andrew
----------------------------------------------------------------------------
Dr. Andrew C.R. Martin University College London
EMAIL: (Work) martin@biochem.ucl.ac.uk (Home) andrew@stagleys.demon.co.uk
URL: http://www.biochem.ucl.ac.uk/~martin
Tel: (Work) +44(0)171 419 3890 (Home) +44(0)1372 275775
^ permalink raw reply [nested|flat] 20+ messages in thread
* Re: [HACKERS] Safe/Fast I/O ...
@ 1998-04-22 14:15 Bruce Momjian <maillist@candle.pha.pa.us>
parent: Andrew Martin <martin@biochemistry.ucl.ac.uk>
0 siblings, 0 replies; 20+ messages in thread
From: Bruce Momjian @ 1998-04-22 14:15 UTC (permalink / raw)
To: martin@biochemistry.ucl.ac.uk; +Cc: pgsql-hackers
>
> > > As David Gould mentioned, we need to do pre-fetching of data pages
> > > somehow.
> > >
> > > When doing a sequential scan on a table, the OS is doing a one-page
> > > prefetch, which is probably enough. The problem is index scans of the
> > > table. Those are not sequential in the main heap table (unless it is
> > > clustered on the index), so a prefetch would help here a lot.
> > >
> > > That is where we need async i/o. I am looking in BSDI, and I don't see
> > > any way to do async i/o. The only way I can think of doing it is via
> > > threads.
> >
> > I found it. It is an fcntl option. From man fcntl:
> >
> > O_ASYNC Enable the SIGIO signal to be sent to the process group when
> > I/O is possible, e.g., upon availability of data to be read.
> >
> > Who else supports this?
> >
>
> under Irix:
>
> man fcntl:
>
> F_SETFL Set file status flags to the third argument, arg, taken as an
> object of type int. Only the following flags can be set [see
> fcntl(5)]: FAPPEND, FSYNC, FNDELAY, FNONBLK, FDIRECT, and
> FASYNC. Since arg is used as a bit vector to set the flags,
> values for all the flags must be specified in arg. (Typically,
> arg may be constructed by obtaining existing values by F_GETFL
> and then changing the particular flags.) FAPPEND is equivalent
> to O_APPEND; FSYNC is equivalent to O_SYNC; FNDELAY is
> equivalent to O_NDELAY; FNONBLK is equivalent to O_NONBLOCK;
> and FDIRECT is equivalent to O_DIRECT. FASYNC is equivalent to
> calling ioctl with the FIOASYNC command (except that with ioctl
> all flags need not be specified). This enables the SIGIO
> facilities and is currently supported only on sockets.
>
> ....but then I can find no details of FIOASYNC on the ioctl page or pages
> referenced therein.
I have found BSDI does not support async i/o. You need a separate
process to do the i/o. The O_ASYNC flag only works on tty files.
--
Bruce Momjian | 830 Blythe Avenue
maillist@candle.pha.pa.us | Drexel Hill, Pennsylvania 19026
+ If your life is a hard drive, | (610) 353-9879(w)
+ Christ can be your backup. | (610) 853-3000(h)
^ permalink raw reply [nested|flat] 20+ messages in thread
end of thread, other threads:[~1998-04-22 14:15 UTC | newest]
Thread overview: 20+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
1998-04-12 03:32 Safe/Fast I/O ... The Hermit Hacker <scrappy@hub.org>
1998-04-12 05:41 ` Matthew N. Dodd <winter@jurai.net>
1998-04-12 05:57 ` The Hermit Hacker <scrappy@hub.org>
1998-04-12 06:08 ` Matthew N. Dodd <winter@jurai.net>
1998-04-12 06:37 ` The Hermit Hacker <scrappy@hub.org>
1998-04-12 07:47 ` David Gould <dg@illustra.com>
1998-04-13 01:23 ` Michal Mosiewicz <mimo@interdata.com.pl>
1998-04-12 13:28 ` Bruce Momjian <maillist@candle.pha.pa.us>
1998-04-12 14:06 ` Bruce Momjian <maillist@candle.pha.pa.us>
1998-04-12 17:24 ` The Hermit Hacker <scrappy@hub.org>
1998-04-12 17:38 ` Matthew N. Dodd <winter@jurai.net>
1998-04-12 14:22 ` Bruce Momjian <maillist@candle.pha.pa.us>
1998-04-12 17:37 ` Jordan Henderson <jordanh@ccia.com>
1998-04-12 17:42 ` Bruce Momjian <maillist@candle.pha.pa.us>
1998-04-12 19:25 ` David Gould <dg@illustra.com>
1998-04-13 00:50 ` ocie@paracel.com
1998-04-13 16:26 Re: [HACKERS] Safe/Fast I/O ... Garrett Wollman <wollman@khavrinen.lcs.mit.edu>
1998-04-14 22:25 Re: [HACKERS] Safe/Fast I/O ... Paul M. Aoki <aoki@CS.Berkeley.EDU>
1998-04-22 11:21 Re: [HACKERS] Safe/Fast I/O ... Andrew Martin <martin@biochemistry.ucl.ac.uk>
1998-04-22 14:15 ` Bruce Momjian <maillist@candle.pha.pa.us>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox