To: "Jeffrey W. Baker" <jwbaker@acm.org>
Cc: Guy Thornley <guy@esphion.com>, Josh Berkus <josh@agliodbs.com>,
	pgsql-performance@postgresql.org, Steve Poe <spoe@sfnet.cc>,
	Chris Browne <cbbrowne@acm.org>
Subject: Re: Read/Write block sizes 
In-reply-to: <1124861121.11270.1.camel@noodles> 
References: <d4b67d7c2b.d7c2bd4b67@bur-mail1.east.sun.com>
	<60k6icpapm.fsf@dba2.int.libertyrms.com>
	<1124846743.12045.94.camel@amd64-laptop-spoe>
	<200508231931.29682.josh@agliodbs.com>
	<1124852854.11147.5.camel@noodles>
	<20050824052023.GB31806@conker.esphion.com>
	<1124861121.11270.1.camel@noodles>
Comments: In-reply-to "Jeffrey W. Baker" <jwbaker@acm.org>
	message dated "Tue, 23 Aug 2005 22:25:21 -0700"
Date: Wed, 24 Aug 2005 01:56:44 -0400
Message-ID: <21700.1124863004@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>

"Jeffrey W. Baker" <jwbaker@acm.org> writes:
> On Wed, 2005-08-24 at 17:20 +1200, Guy Thornley wrote:
>> Dont forget that already in postgres, you have a process per connection, and
>> all the processes take care of their own I/O.

> That's the problem.  Instead you want 1 or 4 or 10 i/o slaves
> coordinating the I/O of all the backends optimally.  For instance, with
> synchronous scanning.

And why exactly are we going to do a better job of I/O scheduling than
the OS itself can do?

There's a fairly basic disconnect in viewpoint involved here.  The
old-school viewpoint (as embodied in Oracle and a few other DBMSes)
is that the OS is too stupid to be worth anything, and the DB should
bypass the OS to the greatest extent possible, doing its own caching,
disk space layout, I/O scheduling, yadda yadda.  That might have been
defensible twenty-odd years ago when Oracle was designed.  Postgres
prefers to lay off to the OS anything that the OS can do well --- and
that definitely includes caching and I/O scheduling.  There are a whole
lot of smart people working on those problems at the OS level.  Maybe we
could make marginal improvements on their results after spending a lot
of effort reinventing the wheel ... but our time will be repaid much
more if we work at levels that the OS cannot have knowledge of, such as
join planning and data statistics.

There are some things we could do to reduce the impedance between us and
the OS --- for instance, the upthread criticism that a seqscan asks the
OS for only 8K at a time is fair enough.  But that doesn't translate
to a conclusion that we should schedule the I/O instead of the OS.

			regards, tom lane