From: Luke Lonergan <llonergan@greenplum.com>
To: Greg Stark <gsstark@mit.edu>
To: stange@rentec.com
Cc: Dave Cramer <pg@fastcrypt.com>
Cc: Joshua Marsh <icub3d@gmail.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Hardware/OS recommendations for large databases (
Date: Fri, 18 Nov 2005 11:24:48 -0800
Message-ID: <BFA36A80.140AE%llonergan@greenplum.com> (raw)
In-Reply-To: <87oe4hpyfd.fsf@stark.xeocode.com>
Greg,
On 11/18/05 11:07 AM, "Greg Stark" <gsstark@mit.edu> wrote:
> That said, 130MB/s is nothing to sneeze at, that's maxing out two high end
> drives and quite respectable for a 3-disk stripe set, even reasonable for a
> 4-disk stripe set. If you're using 5 or more disks in RAID-0 or RAID 1+0 and
> only getting 130MB/s then it does seem likely the cpu is actually holding you
> back here.
With an FC array, it's undoubtedly more like 14 drives, in which case
130MB/s is laughable. On the other hand, I wouldn't be surprised if it were
a single 200MB/s Fibre Channel attachment.
It does make you wonder why people keep recommending 15K RPM drives, like it
would help *not*.
> Still it doesn't show Postgres being nearly so CPU wasteful as the original
> poster claimed.
It's partly about waste, and partly about lack of a concurrent I/O
mechanism. We've profiled it for the waste, we've implemented concurrent
I/O to prove the other point.
>> It's all in the kernel either way; using a different scheduler or file
>> system would change that result. Even better would be using direct IO to not
>> flush everything else from memory and avoid some memory copies from kernel
>> to user space. Note that almost none of the time is user time. Changing
>> postgresql won't change the cpu useage.
>
> Well changing to direct i/o would still be changing Postgres so that's
> unclear. And there are plenty of more mundane ways that Postgres is
> responsible for how efficiently or not the kernel is used. Just using fewer
> syscalls to do the same amount of reading would reduce cpu consumption.
Bingo.
- Luke
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: llonergan@greenplum.com, gsstark@mit.edu, stange@rentec.com, pg@fastcrypt.com, icub3d@gmail.com
Subject: Re: Hardware/OS recommendations for large databases (
In-Reply-To: <BFA36A80.140AE%llonergan@greenplum.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox