From: Ron Peacetree <rjpeace@earthlink.net>
To: Jeffrey W. Baker <jwbaker@acm.org>
To: pgsql-hackers@postgresql.org
To: pgsql-performance@postgresql.org
Subject: Re: [HACKERS] A Better External Sort?
Date: Mon, 3 Oct 2005 17:18:45 -0400 (EDT)
Message-ID: <7911802.1128374325231.JavaMail.root@elwamui-polski.atl.sa.earthlink.net> (raw)
Jeff, are those _burst_ rates from HD buffer or _sustained_ rates from
actual HD media? Rates from IO subsystem buffer or cache are
usually considerably higher than Average Sustained Transfer Rate.
Also, are you measuring _raw_ HD IO (bits straight off the platters, no
FS or other overhead) or _cooked_ HD IO (actual FS or pg IO)?
BTW, it would seem Useful to measure all of raw HD IO, FS HD IO,
and pg HD IO as this would give us an idea of just how much overhead
each layer is imposing on the process.
We may be able to get better IO than we currently are for things like
sorts by the simple expedient of making sure we read enough data per
seek.
For instance, a HD with a 12ms average access time and a ASTR of
50MBps should always read _at least_ 600KB/access or it is impossible
for it to achieve it's rated ASTR.
This number will vary according to the average access time and the
ASTR of your physical IO subsystem, but the concept is valid for _any_
physical IO subsystem.
-----Original Message-----
From: "Jeffrey W. Baker" <jwbaker@acm.org>
Sent: Oct 3, 2005 4:42 PM
To: josh@agliodbs.com
Cc:
Subject: Re: [HACKERS] [PERFORM] A Better External Sort?
On Mon, 2005-10-03 at 13:34 -0700, Josh Berkus wrote:
> Michael,
>
> > >Realistically, you can't do better than about 25MB/s on a
> > > single-threaded I/O on current Linux machines,
> >
> > What on earth gives you that idea? Did you drop a zero?
>
> Nope, LOTS of testing, at OSDL, GreenPlum and Sun. For comparison, A
> Big-Name Proprietary Database doesn't get much more than that either.
I find this claim very suspicious. I get single-threaded reads in
excess of 1GB/sec with XFS and > 250MB/sec with ext3.
-jwb
---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: rjpeace@earthlink.net, jwbaker@acm.org, pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] A Better External Sort?
In-Reply-To: <7911802.1128374325231.JavaMail.root@elwamui-polski.atl.sa.earthlink.net>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox