From: Jeffrey W. Baker <jwbaker@acm.org>
To: Josh Berkus <josh@agliodbs.com>
Cc: pgsql-hackers@postgresql.org, Michael Stone <mstone+postgres@mathom.us>
Cc: pgsql-performance@postgresql.org
Subject: Re: [HACKERS] A Better External Sort?
Date: Mon, 03 Oct 2005 14:32:26 -0700
Message-ID: <1128375146.29080.9.camel@toonses.gghcwest.com> (raw)
In-Reply-To: <200510031416.15620.josh@agliodbs.com>
References: <31833713.1128111650174.JavaMail.root@elwamui-polski.atl.sa.earthlink.net>
<200510031334.02030.josh@agliodbs.com>
<1128372151.28509.1.camel@toonses.gghcwest.com>
<200510031416.15620.josh@agliodbs.com>
On Mon, 2005-10-03 at 14:16 -0700, Josh Berkus wrote:
> Jeff,
>
> > > Nope, LOTS of testing, at OSDL, GreenPlum and Sun. For comparison, A
> > > Big-Name Proprietary Database doesn't get much more than that either.
> >
> > I find this claim very suspicious. I get single-threaded reads in
> > excess of 1GB/sec with XFS and > 250MB/sec with ext3.
>
> Database reads? Or raw FS reads? It's not the same thing.
Just reading files off the filesystem. These are input rates I get with
a specialized sort implementation. 1GB/sec is not even especially
wonderful, I can get that on two controllers with 24-disk stripe set.
I guess database reads are different, but I remain unconvinced that they
are *fundamentally* different. After all, a tab-delimited file (my sort
workload) is a kind of database.
> Also, we're talking *write speed* here, not read speed.
Ok, I did not realize. Still you should see 250-300MB/sec
single-threaded sequential output on ext3, assuming the storage can
provide that rate.
> I also find *your* claim suspicious, since there's no way XFS is 300% faster
> than ext3 for the *general* case.
On a single disk you wouldn't notice, but XFS scales much better when
you throw disks at it. I get a 50MB/sec boost from the 24th disk,
whereas ext3 stops scaling after 16 disks. For writes both XFS and ext3
top out around 8 disks, but in this case XFS tops out at 500MB/sec while
ext3 can't break 350MB/sec.
I'm hopeful that in the future the work being done at ClusterFS will
make ext3 on-par with XFS.
-jwb
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: jwbaker@acm.org, josh@agliodbs.com, mstone+postgres@mathom.us
Subject: Re: [HACKERS] A Better External Sort?
In-Reply-To: <1128375146.29080.9.camel@toonses.gghcwest.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox