From: Josh Berkus <josh@agliodbs.com>
To: Jeffrey W. Baker <jwbaker@acm.org>
Cc: pgsql-hackers@postgresql.org, Michael Stone <mstone+postgres@mathom.us>
Cc: pgsql-performance@postgresql.org
Subject: Re: [HACKERS] A Better External Sort?
Date: Mon, 3 Oct 2005 15:03:11 -0700
Message-ID: <200510031503.12158.josh@agliodbs.com> (raw)
In-Reply-To: <1128375146.29080.9.camel@toonses.gghcwest.com>
References: <31833713.1128111650174.JavaMail.root@elwamui-polski.atl.sa.earthlink.net>
<200510031416.15620.josh@agliodbs.com>
<1128375146.29080.9.camel@toonses.gghcwest.com>
Jeffrey,
> I guess database reads are different, but I remain unconvinced that they
> are *fundamentally* different. After all, a tab-delimited file (my sort
> workload) is a kind of database.
Unfortunately, they are ... because of CPU overheads. I'm basing what's
"reasonable" for data writes on the rates which other high-end DBs can
make. From that, 25mb/s or even 40mb/s for sorts should be achievable
but doing 120mb/s would require some kind of breakthrough.
> On a single disk you wouldn't notice, but XFS scales much better when
> you throw disks at it. I get a 50MB/sec boost from the 24th disk,
> whereas ext3 stops scaling after 16 disks. For writes both XFS and ext3
> top out around 8 disks, but in this case XFS tops out at 500MB/sec while
> ext3 can't break 350MB/sec.
That would explain it. I seldom get more than 6 disks (and 2 channels) to
test with.
--
--Josh
Josh Berkus
Aglio Database Solutions
San Francisco
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: josh@agliodbs.com, jwbaker@acm.org, mstone+postgres@mathom.us
Subject: Re: [HACKERS] A Better External Sort?
In-Reply-To: <200510031503.12158.josh@agliodbs.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox