From: Simon Riggs <simon@2ndquadrant.com>
To: josh@agliodbs.com
Cc: Ron Peacetree <rjpeace@earthlink.net>
Cc: pgsql-hackers@postgresql.org
Cc: pgsql-performance@postgresql.org
Subject: Re: [HACKERS] A Better External Sort?
Date: Fri, 30 Sep 2005 23:21:16 +0100
Message-ID: <1128118876.4045.53.camel@localhost.localdomain> (raw)
In-Reply-To: <200509301341.22795.josh@agliodbs.com>
References: <31833713.1128111650174.JavaMail.root@elwamui-polski.atl.sa.earthlink.net>
<200509301341.22795.josh@agliodbs.com>
On Fri, 2005-09-30 at 13:41 -0700, Josh Berkus wrote:
> Yeah, that's what I thought too. But try sorting an 10GB table, and
> you'll see: disk I/O is practically idle, while CPU averages 90%+. We're
> CPU-bound, because sort is being really inefficient about something. I
> just don't know what yet.
>
> If we move that CPU-binding to a higher level of performance, then we can
> start looking at things like async I/O, O_Direct, pre-allocation etc. that
> will give us incremental improvements. But what we need now is a 5-10x
> improvement and that's somewhere in the algorithms or the code.
I'm trying to keep an open mind about what the causes are, and I think
we need to get a much better characterisation of what happens during a
sort before we start trying to write code. It is always too easy to jump
in and tune the wrong thing, which is not a good use of time.
The actual sort algorithms looks damn fine to me and the code as it
stands is well optimised. That indicates to me that we've come to the
end of the current line of thinking and we need a new approach, possibly
in a number of areas.
For myself, I don't wish to be drawn further on solutions at this stage
but I am collecting performance data, so any test results are most
welcome.
Best Regards, Simon Riggs
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: simon@2ndquadrant.com, josh@agliodbs.com, rjpeace@earthlink.net, pgsql-hackers@postgresql.org
Subject: Re: [HACKERS] A Better External Sort?
In-Reply-To: <1128118876.4045.53.camel@localhost.localdomain>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox