From: Luke Lonergan <llonergan@greenplum.com>
To: Ron Peacetree <rjpeace@earthlink.net>
To: Josh Berkus <josh@agliodbs.com>
To: pgsql-hackers@postgresql.org
To: pgsql-performance@postgresql.org
Subject: Re: [PERFORM] A Better External Sort?
Date: Fri, 30 Sep 2005 13:38:54 -0700
Message-ID: <BF62F06E.10617%llonergan@greenplum.com> (raw)
In-Reply-To: <31833713.1128111650174.JavaMail.root@elwamui-polski.atl.sa.earthlink.net>
Ron,
On 9/30/05 1:20 PM, "Ron Peacetree" <rjpeace@earthlink.net> wrote:
> That 11MBps was your =bulk load= speed. If just loading a table
> is this slow, then there are issues with basic physical IO, not just
> IO during sort operations.
Bulk loading speed is irrelevant here - that is dominated by parsing, which
we have covered copiously (har har) previously and have sped up by 500%,
which still makes Postgres < 1/2 the loading speed of MySQL.
> As I said, the obvious candidates are inefficient physical layout
> and/or flawed IO code.
Yes.
> Until the basic IO issues are addressed, we could replace the
> present sorting code with infinitely fast sorting code and we'd
> still be scrod performance wise.
Postgres' I/O path has many problems that must be micro-optimized away. Too
small of an operand size compared to disk caches, memory, etc etc are the
common problem. Another is lack of micro-parallelism (loops) with long
enough runs to let modern processors pipeline and superscale.
The net problem here is that a simple "select blah from blee order
by(blah.a);" runs at 1/100 of the sequential scan rate.
- Luke
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: llonergan@greenplum.com, rjpeace@earthlink.net, josh@agliodbs.com, pgsql-hackers@postgresql.org
Subject: Re: [PERFORM] A Better External Sort?
In-Reply-To: <BF62F06E.10617%llonergan@greenplum.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox