From: Simon Riggs <simon@2ndquadrant.com>
To: Luke Lonergan <llonergan@greenplum.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Alon Goldshuv <agoldshuv@greenplum.com>
Cc: pgsql-patches@postgresql.org
Subject: Re: COPY FROM performance improvements
Date: Wed, 10 Aug 2005 09:15:00 +0100
Message-ID: <1123661700.3670.621.camel@localhost.localdomain> (raw)
In-Reply-To: <BF1ED512.C208%llonergan@greenplum.com>
References: <BF1ED512.C208%llonergan@greenplum.com>
On Tue, 2005-08-09 at 21:48 -0700, Luke Lonergan wrote:
> The key thing that is missing is the lack of micro-parallelism in the
> character processing in this version. By "inverting the loop", or putting
> the characters into a buffer on the outside, then doing fast character
> scanning inside with special "fix-up" cases, we exposed long runs of
> pipeline-able code to the compiler.
>
> I think there is another way to accomplish the same thing and still preserve
> the current structure, but it requires "strip mining" the character buffer
> into chunks that can be processed with an explicit loop to check for the
> different characters. While it may seem artificial (it is), it will provide
> the compiler with the ability to pipeline the character finding logic over
> long runs. The other necessary element will have to avoid pipeline stalls
> from the "if" conditions as much as possible.
This is a key point, IMHO.
That part of the code was specifically written to take advantage of
processing pipelines in the hardware, not because the actual theoretical
algorithm for that approach was itself faster.
Nobody's said what compiler/hardware they have been using, so since both
Alon and Tom say their character finding logic is faster, it is likely
to be down to that? Name your platforms gentlemen, please.
My feeling is that we may learn something here that applies more widely
across many parts of the code.
Best Regards, Simon Riggs
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: simon@2ndquadrant.com, llonergan@greenplum.com, tgl@sss.pgh.pa.us, agoldshuv@greenplum.com, pgsql-patches@postgresql.org
Subject: Re: COPY FROM performance improvements
In-Reply-To: <1123661700.3670.621.camel@localhost.localdomain>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox