From: Luke Lonergan <llonergan@greenplum.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Alon Goldshuv <agoldshuv@greenplum.com>
Cc: pgsql-patches@postgresql.org
Subject: Re: COPY FROM performance improvements
Date: Tue, 09 Aug 2005 21:48:02 -0700
Message-ID: <BF1ED512.C208%llonergan@greenplum.com> (raw)
In-Reply-To: <29007.1123387696@sss.pgh.pa.us>
Tom,
> As best I can tell, my version of CopyReadAttributes is significantly
> quicker than Alon's, approximately balancing out the fact that my
> version of CopyReadLine is slower. I did the latter first, and would
> now be tempted to rewrite it in the same style as CopyReadAttributes,
> ie one pass of memory-to-memory copy using pointers rather than buffer
> indexes.
I think you are right, with the exception that Alon's results prove out that
the net result of your patch is 20% slower than his.
I think with your speedup of CopyReadAttributes and some additional work on
CopyReadLine the net result could be 50% faster than Alon's patch.
The key thing that is missing is the lack of micro-parallelism in the
character processing in this version. By "inverting the loop", or putting
the characters into a buffer on the outside, then doing fast character
scanning inside with special "fix-up" cases, we exposed long runs of
pipeline-able code to the compiler.
I think there is another way to accomplish the same thing and still preserve
the current structure, but it requires "strip mining" the character buffer
into chunks that can be processed with an explicit loop to check for the
different characters. While it may seem artificial (it is), it will provide
the compiler with the ability to pipeline the character finding logic over
long runs. The other necessary element will have to avoid pipeline stalls
from the "if" conditions as much as possible.
Anyway, thanks for reviewing this code and improving it - it's important to
bring speed increases to our collective customer base. With Bizgres, we're
not satisfied with 12 MB/s, we won't stop until we saturate the I/O bus, so
we may get more extreme with the code than seems reasonable for the general
audience.
- Luke
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: llonergan@greenplum.com, tgl@sss.pgh.pa.us, agoldshuv@greenplum.com, pgsql-patches@postgresql.org
Subject: Re: COPY FROM performance improvements
In-Reply-To: <BF1ED512.C208%llonergan@greenplum.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox