Date: Mon, 11 Dec 2006 13:57:28 -0500
From: Michael Stone <mstone+postgres@mathom.us>
Subject: Re: New to PostgreSQL, performance considerations
In-reply-to: <C1A2E3DF.154B6%llonergan@greenplum.com>
To: pgsql-performance@postgresql.org
Mail-followup-to: pgsql-performance@postgresql.org
Message-id: <20061211185727.GJ16692@mathom.us>
MIME-version: 1.0
Content-type: text/plain; charset=us-ascii; format=flowed
Content-disposition: inline
References: <20061211173105.GH16692@mathom.us>
	<C1A2E3DF.154B6%llonergan@greenplum.com>
User-Agent: Mutt/1.5.13 (2006-08-11)

On Mon, Dec 11, 2006 at 10:30:55AM -0800, Luke Lonergan wrote:
>Here's one - I wrote a general purpose Computational Fluid Dynamics analysis
>method used by hundreds of people to perform aircraft and propulsion systems
>analysis.  

That's kinda the opposite of what I meant by general code. I was trying 
(perhaps poorly) to distinguish between scientific codes and other 
stuff (especially I/O or human interface code).

>Compiler flag tuning would speed it up by factors of 2-3 or even
>more on some architectures.  The reason it was so effective is that the
>structure of the code was designed to be general, but also to expose the
>critical performance sections in a way that the compilers could use - deep
>pipelining/vectorization, unrolling, etc, were carefully made easy for the
>compilers to exploit in critical sections.  

It also sounds like code specifically written to take advantage of 
compiler techniques, rather than random code thrown at a pile of cflags. 
I don't disagree that it is possible to get performance improvements if 
code is written to be performant code; I do (and did) disagree with the 
idea that you'll get huge performance improvements by taking regular old 
C application code and playing with compiler flags. 

>Yes, this made the code in those
>sections harder to read, but it was a common practice because it might 
>take weeks of runtime to get an answer and performance mattered.

IMO that's appropriate for some science codes (although I think even 
that sector is beginning to find that they've gone too far in a lot of 
ways), but for a database I'd rather have people debugging clean, readable 
code than risking my data to something incomprehensible that runs in 
optimal time.

>Column databases like C-Store remove these abstractions at planner time to
>expose native operations in large chunks to the compiler and the IPC
>reflects that - typically 1+ and as high as 2.5.  If we were to redesign the
>executor and planner to emulate that same structure we could achieve similar
>speedups and the compiler would matter more.

gcc --make-it-really-fast-by-rewriting-it-from-the-ground-up?

Mike Stone