User-Agent: Microsoft-Entourage/12.0.0.071130
Date: Mon, 04 Feb 2008 10:47:49 -0800
Subject: Re: Benchmark Data requested
From: Luke Lonergan <llonergan@greenplum.com>
To: Simon Riggs <simon@2ndquadrant.com>,
	<pgsql-performance@postgresql.org>
Message-ID: <C3CC9DD5.52BD9%llonergan@greenplum.com>
Thread-Topic: [PERFORM] Benchmark Data requested
Thread-Index: AchnXnBg0ElUdTP7k0O5rRFrjAaBkQ==
In-Reply-To: <1202150236.4252.521.camel@ebony.site>
Mime-version: 1.0
Content-type: text/plain;
	charset="US-ASCII"
Content-transfer-encoding: 7bit

Hi Simon,

Note that MonetDB/X100 does not have a SQL optimizer, they ran raw
hand-coded plans.  As a consequence, these comparisons should be taken as an
"executor-executor" test and we/you should be sure that the PG planner has
generated the best possible plan.

That said, we've already done the comparisons internally and they've got a
good point to make about L2 cache use and removal of unnecessary
abstractions in the executor.  We've been aware of this since 2005/6 and
have been slowly working these ideas into our/PG executor.

Bottom line: it's a good thing to work to get close to the X100/Monet
executor with a more general purpose DB.  PG is a looong way from being
comparable, mostly due to poor L2 D-cache locality and I-cache thrashing in
the executor.  The only way to cure this is to work on more rows than one at
a time.

- Luke 


On 2/4/08 10:37 AM, "Simon Riggs" <simon@2ndquadrant.com> wrote:

> Can I ask for some help with benchmarking?
> 
> There are some results here that show PostgreSQL is slower in some cases
> than Monet and MySQL. Of course these results were published immediately
> prior to 8.2 being released, plus run out-of-the-box, so without even
> basic performance tuning.
> 
> Would anybody like to repeat these tests with the latest production
> versions of these databases (i.e. with PGSQL 8.3), and with some
> sensible tuning settings for the hardware used? It will be useful to get
> some blind tests with more sensible settings.
> 
> http://monetdb.cwi.nl/projects/monetdb//SQL/Benchmark/TPCH/
> 
> Multiple runs from different people/different hardware is useful since
> they help to iron-out differences in hardware and test methodology. So
> don't worry if you see somebody else doing this also.
> 
> Thanks,