Message-ID: <4DA48768.80605@2ndquadrant.com>
Date: Tue, 12 Apr 2011 10:10:00 -0700
From: Greg Smith <greg@2ndquadrant.com>
User-Agent: Thunderbird 2.0.0.24 (X11/20100317)
MIME-Version: 1.0
To: Scott Marlowe <scott.marlowe@gmail.com>
CC: Glyn Astill <glynastill@yahoo.co.uk>,
	Kevin Grittner <Kevin.Grittner@wicourts.gov>,
	"Joshua D. Drake" <jd@commandprompt.com>, pgsql-performance@postgresql.org
Subject: Re: Linux: more cores = less concurrency.
References: <BANLkTik2+T3xmpStJNVxWzXQWgxU14bVeA@mail.gmail.com>
	<356731.11980.qm@web26005.mail.ukl.yahoo.com>
	<BANLkTi=zS8Lxutk_4n_i=Bgx8B+BuWVPTQ@mail.gmail.com>
In-Reply-To: <BANLkTi=zS8Lxutk_4n_i=Bgx8B+BuWVPTQ@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Scott Marlowe wrote:
> Have you tried running the memory stream benchmark Greg Smith had
> posted here a while back?  It'll let you know if you're memory is
> bottlenecking.  Right now my 48 core machines are the king of that
> benchmark with something like 70+Gig a second.
>   

The big Opterons are still the front-runners here, but not with 70GB/s 
anymore.  Earlier versions of stream-scaling didn't use nearly enough 
data to avoid L3 cache in the processors interfering with results.  More 
recent tests I've gotten in done after I expanded the default test size 
for them show the Opterons normally hitting the same ~35GB/s maximum 
throughput that the Intel processors get out of similar DDR3/1333 sets.  
There are some outliers where >50GB/s still shows up.  I'm not sure if I 
really believe them though; attempts to increase the test size now hit a 
32-bit limit inside stream.c, and I think that's not really big enough 
to avoid L3 cache effects here.

In the table at https://github.com/gregs1104/stream-scaling the 4 X 6172 
server is similar to Scott's system.  I believe the results for 8 
(37613) and 48 cores (32301) there.  I remain somewhat suspicious that 
the higher reuslts of 40 - 51GB/s shown between 16 and 32 cores may be 
inflated by caching.  At this point I'll probably need direct access to 
one of them to resolve this for sure.  I've made a lot of progress with 
other people's servers, but complete trust in those particular results 
still isn't there yet.

-- 
Greg Smith   2ndQuadrant US    greg@2ndQuadrant.com   Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support  www.2ndQuadrant.us
"PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books