DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.co.uk;
	h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding;
	b=bknrKEsqXBmNdtE7ChC+qzKNHzquYeiFYFbrp0ZY3Q1YAIIvSlZ5v7wFUftgYS3O9+RzkKqglZyzdrpFaCNYnEQFOzzJEjTq+CrsDg4MaS5YsuhRuw4PwgI6g9tsOfwBW+tkackDLMXz0bWXYIbi0DDplwpz+fPLq53WzdxN7u0=;
Message-ID: <188622.2398.qm@web26002.mail.ukl.yahoo.com>
Date: Wed, 13 Apr 2011 10:48:01 +0100 (BST)
From: Glyn Astill <glynastill@yahoo.co.uk>
Subject: Re: Linux: more cores = less concurrency.
To: Kevin Grittner <Kevin.Grittner@wicourts.gov>,
	Greg Smith <greg@2ndquadrant.com>
Cc: david@lang.hm, Steve Clark <sclark@netwolves.com>,
	"Joshua D. Drake" <jd@commandprompt.com>,
	Scott Marlowe <scott.marlowe@gmail.com>, pgsql-performance@postgresql.org
In-Reply-To: <4DA48537.7080405@2ndquadrant.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

--- On Tue, 12/4/11, Greg Smith <greg@2ndquadrant.com> wrote:=0A=0A> From: =
Greg Smith <greg@2ndquadrant.com>=0A> Subject: Re: [PERFORM] Linux: more co=
res =3D less concurrency.=0A> To: "Kevin Grittner" <Kevin.Grittner@wicourts=
.gov>=0A> Cc: david@lang.hm, "Steve Clark" <sclark@netwolves.com>, "Glyn As=
till" <glynastill@yahoo.co.uk>, "Joshua D. Drake" <jd@commandprompt.com>, "=
Scott Marlowe" <scott.marlowe@gmail.com>, pgsql-performance@postgresql.org=
=0A> Date: Tuesday, 12 April, 2011, 18:00=0A> Kevin Grittner wrote:=0A> > G=
lyn Astill <glynastill@yahoo.co.uk>=0A> wrote:=0A> >=A0 =A0 =0A> >> Results=
 from Greg Smiths stream_scaling test are=0A> here:=0A> >> =0A> >> http://w=
ww.privatepaste.com/4338aa1196=0A> >>=A0 =A0=A0=A0=0A> >=A0 Well, that pret=
ty much clinches it.=A0 Your=0A> RAM access tops out at 16=0A> > processors=
.=A0 It appears that your processors are=0A> spending most of=0A> > their t=
ime waiting for and contending for the RAM=0A> bus.=0A> >=A0=A0=A0=0A> =0A>=
 I've pulled Glyn's results into https://github.com/gregs1104/stream-scalin=
g so they're=0A> easy to compare against similar processors, his system is=
=0A> the one labled 4 X X7550.=A0 I'm hearing this same story=0A> from mult=
iple people lately:=A0 these 32+ core servers=0A> bottleneck on aggregate m=
emory speed with running PostgreSQL=0A> long before the CPUs are fully util=
ized.=A0 This server=0A> is close to maximum memory utilization at 8 cores,=
 and the=0A> small increase in gross throughput above that doesn't seem=0A>=
 to be making up for the loss in L1 and L2 thrashing from=0A> trying to run=
 more.=A0 These systems with many cores can=0A> only be used fully if you h=
ave a program that can work=0A> efficiency some of the time with just local=
 CPU=0A> resources.=A0 That's very rarely the case for a database=0A> that'=
s moving 8K pages, tuple caches, and other forms of=0A> working memory arou=
nd all the time.=0A> =0A> =0A> > I have gotten machines in where moving a j=
umper,=0A> flipping a DIP=0A> > switch, or changing BIOS options from the d=
efault made=0A> a big=0A> > difference.=A0 I'd be looking at the manuals fo=
r my=0A> motherboard and=0A> > BIOS right now to see what options there mig=
ht be to=0A> improve that=0A> =0A> I already forwarded Glyn a good article =
about tuning these=0A> Dell BIOSs in particular from an interesting blog se=
ries=0A> others here might like too:=0A> =0A> http://bleything.net/articles=
/postgresql-benchmarking-memory.html=0A> =0A> Ben Bleything is doing a very=
 thorough walk-through of=0A> server hardware validation, and as is often t=
he case he's=0A> already found one major problem with the vendor config he=
=0A> had to fix to get expected results.=0A> =0A=0AThanks Greg.  I've been =
through that post, but unfortunately there's no settings that make a differ=
ence.=0A=0AHowever upon further investigation and looking at the manual for=
 the R910 here=0A=0Ahttp://support.dell.com/support/edocs/systems/per910/en=
/HOM/HTML/install.htm#wp1266264=0A=0AI've discovered we only have 4 of the =
8 memory risers, and the manual states that in this configuration we are ru=
nning in "Power Optimized" mode, rather than "Performance Optimized".=0A=0A=
We've got two of these machines, so I've just pulled all the risers from on=
e system, removed half the memory as indicated by that document from Dell a=
bove, and now I'm seeing almost double the throughput.=0A=0A