From: Greg Smith <greg@2ndquadrant.com>
To: Kevin Grittner <Kevin.Grittner@wicourts.gov>
Cc: david@lang.hm, Steve Clark <sclark@netwolves.com>
Cc: Glyn Astill <glynastill@yahoo.co.uk>
Cc: Joshua D. Drake <jd@commandprompt.com>
Cc: Scott Marlowe <scott.marlowe@gmail.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Linux: more cores = less concurrency.
Date: Tue, 12 Apr 2011 10:00:39 -0700
Message-ID: <4DA48537.7080405@2ndquadrant.com> (raw)
In-Reply-To: <4DA43A2B020000250003C702@gw.wicourts.gov>
References: <4DA41EA5020000250003C6E1@gw.wicourts.gov>
<812972.84286.qm@web26008.mail.ukl.yahoo.com>
<4DA43A2B020000250003C702@gw.wicourts.gov>
Kevin Grittner wrote:
> Glyn Astill <glynastill@yahoo.co.uk> wrote:
>
>
>> Results from Greg Smiths stream_scaling test are here:
>>
>> http://www.privatepaste.com/4338aa1196
>>
>
> Well, that pretty much clinches it. Your RAM access tops out at 16
> processors. It appears that your processors are spending most of
> their time waiting for and contending for the RAM bus.
>
I've pulled Glyn's results into
https://github.com/gregs1104/stream-scaling so they're easy to compare
against similar processors, his system is the one labled 4 X X7550. I'm
hearing this same story from multiple people lately: these 32+ core
servers bottleneck on aggregate memory speed with running PostgreSQL
long before the CPUs are fully utilized. This server is close to
maximum memory utilization at 8 cores, and the small increase in gross
throughput above that doesn't seem to be making up for the loss in L1
and L2 thrashing from trying to run more. These systems with many cores
can only be used fully if you have a program that can work efficiency
some of the time with just local CPU resources. That's very rarely the
case for a database that's moving 8K pages, tuple caches, and other
forms of working memory around all the time.
> I have gotten machines in where moving a jumper, flipping a DIP
> switch, or changing BIOS options from the default made a big
> difference. I'd be looking at the manuals for my motherboard and
> BIOS right now to see what options there might be to improve that
I already forwarded Glyn a good article about tuning these Dell BIOSs in
particular from an interesting blog series others here might like too:
http://bleything.net/articles/postgresql-benchmarking-memory.html
Ben Bleything is doing a very thorough walk-through of server hardware
validation, and as is often the case he's already found one major
problem with the vendor config he had to fix to get expected results.
--
Greg Smith 2ndQuadrant US greg@2ndQuadrant.com Baltimore, MD
PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.us
"PostgreSQL 9.0 High Performance": http://www.2ndQuadrant.com/books
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: greg@2ndquadrant.com, Kevin.Grittner@wicourts.gov, sclark@netwolves.com, glynastill@yahoo.co.uk, jd@commandprompt.com, scott.marlowe@gmail.com
Subject: Re: Linux: more cores = less concurrency.
In-Reply-To: <4DA48537.7080405@2ndquadrant.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox