pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedHT on or off for E5-26xx ?
10+ messages / 4 participants
[nested] [flat]
* HT on or off for E5-26xx ?
@ 2012-11-07 03:31 David Boreham <david_list@boreham.org>
0 siblings, 2 replies; 10+ messages in thread
From: David Boreham @ 2012-11-07 03:31 UTC (permalink / raw)
To: pgsql-performance
I'm bringing up a new type of server using Intel E5-2620 (unisocket)
which was selected for good SpecIntRate performance vs cost/power (201
for $410 and 95W).
Was assuming it was 6-core but I just noticed it has HT which is
currently enabled since I see 12 cores in /proc/cpuinfo
Question for the performance experts : is it better to have HT enabled
or disabled for this generation of Xeon ?
Workload will be moderately concurrent, small OLTP type transactions.
We'll also run a few low-load VMs (using KVM) and a big Java application.
Any thoughts welcome.
Thanks.
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: HT on or off for E5-26xx ?
@ 2012-11-07 04:16 Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
parent: David Boreham <david_list@boreham.org>
1 sibling, 1 reply; 10+ messages in thread
From: Mark Kirkwood @ 2012-11-07 04:16 UTC (permalink / raw)
To: pgsql-performance
On 07/11/12 16:31, David Boreham wrote:
>
> I'm bringing up a new type of server using Intel E5-2620 (unisocket)
> which was selected for good SpecIntRate performance vs cost/power (201
> for $410 and 95W).
>
> Was assuming it was 6-core but I just noticed it has HT which is
> currently enabled since I see 12 cores in /proc/cpuinfo
>
> Question for the performance experts : is it better to have HT enabled
> or disabled for this generation of Xeon ?
> Workload will be moderately concurrent, small OLTP type transactions.
> We'll also run a few low-load VMs (using KVM) and a big Java application.
>
>
>
I've been benchmarking a E5-4640 (4 socket) and hyperthreading off gave
much better scaling behaviour in pgbench (gentle rise and flatten off),
whereas with hyperthreading on there was a dramatic falloff after approx
number clients = number of (hyperthreaded) cpus. The box is intended to
be a pure db server, so we are running with hyperthreading off.
Cheers
Mark
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: HT on or off for E5-26xx ?
@ 2012-11-07 13:33 David Boreham <david_list@boreham.org>
parent: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
0 siblings, 1 reply; 10+ messages in thread
From: David Boreham @ 2012-11-07 13:33 UTC (permalink / raw)
To: pgsql-performance
On 11/6/2012 9:16 PM, Mark Kirkwood wrote:
>
>
> I've been benchmarking a E5-4640 (4 socket) and hyperthreading off
> gave much better scaling behaviour in pgbench (gentle rise and flatten
> off), whereas with hyperthreading on there was a dramatic falloff
> after approx number clients = number of (hyperthreaded) cpus. The box
> is intended to be a pure db server, so we are running with
> hyperthreading off.
It looks like this syndrome is not observed on my box, likely due to the
much lower number of cores system-wide (12).
I see pgbench tps increase nicely until #threads/clients == #cores, then
plateau. I tested up to 96 threads btw.
We're waiting on more memory modules to arrive. I'll post some test
results once we have all 4 memory banks populated.
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: HT on or off for E5-26xx ?
@ 2012-11-07 13:37 Devrim GÜNDÜZ <devrim@gunduz.org>
parent: David Boreham <david_list@boreham.org>
1 sibling, 1 reply; 10+ messages in thread
From: Devrim GÜNDÜZ @ 2012-11-07 13:37 UTC (permalink / raw)
To: David Boreham <david_list@boreham.org>; +Cc: pgsql-performance
Hi,
On Tue, 2012-11-06 at 20:31 -0700, David Boreham wrote:
> Was assuming it was 6-core but I just noticed it has HT which is
> currently enabled since I see 12 cores in /proc/cpuinfo
>
> Question for the performance experts : is it better to have H enabled
> or disabled for this generation of Xeon ? Workload will be moderately
> concurrent, small OLTP type transactions. We'll also run a few
> low-load VMs (using KVM) and big Java application.
HT should be good for file servers, or say many of the app servers, or
small web/mail servers. PostgreSQL relies on the CPU power, and since
the HT CPUs don't have the same power as the original CPU, when OS
submits a job to that particular HTed CPU, query will run significantly
slow. To avoid issues, I would suggest you to turn HT off on all
PostgreSQL servers. If you can throw some more money, another 6-core CPU
would give more benefit.
Regards,
--
Devrim GÜNDÜZ
Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com
PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer
Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org Twitter: http://twitter.com/devrimgunduz
Attachments:
signature.asc (application/pgp-signature, 197B) download
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: HT on or off for E5-26xx ?
@ 2012-11-07 13:45 David Boreham <david_list@boreham.org>
parent: Devrim GÜNDÜZ <devrim@gunduz.org>
0 siblings, 0 replies; 10+ messages in thread
From: David Boreham @ 2012-11-07 13:45 UTC (permalink / raw)
To: pgsql-performance
On 11/7/2012 6:37 AM, Devrim GÜNDÜZ wrote:
> HT should be good for file servers, or say many of the app servers, or
> small web/mail servers. PostgreSQL relies on the CPU power, and since
> the HT CPUs don't have the same power as the original CPU, when OS
> submits a job to that particular HTed CPU, query will run significantly
> slow. To avoid issues, I would suggest you to turn HT off on all
> PostgreSQL servers. If you can throw some more money, another 6-core CPU
> would give more benefit.
I realize this is the "received knowledge" but it is not supported by
the evidence before me (which is that I get nearly 2x the throughput
from pgbench using nthreads == nhtcores vs nthreads == nfullcores).
Intel's latest HT implementation seems to suffer less from the kinds of
resource sharing contention issues seen in older generations.
Once I have the machine's full memory installed I'll run pgbench with HT
disabled in the BIOS and post the results.
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: HT on or off for E5-26xx ?
@ 2012-11-07 21:20 Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
parent: David Boreham <david_list@boreham.org>
0 siblings, 1 reply; 10+ messages in thread
From: Mark Kirkwood @ 2012-11-07 21:20 UTC (permalink / raw)
To: pgsql-performance
On 08/11/12 02:33, David Boreham wrote:
> On 11/6/2012 9:16 PM, Mark Kirkwood wrote:
>>
>>
>> I've been benchmarking a E5-4640 (4 socket) and hyperthreading off
>> gave much better scaling behaviour in pgbench (gentle rise and
>> flatten off), whereas with hyperthreading on there was a dramatic
>> falloff after approx number clients = number of (hyperthreaded) cpus.
>> The box is intended to be a pure db server, so we are running with
>> hyperthreading off.
>
> It looks like this syndrome is not observed on my box, likely due to
> the much lower number of cores system-wide (12).
> I see pgbench tps increase nicely until #threads/clients == #cores,
> then plateau. I tested up to 96 threads btw.
>
> We're waiting on more memory modules to arrive. I'll post some test
> results once we have all 4 memory banks populated.
>
>
>
>
Interesting - I was wondering if a single socket board would behave
differently (immediately after posting of course)...I've got an i3 home
system that scales nicely even with hyperthreading on (2 cores, 4
typerthreads).
Cheers
Mark
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: HT on or off for E5-26xx ?
@ 2012-11-08 03:16 David Boreham <david_list@boreham.org>
parent: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
0 siblings, 1 reply; 10+ messages in thread
From: David Boreham @ 2012-11-08 03:16 UTC (permalink / raw)
To: pgsql-performance
Well, the results are in and at least in this particular case
conventional wisdom is overturned. Not a huge benefit, but throughput is
definitely higher with HT enabled and nthreads >> ncores:
HT off :
bash-4.1$ /usr/pgsql-9.2/bin/pgbench -T 600 -j 48 -c 48
starting vacuum...end.
transaction type: TPC-B (sort of)
scaling factor: 100
query mode: simple
number of clients: 48
number of threads: 48
duration: 600 s
number of transactions actually processed: 2435711
tps = 4058.667332 (including connections establishing)
tps = 4058.796309 (excluding connections establishing)
avg-cpu: %user %nice %system %iowait %steal %idle
52.50 0.00 14.79 5.07 0.00 27.64
Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s
avgrq-sz avgqu-sz await svctm %util
sda 0.00 5700.30 0.10 13843.50 0.00 74.78
11.06 48.46 3.50 0.05 65.21
HT on:
bash-4.1$ /usr/pgsql-9.2/bin/pgbench -T 600 -j 48 -c 48
starting vacuum...end.
transaction type: TPC-B (sort of)
scaling factor: 100
query mode: simple
number of clients: 48
number of threads: 48
duration: 600 s
number of transactions actually processed: 2832463
tps = 4720.668984 (including connections establishing)
tps = 4720.750477 (excluding connections establishing)
avg-cpu: %user %nice %system %iowait %steal %idle
40.61 0.00 12.71 3.09 0.00 43.59
Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s
avgrq-sz avgqu-sz await svctm %util
sda 0.00 6197.10 14.80 16389.50 0.14 86.53
10.82 54.11 3.30 0.05 82.35
System details:
E5-2620 (6 core + HT 15Mb LL) 64G (4 channels with 16G 1333 modules),
Intel 710 300G (which is faster than the smaller drives, note),
Supermicro X9SRi-F Motherboard.
CentOS 6.3 64-bit, PG 9.2.1 from the PGDG RPM repository. pgbench
running locally on the server.
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: HT on or off for E5-26xx ?
@ 2012-11-08 13:58 Shaun Thomas <sthomas@optionshouse.com>
parent: David Boreham <david_list@boreham.org>
0 siblings, 1 reply; 10+ messages in thread
From: Shaun Thomas @ 2012-11-08 13:58 UTC (permalink / raw)
To: David Boreham <david_list@boreham.org>; +Cc: pgsql-performance
On 11/07/2012 09:16 PM, David Boreham wrote:
> bash-4.1$ /usr/pgsql-9.2/bin/pgbench -T 600 -j 48 -c 48
Unfortunately without -S, you're not really testing the processors. A
regular pgbench can fluctuate a more than that due to writing and
checkpoints.
For what it's worth, our X5675's perform about 40-50% better with HT
enabled. Not the 2x you might expect by doubling the amount of
"processors", but it definitely didn't make things worse.
--
Shaun Thomas
OptionsHouse | 141 W. Jackson Blvd. | Suite 500 | Chicago IL, 60604
312-444-8534
sthomas@optionshouse.com
______________________________________________
See http://www.peak6.com/email_disclaimer/ for terms and conditions related to this email
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: HT on or off for E5-26xx ?
@ 2012-11-08 14:18 David Boreham <david_list@boreham.org>
parent: Shaun Thomas <sthomas@optionshouse.com>
0 siblings, 1 reply; 10+ messages in thread
From: David Boreham @ 2012-11-08 14:18 UTC (permalink / raw)
To: pgsql-performance
On 11/8/2012 6:58 AM, Shaun Thomas wrote:
> On 11/07/2012 09:16 PM, David Boreham wrote:
>
>> bash-4.1$ /usr/pgsql-9.2/bin/pgbench -T 600 -j 48 -c 48
>
> Unfortunately without -S, you're not really testing the processors. A
> regular pgbench can fluctuate a more than that due to writing and
> checkpoints.
Hmm...my goal was to test with a workload close to our application's
(which is heavy OLTP, small transactions and hence sensitive to I/O
commit rate).
The hypothesis I was testing was that enabling HT positively degrades
performance (which in my case it does not). I wasn't to be honest really
testing the additional benefit from HT, rather observing that it is
non-negative :)
If I have time I can run the select-only test for you and post the
results. The DB fits into memory so it will be a good CPU test.
^ permalink raw reply [nested|flat] 10+ messages in thread
* Re: HT on or off for E5-26xx ?
@ 2012-11-09 14:34 David Boreham <david_list@boreham.org>
parent: David Boreham <david_list@boreham.org>
0 siblings, 0 replies; 10+ messages in thread
From: David Boreham @ 2012-11-09 14:34 UTC (permalink / raw)
To: pgsql-performance
Here are the SELECT only pgbench test results from my E5-2620 machine,
with HT on and off:
HT off:
bash-4.1$ /usr/pgsql-9.2/bin/pgbench -T 600 -j 48 -c 48 -S
starting vacuum...end.
transaction type: SELECT only
scaling factor: 100
query mode: simple
number of clients: 48
number of threads: 48
duration: 600 s
number of transactions actually processed: 25969680
tps = 43281.392273 (including connections establishing)
tps = 43282.476955 (excluding connections establishing)
All 6 cores saturated:
avg-cpu: %user %nice %system %iowait %steal %idle
81.42 0.00 18.21 0.00 0.00 0.37
HT on:
bash-4.1$ /usr/pgsql-9.2/bin/pgbench -T 600 -j 48 -c 48 -S
starting vacuum...end.
transaction type: SELECT only
scaling factor: 100
query mode: simple
number of clients: 48
number of threads: 48
duration: 600 s
number of transactions actually processed: 29934601
tps = 49888.697225 (including connections establishing)
tps = 49889.570754 (excluding connections establishing)
12% of CPU showing as idle (whether that's true or not I'm not sure):
avg-cpu: %user %nice %system %iowait %steal %idle
71.09 0.00 16.99 0.00 0.00 11.92
So for this particular test HT gives us the equivalent of about one
extra core.
It does not reduce performance, rather increases performance slightly.
^ permalink raw reply [nested|flat] 10+ messages in thread
end of thread, other threads:[~2012-11-09 14:34 UTC | newest]
Thread overview: 10+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2012-11-07 03:31 HT on or off for E5-26xx ? David Boreham <david_list@boreham.org>
2012-11-07 04:16 ` Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
2012-11-07 13:33 ` David Boreham <david_list@boreham.org>
2012-11-07 21:20 ` Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
2012-11-08 03:16 ` David Boreham <david_list@boreham.org>
2012-11-08 13:58 ` Shaun Thomas <sthomas@optionshouse.com>
2012-11-08 14:18 ` David Boreham <david_list@boreham.org>
2012-11-09 14:34 ` David Boreham <david_list@boreham.org>
2012-11-07 13:37 ` Devrim GÜNDÜZ <devrim@gunduz.org>
2012-11-07 13:45 ` David Boreham <david_list@boreham.org>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox