From: Mark Kirkwood <mark.kirkwood@catalyst.net.nz>
To: pgsql-performance@postgresql.org
Subject: Re: 60 core performance with 9.3
Date: Fri, 27 Jun 2014 14:28:20 +1200
Message-ID: <53ACD6C4.9050605@catalyst.net.nz> (raw)
In-Reply-To: <CAOR=d=3=Y6yNtwQenwVBGe6ZgjAeA8PwgYb3=aQEKn+vYWhBJA@mail.gmail.com>
References: <53ACB190.2030206@catalyst.net.nz>
<CAOR=d=3=Y6yNtwQenwVBGe6ZgjAeA8PwgYb3=aQEKn+vYWhBJA@mail.gmail.com>
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-performance>
On 27/06/14 14:01, Scott Marlowe wrote:
> On Thu, Jun 26, 2014 at 5:49 PM, Mark Kirkwood
> <mark.kirkwood@catalyst.net.nz> wrote:
>> I have a nice toy to play with: Dell R920 with 60 cores and 1TB ram [1].
>>
>> The context is the current machine in use by the customer is a 32 core one,
>> and due to growth we are looking at something larger (hence 60 cores).
>>
>> Some initial tests show similar pgbench read only performance to what Robert
>> found here
>> http://rhaas.blogspot.co.nz/2012/04/did-i-say-32-cores-how-about-64.html
>> (actually a bit quicker around 400000 tps).
>>
>> However doing a mixed read-write workload is getting results the same or
>> only marginally quicker than the 32 core machine - particularly at higher
>> number of clients (e.g 200 - 500). I have yet to break out the perf toolset,
>> but I'm wondering if any folk has compared 32 and 60 (or 64) core read write
>> pgbench performance?
>
> My guess is that the read only test is CPU / memory bandwidth limited,
> but the mixed test is IO bound.
>
> What's your iostat / vmstat / iotop etc look like when you're doing
> both read only and read/write mixed?
>
>
That was what I would have thought too, but it does not appear to be the
case, here is a typical iostat:
Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s
avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00
nvme0n1 0.00 0.00 0.00 4448.00 0.00 41.47
19.10 0.14 0.03 0.00 0.03 0.03 14.40
nvme1n1 0.00 0.00 0.00 4448.00 0.00 41.47
19.10 0.15 0.03 0.00 0.03 0.03 15.20
nvme2n1 0.00 0.00 0.00 4549.00 0.00 42.20
19.00 0.15 0.03 0.00 0.03 0.03 15.20
nvme3n1 0.00 0.00 0.00 4548.00 0.00 42.19
19.00 0.16 0.04 0.00 0.04 0.04 16.00
dm-0 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00
md0 0.00 0.00 0.00 17961.00 0.00 83.67
9.54 0.00 0.00 0.00 0.00 0.00 0.00
dm-1 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-2 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-3 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-4 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00 0.00
My feeling is spinlock or similar, 'perf top' shows
kernel find_busiest_group
kernel _raw_spin_lock
as the top time users.
--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: mark.kirkwood@catalyst.net.nz
Subject: Re: 60 core performance with 9.3
In-Reply-To: <53ACD6C4.9050605@catalyst.net.nz>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox