From: Glyn Astill <glynastill@yahoo.co.uk>
To: Kevin Grittner <Kevin.Grittner@wicourts.gov>
To: Merlin Moncure <mmoncure@gmail.com>
Cc: david@lang.hm, Steve Clark <sclark@netwolves.com>
Cc: Joshua D. Drake <jd@commandprompt.com>
Cc: Scott Marlowe <scott.marlowe@gmail.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: Linux: more cores = less concurrency.
Date: Tue, 12 Apr 2011 09:54:59 +0100 (BST)
Message-ID: <768150.83213.qm@web26002.mail.ukl.yahoo.com> (raw)
In-Reply-To: <BANLkTi=p=7CaidNh9M1gW7MUmk+UwBO19A@mail.gmail.com>
--- On Tue, 12/4/11, Merlin Moncure <mmoncure@gmail.com> wrote:
> >> The issue I'm seeing is that 8 real cores
> outperform 16 real
> >> cores, which outperform 32 real cores under high
> concurrency.
> >
> > With every benchmark I've done of PostgreSQL, the
> "knee" in the
> > performance graph comes right around ((2 * cores) +
> > effective_spindle_count). With the database fully
> cached (as I
> > believe you mentioned), effective_spindle_count is
> zero. If you
> > don't use a connection pool to limit active
> transactions to the
> > number from that formula, performance drops off. The
> more CPUs you
> > have, the sharper the drop after the knee.
>
> I was about to say something similar with some canned
> advice to use a
> connection pooler to control this. However, OP
> scaling is more or
> less topping out at cores / 4...yikes!. Here are my
> suspicions in
> rough order:
>
> 1. There is scaling problem in client/network/etc.
> Trivially
> disproved, convert the test to pgbench -f and post results
> 2. The test is in fact i/o bound. Scaling is going to be
> hardware/kernel determined. Can we see
> iostat/vmstat/top snipped
> during test run? Maybe no-op is burning you?
This is during my 80 clients test, this is a point at which the performance is well below that of the same machine limited to 8 cores.
http://www.privatepaste.com/dc131ff26e> 3. Locking/concurrency issue in heavy_seat_function()
> (source for
> that?) how much writing does it do?
>
No writing afaik - its a select with a few joins and subqueries - I'm pretty sure it's not writing out temp data either, but all clients are after the same data in the test - maybe theres some locks there?
> Can we see some iobound and cpubound pgbench runs on both
> servers?
>
Of course, I'll post when I've gotten to that.
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: glynastill@yahoo.co.uk, Kevin.Grittner@wicourts.gov, mmoncure@gmail.com, sclark@netwolves.com, jd@commandprompt.com, scott.marlowe@gmail.com
Subject: Re: Linux: more cores = less concurrency.
In-Reply-To: <768150.83213.qm@web26002.mail.ukl.yahoo.com>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox