DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.co.uk;
	h=Message-ID:X-YMail-OSG:Received:X-Mailer:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding;
	b=1mFvFefQfOl4UvoTTO2lKiyZTTVc84f7LrazKmfPEekfJ98caLxmvmrR2Ve7dsodUDbPXrCWTJuwegoFVSzc0a+EU4lmsM+zbPt23Mqa5Fl/sheARXndO1bH8LrxJNTzAI+0fJ50D3Ob85S2bKrYcXWzSCmXfz/ucAbz8Btx8Uk=;
Message-ID: <768150.83213.qm@web26002.mail.ukl.yahoo.com>
Date: Tue, 12 Apr 2011 09:54:59 +0100 (BST)
From: Glyn Astill <glynastill@yahoo.co.uk>
Subject: Re: Linux: more cores = less concurrency.
To: Kevin Grittner <Kevin.Grittner@wicourts.gov>,
	Merlin Moncure <mmoncure@gmail.com>
Cc: david@lang.hm, Steve Clark <sclark@netwolves.com>,
	"Joshua D. Drake" <jd@commandprompt.com>,
	Scott Marlowe <scott.marlowe@gmail.com>, pgsql-performance@postgresql.org
In-Reply-To: <BANLkTi=p=7CaidNh9M1gW7MUmk+UwBO19A@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

--- On Tue, 12/4/11, Merlin Moncure <mmoncure@gmail.com> wrote:=0A=0A> >> T=
he issue I'm seeing is that 8 real cores=0A> outperform 16 real=0A> >> core=
s, which outperform 32 real cores under high=0A> concurrency.=0A> >=0A> > W=
ith every benchmark I've done of PostgreSQL, the=0A> "knee" in the=0A> > pe=
rformance graph comes right around ((2 * cores) +=0A> > effective_spindle_c=
ount). =A0With the database fully=0A> cached (as I=0A> > believe you mentio=
ned), effective_spindle_count is=0A> zero. =A0If you=0A> > don't use a conn=
ection pool to limit active=0A> transactions to the=0A> > number from that =
formula, performance drops off. =A0The=0A> more CPUs you=0A> > have, the sh=
arper the drop after the knee.=0A> =0A> I was about to say something simila=
r with some canned=0A> advice to use a=0A> connection pooler to control thi=
s.=A0 However, OP=0A> scaling is more or=0A> less topping out at cores / 4.=
..yikes!.=A0 Here are my=0A> suspicions in=0A> rough order:=0A> =0A> 1. The=
re is scaling problem in client/network/etc.=A0=0A> Trivially=0A> disproved=
, convert the test to pgbench -f and post results=0A> 2. The test is in fac=
t i/o bound. Scaling is going to be=0A> hardware/kernel determined.=A0 Can =
we see=0A> iostat/vmstat/top snipped=0A> during test run?=A0 Maybe no-op is=
 burning you?=0A=0AThis is during my 80 clients test, this is a point at wh=
ich the performance is well below that of the same machine limited to 8 cor=
es.=0A=0Ahttp://www.privatepaste.com/dc131ff26e=0A=0A> 3. Locking/concurren=
cy issue in heavy_seat_function()=0A> (source for=0A> that?)=A0 how much wr=
iting does it do?=0A> =0A=0ANo writing afaik - its a select with a few join=
s and subqueries - I'm pretty sure it's not writing out temp data either, b=
ut all clients are after the same data in the test - maybe theres some lock=
s there?=0A=0A> Can we see some iobound and cpubound pgbench runs on both=
=0A> servers?=0A> =0A=0AOf course, I'll post when I've gotten to that.=0A