From: Tom Lane <tgl@sss.pgh.pa.us>
To: Gavin Hamill <gdh@laterooms.com>
Cc: pgsql-performance@postgresql.org
Subject: Re: pg 8.1.3, AIX, huge box, painfully slow.
Date: Fri, 07 Apr 2006 18:52:20 -0400
Message-ID: <17515.1144450340@sss.pgh.pa.us> (raw)
In-Reply-To: <20060407232759.45581362.gdh@laterooms.com>
References: <20060407185823.abe93017.gdh@laterooms.com>
<1144436060.32269.13.camel@state.g2switchworks.com>
<20060407205919.a091da26.gdh@laterooms.com>
<15772.1144440362@sss.pgh.pa.us>
<20060407223635.9c4a0b38.gdh@laterooms.com>
<17055.1144447009@sss.pgh.pa.us>
<20060407232759.45581362.gdh@laterooms.com>
Gavin Hamill <gdh@laterooms.com> writes:
> On Fri, 07 Apr 2006 17:56:49 -0400
> Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> This is not good. Did the semop storms coincide with visible
>> slowdown? (I'd assume so, but you didn't actually say...)
> Yes, there's a definate correlation here.. I attached truss to the
> main postmaster..
> ...
> And when I saw a flood of semop's for any particular PID, a second later
> in the 'topas' process list would show that PID at a 100% CPU ...
So apparently we've still got a problem with multiprocess contention for
an LWLock somewhere. It's not the BufMgrLock because that's gone in 8.1.
It could be one of the finer-grain locks that are still there, or it
could be someplace else.
Are you in a position to try your workload using PG CVS tip? There's a
nontrivial possibility that we've already fixed this --- a couple months
ago I did some work to reduce contention in the lock manager:
2005-12-11 16:02 tgl
* src/: backend/access/transam/twophase.c,
backend/storage/ipc/procarray.c, backend/storage/lmgr/README,
backend/storage/lmgr/deadlock.c, backend/storage/lmgr/lock.c,
backend/storage/lmgr/lwlock.c, backend/storage/lmgr/proc.c,
include/storage/lock.h, include/storage/lwlock.h,
include/storage/proc.h: Divide the lock manager's shared state into
'partitions', so as to reduce contention for the former single
LockMgrLock. Per my recent proposal. I set it up for 16
partitions, but on a pgbench test this gives only a marginal
further improvement over 4 partitions --- we need to test more
scenarios to choose the number of partitions.
This is unfortunately not going to help you as far as getting that
machine into production now (unless you're brave enough to run CVS tip
as production, which I certainly am not). I'm afraid you're most likely
going to have to ship that pSeries back at the end of the month, but
while you've got it it'd be awfully nice if we could use it as a testbed
...
regards, tom lane
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Reply to all the recipients using the --to and --cc options:
reply via email
To: pgsql-performance@postgresql.org
Cc: tgl@sss.pgh.pa.us, gdh@laterooms.com
Subject: Re: pg 8.1.3, AIX, huge box, painfully slow.
In-Reply-To: <17515.1144450340@sss.pgh.pa.us>
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox