Date: Tue, 17 Sep 2013 15:35:46 +0200
From: Andres Freund <andres@2ndquadrant.com>
To: Merlin Moncure <mmoncure@gmail.com>
Cc: =?utf-8?B?0JTQvNC40YLRgNC40Lkg0JTQtdCz0YLRj9GA0ZHQsg==?=
 <degtyaryov@gmail.com>,
 postgres performance list <pgsql-performance@postgresql.org>
Subject: Re: Cpu usage 100% on slave. s_lock problem.
Message-ID: <20130917133546.GE5452@awork2.anarazel.de>
References: 
 <CAHyXU0wZZPDU5AfsnC_z_kg5LVLzC9cXr+T84N2n+u=GDxNi8w@mail.gmail.com>
 <CAHyXU0wavt2APZ89C7x9b3y-p-2trkdk6UpyQotiPXgPVKd_UQ@mail.gmail.com>
 <CAHyXU0z3BbK=-Qu342Nxr9EbLtASKWq7XH+9tyaq17MQd+YP0w@mail.gmail.com>
 <20130827155556.GA9061@awork2.anarazel.de>
 <CAHyXU0yLWU4O=0n6qm5tDTDxBdNwzjhUfA456cYggWY6zUXDMg@mail.gmail.com>
 <CAFcrtwHU_iEfhoz-atgNXwVsw=3F4k8x5hfsrAxtB=ftOnWYBw@mail.gmail.com>
 <20130917115926.GA5452@awork2.anarazel.de>
 <CAHyXU0weAh-qZQ_CCCNoded9wd9WrzVdmnyto_JN+kESjN-1Jw@mail.gmail.com>
 <20130917132439.GD5452@awork2.anarazel.de>
 <CAHyXU0x3G3my-L5Yy=Qrp8XFPc_7MOQEWao47W-opF44O9WP0g@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: 
 <CAHyXU0x3G3my-L5Yy=Qrp8XFPc_7MOQEWao47W-opF44O9WP0g@mail.gmail.com>
Precedence: bulk
Sender: pgsql-performance-owner@postgresql.org

On 2013-09-17 08:32:30 -0500, Merlin Moncure wrote:
> On Tue, Sep 17, 2013 at 8:24 AM, Andres Freund <andres@2ndquadrant.com> wrote:
> > On 2013-09-17 08:18:54 -0500, Merlin Moncure wrote:
> >> Do you think it's worth submitting the lock avoidance patch for formal review?
> >
> > You mean the bufmgr.c thing? Generally I think that that code needs a
> > good of scalability work - there's a whole thread about it
> > somewhere. But TBH the theories you've voiced about the issues you've
> > seen haven't convinced me so far.
> 
> er, no (but I share your skepticism -- my challenge right now is to
> demonstrate measurable benefit which so far I've been unable to do).
> I was talking about the patch on  *this* thread which bypasses the
> s_lock in RecoveryInProgress()  :-).

Ah, yes. Sorry confused issues ;). Yes, I think that'd made sense.

> > Quick question: Do you happen to have pg_locks output from back then
> > around? We've recently found servers going into somewhat similar
> > slowdowns because they exhausted the fastpath locks which made lwlocks
> > far more expensive and made s_lock go up very high in the profle.
> 
> I do. Unfortunately I don't have profile info.   Not sure how useful
> it is -- I'll send it off-list.

Great.

The primary thing I'd like to know is whether there are lots of
non-fastpath locks...

If you ever get into the situation I mistakenly referred to again, I'd
strongly suggest recompling postgres with -fno-omit-frame-pointer. That
makes hierarchical profiles actually useful which can help tremendously
with diagnosing issues like this...

Greetings,

Andres Freund

-- 
 Andres Freund	                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services


-- 
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance