Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1ToHUc-0007bi-QV for pgsql-performance@arkaria.postgresql.org; Thu, 27 Dec 2012 17:43:43 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.72) (envelope-from ) id 1ToHUb-00083b-RJ for pgsql-performance@arkaria.postgresql.org; Thu, 27 Dec 2012 17:43:41 +0000 Received: from magus.postgresql.org ([2a02:c0:301:0:ffff::29]) by malur.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1ToHUa-00082q-WF for pgsql-performance@postgresql.org; Thu, 27 Dec 2012 17:43:41 +0000 Received: from smtp.01.com ([199.36.142.181]) by magus.postgresql.org with esmtp (Exim 4.72) (envelope-from ) id 1ToHUU-0001SF-F3 for pgsql-performance@postgresql.org; Thu, 27 Dec 2012 17:43:40 +0000 Received: from localhost (localhost.localdomain [127.0.0.1]) by smtp-out-2.01.com (Postfix) with ESMTP id BD20B39014E; Thu, 27 Dec 2012 11:43:32 -0600 (CST) X-Virus-Scanned: amavisd-new at smtp-out-2.01.com Received: from smtp.01.com ([127.0.0.1]) by localhost (smtp-out-2.01.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tTrJzVtztZTG; Thu, 27 Dec 2012 11:43:32 -0600 (CST) Received: from smtp.01.com (localhost.localdomain [127.0.0.1]) by smtp-out-2.01.com (Postfix) with ESMTP id 95C66390153; Thu, 27 Dec 2012 11:43:32 -0600 (CST) Received: from [192.168.10.146] (cpe-098-027-058-255.nc.res.rr.com [98.27.58.255]) by smtp-out-2.01.com (Postfix) with ESMTPSA id 0988339014F; Thu, 27 Dec 2012 11:43:31 -0600 (CST) Message-ID: <50DC88C3.8060308@pgexperts.com> Date: Thu, 27 Dec 2012 12:43:31 -0500 From: Andrew Dunstan User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120827 Thunderbird/15.0 MIME-Version: 1.0 To: Jeff Janes CC: pgsql-perform Subject: Re: backend suddenly becomes slow, then remains slow References: <50CB7284.3020008@pgexperts.com> <5309.1355514960@sss.pgh.pa.us> <50CB8913.8050202@dunslane.net> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Pg-Spam-Score: -1.9 (-) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-performance Precedence: bulk Sender: pgsql-performance-owner@postgresql.org On 12/26/2012 11:03 PM, Jeff Janes wrote: > On Fri, Dec 14, 2012 at 10:40 AM, Andrew Dunstan > wrote: > > One of my clients has an odd problem. Every so often a backend will > suddenly > > become very slow. The odd thing is that once this has happened it > remains > > slowed down, for all subsequent queries. Zone reclaim is off. There > is no IO > > or CPU spike, no checkpoint issues or stats timeouts, no other > symptom that > > we can see. > > By "no spike", do you mean that the system as a whole is not using an > unusual amount of IO or CPU, or that this specific slow back-end is > not using an unusual amount? both, really. > > Could you strace is and see what it is doing? Not very easily, because it's a pool connection and we've lowered the pool session lifetime as part of the amelioration :-) So it's not happening very much any more. > > > The problem was a lot worse that it is now, but two steps have > > alleviated it mostly, but not completely: much less aggressive > autovacuuming > > and reducing the maximum lifetime of backends in the connection > pooler to 30 > > minutes. > > Do you have a huge number of tables? Maybe over the course of a > long-lived connection, it touches enough tables to bloat the relcache > / syscache. I don't know how the autovac would be involved in that, > though. > > Yes, we do indeed have a huge number of tables. This seems a plausible thesis. cheers andrew -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance