Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1WUZPm-0006QI-Fz for pgsql-performance@arkaria.postgresql.org; Mon, 31 Mar 2014 10:26:02 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.80) (envelope-from ) id 1WUZPl-0000Yy-WE for pgsql-performance@arkaria.postgresql.org; Mon, 31 Mar 2014 10:26:02 +0000 Received: from makus.postgresql.org ([2001:4800:7903:4::125]) by malur.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1WUZPl-0000Ys-4Y for pgsql-performance@postgresql.org; Mon, 31 Mar 2014 10:26:01 +0000 Received: from mail-lb0-f171.google.com ([209.85.217.171]) by makus.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1WUZPi-0004Fu-PS for pgsql-performance@postgresql.org; Mon, 31 Mar 2014 10:26:00 +0000 Received: by mail-lb0-f171.google.com with SMTP id w7so5726107lbi.30 for ; Mon, 31 Mar 2014 03:25:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:content-type:content-transfer-encoding :subject:message-id:date:to:mime-version; bh=iAdw6X/TveiIA+sBWFSNVH5RZNhM8ENOyKCikjQ0M1U=; b=Y78/VOKpbwm6+s+w0jd7copu4TuZKBOgxm+bcMURjAygPtRn1yt/fmj8OwTVr58YpD UVCUAilvvR94e72ZbR5752GC3trhqxDjpva+ISpUWtgEpWvSJHhxEmBntsJly3QvKCEt 5FGtEbF/Bg9tuDdY7a30+AzjMOZTKqTfEJbwSheWxMV5LE/uMyTAFRiV3WLawoGmo3jV Uf5BBXlVS97oH7A91t984556M1rtiRNM9k+gknUnywLrA8v/OFgPapgJUnyK43nDvhKC 191z1cWS6LOzscpO6+1KP5k+kED2Fr6nobb0+dgpwdXA+TSPRY7Wz+RZy3wxFJbBsjH0 aKYw== X-Gm-Message-State: ALoCoQmptcNm0yuKwto3+yI1u9twweYxtdKNSri6X9RpcxpSGVeurnGaApHLMuURXUhAdLPsP/OG X-Received: by 10.152.7.97 with SMTP id i1mr1148894laa.36.1396261556810; Mon, 31 Mar 2014 03:25:56 -0700 (PDT) Received: from [192.168.1.45] (0905ds8-ar.0.fullrate.dk. [90.185.23.222]) by mx.google.com with ESMTPSA id cr6sm9640943lbb.19.2014.03.31.03.25.54 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 31 Mar 2014 03:25:55 -0700 (PDT) From: =?windows-1252?Q?Niels_Kristian_Schj=F8dt?= Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Subject: Sudden crazy high CPU usage Message-Id: <24BBD031-7384-4FFF-848B-90F5664C0063@autouncle.com> Date: Mon, 31 Mar 2014 12:25:52 +0200 To: "pgsql-performance@postgresql.org list" Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) X-Mailer: Apple Mail (2.1874) X-Pg-Spam-Score: -2.6 (--) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-performance Precedence: bulk Sender: pgsql-performance-owner@postgresql.org I'm running postgresql 9.3 on a production server. An hour ago, out of the = "blue", I ran into an issue I have never encountered before: my server star= ted to use CPU as crazy. The server is a standard ubuntu 12.04 LTE installa= tion running only Postgres and Redis. The incident can be seen on the in numbers below:=20 https://s3-eu-west-1.amazonaws.com/autouncle-public/other/cpu.png I imidiatly took a look at pg_stat_activity but nothing in there seemed sus= picious. I also had a look at the postgres log, but nothing was in there to= o. I have pg_stat_statements running, so I reseted that one, and nothing re= ally suspicious occurred in there, expect for the fact, that all queries we= re taking 100x times longer than usual. I have tried the following with no luck: =95 Restart clients connecting to the db =95 Restart postgres =95 Restart the whole server I have run memory tests on the server as well, and nothing seems to be wron= g. No changes in any software running on the servers has been made within the = last 24 hours. The question is: I have a streaming replication server running, which I hav= e now done a failover to, and it runs fine. However I still have no clue wh= y my master suddenly has become so CPU consuming, and how I can debug / tra= ce it further down? --=20 Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-performance