Received: from malur.postgresql.org ([217.196.149.56]) by arkaria.postgresql.org with esmtp (Exim 4.80) (envelope-from ) id 1YQLxk-0008Co-0H for pgsql-admin@arkaria.postgresql.org; Tue, 24 Feb 2015 20:20:12 +0000 Received: from localhost ([127.0.0.1] helo=postgresql.org) by malur.postgresql.org with smtp (Exim 4.80) (envelope-from ) id 1YQLxj-0005QN-Ew for pgsql-admin@arkaria.postgresql.org; Tue, 24 Feb 2015 20:20:11 +0000 Received: from makus.postgresql.org ([2001:4800:1501:1::229]) by malur.postgresql.org with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256) (Exim 4.80) (envelope-from ) id 1YQLxi-0005PW-9d for pgsql-admin@postgresql.org; Tue, 24 Feb 2015 20:20:10 +0000 Received: from 188-178-198-210-static.dk.customer.tdc.net ([188.178.198.210] helo=shrek.krogh.cc) by makus.postgresql.org with esmtps (TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256) (Exim 4.80) (envelope-from ) id 1YQLxe-0003q4-Ew for pgsql-admin@postgresql.org; Tue, 24 Feb 2015 20:20:08 +0000 Received: from localhost (localhost [127.0.0.1]) by shrek.krogh.cc (Postfix) with ESMTP id EFE3C1F01E20 for ; Tue, 24 Feb 2015 21:20:01 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at shrek.krogh.cc Received: from shrek.krogh.cc ([127.0.0.1]) by localhost (shrek.krogh.cc [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id wRTN2TeKkFm1 for ; Tue, 24 Feb 2015 21:19:59 +0100 (CET) Received: from shrek.krogh.cc (localhost [IPv6:::1]) by shrek.krogh.cc (Postfix) with ESMTP id 7B0991F00DBF for ; Tue, 24 Feb 2015 21:19:59 +0100 (CET) Received: from 188.178.198.210 (SquirrelMail authenticated user jesper@krogh.cc) by shrek.krogh.cc with HTTP; Tue, 24 Feb 2015 21:19:59 +0100 Message-ID: Date: Tue, 24 Feb 2015 21:19:59 +0100 Subject: Large memory machine and PG 9.2.9 From: jesper@krogh.cc To: "pgsql-admin@postgresql.org" User-Agent: SquirrelMail/1.4.23 [SVN] MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-Pg-Spam-Score: -0.0 (/) List-Archive: List-Help: List-ID: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: X-Mailing-List: pgsql-admin Precedence: bulk Sender: pgsql-admin-owner@postgresql.org Hi. We have just moved our 9.2.9 instance onto new beefy iron. The machine has 48 Intel cores and 3TB of memory. Running linux 3.13.0-43 (Ubuntu 12.04). The problem is a bit hard to describe, but I do suspect it is related to the large memory and probably kernel og pg-kernel interaction. Performance is nice up and until the point where the memory is full at which point sluggish behaviour comes up. sar -B output is. 17:30:01 29513.79 140026.51 634164.09 0.00 160093.42 0.00 0.00 0.00 0.00 17:35:01 31351.13 154801.18 638880.17 0.12 184323.22 0.00 0.00 0.00 0.00 17:40:01 38269.69 128701.23 652375.35 0.23 176369.40 0.00 0.00 0.00 0.00 17:45:01 34834.14 135371.82 627765.26 0.00 169779.58 0.00 0.00 0.00 0.00 17:50:01 34039.17 134630.64 627500.30 0.00 174259.04 0.00 0.00 0.00 0.00 17:55:01 30318.70 150791.13 612534.75 0.00 163425.51 0.00 0.00 0.00 0.00 18:05:01 28446.80 122103.38 549891.01 0.26 141756.52 0.00 612.77 556.60 90.83 18:10:01 12944.16 39222.64 332848.27 4.27 82317.48 0.00 3037.11 2725.14 89.73 18:15:01 12955.10 47841.51 453714.33 3.95 106397.71 1018.39 4811.64 5421.81 93.00 18:20:01 16393.43 64063.10 548341.21 2.48 149489.93 6447.13 2238.06 8537.87 98.30 18:25:01 15725.89 59096.20 502043.56 0.27 152932.96 5197.78 2783.81 7782.02 97.50 18:30:01 12735.95 50460.08 394507.90 0.09 143488.71 4645.20 2141.35 6621.27 97.56 18:35:01 11995.37 52743.57 414669.31 0.02 134363.87 5096.32 1708.52 6668.57 98.00 18:40:01 11448.30 43185.84 373441.27 0.35 109712.93 3247.41 1772.93 4902.79 97.66 18:45:01 10959.95 44993.48 402033.19 0.04 115914.63 3157.24 2393.26 5378.58 96.90 18:50:01 11270.25 50853.00 431117.15 0.30 105697.26 3951.30 1951.31 5778.41 97.90 18:55:01 10086.69 59206.44 362027.12 0.70 104760.65 6928.91 1684.04 8497.73 98.66 Sluggish'ness starts at 18:10'ish and continues. Load drops and IO is very small.. All good suggestions welcome. sysctl changes $ grep '^vm' /etc/sysctl.conf vm.swappiness = 0 vm.dirty_background_ratio = 3 vm.dirty_ratio = 15 vm.dirty_expire_centisecs = 500 vm.dirty_writeback_centisecs = 100 Thanks Jesper -- Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-admin