pg.ddx.io  pgsql-admin@postgresql.org mailing list archive  
help / color / mirror / Atom feed
From: jesper@krogh.cc
To: pgsql-admin@postgresql.org <pgsql-admin@postgresql.org>
Subject: Large memory machine and PG 9.2.9
Date: Tue, 24 Feb 2015 21:19:59 +0100
Message-ID: <fa77ad64f4b8bd5f0f5f0a1ef5dd9871.squirrel@shrek.krogh.cc> (raw)
List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-admin>

Hi.

We have just moved our 9.2.9 instance onto new beefy iron. The machine has
48 Intel cores and 3TB of memory. Running linux 3.13.0-43 (Ubuntu 12.04).

The problem is a bit hard to describe, but I do suspect it is related to
the large memory and probably kernel og pg-kernel interaction.

Performance is nice up and until the point where the memory is full at
which point sluggish behaviour comes up. sar -B output is.

17:30:01     29513.79 140026.51 634164.09      0.00 160093.42      0.00   
  0.00      0.00      0.00
17:35:01     31351.13 154801.18 638880.17      0.12 184323.22      0.00   
  0.00      0.00      0.00
17:40:01     38269.69 128701.23 652375.35      0.23 176369.40      0.00   
  0.00      0.00      0.00
17:45:01     34834.14 135371.82 627765.26      0.00 169779.58      0.00   
  0.00      0.00      0.00
17:50:01     34039.17 134630.64 627500.30      0.00 174259.04      0.00   
  0.00      0.00      0.00
17:55:01     30318.70 150791.13 612534.75      0.00 163425.51      0.00   
  0.00      0.00      0.00
18:05:01     28446.80 122103.38 549891.01      0.26 141756.52      0.00   
612.77    556.60     90.83
18:10:01     12944.16  39222.64 332848.27      4.27  82317.48      0.00  
3037.11   2725.14     89.73
18:15:01     12955.10  47841.51 453714.33      3.95 106397.71   1018.39  
4811.64   5421.81     93.00
18:20:01     16393.43  64063.10 548341.21      2.48 149489.93   6447.13  
2238.06   8537.87     98.30
18:25:01     15725.89  59096.20 502043.56      0.27 152932.96   5197.78  
2783.81   7782.02     97.50
18:30:01     12735.95  50460.08 394507.90      0.09 143488.71   4645.20  
2141.35   6621.27     97.56
18:35:01     11995.37  52743.57 414669.31      0.02 134363.87   5096.32  
1708.52   6668.57     98.00
18:40:01     11448.30  43185.84 373441.27      0.35 109712.93   3247.41  
1772.93   4902.79     97.66
18:45:01     10959.95  44993.48 402033.19      0.04 115914.63   3157.24  
2393.26   5378.58     96.90
18:50:01     11270.25  50853.00 431117.15      0.30 105697.26   3951.30  
1951.31   5778.41     97.90
18:55:01     10086.69  59206.44 362027.12      0.70 104760.65   6928.91  
1684.04   8497.73     98.66

Sluggish'ness starts at 18:10'ish and continues. Load drops and IO is very
small..

All good suggestions welcome.

sysctl changes
$ grep '^vm' /etc/sysctl.conf
vm.swappiness = 0
vm.dirty_background_ratio = 3
vm.dirty_ratio = 15
vm.dirty_expire_centisecs = 500
vm.dirty_writeback_centisecs = 100

Thanks

Jesper



-- 
Sent via pgsql-admin mailing list (pgsql-admin@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-admin



view thread (2+ messages)  latest in thread

Message-ID: <fa77ad64f4b8bd5f0f5f0a1ef5dd9871.squirrel@shrek.krogh.cc>
Permalink:  ../fa77ad64f4b8bd5f0f5f0a1ef5dd9871.squirrel@shrek.krogh.cc/
Also on:    postgresql.org/message-id/fa77ad64f4b8bd5f0f5f0a1ef5dd9871.squirrel@shrek.krogh.cc

 · 

reply

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Reply to all the recipients using the --to and --cc options:
  reply via email

  To: pgsql-admin@postgresql.org
  Cc: jesper@krogh.cc
  Subject: Re: Large memory machine and PG 9.2.9
  In-Reply-To: <fa77ad64f4b8bd5f0f5f0a1ef5dd9871.squirrel@shrek.krogh.cc>

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox